Clean Code Principles And Patterns: Python Edition

1: About the Author....12

2: Introduction....13

3: Architectural Principles and Patterns....19

3.1: Software Hierarchy....20

3.2: Single Responsibility Principle....23

3.3: Uniform Naming Principle....28

3.4: Encapsulation Principle....30

3.5: Service Aggregation Principle....31

3.6: High Cohesion, Low Coupling Principle....47

3.7: Library Composition Principle....48

3.8: Avoid Duplication Principle....49

3.9: Externalized Service Configuration Principle....50

3.9.1: Environment Variables....51

3.9.2: Kubernetes ConfigMaps....53

3.9.3: Kubernetes Secrets....55

3.10: Service Substitution Principle....56

3.11: Inter-Service Communication Methods....57

3.11.1: Synchronous Communication Method....57

3.11.2: Asynchronous Communication Method....58

3.11.3: Shared Data Communication Method....60

3.12: Strategic Domain-Driven Design Principle....61

3.12.1: Design Example 1: Mobile Telecom Network Analytics Software System....64

3.12.2: Design Example 2: Banking Software System....69

3.13: Autopilot Microservices Principle....71

3.13.1: Stateless Microservices Principle....72

3.13.2: Resilient Microservices Principle....72

3.13.3: Horizontally Autoscaling Microservices Principle....75

3.13.4: Highly-Available Microservices Principle....76

3.13.5: Observable Microservices Principle....78

3.14: Software Versioning Principles....78

3.14.1: Use Semantic Versioning Principle....79

3.14.2: Avoid Using 0.x Versions Principle....79

3.14.3: Don’t Increase Major Version Principle....79

3.14.4: Implement Security Patches and Bug Corrections to All Major Versions Principle....81

3.14.5: Avoid Using Non-LTS Versions in Production Principle....81

3.15: Git Version Control Principle....81

3.15.1: Feature Branch....82

3.15.2: Feature Toggle....83

3.16: Architectural Patterns....85

3.16.1: Event Sourcing Pattern....85

3.16.2: Command Query Responsibility Segregation (CQRS) Pattern....87

3.16.3: Distributed Transaction Patterns....88

3.16.3.1: Saga Orchestration Pattern....89

3.16.3.2: Saga Choreography Pattern....93

3.17: Preferred Technology Stacks Principle....95

4: Object-Oriented Design Principles....99

4.1: Object-Oriented Programming Concepts....100

4.1.1: Classes/Objects....101

4.1.2: Encapsulation....102

4.1.3: Abstraction....102

4.1.4: Inheritance....102

4.1.5: Interface....103

4.1.5.1: Interface evolution....105

4.1.6: Polymorphism....106

4.2: Programming Paradigms....107

4.2.1: Imperative Programming....107

4.2.2: Functional Programming....108

4.3: Multi-Paradigm Programming Principle....112

4.4: Why is Object-Oriented Programming Hard?....113

4.5: SOLID Principles....115

4.5.1: Single Responsibility Principle....116

4.5.2: Open-Closed Principle....122

4.5.3: Liskov’s Substitution Principle....127

4.5.4: Interface Segregation and Multiple Inheritance Principle....130

4.5.5: Program Against Interfaces Principle (Generalized Dependency Inversion Principle)....139

4.6: Clean Microservice Design Principle....148

4.6.1: Real-Life Example....152

4.7: Vertical Slice Design Principle....164

4.8: Class Organization Principle....166

4.9: Uniform Naming Principle....168

4.9.1: Naming Interfaces and Classes....168

4.9.2: Naming Functions....171

4.9.2.1: Preposition in Function Name....174

4.9.2.2: Naming Method Pairs....176

4.9.2.3: Naming Boolean Functions (Predicates)....178

4.9.2.4: Naming Builder Methods....181

4.9.2.5: Naming Methods with Implicit Verbs....182

4.9.2.6: Naming Lifecycle Methods....183

4.9.2.7: Naming Function Parameters....183

4.10: Encapsulation Principle....184

4.10.1: Immutable Objects....185

4.10.2: Don’t Leak Modifiable Internal State Outside an Object Principle....186

4.10.3: Don’t Assign From a Method Parameter to a Modifiable Attribute....188

4.11: Prefer Composition Over Inheritance Principle....188

4.12: Tactical Domain-Driven Design Principle....196

4.12.1: Tactical DDD Concepts....197

4.12.1.1: Entities....197

4.12.1.2: Value Objects....197

4.12.1.3: Aggregates....198

4.12.1.4: Aggregate Roots....199

4.12.2: Actors....201

4.12.2.1: Factories....201

4.12.2.2: Repositories....201

4.12.2.3: Services....201

4.12.2.4: Events....202

4.12.2.5: Event Storming....202

4.12.3: Domain-Driven Design Example: Data Exporter Microservice....202

4.12.4: Domain-Driven Design Example 2: Anomaly Detection Microservice....216

4.13: Design Patterns....219

4.13.1: Design Patterns for Creating Objects....220

4.13.1.1: Factory Pattern....220

4.13.1.2: Abstract Factory Pattern....222

4.13.1.3: Static Factory Method Pattern....225

4.13.1.4: Builder Pattern....229

4.13.1.5: Singleton Pattern....232

4.13.1.6: Prototype Pattern....235

4.13.1.7: Object Pool Pattern....237

4.13.2: Structural Design Patterns....239

4.13.2.1: Composite Pattern....239

4.13.2.2: Facade Pattern....242

4.13.2.3: Bridge Pattern....243

4.13.2.4: Strategy Pattern....248

4.13.2.5: Adapter Pattern....249

4.13.2.6: Proxy Pattern....254

4.13.2.7: Decorator Pattern....257

4.13.2.8: Flyweight Pattern....261

4.13.3: Behavioral Design Patterns....264

4.13.3.1: Chain of Responsibility Pattern....264

4.13.3.2: Observer Pattern....266

4.13.3.3: Command/Action Pattern....268

4.13.3.4: Iterator Pattern....271

4.13.3.5: State Pattern....272

4.13.3.6: Mediator Pattern....275

4.13.3.7: Template Method Pattern....295

4.13.3.8: Memento Pattern....297

4.13.3.9: Visitor Pattern....298

4.13.3.10: Null Object Pattern....303

4.14: Don’t Ask, Tell Principle....304

4.15: Law of Demeter....307

4.16: Avoid Primitive Obsession Principle....308

4.17: Dependency Injection (DI) Principle....317

4.18: Avoid Code Duplication Principle....324

4.19: Inheritance in Cascading Style Sheets (CSS)....327

5: Coding Principles....330

5.1: Uniform Variable Naming Principle....330

5.1.1: Naming Integer Variables....331

5.1.2: Naming Floating-Point Number Variables....332

5.1.3: Naming Boolean Variables....333

5.1.4: Naming String Variables....335

5.1.5: Naming Enum Variables....336

5.1.6: Naming Collection (List and Set) Variables....336

5.1.7: Naming Dictionary Variables....337

5.1.8: Naming Pair and Tuple Variables....338

5.1.9: Naming Object Variables....339

5.1.10: Naming Optional Variables....339

5.1.11: Naming Function Variables (Callbacks)....340

5.1.12: Naming Class Attributes....342

5.1.13: General Naming Rules....343

5.1.13.1: Use Short, Common Names....343

5.1.13.2: Pick One Term And Use It Consistently....343

5.1.13.3: Avoid Obscure Abbreviations....343

5.1.13.4: Avoid Too Short Or Meaningless Names....344

5.2: Uniform Source Code Repository Structure Principle....345

5.3: Domain-Based Source Code Structure Principle....346

5.4: Avoid Comments Principle....356

5.4.1: Name Things Properly....357

5.4.2: Single Return Of Named Value At The End Of Function....360

5.4.3: Return Type Aliasing....362

5.4.4: Extract Constant for Boolean Expression....363

5.4.5: Extract Named Constant or Enumerated Type....363

5.4.6: Extract Function....364

5.4.7: Avoid Comments for Regular Expression....365

5.4.8: Avoiding Comments in Bash Shell Scripts....366

5.5: Function Single Return Principle....368

5.6: Use Type Annotations for Production Code Principle....372

5.6.1: Function Arguments Might Be Given in Wrong Order....372

5.6.2: Function Argument Might Be Given with Wrong Type....373

5.6.3: Function Return Value Type Might Be Misunderstood....373

5.6.4: Refactoring Code Is More Difficult....373

5.6.5: Forced to Write Public API Comments....374

5.6.6: Type Errors Are Not Found in Testing....374

5.7: Refactoring Principle....374

5.7.1: Rename....376

5.7.2: Extract Method....377

5.7.3: Extract Class....377

5.7.4: Extract Constant....379

5.7.5: Replace Conditionals with Polymorphism....381

5.7.6: Introduce Parameter Object....383

5.7.7: Invert If-Statement....384

5.7.8: Creating Rich Object....385

5.8: Static Code Analysis Principle....389

5.8.1: Common Static Code Analysis Issues....390

5.9: Error/Exception Handling Principle....393

5.9.1: Returning Errors....405

5.9.1.1: Returning Failure Indicator....406

5.9.1.2: Returning an Optional Value....406

5.9.1.3: Returning an Error Object....407

5.9.1.4: Adapt to Wanted Error Handling Mechanism....411

5.9.1.5: Functional Exception Handling....412

5.10: Avoid Off-By-One Errors Principle....418

5.11: Be Critical When Googling or Using Generative AI Principle....419

5.12: Make One Change At A Time Principle....421

5.13: Use Appropriate Data Structure Principle....422

5.13.1: List....422

5.13.2: Dictionary....423

5.13.3: Tuple....423

5.13.4: Set....423

5.13.5: String....424

5.13.6: Bytearray....424

5.13.7: Counter....424

5.13.8: OrderedDict....424

5.13.9: Deque (Double Ended Queue)....425

5.13.10: Customized Built-In Collections....425

5.13.11: Stack (LIFO Queue)....425

5.13.12: Queue (FIFO Queue)....425

5.13.13: Priority Queue....426

5.13.14: Synchronized Queues (LIFO/FIFO)....426

5.14: Optimization Principle....426

5.14.1: Optimization Patterns....427

5.14.1.1: Optimize Busy Loops Only Pattern....427

5.14.1.2: Remove Unnecessary Functionality Pattern....428

5.14.1.3: Object Pool Pattern....429

5.14.1.4: Use Optimal Data Structures Pattern....429

5.14.1.5: Algorithm Complexity Reduction Pattern....429

5.14.1.6: Cache Function Results Pattern....430

5.14.1.7: Buffer File I/O Pattern....431

5.14.1.8: Share Identical Objects a.k.a Flyweight Pattern....431

6: Testing Principles....432

6.1: Functional Testing Principles....432

6.1.1: Unit Testing Principle....433

6.1.1.1: Test-Driven Development (TDD)....437

6.1.1.2: Unit Specification-Driven Development (USDD)....451

6.1.1.3: Naming Conventions....457

6.1.1.4: Mocking....457

6.1.1.5: Web UI Component Unit Testing....473

6.1.2: Software Component Integration Testing Principle....475

6.1.2.1: Web UI Integration Testing....484

6.1.2.2: Setting Up Integration Testing Environment....486

6.1.3: Complete Example with BDD, ATDD, DDD, OOD and TDD....489

6.1.4: End-to-End (E2E) Testing Principle....530

6.2: Non-Functional Testing Principle....534

6.2.1: Performance Testing....534

6.2.2: Data Volume Testing....536

6.2.3: Stability Testing....536

6.2.4: Reliability Testing....537

6.2.5: Stress and Scalability Testing....538

6.2.6: Security Testing....539

6.2.7: Other Non-Functional Testing....540

6.2.7.1: Visual Testing....540

7: Security Principles....541

7.1: Shift Security to Left Principle....541

7.2: Have a Product Security Lead Principle....541

7.3: Use Threat Modelling Process Principle....542

7.3.1: Decompose Application....542

7.3.2: Determine and Rank Threats....542

7.3.2.1: STRIDE Threat Examples....543

7.3.3: Determine Countermeasures and Mitigation....545

7.3.4: Threat Modeling Example using STRIDE....546

7.3.4.1: Decompose Application....546

7.3.4.2: Determine and Rank Threats....548

7.3.5: Determine Countermeasures and Mitigation....550

7.3.6: Threat Modeling Example Using ASF....553

7.4: Security Features....555

7.4.1: Authentication and Authorization....556

7.4.1.1: OpenID Connect Authentication and Authorization in Frontend....556

7.4.1.2: OAuth2 Authorization in Backend....570

7.4.2: Password Policy....579

7.4.3: Cryptography....580

7.4.3.1: Encryption Key Lifetime and Rotation....581

7.4.4: Denial-of-service (DoS) Prevention....582

7.4.5: Database Security....582

7.4.6: SQL Injection Prevention....583

7.4.7: OS Command Injection Prevention....583

7.4.8: Security Configuration....584

7.4.9: Automatic Vulnerability Scanning....585

7.4.10: Integrity....586

7.4.11: Error Handling....586

7.4.12: Logging....586

7.4.13: Audit Logging....587

7.4.14: Input Validation....587

7.4.14.1: Validating Numbers....589

7.4.14.2: Validating Strings....589

7.4.14.3: Validating Timestamps....589

7.4.14.4: Validating Arrays....589

7.4.14.5: Validating Objects....589

7.4.14.6: Validating Files Uploaded to Server....590

8: API Design Principles....591

8.1: Frontend Facing API Design Principles....591

8.1.1: JSON-based RPC API Design Principle....591

8.1.2: REST API Design Principle....596

8.1.2.1: Creating a Resource....596

8.1.2.2: Reading Resources....599

8.1.2.3: Updating Resources....603

8.1.2.4: Deleting Resources....606

8.1.2.5: Executing Non-CRUD Actions on Resources....606

8.1.2.6: Resource Composition....607

8.1.2.7: HTTP Status Codes....609

8.1.2.8: HATEOAS and HAL....611

8.1.2.9: API Versioning....612

8.1.2.10: Documentation....613

8.1.2.11: Implementation Example....614

8.1.3: GraphQL API Design....627

8.1.4: Subscription-Based API Design....644

8.1.4.1: Server-Sent Events (SSE)....644

8.1.4.2: GraphQL Subscriptions....646

8.1.5: WebSocket Example....648

8.2: Inter-Microservice API Design Principles....663

8.2.1: Synchronous API Design Principle....663

8.2.1.1: gRPC-Based API Design Example....664

8.2.2: Asynchronous API Design Principle....672

8.2.2.1: Request-Only Asynchronous API Design....672

8.2.2.2: Request-Response Asynchronous API Design....674

9: Databases And Database Principles....677

9.1: Relational Databases....677

9.1.1: Structure of Relational Database....677

9.1.2: Use Object Relational Mapper (ORM) Principle....678

9.1.3: Entity/Table Relationships....684

9.1.3.1: One-To-One/Many Relationships....684

9.1.3.2: Many-To-Many Relationships....685

9.1.3.3: Sales Item Repository Example....688

9.1.4: Use Parameterized SQL Statements Principle....690

9.1.5: Normalization Rules....699

9.1.5.1: First Normal Form (1NF)....700

9.1.5.2: Second Normal Form (2NF)....700

9.1.5.3: Third Normal Form (3NF)....700

9.2: Document Databases....701

9.3: Key-Value Database Principle....706

9.4: Wide-Column Databases....707

9.5: Search Engines....713

10: Concurrent Programming Principles....714

10.1: Threading Principle....714

10.1.1: Parallel Executors....716

10.2: Thread Safety Principle....716

10.2.1: Use Locking for Mutual Exclusion (Mutex)....717

10.2.2: Atomic Variables....717

10.2.3: Concurrent Collections....718

10.3: Publish/Subscribe Shared State Change Principle....719

11: Teamwork Principles....721

11.1: Use Agile Framework Principle....721

11.2: Define the Done Principle....723

11.3: You Write Code for Other People Principle....725

11.4: Avoid Technical Debt Principle....725

11.5: Software Component Documentation Principle....727

11.6: Code Review Principle....730

11.6.1: Focus on Object-Oriented Design....730

11.6.2: Focus on Function Specification by Unit Tests....731

11.6.3: Focus on Proper and Uniform Naming....731

11.6.4: Don’t Focus on Premature Optimization....731

11.6.5: Detect Possible Malicious Code....731

11.7: Uniform Code Formatting Principle....732

11.8: Highly Concurrent Development Principle....732

11.8.1: Dedicated Microservices and Microlibraries....732

11.8.2: Dedicated Domains....733

11.8.3: Follow Open-Closed Principle....733

11.9: Pair Programming Principle....734

11.10: Well-Defined Development Team Roles Principle....735

11.10.1: Product Owner....735

11.10.2: Scrum Master....736

11.10.3: Software Developer....736

11.10.4: Test Automation Developer....737

11.10.5: DevOps Engineer....737

11.10.6: UI Designer....738

11.11: Competence Transfer Principle....738

12: DevSecOps....740

12.1: SecOps Lifecycle....741

12.2: DevOps Lifecycle....741

12.2.1: Plan....742

12.2.2: Code....742

12.2.3: Build and Test....743

12.2.4: Release....744

12.2.4.1: Example Dockerfile....745

12.2.4.2: Example Kubernetes Deployment....745

12.2.4.3: Example CI/CD Pipeline....749

12.2.5: Deploy....755

12.2.6: Operate....756

12.2.7: Monitor....757

12.2.7.1: Logging to Standard Input....757

12.2.7.2: Distributed Tracing....758

12.2.7.3: Metrics Collection....758

12.2.7.4: Metrics Visualization....759

12.2.7.5: Alerting....759

12.2.8: Software System Alerts Dashboard Example....760

12.2.9: Microservice Grafana Dashboard Example....760

12.2.9.1: Logging....762

12.2.9.2: OpenTelemetry Log Data Model....763

12.2.9.3: PrometheusRule Example....764

13: Conclusion....765

14: Appendix A....769

15: Appendix B....773

Clean Code Principles and Patterns is one of the most comprehensive no-fluff guides for software developers to help them write clean code every day. The author, Petri Silén, has almost 30 years of industry experience in designing and implementing software, and now he puts all his knowledge gained during the years into this book. The book is packed with principles and patterns that help developers, from novices and juniors to seniors and experts, to write cleaner code. The principles and patterns presented in the book are accompanied by realistic yet straightforward examples to help the reader understand them better. Examples are written in Python. Most of the examples are directly applicable to other programming languages, too. The reader should have basic knowledge of Python to get the full benefit from this book. The source code for all major examples presented in the book is available in a public GitHub repository.

After reading this book, you will know the following and much more:

• How to design modern cloud-native microservices
• What are autopilot microservices
• What are event sourcing, CQRS, distributed transactions, saga orchestration pattern, and saga choreography pattern
• What are the five SOLID principles, and how to put them into use in real-life code
• What are the 25 design patterns, and how to use them
• What is the MVC pattern, and how MVP and MVVM differ from each other
• How to achieve a clean design for a microservice
• Why and how to use dependency injection
• Detailed instructions with concrete examples on how to uniformly name various software entities like classes, functions, and variables
• Why you should prefer composition over inheritance
• How to split a software system into domains using domain-driven design
• How to organize a source code repository
• How to organize code into directories
• Concrete ways how to avoid writing comments and refactor comments away
• What are the most common issues that static code analyzers find, and how to correct them
• The most important refactoring techniques for everyday use
• Why you should use a statically typed language
• How to correctly handle errors and exceptions
• How to not forget to handle errors and exceptions
• Why you should never pass or return a null value
• How to avoid off-by-one errors effectively
• What you should remember when using a Google search to get answers
• When and how to optimize code
• TDD, Unit testing, mocking, integration testing, E2E testing, and non-functional testing
• What is threat modeling and how to conduct it
• Authentication and authorization using OpenID Connect and OAuth2
• What are the essential security features to implement in an application
• How to design APIs using technologies like JSON-RPC, REST, GraphQL, SSE, WebSocket, gRPC, and event-driven services
• When and how to use a relational database, document database, key-value store, or wide-column database
• How to avoid SQL injection attacks using ORM or parameterized SQL queries
• When to use threading or parallel algorithms and how to ensure thread safety
• What principles to follow when working in a software development team
• What are DevOps, SecOps, and continuous integration (CI), and what is the difference between continuous delivery (CD) and continuous deployment (CD)