Dead Simple Python: Idiomatic Python for the Impatient Programmer

Title Page....25

Copyright....26

Dedication....28

About the Author....29

Foreword....31

Acknowledgments....33

Introduction....35

Who Is This Book For?....37

What Does “Simple” Mean?....37

What’s in This Book?....38

What’s NOT in This Book....39

How to Read This Book....40

About the Vocabulary....40

Theory Recaps....41

Objective or Opinionated?....41

The Examples....42

What About a Project?....42

Prerequisites....43

Part I: THE PYTHON ENVIRONMENT....45

Chapter 1: The Python Philosophy....46

What Is Python, Exactly?....46

Myths: What Python Isn’t....47

Myth #1: Python Is Merely a Scripting Language....47

Myth #2: Python Is Slow....48

Myth #3: Python Cannot Be Compiled....48

Myth #4: Python Gets Compiled Behind the Scenes....49

Myth #5: Python Is Unsuitable for Large Projects....50

Python 2 vs. Python 3....51

Defining “Pythonic” Code....52

The Zen of Python....53

Documentation, PEPs, and You....54

Who Calls the Shots?....54

The Python Community....55

The Pursuit of the One Obvious Way....56

Wrapping Up....58

Chapter 2: Your Workbench....59

Installing Python....59

Installing on Windows....60

Installing on macOS....60

Installing on Linux....61

Installing from Source....61

Meet the Interpreter....64

Interactive Session....64

Running Python Files....65

Packages and Virtual Environments....66

Creating a Virtual Environment....67

Activating a Virtual Environment....68

Leaving a Virtual Environment....69

Introducing pip....69

System-Wide Packages....69

Installing Packages....70

requirements.txt....71

Upgrading Packages....71

Removing Packages....72

Finding Packages....72

One Warning About pip . . .....72

Virtual Environments and Git....73

The Whole Shebang....74

File Encoding....76

A Few Extra Virtual Environment Tricks....77

Using a Virtual Environment Without Activating....77

The Alternatives....77

The Line Limit Debate....79

Tabs or Spaces?....80

Quality Control: Static Analyzers....81

Pylint....81

Flake8....84

Mypy....87

Style Janitors: Autoformatting Tools....87

autopep8....87

Black....88

Testing Frameworks....88

An Exhibition of Code Editors....88

IDLE....89

Emacs and Vim....89

PyCharm....89

Visual Studio Code....90

Sublime Text....91

Spyder....91

Eclipse + PyDev/LiClipse....91

The Eric Python IDE....92

Wrapping Up....92

Chapter 3: Syntax Crash Course....94

Hello, World!....94

Statements and Expression....95

The Importance of Whitespace....96

Doing Nothing....98

Comments and Docstrings....98

Docstrings....99

Declaring Variables....100

What About Constants?....102

Mathematics....102

Meet the Number Types....102

Operators....104

The math Module....106

Logic....107

Conditionals....107

Comparison Operators....108

Boolean, None, and Identity Operators....109

Truthiness....110

Logical Operators....111

The Walrus Operator....112

Ellipsis....113

Strings....113

String Literals....114

Raw Strings....116

Formatted Strings....117

Template Strings....123

String Conversion....124

A Note on String Concatenation....125

Functions....126

Classes and Objects....127

Error Handling....129

Tuples and Lists....129

Loops....131

while Loop....131

Loop Control....132

for Loop....132

Structural Pattern Matching....133

Literal Patterns and Wildcards....134

Or Patterns....134

Capture Patterns....135

Guard Statements....137

More About Structural Pattern Matching....137

Wrapping Up....138

Chapter 4: Project Structure and Imports....139

Setting Up the Repository....140

Modules and Packages....140

PEP 8 and Naming....142

Project Directory Structure....142

How import Works....144

Import Dos and Don’ts....145

Importing Functions from Modules....146

The Problem of Shadowing....147

The Trouble with Nested Packages....149

Beware of Importing All....150

Importing Within Your Project....152

Absolute Imports....152

Relative Imports....153

Importing from the Same Package....154

Entry Points....155

Module Entry Points....155

Package Entry Points....157

Controlling Package Imports....158

Program Entry Points....160

The Python Module Search Path....161

What Really Happens....163

Wrapping Up....165

Part II: ESSENTIAL STRUCTURES....166

Chapter 5: Variables and Types....167

Variables According to Python: Names and Values....167

Assignment....168

Data Types....171

The type() Function....172

Duck Typing....173

Scope and Garbage Collection....174

Local Scope and the Reference-Counting Garbage Collector....174

Interpreter Shutdown....176

Global Scope....176

The Dangers of Global Scope....179

The nonlocal Keyword....179

Scope Resolution....180

The Curious Case of the Class....181

Generational Garbage Collector....182

The Immutable Truth....182

Passing by Assignment....183

Collections and References....186

Shallow Copy....189

Deep Copy....191

Coercion and Conversion....193

A Note About Systems Hungarian Notation....195

Terminology Review....196

Wrapping Up....198

Chapter 6: Functions and Lambdas....200

Python Function Essentials....203

Recursion....205

Default Argument Values....208

Keyword Arguments....211

On Overloaded Functions....213

Variadic Arguments....214

Keyword Variadic Arguments....216

Keyword-Only Parameters....217

Positional-Only Parameters....218

Argument Types: All Together Now!....219

Nested Functions....220

Closures....221

Recursion with Closures....222

Stateful Closures....223

Lambdas....225

Why Lambdas Are Useful....226

Lambdas as Sorting Keys....229

Decorators....230

Type Hints and Function Annotations....234

Duck Typing and Type Hints....237

Should You Use Type Hinting?....237

Wrapping Up....238

Chapter 7: Objects and Classes....239

Declaring a Class....241

The Initializer....242

The Constructor....243

The Finalizer....244

Attributes....246

Instance Attributes....246

Class Attributes....247

Scope-Naming Conventions....248

Nonpublic....248

Public....249

Name Mangling....249

Public, Nonpublic, or Name Mangled?....250

Methods....251

Instance Methods....251

Class Methods....252

Static Methods....253

Properties....254

Setting Up the Scenario....254

Defining a Property....256

Property with Decorators....260

When Not to Use Properties....262

Special Methods....263

Scenario Setup....264

Conversion Methods....265

Comparison Methods....270

Binary Operator Support....272

Unary Operator Support....273

Making Callable....274

More Special Methods: Looking Ahead....276

Class Decorators....276

Structural Pattern Matching with Objects....278

Functional Meets Object Oriented....281

When to Use Classes....282

Classes Aren’t Modules....282

Single Responsibility....283

Sharing State....283

Are Objects Right for You?....284

Wrapping Up....284

Chapter 8: Errors and Exceptions....286

Exceptions in Python....286

Reading Tracebacks....289

Catching Exceptions: LBYL vs. EAFP....291

Multiple Exceptions....293

Beware the Diaper Anti-pattern....295

Raising Exceptions....298

Using Exceptions....300

Exceptions and Logging....302

Bubbling Up....308

Exception Chaining....309

Else and Finally....311

Else: “If All Goes Well”....312

Finally: “After Everything”....314

Creating Exceptions....318

A Gallery of Exceptions....320

Wrapping Up....323

Part III: DATA AND FLOW....324

Chapter 9: Collections and Iteration....325

Loops....325

while Loops....326

for Loops....328

Collections....329

Tuples....329

Named Tuples....330

Lists....331

Deques....333

Sets....334

frozenset....336

Dictionaries....337

Check or Except?....338

Dictionary Variants....340

Unpacking Collections....341

Starred Expressions....343

Unpacking Dictionaries....344

Structural Pattern Matching on Collections....345

Accessing by Index or Key....348

Slice Notation....349

Start and Stop....350

Negative Indices....351

Steps....352

Copy with Slice....354

Slice Objects....354

Slicing on Custom Objects....355

Using islice....355

The in Operator....356

Checking Collection Length....357

Iteration....358

Iterables and Iterators....359

Manually Using Iterators....359

Iterating with for Loops....363

Sorting Collections in Loops....365

Enumerating Loops....366

Mutation in Loops....367

Loop Nesting and Alternatives....370

Iteration Tools....372

Basic Built-in Tools....372

Filter....373

Map....374

Zip....376

Itertools....377

Custom Iterable Classes....379

Wrapping Up....384

Chapter 10: Generators and Comprehensions....385

Lazy Evaluation and Eager Iterables....385

Infinite Iterators....386

Generators....387

Generators vs. Iterator Classes....390

Closing Generators....394

Behavior on Close....396

Throwing Exceptions....397

yield from....400

Generator Expressions....402

Generator Objects Are Lazy....404

Generator Expressions with Multiple Loops....406

Conditionals in Generator Expressions....407

Nested Generator Expressions....411

List Comprehensions....412

Set Comprehensions....414

Dictionary Comprehensions....414

Hazards of Generator Expressions....415

They Quickly Become Unreadable....416

They Don’t Replace Loops....417

They Can Be Hard to Debug....418

When to Use Generator Expressions....419

Simple Coroutines....420

Returning Values from a Coroutine....423

Sequence of Behavior....424

What About Async?....425

Wrapping Up....426

Chapter 11: Text IO and Context Managers....427

Standard Input and Output....427

Revisiting print()....428

Revisiting input()....434

Streams....435

Context Manager Basics....437

File Modes....438

Reading Files....441

The read() Method....441

The readline() Method....442

The readlines() Method....443

Reading with Iteration....444

Stream Position....445

Writing Files....446

The write() Method....447

The writelines() Method....449

Writing Files with print()....451

Line Separators....452

Context Manager Details....453

How Context Managers Work....453

Using Multiple Context Managers....455

Implementing the Context Management Protocol....455

The __enter__() Method....457

The __exit__() Method....458

Using the Custom Class....459

Paths....462

Path Objects....463

Parts of a Path....465

Creating a Path....470

Relative Paths....473

Paths Relative to Package....474

Path Operations....478

Out-of-Place File Writes....480

The os Module....482

File Formats....482

JSON....483

Other Formats....487

Wrapping Up....492

Chapter 12: Binary and Serialization....493

Binary Notation and Bitwise....493

Number Systems Refresher....494

Python Integers and Binary....501

Bitwise Operations....502

Bytes Literals....505

Bytes-Like Objects....507

Creating a bytes Object....508

Using int.to_bytes()....510

Sequence Operations....512

Converting bytes to int....513

struct....513

struct Format String and Packing....514

Unpacking with struct....518

struct objects....520

Bitwise on Bytes-Like Objects....521

Bitwise Operations via Integers....521

Bitwise Operations via Iteration....523

memoryview....526

Reading and Writing Binary Files....529

Organizing the Data....530

Writing to a File....535

Reading from a Binary File....536

Seek with Binary Stream....540

BufferedRWPair....541

Serialization Techniques....543

Forbidden Tools: pickle, marshal, and shelve....545

Serialization Formats....547

Wrapping Up....550

Part IV: ADVANCED CONCEPTS....552

Chapter 13: Inheritance and Mixins....553

When to Use Inheritance....556

Crimes of Inheritance....557

Basic Inheritance in Python....558

Multiple Inheritance....561

Method Resolution Order....562

Ensuring Consistent Method Resolution Order....567

Explicit Resolution Order....571

Resolving Base Class in Multiple Inheritance....572

Mixins....576

Wrapping Up....579

Chapter 14: Metaclasses and ABCs....581

Metaclasses....581

Creating Classes with type....582

Custom Metaclasses....584

Type Expectations with Duck Typing....587

EAFP: Catching Exceptions....588

LBYL: Checking for Attributes....588

Abstract Classes....591

Built-in ABCs....592

Deriving from ABCs....593

Implementing Custom ABCs....597

Virtual Subclasses....601

Setting Up the Example....601

Using Virtual Subclassing....604

Wrapping Up....608

Chapter 15: Introspection and Generics....609

Special Attributes....609

Inside Object Attribute Access: The __dict__ Special Attribute....610

Listing Attributes....613

Getting an Attribute....614

Checking for an Attribute....617

Setting an Attribute....618

Deleting an Attribute....620

Function Attributes....620

The Wrong Way to Use Function Attributes....621

Mutability and Function Attributes....623

Descriptors....625

The Descriptor Protocol....625

Writing a Descriptor Class (the Slightly Wrong Way)....626

Using a Descriptor....628

Writing a Descriptor Class the Right Way....630

Using Multiple Descriptors in the Same Class....632

Slots....635

Binding Attribute Names to Values....636

Using Arbitrary Attributes with Slots....637

Slots and Inheritance....638

Immutable Classes....638

Single-Dispatch Generic Functions....642

Registering Single-Dispatch Functions with Type Hints....644

Registering Single-Dispatch Functions with Explicit Type....645

Registering Single-Dispatch Functions with the register() Method....646

Using the Element Class....647

Arbitrary Execution....649

Wrapping Up....651

Chapter 16: Asynchrony and Concurrency....653

Asynchrony in Python....655

The Example Scenario: Collatz Game, Synchronous Version....658

Asynchrony....662

Native Coroutines....663

Tasks....666

The Event Loop....667

Making It (Actually) Asynchronous....669

Scheduling and Asynchronous Execution Flow....671

Simplifying the Code....673

Asynchronous Iteration....674

Asynchronous Context Managers....678

Asynchronous Generators....678

Other Asynchrony Concepts....678

Wrapping Up....679

Chapter 17: Threading and Parallelism....681

Threading....682

Concurrency vs. Parallelism....682

Basic Threading....683

Timeouts....687

Daemonic Threads....688

Futures and Executors....689

Timeouts with Futures....693

Race Conditions....695

A Race Condition Example....697

Creating Multiple Threads with ThreadPoolExecutor....699

Locks....701

Deadlock, Livelock, and Starvation....702

Passing Messages with Queue....705

Futures with Multiple Workers....707

Achieving Parallelism with Multiprocessing....709

Pickling Data....710

Speed Considerations and ProcessPoolExecutor....712

The Producer/Consumer Problem....714

Importing the Modules....716

Monitoring the Queue....717

Subprocess Cleanup....717

Consumers....718

Checking for an Empty Queue....719

Producers....720

Starting the Processes....721

Performance Results....722

Logging with Multiprocessing....723

Wrapping Up....723

Part V: BEYOND THE CODE....726

Chapter 18: Packaging and Distribution....727

Planning Your Packaging....728

The Dangers of Cargo Cult Programming....729

A Note on Packaging Opinions....730

Determining Your Packaging Goals....730

Project Structure: src or src-less....733

Packaging a Distribution Package with setuptools....734

Project Files and Structure....735

Where Metadata Belongs....735

The README.md and LICENSE Files....736

The setup.cfg File....737

The setup.py File....746

The MANIFEST.in File....747

The requirements.txt File....748

The pyproject.toml File....750

Testing the Setup Configuration....751

Building Your Package....753

Publishing on pip (Twine)....754

Uploading to Test PyPI....754

Installing Your Uploaded Package....756

Uploading to PyPI....757

Alternative Packaging Tools....757

Poetry....758

Flit....758

Distributing to End Users....758

PEX....759

Freezers....760

Images and Containers....762

A Note on Native Linux Packaging....766

Documentation....767

Wrapping Up....768

Chapter 19: Debugging and Logging....770

Warnings....771

Types of Warnings....772

Filtering Warnings....774

Converting Warnings into Exceptions....776

Logging....777

Logger Objects....778

Handler Objects....780

Logging with Levels....781

Controlling the Log Level....783

Running the Example....785

Filter, Formatter, and Configuration....787

Assert Statements....787

Proper Use of assert....788

Wrong Use of assert....790

Seeing assert in Action....792

The inspect Module....793

Using pdb....793

A Debugging Example....794

Starting the Debugger....796

Debugger Shell Commands....797

Stepping Through the Code....798

Setting a Breakpoint and Stepping into a Function....799

Moving Through the Execution Stack....800

Inspecting the Source....802

Checking a Solution....804

Postmortem Debugging....805

Using faulthandler....806

Evaluating Your Program’s Security with Bandit....809

Reporting Bugs to Python....812

Wrapping Up....813

Chapter 20: Testing and Profiling....814

What About TDD?....815

Test Frameworks....815

The Example Project....817

Testing and Project Structure....818

Testing Basics....820

Starting the Example....821

Unit Testing....822

Executing the Tests with pytest....823

Testing for Exceptions....824

Test Fixtures....826

Continuing the Example: Using the API....829

Sharing Data Between Test Modules....831

Flaky Tests and Conditionally Skipping Tests....832

Advanced Fixtures: Mocking and Parametrizing....834

Continuing the Example: Representing a Typo....834

Parametrizing....837

Indirect Parametrization of Fixtures....838

Mocking Inputs with Monkeypatch....841

Marking....842

Capturing from Standard Streams....845

GUI Testing....847

Continuing the Example: Connecting the API to Typo....848

Autouse Fixtures....849

Mixed Parametrization....852

Fuzzing....853

Wrapping Up the Example....853

Code Coverage....855

Automating Testing with tox....858

Benchmarking and Profiling....861

Benchmarking with timeit....861

Profiling with cProfile or profile....863

tracemalloc....868

Wrapping Up....869

Chapter 21: The Parting of the Ways....870

About the Future....871

Where Do You Go from Here?....872

Application Development in Python....872

Game Development in Python....874

Web Development in Python....874

Client-Side Python....875

Data Science in Python....876

Machine Learning in Python....878

Security....880

Embedded Development in Python....881

Scripting....882

Python Flavors....882

Developing for Python....884

Developing Python Packages and Tools....884

Developing Python Extensions....885

Contributing to Python....887

Getting Involved with Python....888

Asking Questions....889

Answering Questions....890

User Groups....892

PyLadies....892

Conferences....892

Joining the Python Software Foundation....893

And the Road Goes Ever On . . .....894

Appendix A: Special Attributes and Methods....896

Special Methods....896

Special Attributes....907

Appendix B: Python Debugger (pdb) Commands....911

Appendix : Glossary....914

A....914

B....917

C....920

D....925

E....928

F....930

G....933

H....935

I....936

K....939

L....939

M....940

N....943

O....944

P....945

Q....950

R....950

S....951

T....957

U....959

V....960

W....960

Y....961

Z....961

Index....963

Dead Simple Python is a thorough introduction to every feature of the Python language for programmers who are impatient to write production code. Instead of revisiting elementary computer science topics, you’ll dive deep into idiomatic Python patterns so you can write professional Python programs in no time.After speeding through Python’s basic syntax and setting up a complete programming environment, you’ll learn to work with Python’s dynamic data typing, its support for both functional and object-oriented programming techniques, special features like generator expressions, and advanced topics like concurrency. You’ll also learn how to package, distribute, debug, and test your Python project.

Master how to:

• Make Python's dynamic typing work for you to produce cleaner, more adaptive code.
• Harness advanced iteration techniques to structure and process your data.
• Design classes and functions that work without unwanted surprises or arbitrary constraints.
• Use multiple inheritance and introspection to write classes that work intuitively.
• Improve your code's responsiveness and performance with asynchrony, concurrency, and parallelism.
• Structure your Python project for production-grade testing and distribution

The most pedantically pythonic primer ever printed, Dead Simple Python will take you from working with the absolute basics to coding applications worthy of publication.