Speed Up Your Python with Rust: Optimize Python performance by creating Python pip modules in Rust with Py03

Speed Up Your Python with Rust....2

Contributors....5

About the author....5

About the reviewers....6

Preface....22

Who this book is for....23

What this book covers....24

To get the most out of this book....25

Download the example code files....25

Download the color images....25

Conventions used....25

Get in touch....26

Share Your Thoughts....27

Section 1: Getting to Understand Rust....28

Chapter 1: An Introduction to Rust from a Python Perspective....29

Technical requirements....29

Understanding the differences between Python and Rust....30

Why fuse Python with Rust?....30

Passing strings in Rust....32

Sizing up floats and integers in Rust....35

Managing data in Rust's vectors and arrays....37

Replacing dictionaries with hashmaps....38

Error handling in Rust....42

Understanding variable ownership....45

Copy....46

Move....47

Immutable borrow....48

Mutable borrow....50

Keeping track of scopes and lifetimes....51

Building structs instead of objects....55

Metaprogramming with macros instead of decorators....58

Summary....61

Questions....62

Answers....62

Further reading....62

Chapter 2: Structuring Code in Rust....63

Technical requirements....63

Managing our code with crates and Cargo instead of pip....63

Structuring code over multiple files and modules....70

Building module interfaces....75

Benefits of documentation when coding....83

Interacting with the environment....84

Summary....87

Questions....87

Answers....88

Further reading....88

Chapter 3: Understanding Concurrency....89

Technical requirements....89

Introducing concurrency....89

Threads....89

Processes....91

Basic asynchronous programming with threads....92

Running multiple processes....99

Customizing threads and processes safely....112

Amdahl's law....113

Deadlocks....114

Race conditions....117

Summary....117

Questions....118

Answers....118

Further reading....119

Section 2: Fusing Rust with Python....120

Chapter 4: Building pip Modules in Python....121

Technical requirements....121

Configuring setup tools for a Python pip module....121

Creating a GitHub repository....122

Defining the basic parameters....125

Defining a README file....126

Defining a basic module....127

Packaging Python code in a pip module....128

Building our Fibonacci calculation code....129

Creating a command-line interface....131

Building unit tests....133

Configuring continuous integration....138

Manually deploying onto PyPI....139

Managing dependencies....141

Setting up type checking for Python....142

Setting up and running tests and type-checking with GitHub Actions....143

Create automatic versioning for our pip package....147

Deploying onto PyPI using GitHub Actions....150

Summary....153

Questions....153

Answers....154

Further reading....154

Chapter 5: Creating a Rust Interface for Our pip Module....155

Technical requirements....155

Packaging Rust with pip....155

Define gitignore and Cargo for our package....156

Configuring the Python setup process for our package....157

Installing our Rust library for our package....160

Building a Rust interface with the pyO3 crate....161

Building our Fibonacci Rust code....162

Creating command-line tools for our package....165

Creating adapters for our package....167

Building tests for our Rust package....175

Comparing speed with Python, Rust, and Numba....178

Summary....180

Questions....181

Answers....181

Further reading....181

Chapter 6: Working with Python Objects in Rust....182

Technical requirements....182

Passing complex Python objects into Rust....182

Updating our setup.py file to support .yml loading....183

Defining our .yml loading command....184

Processing data from our Python dictionary....185

Extracting data from our config file....188

Returning our Rust dictionary to our Python system....189

Inspecting and working with custom Python objects....192

Creating an object for our Rust interface....192

Acquiring the Python GIL in Rust....193

Adding data to our newly created PyDict struct....194

Setting the attributes of our custom object....197

Constructing our own custom Python objects in Rust....198

Defining a Python class with the required attributes....199

Defining class static methods to process input numbers....199

Defining a class constructor....200

Wrapping up and testing our module....201

Summary....204

Questions....205

Answers....205

Further reading....205

Chapter 7: Using Python Modules with Rust....206

Technical requirements....206

Exploring NumPy....206

Adding vectors in NumPy....207

Adding vectors in pure Python....208

Adding vectors using NumPy in Rust....210

Building a model in NumPy....213

Defining our model....213

Building a Python object that executes our model....216

Using NumPy and other Python modules in Rust....219

Recreating our NumPy model in Rust....222

Building get_weight_matrix and inverse_weight_matrix functions....223

Building get_parameters, get_times, and get_input_vector functions....224

Building calculate_parameters and calculate_times functions....225

Adding calculate functions to the Python bindings and adding a NumPy dependency to our setup.py file....226

Building our Python interface....227

Summary....228

Questions....229

Answers....229

Further reading....230

Chapter 8: Structuring an End-to-End Python Package in Rust....231

Technical requirements....231

Breaking down a catastrophe modeling problem for our package....231

Building an end-to-end solution as a package....237

Building a footprint merging process....238

Building the vulnerability merge process....241

Building a Python interface in Rust....244

Building our interface in Python....246

Building package installation instructions....247

Utilizing and testing our package....249

Building a Python construct model using pandas....250

Building a random event ID generator function....251

Timing our Python and Rust implementations with a series of different data sizes....252

Summary....254

Further reading....254

Section 3: Infusing Rust into a Web Application....256

Chapter 9: Structuring a Python Flask App for Rust....257

Technical requirements....257

Building a basic Flask application....257

Building an entry point for our application....258

Building our Fibonacci number calculator module....259

Building a Docker image for our application....261

Building our NGINX service....263

Connecting and running our Nginx service....264

Defining our data access layer....267

Defining a PostgreSQL database in docker-compose....268

Building a config loading system....269

Building our data access layer....271

Setting up the application database migration system....273

Building database models....275

Applying the database access layer to the fib calculation view....277

Building a message bus....279

Building a Celery broker for Flask....280

Building a Fibonacci calculation task for Celery....282

Updating our calculation view....283

Defining our Celery service in Docker....283

Summary....287

Questions....287

Answers....287

Further reading....288

Chapter 10: Injecting Rust into a Python Flask App....289

Technical requirements....289

Fusing Rust into Flask and Celery....289

Defining our dependency on the Rust Fibonacci number calculation package....290

Building our calculation model with Rust....290

Creating a calculation view using Rust....293

Inserting Rust into our Celery task....294

Deploying Flask and Celery with Rust....295

Deploying with a private GitHub repository....297

Building a Bash script that orchestrates the whole process....299

Reconfiguring the Rust Fib package installment in our Dockerfile....299

Fusing Rust with data access....300

Setting up our database cloning package....301

Setting up the diesel environment....303

Autogenerating and configuring our database models and schema....304

Defining our database connection in Rust....306

Creating a Rust function that gets all the Fibonacci records and returns them....307

Deploying Rust nightly in Flask....309

Summary....310

Questions....311

Answers....311

Further reading....311

Chapter 11: Best Practices for Integrating Rust....312

Technical requirements....312

Keeping our Rust implementation simple by piping data to and from Rust....312

Building a Python script that formulates the numbers for calculation....313

Building a Rust file that accepts the numbers, calculates the Fibonacci numbers, and returns the calculated numbers....314

Building another Python script that accepts the calculated numbers and prints them out....315

Giving the interface a native feel with objects....317

Defining traits....323

Defining struct behavior with traits....325

Passing traits through functions....327

Storing structs with common traits....329

Running our traits in the main file....329

Keeping data-parallelism simple with Rayon....332

Further reading....334

Why subscribe?....335

Other Books You May Enjoy....335

Packt is searching for authors like you....338

Share Your Thoughts....338

Python has made software development easier, but it falls short in several areas including memory management that lead to poor performance and security. Rust, on the other hand, provides memory safety without using a garbage collector, which means that with its low memory footprint, you can build high-performant and secure apps relatively easily. However, rewriting everything in Rust can be expensive and risky as there might not be package support in Rust for the problem being solved. This is where Python bindings and pip come in.

This book will help you, as a Python developer, to start using Rust in your Python projects without having to manage a separate Rust server or application. Seeing as you'll already understand concepts like functions and loops, this book covers the quirks of Rust such as memory management to code Rust in a productive and structured manner. You'll explore the PyO3 crate to fuse Rust code with Python, learn how to package your fused Rust code in a pip package, and then deploy a Python Flask application in Docker that uses a private Rust pip module. Finally, you'll get to grips with advanced Rust binding topics such as inspecting Python objects and modules in Rust.

By the end of this Rust book, you'll be able to develop safe and high-performant applications with better concurrency support.

What You Will Learn

• Explore the quirks of the Rust programming language that a Python developer needs to understand to code in Rust
• Understand the trade-offs for multiprocessing and thread safety to write concurrent code
• Build and manage a software project with cargo and crates
• Fuse Rust code with Python so that Python can import and run Rust code
• Deploy a Python Flask application in Docker that utilizes a private Rust pip module
• Inspect and create your own Python objects in Rust

Who this book is for:

This book is for Python developers who want to speed up their Python code with Rust and implement Rust in a Python system without altering the entire system. You'll be able to learn about all topics relating to Rust programming. Basic knowledge of Python is required to get the most out of this book.