Machine Learning Model Serving Patterns and Best Practices: A definitive guide to deploying, monitoring, and providing accessibility to ML models in production

Cover Page....2

Table of Contents....3

Preface....5

Part 1:Introduction to Model Serving....13

Chapter 1: Introducing Model Serving....14

Technical requirements....14

What is serving?....15

What are models?....17

What is model serving?....19

Understanding the importance of model serving....24

Using existing tools to serve models....26

Summary....27

Chapter 2: Introducing Model Serving Patterns....29

Design patterns in software engineering....29

Understanding the value of model serving patterns....33

ML serving patterns....37

Summary....51

Further reading....52

Part 2:Patterns and Best Practices of Model Serving....53

Chapter 3: Stateless Model Serving....54

Technical requirements....54

Understanding stateful and stateless functions....55

States in machine learning models....62

Summary....97

Chapter 4: Continuous Model Evaluation....99

Technical requirements....99

Introducing continuous model evaluation....100

The necessity of continuous model evaluation....105

Continuous model evaluation use cases....124

Evaluating a model continuously....130

Monitoring model performance when predicting rare classes....137

Summary....139

Further reading....140

Chapter 5: Keyed Prediction....141

Technical requirements....141

Introducing keyed prediction....142

Exploring keyed prediction use cases....145

Exploring techniques for keyed prediction....162

Summary....174

Further reading....174

Chapter 6: Batch Model Serving....175

Technical requirements....175

Introducing batch model serving....176

Different types of batch model serving....179

Example scenarios of batch model serving....191

Techniques in batch model serving....193

Limitations of batch serving....199

Summary....201

Further reading....201

Chapter 7: Online Learning Model Serving....202

Technical requirements....202

Introducing online model serving....202

Use cases for online model serving....215

Challenges in online model serving....219

Implementing online model serving....225

Summary....229

Further reading....229

Chapter 8: Two-Phase Model Serving....231

Technical requirements....231

Introducing two-phase model serving....232

Exploring two-phase model serving techniques....234

Use cases of two-phase model serving....249

Summary....254

Further reading....254

Chapter 9: Pipeline Pattern Model Serving....256

Technical requirements....256

Introducing the pipeline pattern....257

Introducing Apache Airflow....260

Demonstrating a machine learning pipeline using Airflow....271

Advantages and disadvantages of the pipeline pattern....276

Summary....277

Further reading....278

Chapter 10: Ensemble Model Serving Pattern....279

Technical requirements....279

Introducing the ensemble pattern....280

Using ensemble pattern techniques....282

End-to-end dummy example of serving the model....291

Summary....293

Chapter 11: Business Logic Pattern....295

Technical requirements....295

Introducing the business logic pattern....295

Technical approaches to business logic in model serving....299

Summary....304

Part 3:Introduction to Tools for Model Serving....306

Chapter 12: Exploring TensorFlow Serving....307

Technical requirements....307

Introducing TensorFlow Serving....308

Using TensorFlow Serving to serve models....312

Summary....323

Further reading....324

Chapter 13: Using Ray Serve....325

Technical requirements....325

Introducing Ray Serve....325

Using Ray Serve to serve a model....335

Summary....345

Further reading....346

Chapter 14: Using BentoML....347

Technical requirements....347

Introducing BentoML....347

Using BentoML to serve a model....362

Summary....366

Further reading....366

Part 4:Exploring Cloud Solutions....367

Chapter 15: Serving ML Models using a Fully Managed AWS Sagemaker Cloud Solution....368

Technical requirements....368

Introducing Amazon SageMaker....368

Using Amazon SageMaker to serve a model....373

Summary....386

Index....388

Why subscribe?....410

Other Books You May Enjoy....412

Packt is searching for authors like you....416

Share Your Thoughts....416

Download a free PDF copy of this book....417

Serving patterns enable data science and ML teams to bring their models to production. Most ML models are not deployed for consumers, so ML engineers need to know the critical steps for how to serve an ML model.

This book will cover the whole process, from the basic concepts like stateful and stateless serving to the advantages and challenges of each. Batch, real-time, and continuous model serving techniques will also be covered in detail. Later chapters will give detailed examples of keyed prediction techniques and ensemble patterns. Valuable associated technologies like TensorFlow severing, BentoML, and RayServe will also be discussed, making sure that you have a good understanding of the most important methods and techniques in model serving. Later, you'll cover topics such as monitoring and performance optimization, as well as strategies for managing model drift and handling updates and versioning. The book will provide practical guidance and best practices for ensuring that your model serving pipeline is robust, scalable, and reliable. Additionally, this book will explore the use of cloud-based platforms and services for model serving using AWS SageMaker with the help of detailed examples.

By the end of this book, you'll be able to save and serve your model using state-of-the-art techniques.

What you will learn

• Explore specific patterns in model serving that are crucial for every data science professional
• Understand how to serve machine learning models using different techniques
• Discover the various approaches to stateless serving
• Implement advanced techniques for batch and streaming model serving
• Get to grips with the fundamental concepts in continued model evaluation
• Serve machine learning models using a fully managed AWS Sagemaker cloud solution

Who this book is for

This book is for machine learning engineers and data scientists who want to bring their models into production. Those who are familiar with machine learning and have experience of using machine learning techniques but are looking for options and strategies to bring their models to production will find great value in this book. Working knowledge of Python programming is a must to get started.