Active Machine Learning with Python: Refine and elevate data quality over quantity with active learning

Cover....1

Title Page....2

Copyright and Credits....3

Contributors....5

Table of Contents....8

Preface....12

Part 1: Fundamentals of Active Machine Learning....18

Chapter 1: Introducing Active Machine Learning....20

Understanding active machine learning systems....20

Definition....21

Potential range of applications....21

Key components of active machine learning systems....22

Exploring query strategies scenarios....24

Membership query synthesis....24

Stream-based selective sampling....25

Pool-based sampling....28

Comparing active and passive learning....30

Summary....31

Chapter 2: Designing Query Strategy Frameworks....34

Technical requirements....35

Exploring uncertainty sampling methods....35

Understanding query-by-committee approaches....42

Maximum disagreement....43

Vote entropy....45

Average KL divergence....47

Labeling with EMC sampling....51

Sampling with EER....54

Understanding density-weighted sampling methods....55

Summary....61

Chapter 3: Managing the Human in the Loop....62

Technical requirements....62

Designing interactive learning systems and workflows....63

Exploring human-in-the-loop labeling tools....67

Common labeling platforms....68

Handling model-label disagreements....69

Programmatically identifying mismatches....69

Manual review of conflicts....71

Effectively managing human-in-the-loop systems....72

Ensuring annotation quality and dataset balance....74

Assess annotator skills....74

Use multiple annotators....75

Balanced sampling....76

Summary....78

Part 2: Active Machine Learning in Practice....80

Chapter 4: Applying Active Learning to Computer Vision....82

Technical requirements....82

Implementing active ML for an image classification project....83

Building a CNN for the CIFAR dataset....84

Applying uncertainty sampling to improve classification performance....90

Applying active ML to an object detection project....93

Preparing and training our model....94

Analyzing the evaluation metrics....96

Implementing an active ML strategy....97

Using active ML for a segmentation project....101

Summary....105

Chapter 5: Leveraging Active Learning for Big Data....106

Technical requirements....106

Implementing ML models for video analysis....107

Selecting the most informative frames with Lightly....109

Using Lightly to select the best frames to label for object detection....110

SSL with active ML....132

Summary....135

Part 3: Applying Active Machine Learning to Real-World Projects....136

Chapter 6: Evaluating and Enhancing Efficiency....138

Technical requirements....138

Creating efficient active ML pipelines....139

Monitoring active ML pipelines....141

Determining when to stop active ML runs....144

Enhancing production model monitoring with active ML....145

Challenges in monitoring production models....145

Active ML to monitor models in production....147

Early detection for data drift and model decay....149

Summary....150

Chapter 7: Utilizing Tools and Packages for Active ML....152

Technical requirements....152

Mastering Python packages for enhanced active ML....153

scikit-learn....153

modAL....156

Getting familiar with the active ML tools....162

Summary....164

Index....166

Other Books You May Enjoy....173

Building accurate machine learning models requires quality data-lots of it. However, for most teams, assembling massive datasets is time-consuming, expensive, or downright impossible. Led by Margaux Masson-Forsythe, a seasoned ML engineer and advocate for surgical data science and climate AI advancements, this hands-on guide to active machine learning demonstrates how to train robust models with just a fraction of the data using Python's powerful active learning tools.

You'll master the fundamental techniques of active learning, such as membership query synthesis, stream-based sampling, and pool-based sampling and gain insights for designing and implementing active learning algorithms with query strategy and Human-in-the-Loop frameworks. Exploring various active machine learning techniques, you'll learn how to enhance the performance of computer vision models like image classification, object detection, and semantic segmentation and delve into a machine AL method for selecting the most informative frames for labeling large videos, addressing duplicated data. You'll also assess the effectiveness and efficiency of active machine learning systems through performance evaluation.

By the end of the book, you'll be able to enhance your active learning projects by leveraging Python libraries, frameworks, and commonly used tools.

What you will learn

• Master the fundamentals of active machine learning
• Understand query strategies for optimal model training with minimal data
• Tackle class imbalance, concept drift, and other data challenges
• Evaluate and analyze active learning model performance
• Integrate active learning libraries into workflows effectively
• Optimize workflows for human labelers
• Explore the finest active learning tools available today

Who this book is for

Ideal for data scientists and ML engineers aiming to maximize model performance while minimizing costly data labeling, this book is your guide to optimizing ML workflows and prioritizing quality over quantity. Whether you're a technical practitioner or team lead, you'll benefit from the proven methods presented in this book to slash data requirements and iterate faster.

Basic Python proficiency and familiarity with machine learning concepts such as datasets and convolutional neural networks is all you need to get started.