Deep Learning Crash Course: A Hands-On, Project-Based Introduction to Artificial Intelligence

ABOUT THE AUTHORS....7

BRIEF CONTENTS....9

CONTENTS IN DETAIL....11

ACKNOWLEDGMENTS....21

INTRODUCTION....23

Who Should Read This Book?....25

This Book’s Approach....25

Setting Up Your Learning Environment....26

Obtaining the Data and Code Used in This Book....27

What’s in This Book?....27

Online Resources....30

Time to Start Your Journey....30

CHAPTER 1: BUILDING AND TRAINING YOUR FIRST NEURAL NETWORK....31

Classifying Data with a Single Neuron....32

Classifying 1D Data....33

Classifying 2D Data....41

Adding a Bias as Another Dimension....47

Using Weight Regularization to Stabilize the Training....48

Using Different Activation Functions....49

Classifying Data with a Two-Layer Neural Network....50

Implementing a Dense Neural Network....52

Training with Error Backpropagation....55

Attempting an Impossible Classification....60

Classifying Data with a Three-Layer Neural Network....61

Project 1A: Classifying Handwritten Digits....64

Using the MNIST Digit Dataset....64

Implementing the Neural Network....66

Training and Testing....68

Making Improvements....71

Performing Failure Analysis....76

Summary....78

Seminal Works and Further Reading....78

CHAPTER 2: CAPTURING TRENDS AND RECOGNIZING PATTERNS WITH DENSENEURAL NETWORKS....81

Regressing Data with a Single Neuron....82

Regressing 1D Data....82

Regressing 2D Data....87

Regressing Data with a Two-Layer Neural Network....91

Implementing a Dense Neural Network....92

Training with Error Backpropagation....93

Training Using Batches....95

Plotting Predictions vs. Ground Truth....95

Training with Mini-Batches....97

Tracking Training Metrics....99

Randomizing the Batches....101

Dividing Data into Multiple Datasets....103

Splitting the Data....104

Training and Validating the Neural Network....105

Project 2A: Emulating a Physical System....109

Loading the Theoretical and Geometrical–Optics Optical Forces....110

Loading and Preparing the Training Data....114

Implementing the Neural Network....117

Implementing Data Loaders....118

Training the Neural Network....118

Evaluating Performance....121

Summary....122

Seminal Works and Further Reading....123

CHAPTER 3: PROCESSING IMAGES WITH CONVOLUTIONAL NEURALNETWORKS....125

Understanding Convolutions....126

Convolving 1D Data....126

Convolving 2D Data....128

Using Convolutions in a Neural Network....130

Implementing Neural Networks in PyTorch....130

Defining Convolutional Layers....132

Adding ReLU Activation....134

Adding Pooling Layers....135

Using Upsampling Layers....136

Transforming Images....138

Using Dense Layers to Classify Images....139

Project 3A: Classifying Malaria-Infected Blood Smears....140

Loading the Malaria Dataset....141

Classifying with Dense Neural Networks....145

Classifying with Convolutional Neural Networks....148

Checking the Values of the Filters....150

Visualizing Activations of Convolutional Layers....151

Visualizing Heatmaps....154

Project 3B: Localizing Microscopic Particles....158

Loading the Videos....158

Manually Annotating the Videos....160

Implementing a Convolutional Neural Network....164

Training with Annotated Data....165

Simulating the Training Data....167

Training with Simulated Data....173

Project 3C: Creating DeepDreams....175

Loading an Image....176

Loading a Pretrained Neural Network....177

Implementing the DeepDreams Algorithm....179

Using Multiple Layers at Once....186

Project 3D: Transferring the Style of Images....188

Loading the Content and Style Images....188

Loading a Pretrained Neural Network....189

Implementing Style Transfer....190

Creating an Image in Gaudí's Style....193

Summary....194

Seminal Works and Further Reading....195

CHAPTER 4: ENHANCING, GENERATING, AND ANALYZING DATA WITHAUTOENCODERS....197

Project 4A: Generating Images with Variational Autoencoders....209

Understanding Variational Autoencoders....209

Loading the MNIST Dataset....211

Training the Variational Autoencoder....212

Generating Images with the Decoder....213

Clustering Images with the Encoder....216

Understanding Encoder-Decoders....198

Implementing a Denoising Encoder-Decoder....199

Generating the Data....199

Creating a Simulated Dataset....202

Defining and Training the Encoder-Decoder....203

Denoising the Image....203

Checking for Absence of Mode Collapse....205

Checking Generalization Capabilities....206

Retraining....207

Project 4B: Morphing Images with Wasserstein Autoencoders....218

Understanding Wasserstein Autoencoders....218

Loading the Fashion-MNIST Dataset....219

Training the Wasserstein Autoencoder....221

Reconstructing the Fashion-MNIST Images....222

Creating New Images....222

Morphing Images....223

Project 4C: Detecting Anomalies in ECG Data....225

Understanding Anomaly Detection....226

Loading the ECG Dataset....226

Defining and Training the Autoencoder....229

Testing with Normal and Anomalous ECGs....230

Detecting Anomalies....231

Summary....236

Seminal Works and Further Reading....237

CHAPTER 5: SEGMENTING AND ANALYZING IMAGES WITH U-NETS....239

Introducing U-Nets....240

Understanding Semantic Segmentation with U-Nets....241

Segmenting Images of Biological Tissues....242

Loading the Segmented Tissue Images....243

Creating the Data Pipelines....244

Defining and Training the U-Net....246

Plotting the Training Metrics....249

Preventing Overfitting....250

Evaluating the Trained U-Net....252

Project 5A: Detecting Quantum Dots in Fluorescence Images....254

Loading the Image....254

Simulating Quantum Dots....255

Defining and Training the U-Net....261

Evaluating the Trained U-Net....262

Project 5B: Counting Cells....264

Loading the Dataset....264

Creating a Pipeline....265

Simulating the Cell Images....267

Implementing and Training the U-Net....274

Testing the Trained U-Net....275

Summary....278

Seminal Works and Further Reading....278

CHAPTER 6: TRAINING NEURAL NETWORKS WITH SELF-SUPERVISEDLEARNING....281

Understanding Self-Supervised Learning....282

Self-Supervised Contrastive Learning....282

Self-Supervised Non-Contrastive Learning....283

Self-Supervised Geometric Learning....284

Determining the Position of a Particle in an Image....285

Creating the Dataset....285

Learning from Translations....287

Learning from Flipping the Image....292

Improving Performance with LodeSTAR....295

Project 6A: Localizing Mouse Stem Cells with LodeSTAR....298

Using the Cell Tracking Challenge Dataset....298

Preparing the Training Crop....301

Creating the Training Pipeline and Data Loader....302

Training the Neural Network....303

Evaluating Performance....304

Summary....307

Seminal Works and Further Reading....308

CHAPTER 7: PROCESSING TIME SERIES AND LANGUAGE WITH RECURRENTNEURAL NETWORKS....309

Understanding Recurrent Neural Networks....309

Using a Comb Filter....310

Understanding a Simple Recurrent Neural Network....311

Predicting Temperature with Recurrent Neural Networks....312

Loading the Jena Climate Dataset....312

Preprocessing the Data....314

Implementing a Commonsense Benchmark....317

Determining the Computational Device....317

Predicting with a Simple Recurrent Neural Network....318

Stacking Multiple Recurrent Layers....323

Using Gated Recurrent Units....324

Using Long Short-Term Memory Networks....326

Project 7A: Translating with Recurrent Neural Networks....329

Preparing the Bilingual Dataset....329

Defining the Sequence-to-Sequence Application....338

Loading Pretrained Embeddings....346

Training the Sequence-to-Sequence Application....348

Testing the Model Performance....349

Evaluating the Model with the BLEU Score....351

Summary....352

Seminal Works and Further Reading....353

CHAPTER 8: PROCESSING LANGUAGE AND CLASSIFYING IMAGES WITHATTENTION AND TRANSFORMERS....355

Understanding Attention....356

Implementing Dot-Product Attention....356

Visualizing Attention....361

Making the Attention Mechanism Trainable....363

Implementing Other Attention Mechanisms....365

Project 8A: Using Attention to Improve Language Translation....366

Incorporating Attention....367

Training and Testing the Seq2Seq Model with Attention....369

Interpreting the Attention Matrix....370

Project 8B: Performing Sentiment Analysis with a Transformer....372

Breaking Down Multi-Head Attention....372

Understanding the Transformer Structure....374

Loading the IMDb Dataset....375

Preprocessing the Reviews....377

Defining the Data Loaders....379

Building an Encoder-Only Transformer ....380

Training the Model....386

Evaluating the Trained Model....386

Project 8C: Classifying Images with a Vision Transformer....388

Using the CIFAR-10 Dataset....388

Data Preprocessing....390

Building the ViT Model....391

Training and Evaluating the ViT Model....393

Improving the ViT Model with CutMix....394

Using a Pretrained ViT Model....400

Summary....402

Seminal Works and Further Reading....403

CHAPTER 9: CREATING AND TRANSFORMING IMAGES WITH GENERATIVEADVERSARIAL NETWORKS....405

Understanding GANs....406

Discriminating Between Real and Fake Data....407

Generating Realistic Fake Data....408

Training a GAN....409

Generating Digits with a GAN....409

Loading the MNIST Dataset with PyTorch....410

Defining the Generator and Discriminator....411

Training the GAN....412

Plotting the Intermediate Results....416

Plotting the Training Losses....418

Project 9A: Generating Digits with a Conditional GAN....419

Defining the Conditional Generator and Discriminator....420

Training the Conditional GAN....422

Plotting the Generated Digits....423

Project 9B: Virtually Staining a Biological Tissue....424

Downloading the Human Motor Neurons Dataset....424

Creating a Dataset....425

Instantiating the Generator and Discriminator....430

Compiling the Conditional GAN....432

Training the Conditional GAN....433

Evaluating the Trained Conditional GAN....438

Project 9C: Converting Between Holographic and Bright-Field Microscopy Images....439

Understanding CycleGANs....440

Using the Holo2Bright Dataset....441

Instantiating the CycleGAN Generators and Discriminators....444

Training the CycleGAN....446

Evaluating the Trained CycleGAN....451

Summary....453

Seminal Works and Further Reading....453

CHAPTER 10: IMPLEMENTING GENERATIVE AI WITH DIFFUSION MODELS....455

Understanding Diffusion....456

Breaking Down Denoising Diffusion Probabilistic Models....457

Modeling the Forward Diffusion Process....458

Deriving the Fast Forward Process....459

Modeling the Reverse Diffusion Process....460

Generating Digits with a Diffusion Model....462

Loading the MNIST Dataset....462

Implementing the Forward Diffusion Process....463

Applying the Reverse Diffusion Process....465

Defining the Positional Encoding Function....466

Instantiating the Attention U-Net....468

Training the Diffusion Model....469

Sampling Images from Fixed Noise....474

Project 10A: Generating Bespoke Digits....476

Guiding the Diffusion Model....476

Defining the Conditional Attention U-Net....477

Training the Conditional Diffusion Model....478

Plotting the Intermediate Training Results....478

Project 10B: Generating Images from Text Prompts....480

Providing a Conditional Text Input....480

Defining the Conditional Attention U-Net....487

Training the Conditional Diffusion Model....488

Plotting the Intermediate Training Results....489

Training with the CLIP Tokenizer and CLIP Text Encoder....490

Project 10C: Generating Super-Resolution Images....493

Downloading the BioSR Dataset....493

Managing the Dataset....494

Preprocessing the Images....495

Creating the Training and Test Datasets....495

Adapting the Diffusion Process for Super-Resolution....496

Defining the Conditional Attention U-Net....497

Training the Conditional Diffusion Model....498

Summary....501

Seminal Works and Further Reading....501

CHAPTER 11: MODELING MOLECULES AND COMPLEX SYSTEMS WITH GRAPHNEURAL NETWORKS....505

Understanding Graph Convolutions....506

Predicting Molecular Properties with Graph Convolutions....509

Implementing a Graph Convolution Layer....510

Representing a Molecule as a Graph....512

Using the ZINC Dataset....513

Applying a Graph Convolutional Network....516

Training the Graph Convolutional Network....519

Evaluating the Graph Convolutional Network....521

Predicting Molecular Properties with Message Passing....521

Implementing a Message-Passing Layer....522

Implementing a Message-Passing Network....525

Training and Evaluating the Message-Passing Network....526

Project 11A: Simulating Complex Physical Phenomena....527

Working with the SAND Dataset....527

Building a Graph Network–Based Simulator....530

Building the Dataset....532

Training the Model....537

Testing the Model....538

Simulating the System....539

Project 11B: Identifying Cell Trajectories....541

Exploring the Cell-Tracking Data....542

Creating a Graph from Segmented Images....544

Building a Training Dataset....549

Making MAGIK....551

Evaluating Performance....552

Summary....556

Seminal Works and Further Reading....557

CHAPTER 12: CONTINUOUSLY IMPROVING PERFORMANCE WITH ACTIVELEARNING....559

Understanding Active Learning....560

Performing Binary Classification....561

Creating a Dataset with Two Groups of Data Points....561

Classifying the Data Points with a Logistic Regression....562

Implementing the Active Learning Process....563

Comparing Random and Uncertainty Sampling Strategies....568

Performing Multiclass Classification....569

Creating a Dataset with Three Groups of Data Points....570

Implementing the Active Learning Process with Multiple Classes....571

Comparing Sampling Strategies....575

Project 12A: Classifying MNIST Digits with Active Learning....577

Training a Baseline Model....577

Implementing Multiple Active Learning Strategies....578

Comparing the Performance of the Active Learning Strategies....583

Summary....585

Seminal Works and Further Reading....586

CHAPTER 13: MASTERING DECISION-MAKING WITH DEEP REINFORCEMENTLEARNING....587

Understanding Reinforcement Learning and Q-Learning....588

Implementing Tetris....591

Constructing a Simplified Tetris....591

Playing Tetris with the Command Line....598

Playing Tetris with a Pygame Graphical Interface....599

Making an Agent Play Tetris with Q-Learning....603

Adapting the Actions of Tetris for Q-Learning....603

Implementing the Q-Learning Agent....604

Training the Q-Learning Agent....611

Training with a Random Tile Sequence....614

Making an Agent Play Tetris with Deep Q-Learning....615

Understanding Deep Q-Learning....616

Implementing the Deep Q-Learning Agent....617

Training the Deep Q-Learning Agent....625

Summary....628

Seminal Works and Further Reading....628

CHAPTER 14: PREDICTING CHAOS WITH RESERVOIR COMPUTING....631

Introducing Reservoir Computing....631

Defining the Lorenz System....632

Numerically Integrating the Lorenz System....633

Visualizing Time Evolution....634

Visualizing the Lorenz Attractor....636

Demonstrating the Butterfly Effect....637

Implementing a Reservoir Computer....640

Setting Up the Reservoir....640

Preparing the Training and Validation Data....642

Training the Reservoir Computer....643

Evaluating the Performance of the Reservoir Computer....646

Summary....649

Seminal Works and Further Reading....650

CONCLUSION....653

INDEX....655

Deep Learning Crash Course is a fast-paced, thorough introduction that will have you building today’s most powerful AI models from scratch. No experience with deep learning required!

Designed for programmers who may be new to deep learning, this book offers practical, hands-on experience, not just an abstract understanding of theory.

You’ll start from the basics, and using PyTorch with real datasets, you’ll quickly progress from your first neural network to advanced architectures like convolutional neural networks (CNNs), transformers, diffusion models, and graph neural networks (GNNs). Each project can be run on your own hardware or in the cloud, with annotated code available on GitHub.

You’ll build and train models to: 

• Classify and analyze images, sequences, and time series
• Generate and transform data with autoencoders, GANs (generative adversarial networks), and diffusion models
• Process natural language with recurrent neural networks and transformers
• Model molecules and physical systems with graph neural networks
• Improve continuously through reinforcement and active learning
• Predict chaotic systems with reservoir computing

Whether you’re an engineer, scientist, or professional developer, you’ll gain fluency in deep learning and the confidence to apply it to ambitious, real-world problems. With Deep Learning Crash Course, you’ll move from using AI tools to creating them.