Python Deep Learning: Understand how deep neural networks work and apply them to real-world tasks. 3 Ed

Cover....1

Title Page....2

Copyright and Credit....3

Contributors ....4

Table of Contents....6

Preface....12

Part 1: Introduction to Neural Networks....18

Chapter 1: Machine Learning – an Introduction....20

Technical requirements....20

Introduction to ML....21

Different ML approaches....22

Supervised learning....22

Unsupervised learning....28

Reinforcement learning....32

Components of an ML solution....35

Neural networks....38

Introducing PyTorch....39

Summary....43

Chapter 2: Neural Networks....44

Technical requirements....44

The need for NNs....45

The math of NNs....45

Linear algebra....46

An introduction to probability....50

Differential calculus....56

An introduction to NNs....58

Units – the smallest NN building block....59

Layers as operations....61

Multi-layer NNs....63

Activation functions....64

The universal approximation theorem....66

Training NNs....69

GD....69

Backpropagation....73

A code example of an NN for the XOR function....75

Summary....81

Chapter 3: Deep Learning Fundamentals....82

Technical requirements....82

Introduction to DL....83

Fundamental DL concepts....84

Feature learning....85

The reasons for DL’s popularity....86

Deep neural networks....87

Training deep neural networks....88

Improved activation functions....89

DNN regularization....93

Applications of DL....96

Introducing popular DL libraries....99

Classifying digits with Keras....99

Classifying digits with PyTorch....103

Summary....106

Part 2: Deep Neural Networks for Computer Vision....108

Chapter 4: Computer Vision with Convolutional Networks....110

Technical requirements....111

Intuition and justification for CNNs....111

Convolutional layers....112

A coding example of the convolution operation....115

Cross-channel and depthwise convolutions....117

Stride and padding in convolutional layers....120

Pooling layers....121

The structure of a convolutional network....123

Classifying images with PyTorch and Keras....124

Convolutional layers in deep learning libraries....124

Data augmentation....124

Classifying images with PyTorch....125

Classifying images with Keras....128

Advanced types of convolutions....130

1D, 2D, and 3D convolutions....130

1×1 convolutions....131

Depthwise separable convolutions....131

Dilated convolutions....132

Transposed convolutions....133

Advanced CNN models....136

Introducing residual networks....137

Inception networks....140

Introducing Xception....145

Squeeze-and-Excitation Networks....146

Introducing MobileNet....147

EfficientNet....149

Using pre-trained models with PyTorch and Keras....150

Summary....151

Chapter 5: Advanced Computer Vision Applications....152

Technical requirements....153

Transfer learning (TL)....153

Transfer learning with PyTorch....155

Transfer learning with Keras....158

Object detection....162

Approaches to object detection....163

Object detection with YOLO....165

Object detection with Faster R-CNN....170

Introducing image segmentation....176

Semantic segmentation with U-Net....177

Instance segmentation with Mask R-CNN....179

Image generation with diffusion models....182

Introducing generative models....183

Denoising Diffusion Probabilistic Models....184

Summary....187

Part 3: Natural Language Processing and Transformers....188

Chapter 6: Natural Language Processing and Recurrent Neural Networks....190

Technical requirements....191

Natural language processing....191

Tokenization....192

Introducing word embeddings....197

Word2Vec....199

Visualizing embedding vectors....203

Language modeling....204

Introducing RNNs....206

RNN implementation and training....209

Backpropagation through time....211

Vanishing and exploding gradients....214

Long-short term memory....216

Gated recurrent units....220

Implementing text classification....221

Summary....226

Chapter 7: The Attention Mechanism and Transformers....228

Technical requirements....228

Introducing seq2seq models....229

Understanding the attention mechanism....231

Bahdanau attention....231

Luong attention....234

General attention....235

Transformer attention....237

Implementing TA....241

Building transformers with attention....244

Transformer encoder....245

Transformer decoder....248

Putting it all together....251

Decoder-only and encoder-only models....253

Bidirectional Encoder Representations from Transformers....253

Generative Pre-trained Transformer....258

Summary....261

Chapter 8: Exploring Large Language Models in Depth....262

Technical requirements....263

Introducing LLMs....263

LLM architecture....264

LLM attention variants....264

Prefix decoder....271

Transformer nuts and bolts....272

Models....275

Training LLMs....276

Training datasets....277

Pre-training properties....280

FT with RLHF....285

Emergent abilities of LLMs....287

Introducing Hugging Face Transformers....289

Summary....293

Chapter 9: Advanced Applications of Large Language Models....294

Technical requirements....294

Classifying images with Vision Transformer....295

Using ViT with Hugging Face Transformers....297

Understanding the DEtection TRansformer....299

Using DetR with Hugging Face Transformers....303

Generating images with stable diffusion....305

Autoencoder....306

Conditioning transformer....307

Diffusion model....309

Using stable diffusion with Hugging Face Transformers....310

Exploring fine-tuning transformers....313

Harnessing the power of LLMs with LangChain....315

Using LangChain in practice....316

Summary....319

Part 4: Developing and Deploying Deep Neural Networks....320

Chapter 10: Machine Learning Operations (MLOps)....322

Technical requirements....323

Understanding model development....323

Choosing an NN framework....323

PyTorch versus TensorFlow versus JAX....323

Open Neural Network Exchange....324

Introducing TensorBoard....329

Developing NN models for edge devices with TF Lite....333

Mixed-precision training with PyTorch....336

Exploring model deployment....337

Deploying NN models with Flask....337

Building ML web apps with Gradio....339

Summary....342

Index....344

Other Books You May Enjoy....359

Master effective navigation of neural networks, including convolutions and transformers, to tackle computer vision and NLP tasks using Python

Key Features

• Understand the theory, mathematical foundations and the structure of deep neural networks
• Become familiar with transformers, large language models, and convolutional networks
• Learn how to apply them on various computer vision and natural language processing problems Purchase of the print or Kindle book includes a free PDF eBook

Book Description

The field of deep learning has developed rapidly in the past years and today covers broad range of applications. This makes it challenging to navigate and hard to understand without solid foundations. This book will guide you from the basics of neural networks to the state-of-the-art large language models in use today.

The first part of the book introduces the main machine learning concepts and paradigms. It covers the mathematical foundations, the structure, and the training algorithms of neural networks and dives into the essence of deep learning.

The second part of the book introduces convolutional networks for computer vision. We'll learn how to solve image classification, object detection, instance segmentation, and image generation tasks.

The third part focuses on the attention mechanism and transformers - the core network architecture of large language models. We'll discuss new types of advanced tasks, they can solve, such as chat bots and text-to-image generation.

By the end of this book, you'll have a thorough understanding of the inner workings of deep neural networks. You'll have the ability to develop new models or adapt existing ones to solve your tasks. You'll also have sufficient understanding to continue your research and stay up to date with the latest advancements in the field.

What you will learn

• Establish theoretical foundations of deep neural networks
• Understand convolutional networks and apply them in computer vision applications
• Become well versed with natural language processing and recurrent networks
• Explore the attention mechanism and transformers
• Apply transformers and large language models for natural language and computer vision
• Implement coding examples with PyTorch, Keras, and Hugging Face Transformers
• Use MLOps to develop and deploy neural network models

Who this book is for

This book is for software developers/engineers, students, data scientists, data analysts, machine learning engineers, statisticians, and anyone interested in deep learning. Prior experience with Python programming is a prerequisite.