Advanced Forecasting with Python: Mastering Modern Forecasting Techniques with Machine Learning and Cloud Tools. 2 Ed

Advanced Forecasting with Python....2

Introduction....5

Table of Contents....7

About the Author....19

About the Technical Reviewer....20

Part I: Machine Learning for Forecasting....21

1. Models for Forecasting....22

Reading Guide for This Book....23

Machine Learning Landscape....23

Univariate Time Series Models....23

A Quick Example of the Time Series Approach....24

Supervised Machine Learning Models....29

A Quick Example of the Supervised Machine Learning Approach....29

Correlation Coefficient....36

Other Distinctions in Machine Learning Models....37

Supervised vs. Unsupervised Models....37

Classification vs. Regression Models....38

Univariate vs. Multivariate Models....38

Key Takeaways....39

2. Model Evaluation for Forecasting....40

Evaluation with an Example Forecast....40

Error Metrics....43

Error Metric 1: MSE....43

Error Metric 2: RMSE....44

Error Metric 3: R2....44

Error Metric 4: MAE....45

Error Metric 5: MAPE....46

Model Evaluation Strategies....46

Overfit and the Out of Sample Error....47

Strategy 1: Train-Test Split....47

Strategy 2: Train-Validation-Test Split....49

Strategy 3: Cross-Validation for Forecasting....51

K-Fold Cross-Validation....51

Time Series Cross-Validation....53

Rolling Time Series Cross-Validation....55

Backtesting....57

Which Strategy to Use for Safe Forecasts?....58

Final Considerations on Model Evaluation....58

Key Takeaways....59

3. Model Management and Benchmarking Using MLflow....61

Introduction to MLflow....61

Local vs. Hosted MLflow....62

Setting Up MLflow Locally....62

Experiment Tracking Using MLflow....63

MLflow Data....64

Artifacts....65

Metrics....65

Params....66

Tags....66

Inspecting the Model Logs Through Python....66

The MLflow UI....67

MLflow Throughout This Book....68

Key Takeaways....69

Part II: Univariate Time Series Models....70

4. The AR Model....71

Autocorrelation: The Past Influences the Present....72

Compute Autocorrelation in Earthquake Counts....72

Positive and Negative Autocorrelation....77

Stationarity and the ADF Test....78

Differencing a Time Series....79

Lags in Autocorrelation....81

Partial Autocorrelation....84

How Many Lags to Include?....86

AR Model Definition....86

Estimating the AR Using Yule–Walker Equations....87

The Yule–Walker Method....87

Train, Test, Evaluation, and Tuning....92

Saving the Final Model Using MLflow....97

Key Takeaways....99

5. The MA Model....100

The Model Definition....101

Fitting the MA Model....102

Stationarity....103

Choosing Between an AR and an MA Model....103

Application of the MA Model....105

Multistep Forecasting with Model Retraining....113

Grid Search to Find the Best MA Order....116

Saving This Model in MLflow....118

Key Takeaways....119

6. The ARMA Model....121

The Idea Behind the ARMA Model....121

The Mathematical Definition of the ARMA Model....122

An Example: Predicting Births Using ARMA....122

Fitting an ARMA(1,1) Model....128

More Model Evaluation KPIs....129

Automated Hyperparameter Tuning....133

Grid Search: Tuning for Predictive Performance....134

Saving This Model in MLflow....138

Key Takeaways....140

7. The ARIMA Model....141

ARIMA Model Definition....141

Model Definition....142

ARIMA on the CO2 Example....142

Key Takeaways....149

8. The SARIMA Model....151

Univariate Time Series Model Breakdown....151

The SARIMA Model Definition....152

Example: SARIMA on Walmart Sales....153

Key Takeaways....158

Part III: Multivariate Time Series Models....159

9. The SARIMAX Model....160

Time Series Building Blocks....160

Model Definition....161

Supervised Models vs. SARIMAX....161

Example of SARIMAX on the Walmart Dataset....162

Key Takeaways....166

10. The VAR Model....168

The Model Definition....168

Order: Only One Hyperparameter....169

Stationarity....169

Estimation of the VAR Coefficients....169

One Multivariate Model vs. Multiple Univariate Models....170

An Example: VAR for Forecasting Walmart Sales....170

Key Takeaways....174

11. The VARMAX Model....175

Model Definition....176

Multiple Time Series with Exogenous Variables....176

Key Takeaways....179

Part IV: Supervised Models....180

12. The Linear Regression....181

Linear Regression....182

Model Definition....182

Example: Linear Model to Forecast CO2 Levels....184

Key Takeaways....190

13. The Decision Tree Model....192

Mathematics....193

Splitting....193

Pruning and Reducing Complexity....194

Example....194

Key Takeaways....202

14. The kNN Model....203

Intuitive Explanation....203

Mathematical Definition of Nearest Neighbors....204

Combining k Neighbors into One Forecast....205

Deciding on the Number of Neighbors k....206

Predicting Traffic Using kNN....207

Grid Search on kNN....210

Random Search: An Alternative to Grid Search....211

Key Takeaways....212

15. The Random Forest....213

Intuitive Idea Behind Random Forests....213

Random Forest Concept 1: Ensemble Learning....214

Bagging Concept 1: Bootstrap....214

Bagging Concept 2: Aggregation....215

Random Forest Concept 2: Variable Subsets....216

Predicting Births Using a Random Forest....216

Grid Search on the Two Main Hyperparameters of the Random Forest....218

Random Search CV Using Distributions....220

Distribution for max_features....220

Distribution for n_estimators....222

Fitting the RandomizedSearchCV....223

Interpretation of Random Forests: Feature Importance....224

Key Takeaways....227

16. Gradient Boosting with XGBoost, LightGBM, and CatBoost....228

Boosting: A Different Way of Ensemble Learning....228

Gradient Boosting....229

The Difference Between XGBoost and LightGBM....230

Adding CatBoost to the Mix....232

Forecasting Traffic Volume with XGBoost....232

Forecasting Traffic Volume with LightGBM....234

Forecasting Traffic Volume with CatBoost....234

Inspecting the Differences in Predictions....235

Hyperparameter Tuning Using Bayesian Optimization....236

The Theory of Bayesian Optimization....236

Bayesian Optimization Using scikit-optimize....237

Conclusion....243

Key Takeaways....243

17. Bayesian Models with pyBATS....245

Intuitive Idea Behind Bayesian Models....245

Bayesian vs. Frequentist Statistics....246

pyBATS As an Implementation of Bayesian Forecasting....247

Forecasting Sales Using pyBATS....248

Sales Forecast: Exploratory Data Analysis....248

Sales Forecast: Univariate Poisson DGLM....253

Sales Forecast: Normal DGLM with X Variable....256

Key Takeaways....258

Part V: Neural Networks....260

18. Neural Networks....261

Fully Connected Neural Networks....262

Activation Functions....262

The Weights: Backpropagation....263

Optimizers....264

Learning Rate of the Optimizer....264

Hyperparameters at Play in Developing a NN....264

Introducing the Example Data....265

Specific Data Prep Needs for NN....267

Scaling and Standardization....267

Principal Component Analysis (PCA)....267

The Neural Network Using Keras....270

Conclusion....276

Key Takeaways....277

19. RNNs Using SimpleRNN and GRU....278

What Are RNNs: Architecture....278

Inside the SimpleRNN Unit....279

The Example....280

Predicting a Sequence Rather Than a Value....280

Univariate Model Rather Than Multivariable....280

Preparing the Data....281

A Simple SimpleRNN....283

SimpleRNN with Hidden Layers....285

Simple GRU....287

GRU with Hidden Layers....290

Key Takeaways....292

20. LSTM RNNs....293

What Is LSTM....293

The LSTM Cell....293

Example....294

LSTM with 1 Layer of 8....296

LSTM with 3 Layers of 64....299

Conclusion....301

Key Takeaways....301

Part VI: Black Box and Cloud-Based Models....303

21. The N-BEATS Model with Darts....304

Intuition and Mathematics of N-BEATS....304

Interpretability....304

Stacking....305

Automatic Feature Engineering and Decomposition....305

The Darts Package and Its Implementation of N-BEATS....305

Forecasting Sales Using N-BEATS in Darts....306

Sales Forecast: Preparing the Data....306

Sales Forecast: Create Default N-BEATS Model....310

Sales Forecast: Create Multivariate N-BEATS Model....312

Sales Forecast: Tuning the Multivariate N-BEATS Model....315

Benchmark Results....319

Key Takeaways....319

22. The Transformer Model with Darts....321

Intuition and Mathematics of Transformers....321

Attention Is All You Need....322

Transformers Move Away from Recurrent Units....322

Positional Encoding....322

Self-Attention....323

The Darts Package and Its Implementation of Transformers....323

Forecasting Sales Using Transformer in Darts....323

Sales Forecast: Preparing the Data....324

Sales Forecast: Create a Default N-BEATS Model....326

Sales Forecast: Create the Multivariate Transformer Model....329

Sales Forecast: Tuning the Multivariate Transformer Model....330

Benchmark Results....336

Key Takeaways....337

23. The NeuralProphet Model....339

The NeuralProphet Model....340

Predicting Heat Waves with NeuralProphet....340

Wikipedia Pageview Tool....340

Preparing the Data in Python....341

Time Series Decomposition Using NeuralProphet....342

Advanced Decomposition Setting....345

Train-Test Split....348

Default Model....349

Tuned Model....352

MLflow....356

Key Takeaways....358

24. The DeepAR Model and AWS SageMaker AI....360

About DeepAR....360

Forecasting Heat Wave Page Views Using gluonts DeepAR Locally....360

Tuning the DeepAR Model Using Ray HyperOptSearch....362

What Is Ray HyperOptSearch?....362

DeepAR vs. AWS SageMaker AI....364

Forecasting Heat Wave Page Views Using DeepAR Inside an AWS SageMaker AI Training Job....365

Step 1: Prepare the Data File....365

Step 2: Upload the Data to S3....365

Step 3: SageMaker AI....365

Key Takeaways....371

25. Uber’s Orbit Model....372

About Orbit....372

Forecasting Heat Wave Page Views Using Orbit’s Local-Global Trend Model....372

Tuning Orbit’s GLT....376

Forecasting Heat Wave Page Views Using Orbit’s Damped Local Trend Model....379

Tuning Orbit’s DLT....382

Key Takeaways....385

26. AutoML with Microsoft Azure....387

Cloud Computing....387

From Cloud-Based Maths to AutoML....388

Microsoft Azure Cloud....388

Forecasting Heat Wave Page Views with Microsoft Azure Cloud AutoML....389

Step 1: Create an Azure Machine Learning Studio....389

Step 2: Create an Automated ML Run....390

Optional: Download MLflow Artifact of the Model....401

Key Takeaways....402

27. AutoML with Vertex AI on Google Cloud Platform....403

GCP and BigQuery....403

Vertex AI....404

Forecasting Heat Wave Page Views with Google Cloud Platform’s Vertex AI AutoML....404

Step 1: Data Preparation for GCP....404

Step 2: Create a Bucket....406

Step 3: Uploading the Data....408

Optional Next Steps....420

Key Takeaways....422

28. Nixtla Suite and TimeGPT....423

The NixtlaVerse....423

Simple Use Case with the NixtlaVerse....424

The GPT in TimeGPT....429

Nixtla’s TimeGPT API....430

Key Takeaways....431

29. Model Selection....432

Model Selection Based on Metrics....432

Model Structure and Inputs....433

One-Step Forecasts vs. Multistep Forecasts....434

Model Complexity vs. Gain....434

Model Complexity vs. Interpretability....435

Model Stability and Variation....436

Open Source vs. Cloud Services Black Box....436

Conclusion....437

Key Takeaways....437

Index....439

Advanced Forecasting with Python, Second Edition, is a comprehensive and practical guide to mastering modern forecasting techniques using Python. Designed for data scientists, analysts, and machine learning practitioners, this updated edition bridges the gap between classical forecasting models and cutting-edge, AI-powered techniques that are reshaping the field.

The book begins with foundational models like AR, MA, ARIMA, and SARIMA, offering intuitive and mathematical explanations alongside hands-on Python implementations. It then expands into multivariate models (VAR, VARMAX), supervised machine learning (Random Forests, XGBoost, LightGBM, CatBoost), and deep learning architectures such as LSTMs, NBEATS, and Transformers. Each chapter not only teaches the theory and code but also tracks model performance using MLflow, enabling efficient benchmarking and experimentation management. The second edition stands out for its extensive new content. Readers will now explore Orbit by Uber, AutoGluon by AWS, Prophet by Meta, Microsoft Azure AutoML, Google GCP AutoML, and TimeGPT by Nixtla, equipping them with the latest tools from top cloud providers. These additions make sure that readers stay current in an ever-evolving landscape. Moreover, the new chapters highlight practical deployment strategies and trade-offs between performance, explainability, and scalability.

Whether you are just beginning your forecasting journey or seeking to enhance your expertise with state-of-the-art tools and cloud-based solutions, this book offers a rich, hands-on learning experience. With step-by-step Python examples, detailed model insights, and modern forecasting workflows, it is an indispensable resource for staying ahead in the realm of predictive analytics.

You Will:

• Build robust forecasting solutions using Python
• Gain both intuitive and mathematical insights into traditional and cutting-edge forecasting models 
• Master model evaluation through cross-validation, backtesting, and MLflow-based tracking
• Leverage cloud-based platforms and Model-as-a-Service tools for scalable forecasting deployments

Who this book is for:

This book is ideal for data scientists, analysts, and ML practitioners working on real-world forecasting problems. It suits both intermediate learners and experienced professionals looking to master state-of-the-art forecasting techniques.