Ultimate Enterprise Data Analysis and Forecasting using Python: Leverage Cloud platforms with Azure Time Series Insights and AWS Forecast Components for Time Series Analysis and Forecasting with Deep learning Modeling using Python

Cover Page....2

Title Page....3

Copyright Page....4

Dedication Page....5

About the Author....6

About the Technical Reviewers....8

Preface....9

Errata....14

Table of Contents....18

1. Introduction to Python and its key packages for DS and ML Projects....26

Introduction....26

Structure....26

Introduction to Python programming language....26

Key features of Python....27

Python programming IDEs and comparisons....28

Jupyter Notebook....28

PyCharm....29

Spyder....29

Installing Jupyter notebook....30

Python libraries....35

Pandas....35

Panel + Data = Pandas....35

Reshaping DataFrame....39

Combining DataFrame....46

Working with categorical data....55

Encoding....56

Date and time data....57

Converting data types....60

NumPy....61

Python statistics libraries....76

Working with various files in Python....77

Conclusion....81

Points to remember....81

2. Python for Time Series Data Analysis....83

Introduction....83

Structure....83

Data analysis and its benefits....83

Benefits....84

Advanced Analytics....85

Python - the best choice for data analytics....86

Time series data....86

Time series data management....88

Data lifecycle management (DLM)....90

Data acquisition or collection....91

Ingesting data....91

Transforming data....91

Storing data....92

Actionable information....92

Data remediation....92

Data cleansing and preparation....93

Handling missing and duplicate data....94

Handling uniform format....104

Handling categorical columns....107

Transformation of data....108

Handling time series data....115

Exploratory data analysis (EDA)....121

EDA for time series....128

Conclusion....132

Points to remember....133

3. Time Series Analysis and its Components....134

Introduction....134

Structure....134

Time series data analysis....135

Significance of time series data....137

Trend....138

Seasonality....139

Components of time series data....139

Stationarity versus non-stationarity....140

Augmented Dickey-Fuller (ADF)....141

Kwiatkowski–Phillips–Schmidt–Shin (KPSS) test....146

Converting non-stationary data into stationary....148

Conclusion....163

Points to remember....164

4. Time Series Analysis and Forecasting Opportunities in Various Industries....165

Introduction....165

Structure....165

Opportunity and scope of TSA&F....165

Scope of price prediction....168

Scope of forecasting in healthcare for diagnosis....169

Scope of predictive maintenance....171

Challenges with TSA&F....173

Case studies....175

Price prediction in retail and use case....175

EDA and Stationarity Analysis....178

Forecasting in healthcare for diagnosis and use case....180

Predictive maintenance and anomaly detection use case....185

Conclusion....189

Points to remember....189

5. Exploring various aspects of Time Series Analysis and Forecasting....191

Introduction....191

Structure....191

Understanding time series analysis (TSA)....191

Statistical analysis....192

The measure of central tendency....196

Measure of variability....196

Boxplots....198

Histogram....199

Inferential Statistics....200

Regression analysis....201

Linear model....201

Hypothesis testing....203

Confidence intervals....210

Confidence intervals assessment at the speed of the motor....211

Study the shapes of Time Series....212

Transforms for TSA&F....213

Box-Cox transformation....218

Overview of Naive forecasting....226

Conclusion....228

Points to remember....229

6. Exploring Time Series Models - AR, MA, ARMA, and ARIMA....230

Introduction....230

Structure....230

Overview and time series models....230

Statistical models for TSA&F....232

Autoregressive (AR) Model....233

Autoregressive (AR) Implementation....234

Analysis of the P-value....242

Auto-regression model for TSA&F with Python....245

Moving Average (MA) Model....255

Moving Average (MA) Implementation....256

ARMA Model....275

Auto-Regressive Integrated Moving Average (ARIMA)....276

Time Series Analysis and Forecasting Process Workflow....282

Pertinency of the model....283

Conclusion....283

Points to remember....284

7. Understanding Exponential Smoothing and ETS Methods in TSA....285

Introduction....285

Structure....285

Understanding Exponential Smoothing....285

Exponential Smoothing implementation using Excel....286

Exponential Smoothing types....293

Triple Exponential Smoothing Model for TSA&F using Excel....296

SES and DES Model implementation using Python....301

Error - Trend-Seasonality (ETS)....307

Exponentially Weighted Moving Averages (EWMA)....316

Benefits of EWMA....317

Limitations of EWMA....317

EWMA Model for TSA&F with Excel (simple method)....317

EWMA Model implementation using Python....320

Conclusion....322

Points to remember....323

8. Exploring Vector Autoregression and its Subsets (VAR, VMA, and VARMA)....325

Introduction....325

Structure....325

Understanding Vector Autoregression....326

VAR implementation using Python....328

1. Analyze the time series data and its characteristics....329

2. Test for data stationarity using the ADF method....332

Augmented Dickey-Fuller (ADF) test....332

3. Train-test split....335

4. Re-run the ADF test....336

5. Apply the VAR algorithm....339

6. Optimal order (p) selection process....340

7. Analysis of Serial Correlation of Residuals [ScoR]....344

8. Building forecast VAR model....345

9. Model evaluation....348

Vector Autoregression Moving - Average (VARMA)....349

VARMA implementation using Python....351

Vector Autoregression Moving-Average with Exogenous Regressors (VARMAX)....377

VARMAX implementation using Python....378

Seasonal Autoregressive Integrated Moving-Average (SARIMA)....382

SARIMA implementation using Python....382

SARIMAX implementation using Python....385

Fractional-Autoregressive-Integrated-Moving Average model (FARIMA)....387

Conclusion....388

Points to remember....389

9. Deep Learning for Time Series Analysis and Forecasting....391

Introduction....391

Structure....391

Deep learning in time series analysis....391

Neural Networks....394

Artificial neural networks (ANN)....400

Long short-term memory (LSTM)....402

Convolutional neural networks (CNN)....423

Recurrent Neural Network (RNN)....443

Backpropagation through time (BPTT)....445

Conclusion....456

Points to remember....457

10. Azure Time Series Insights....461

Introduction....461

Structure....461

Prerequisites....461

Understanding Azure - Time Series Insights (Azure-TSI) Gen2 component....462

Components of Azure TSI and its major jobs....463

Azure TSI – versions (Gen 1 and Gen 2)....464

Azure TSI – Capabilities....465

Exploring Azure TSI Data Storage....466

High-level architecture of Azure TSI....468

Creating Azure IoT hub instance....470

Creating Azure TSI Gen2 environment....478

Exploring Azure TSI Explorer....487

Conclusion....489

Points to remember....489

11. AWS Forecast....491

Introduction....491

Structure....491

Prerequisites....492

Understanding Amazon Forecast Service (AFS)....492

Workflow for Amazon Forecast Service....492

Data Preparations....493

Dataset Guidelines for Forecast....499

Quality of Data....502

Importing data....503

Training data....507

Forecast creation and selection....508

Retrieve the Forecast....509

Orchestration of Amazon Forecast....511

Conclusion....511

Points to remember....512

Index....513

Embark on a transformative journey through the intricacies of time series analysis and forecasting with this comprehensive handbook. Beginning with the essential packages for data science and machine learning projects you will delve into Python's prowess for efficient time series data analysis, exploring the core components and real-world applications across various industries through compelling use-case studies. From understanding classical models like AR, MA, ARMA, and ARIMA to exploring advanced techniques such as exponential smoothing and ETS methods, this guide ensures a deep understanding of the subject.It will help you navigate the complexities of vector autoregression (VAR, VMA, VARMA) and elevate your skills with a deep dive into deep learning techniques for time series analysis. By the end of this book, you will be able to harness the capabilities of Azure Time Series Insights and explore the cutting-edge AWS Forecast components, unlocking the cloud's power for advanced and scalable time series forecasting.

What you will learn

• Explore Time Series Data Analysis and Forecasting, covering components and significance.
• Gain a practical understanding through hands-on examples and real-world case studies.
• Master Time Series Models (AR, MA, ARMA, ARIMA, VAR, VMA, VARMA) with executable samples.
• Delve into Deep Learning for Time Series Analysis, demystified with classical examples.
• Actively engage with Azure Time Series Insights and AWS Forecast components for a contemporary perspective.

Who is this book for?

This book caters to beginners, intermediates, and practitioners in data-related fields such as Data Analysts, Data Scientists, and Machine Learning Engineers, as well as those venturing into Time Series Analysis and Forecasting. It assumes readers have a foundational understanding of programming languages (C, C++, Python), data structures, statistics, and visualization concepts. With a focus on specific projects, it also functions as a quick reference for advanced users.