Bayesian Analysis with Python: A practical guide to probabilistic modeling. 3 Ed

Cover....1

Copyright....3

Foreword....5

Contributors....8

Table of Contents....12

Preface....20

Who this book is for....21

What this book covers....21

What's new in this edition?....23

Installation instructions....25

Conventions used....27

Chapter 1: Thinking Probabilistically....30

Statistics, models, and this book's approach....31

Working with data....32

Bayesian modeling....33

A probability primer for Bayesian practitioners....34

Sample space and events....34

Random variables....38

Discrete random variables and their distributions....40

Continuous random variables and their distributions....45

Cumulative distribution function....47

Conditional probability....49

Expected values....51

Bayes' theorem....52

Interpreting probabilities....55

Probabilities, uncertainty, and logic....57

Single-parameter inference....58

The coin-flipping problem....58

Choosing the likelihood....59

Choosing the prior....60

Getting the posterior....62

The influence of the prior....66

How to choose priors....67

Communicating a Bayesian analysis....70

Model notation and visualization....70

Summarizing the posterior....71

Summary....73

Exercises....74

Chapter 2: Programming Probabilistically....78

Probabilistic programming....79

Flipping coins the PyMC way....80

Summarizing the posterior....83

Posterior-based decisions....86

Savage-Dickey density ratio....87

Region Of Practical Equivalence....88

Loss functions....90

Gaussians all the way down....93

Gaussian inferences....93

Posterior predictive checks....97

Robust inferences....99

Degrees of normality....100

A robust version of the Normal model....101

InferenceData....105

Groups comparison....107

The tips dataset....109

Cohen's d....112

Probability of superiority....113

Posterior analysis of mean differences....114

Summary....116

Exercises....117

Chapter 3: Hierarchical Models....120

Sharing information, sharing priors....121

Hierarchical shifts....122

Water quality....126

Shrinkage....129

Hierarchies all the way up....132

Summary....136

Exercises....137

Chapter 4: Modeling with Lines....140

Simple linear regression....141

Linear bikes....143

Interpreting the posterior mean....145

Interpreting the posterior predictions....148

Generalizing the linear model....149

Counting bikes....150

Robust regression....152

Logistic regression....155

The logistic model....155

Classification with logistic regression....158

Interpreting the coefficients of logistic regression....160

Variable variance....162

Hierarchical linear regression....165

Centered vs. noncentered hierarchical models....168

Multiple linear regression....170

Summary....173

Exercises....174

Chapter 5: Comparing Models....176

Posterior predictive checks....177

The balance between simplicity and accuracy....183

Many parameters (may) lead to overfitting....183

Too few parameters lead to underfitting....185

Measures of predictive accuracy....186

Information criteria....187

Akaike Information Criterion....188

Widely applicable information criteria....189

Other information criteria....189

Cross-validation....190

Approximating cross-validation....191

Calculating predictive accuracy with ArviZ....193

Model averaging....196

Bayes factors....197

Some observations....199

Calculation of Bayes factors....200

Analytically....200

Sequential Monte Carlo....203

Savage–Dickey ratio....204

Bayes factors and inference....207

Regularizing priors....208

Summary....210

Exercises....211

Chapter 6: Modeling with Bambi....214

One syntax to rule them all....215

The bikes model, Bambi's version....219

Polynomial regression....222

Splines....224

Distributional models....227

Categorical predictors....229

Categorical penguins....229

Relation to hierarchical models....232

Interactions....233

Interpreting models with Bambi....236

Variable selection....238

Projection predictive inference....240

Projection predictive with Kulprit....241

Summary....246

Exercises....247

Chapter 7: Mixture Models....250

Understanding mixture models....251

Finite mixture models....253

The Categorical distribution....255

The Dirichlet distribution....255

Chemical mixture....256

The non-identifiability of mixture models....258

How to choose K....260

Zero-Inflated and hurdle models....263

Zero-Inflated Poisson regression....264

Hurdle models....266

Mixture models and clustering....269

Non-finite mixture model....270

Dirichlet process....270

Continuous mixtures....276

Some common distributions are mixtures....276

Summary....277

Exercises....279

Chapter 8: Gaussian Processes....282

Linear models and non-linear data....283

Modeling functions....284

Multivariate Gaussians and functions....286

Covariance functions and kernels....287

Gaussian processes....290

Gaussian process regression....291

Gaussian process regression with PyMC....291

Setting priors for the length scale....295

Gaussian process classification....296

GPs for space flu....299

Cox processes....300

Coal mining disasters....301

Red wood....303

Regression with spatial autocorrelation....306

Hilbert space GPs....311

HSGP with Bambi....314

Summary....315

Exercises....316

Chapter 9: Bayesian Additive Regression Trees....318

Decision trees....319

BART models....321

Bartian penguins....322

Partial dependence plots....324

Individual conditional plots....325

Variable selection with BART....326

Distributional BART models....329

Constant and linear response....331

Choosing the number of trees....333

Summary....334

Exercises....334

Chapter 10: Inference Engines....336

Inference engines....337

The grid method....338

Quadratic method....341

Markovian methods....343

Monte Carlo....343

Markov chain....345

Metropolis-Hastings....346

Hamiltonian Monte Carlo....351

Sequential Monte Carlo....353

Diagnosing the samples....356

Convergence....357

Trace plot....357

Rank plot....359

, (R hat)....360

Effective Sample Size (ESS)....362

Monte Carlo standard error....364

Divergences....365

Keep calm and keep trying....367

Summary....368

Exercises....369

Chapter 11: Where to Go Next....372

Other Books You May Enjoy....383

Index....388

The third edition of Bayesian Analysis with Python serves as an introduction to the main concepts of applied Bayesian modeling using PyMC, a state-of-the-art probabilistic programming library, and other libraries that support and facilitate modeling like ArviZ, for exploratory analysis of Bayesian models; Bambi, for flexible and easy hierarchical linear modeling; PreliZ, for prior elicitation; PyMC-BART, for flexible non-parametric regression; and Kulprit, for variable selection.

In this updated edition, a brief and conceptual introduction to probability theory enhances your learning journey by introducing new topics like Bayesian additive regression trees (BART), featuring updated examples. Refined explanations, informed by feedback and experience from previous editions, underscore the book's emphasis on Bayesian statistics. You will explore various models, including hierarchical models, generalized linear models for regression and classification, mixture models, Gaussian processes, and BART, using synthetic and real datasets.

By the end of this book, you will possess a functional understanding of probabilistic modeling, enabling you to design and implement Bayesian models for your data science challenges. You'll be well-prepared to delve into more advanced material or specialized statistical modeling if the need arises.

What you will learn

• Build probabilistic models using PyMC and Bambi
• Analyze and interpret probabilistic models with ArviZ
• Acquire the skills to sanity-check models and modify them if necessary
• Build better models with prior and posterior predictive checks
• Learn the advantages and caveats of hierarchical models
• Compare models and choose between alternative ones
• Interpret results and apply your knowledge to real-world problems
• Explore common models from a unified probabilistic perspective
• Apply the Bayesian framework's flexibility for probabilistic thinking

Who this book is for

If you are a student, data scientist, researcher, or developer looking to get started with Bayesian data analysis and probabilistic programming, this book is for you. The book is introductory, so no previous statistical knowledge is required, although some experience in using Python and scientific libraries like NumPy is expected.