Pandas Workout: 200 exercises to make you a stronger data analyst

Pandas Workout....1

brief contents....6

contents....7

preface....10

acknowledgments....12

about this book....14

Who should read this book....15

How this book is organized: A road map....16

About the code....17

Software/hardware requirements....17

liveBook discussion forum....18

about the author....19

about the cover illustration....20

1 Series....21

Useful references....23

Exercise 1 - Test scores....24

Working it out....25

Solution....30

Beyond the exercise....31

Exercise 2 - Scaling test scores....36

Working it out....36

Solution....39

Beyond the exercise....39

Exercise 3 - Counting tens digits....39

Working it out....39

Solution....43

Beyond the exercise....43

Exercise 4 - Descriptive statistics....46

Working it out....47

Solution....49

Beyond the exercise....49

Exercise 5 - Monday temperatures....49

Working it out....50

Solution....51

Beyond the exercise....51

Exercise 6 - Passenger frequency....52

Working it out....52

Solution....53

Beyond the exercise....54

Exercise 7 - Long, medium, and short taxi rides....54

Working it out....54

Solution....56

Beyond the exercise....56

Summary....56

2 Data frames....57

Exercise 8 - Net revenue....61

Working it out....61

Solution....64

Beyond the exercise....64

Exercise 9 - Tax planning....64

Working it out....64

Solution....66

Beyond the exercise....66

Exercise 10 - Adding new products....73

Working it out....74

Solution....77

Beyond the exercise....77

Exercise 11 - Bestsellers....78

Working it out....78

Solution....80

Beyond the exercise....80

Exercise 12 - Finding outliers....80

Working it out....81

Solution....83

Beyond the exercise....83

Exercise 13 - Interpolation....85

Working it out....85

Solution....87

Beyond the exercise....87

Exercise 14 - Selective updating....87

Working it out....87

Solution....88

Beyond the exercise....89

Summary....89

3 Importing and exporting data....90

Exercise 15 - Weird taxi rides....93

Working it out....94

Solution....98

Beyond the exercise....98

Exercise 16 - Pandemic taxis....99

Working it out....100

Solution....104

Beyond the exercise....105

Exercise 17 - Setting column types....107

Working it out....107

Beyond the exercise....109

Exercise 18 - passwd to df....109

Working it out....110

Solution....111

Beyond the exercise....112

Exercise 19 - Bitcoin values....112

Working it out....113

Solution....116

Beyond the exercise....116

Exercise 20 - Big cities....116

Working it out....117

Solution....118

Beyond the exercise....118

Summary....119

4 Indexes....120

Exercise 21 - Parking tickets....122

Working it out....123

Solution....127

Beyond the exercise....127

Exercise 22 - State SAT scores....132

Working it out....133

Solution....134

Beyond the exercise....135

Exercise 23 - Olympic games....136

Working it out....137

Solution....141

Beyond the exercise....143

Exercise 24 - Olympic pivots....146

Working it out....146

Solution....149

Beyond the exercise....149

Summary....150

5 Cleaning data....151

Exercise 25 - Parking cleanup....155

Working it out....156

Solution....161

Beyond the exercise....162

Exercise 26 - Celebrity deaths....163

Working it out....164

Solution....167

Beyond the exercise....168

Exercise 27 - Titanic interpolation....168

Working it out....169

Solution....173

Beyond the exercise....173

Exercise 28 - Inconsistent data....174

Working it out....175

Solution....177

Beyond the exercise....177

Summary....178

6 Grouping, joining, and sorting....179

Exercise 29 - Longest taxi rides....182

Working it out....183

Solution....189

Beyond the exercise....189

Exercise 30 - Taxi ride comparison....192

Working it out....192

Solution....196

Beyond the exercise....197

Exercise 31 - Tourist spending per country....202

Working it out....203

Solution....209

Beyond the exercise....210

Summary....210

7 Advanced grouping, joining, and sorting....211

Exercise 32 - Multicity temperatures....214

Working it out....214

Solution....220

Beyond the exercise....221

Exercise 33 - SAT scores, revisited....224

Working it out....224

Solution....230

Beyond the exercise....232

Exercise 34 - Snowy, rainy cities....235

Working it out....236

Solution....240

Beyond the exercise....242

Exercise 35 - Wine scores and tourism spending....242

Working it out....243

Solution....249

Beyond the exercise....250

Summary....250

8 Midway project....251

Problem....252

Working it out....254

Solution....267

Summary....270

9 Strings....271

Exercise 36 - Analyzing Alice....276

Working it out....277

Solution....280

Beyond the exercise....281

Exercise 37 - Wine words....281

Working it out....282

Solution....287

Beyond the exercise....288

Exercise 38 - Programmer salaries....288

Working it out....289

Solution....296

Beyond the exercise....297

Summary....298

10 Dates and times....299

Exercise 39 - Short, medium, and long taxi rides....305

Working it out....306

Solution....310

Beyond the exercise....311

Exercise 40 - Writing dates, reading dates....311

Working it out....311

Solution....313

Beyond the exercise....314

Exercise 41 - Oil prices....317

Working it out....317

Solution....320

Beyond the exercise....320

Exercise 42 - Best tippers....320

Working it out....321

Solution....325

Beyond the exercise....326

Summary....326

11 Visualization....327

Exercise 43 - Cities....329

Working it out....330

Solution....336

Beyond the exercise....337

Exercise 44 - Boxplotting weather....340

Working it out....340

Solution....345

Beyond the exercise....346

Exercise 45 - Taxi fare breakdown....347

Working it out....347

Solution....356

Beyond the exercise....357

Exercise 46 - Cars, oil, and ice cream....361

Working it out....362

Solution....366

Beyond the exercise....367

Exercise 47 - Seaborn taxi plots....378

Working it out....378

Solution....383

Beyond the exercise....384

Summary....384

12 Performance....385

Exercise 48 - Categories....390

Working it out....391

Solution....394

Beyond the exercise....394

Exercise 49 - Faster reading and writing....396

Working it out....396

Solution....400

Beyond the exercise....401

Exercise 50 - “query” and “eval”....404

Working it out....404

Solution....410

Beyond the exercise....411

Summary....411

13 Final project....412

Problem....412

Column names and meanings....414

Working it out....416

Summary....436

index....437

Symbols....437

A....437

B....437

C....437

D....437

E....438

F....438

G....438

H....438

I....438

J....438

K....438

L....438

M....438

N....439

O....439

P....439

Q....439

R....439

S....439

T....440

U....440

V....440

W....440

X....440

Pandas Workout - back....442

Practice makes perfect pandas!

Work out your pandas skills against dozens of real-world challenges, each carefully designed to build an intuitive knowledge of essential pandas tasks.

In Pandas Workout you’ll learn how to:

• Clean your data for accurate analysis
• Work with rows and columns for retrieving and assigning data
• Handle indexes, including hierarchical indexes
• Read and write data with a number of common formats, such as CSV and JSON
• Process and manipulate textual data from within pandas
• Work with dates and times in pandas
• Perform aggregate calculations on selected subsets of data
• Produce attractive and useful visualizations that make your data come alive

Pandas Workout hones your pandas skills to a professional-level through two hundred exercises, each designed to strengthen your pandas skills. You’ll test your abilities against common pandas challenges such as importing and exporting, data cleaning, visualization, and performance optimization. Each exercise utilizes a real-world scenario based on real-world data, from tracking the parking tickets in New York City, to working out which country makes the best wines. You’ll soon find your pandas skills becoming second nature—no more trips to StackOverflow for what is now a natural part of your skillset.

About the technology

Python’s pandas library can massively reduce the time you spend analyzing, cleaning, exploring, and manipulating data. And the only path to pandas mastery is practice, practice, and, you guessed it, more practice. In this book, Python guru Reuven Lerner is your personal trainer and guide through over 200 exercises guaranteed to boost your pandas skills.

About the book

Pandas Workout is a thoughtful collection of practice problems, challenges, and mini-projects designed to build your data analysis skills using Python and pandas. The workouts use realistic data from many sources: the New York taxi fleet, Olympic athletes, SAT scores, oil prices, and more. Each can be completed in ten minutes or less. You’ll explore pandas’ rich functionality for string and date/time handling, complex indexing, and visualization, along with practical tips for every stage of a data analysis project.

What's inside

• Clean data with less manual labor
• Retrieving and assigning data
• Process and manipulate text
• Calculations on selected data subsets

About the reader

For Python programmers and data analysts.