Software Testing Strategies: A testing guide for the 2020s

Software Testing Strategies....2

Contributors....7

About the authors....7

About the reviewers:....9

Preface....23

Our scope - beyond button-pushing....23

What is Testing?....25

Who this book is for....28

What this book covers....29

To get the most out of this book....30

Download the example code files....31

Download the color images....31

Conventions used....31

Get in touch....32

Share your thoughts....32

Download a free PDF copy of this book....32

Part 1:The Practice of Software Testing....34

Chapter 1: Testing and Designing Tests....35

Jumping into testing....35

The impossibility of complete testing....36

What is the EVP asking for?....37

Toward a theory of error....43

Testing software – an example....44

Testing software – an analysis....51

Quick attacks – the airdropped tester....52

Test design – input space coverage....54

Equivalence classes and boundaries....57

Decision tables....61

Decision trees....63

All-pairs and pairwise testing....67

High volume automated approaches....71

Other approaches....73

Data and predictability – the oracle problem....74

Summary....76

Further reading....76

Chapter 2: Fundamental Issues in Tooling and Automation....77

Technical requirements....77

No silver bullets – you can’t radically fix the test cycle....78

The minefield regression problem....79

Coverage model....80

The Battleships problem – testing versus checking....84

Comparing Battleships to bugs....86

Automation rarely beats human intuition....87

The maintenance problem....88

The dangers of comprehensive tests....92

The money problem....93

Lessons from the fundamental issues....96

Summary....97

Chapter 3: Programmer-Facing Testing....99

Technical requirements....99

The programmer’s view....100

Testing and reliability....101

The hexagonal architecture....101

Introducing FizzBuzz....104

Unit tests....106

TDD....110

Consequences of TDD....112

Unit test and unit code design....113

Using test doubles to create seams....115

Mutation testing....116

Web APIs from a test perspective....120

Web API testing strategy....121

Testing functional and legacy code....123

A Roman Numerals Kata....123

Summary....124

Chapter 4: Customer-Facing Tests....126

Technical requirements....126

A word of warning....126

Human Or Tooling—is it either/or?....128

GUI test automation patterns....129

Eliminating redundancy with domain-specific libraries....132

Eliminating redundancy through object locators....133

Do you need conditionals, looping structures, and variables?....134

The tradeoff between assertion and image recognition....137

Designing your own system....139

Toward specification by example....140

Specification by example....142

Low-code and no-code test automation....144

Batch- and model-driven test automation....145

This chapter is completely wrong....149

Summary....149

Chapter 5: Specialized Testing....151

Technical requirements....151

Understanding load and performance testing....151

Getting to know the basics of load testing....152

Setting up a load test....152

Exploring security testing....156

Security concepts....156

Checking out common security vulnerabilities....157

Learning about industry standards....157

Investigating some security testing tools....158

Delving into accessibility testing....158

What is accessibility?....158

Advocating for accessibility....159

Investigating the distinctions between accessibility and inclusive design....160

Learning about the WCAG standard....160

Investigating some accessibility testing tools....160

Internationalization and localization....161

Preparing for internationalization and localization....162

Investigating tools for internationalization and localization....162

CI....163

CI and the pipeline....163

Getting involved with build management as a tester....163

Investigating CI tools....163

Regulated testing....164

Navigating regulatory requirements....164

Summary....165

Chapter 6: Testing Related Skills....166

Technical requirements....166

Finding bugs....167

Oracles in software testing....168

Inattentional blindness and Oracles....170

About the word bug....172

Writing bug reports....172

Effective bug reports....174

Effective reproduction steps....176

Planning testing – cases and plans....179

Test cases in practice....187

Metrics and measurement....188

Metric dysfunction....189

Project projections....191

Influencing change....192

Summarizing information....192

Summary....194

Chapter 7: Test Data Management....196

Technical requirements....196

The test data problem....197

Test data and database apps....199

The standard data seed....201

Table-driven – when test data drives behavior....203

Scriptable users and structure....204

Exploring synthetic users....204

Leveraging production refreshes....207

Exploring development, test, and production environments....208

Understanding the regulatory issues in test data management....209

The user spy feature....210

Summary....210

Part 2:Testing and Software Delivery....212

Chapter 8: Delivery Models and Testing....213

Technical requirements....214

Waterfall....214

The advantages of and place for the waterfall....217

The V-Model....218

Iterative, incremental, and mini waterfalls....219

Extreme Programming (XP)....222

The context of XP....222

Scrum and SAFe....223

The context of Scrum....227

SAFe and its context....228

House-rules software development....232

The Agile Manifesto....242

Context-driven testing....244

Illustrations of the principles in action....244

Kanban as a house rule....245

Continuous delivery and deployment....246

DevOps, Platform Engineering, SRE....247

Summary....248

Further reading....249

Chapter 9: The Puzzle Pieces of Good Testing....250

Technical requirements....250

Recipes – how to do hard things....251

Defining recipes....251

Shouldn’t recipes just be automated?....254

Do recipes overlap with technical documentation?....255

Coverage – did we test the right things well enough?....255

Precise code coverage measures....259

Closing out coverage....260

Defects – what is the status of the software?....260

Schedule and risk – too many test ideas, not enough time....261

Iterative testing....262

Strategy – what are our risks and priorities?....263

Dashboard – how do we communicate what we know?....264

Summary....269

Chapter 10: Putting Your Test Strategy Together....271

What are we doing now?....271

Getting the form filled out....276

The elevator pitch....276

A census of risk....279

Defining a real risk census....281

Setting priorities, time management, and scope....283

Today’s strategy versus tomorrow’s goals....285

Summary....286

Chapter 11: Lean Software Testing....288

Lean software testing defined....288

From ideas in practice to the term “Lean”....289

The seven wastes....291

Waste #1 – transport....292

Waste #2 – inventory....293

Waste #3 – motion....294

Waste #4 – waiting....296

Waste #5 – overprocessing....297

Waste #6 – overproduction....298

Waste #7 – defects....298

(New) waste #8 – ability....300

Removing waste in testing....301

Flow....302

Visualizing flow – an example....303

Multitasking....305

Measurement – lead time versus cycle time....307

Efficiency and congestion....308

Metric – touch time....308

Batch size....309

Queues and efficiency....310

Arrival time pacing....311

Limiting work in progress to create a pull system....311

Release cadence....312

One-piece flow and CD....313

Summary....314

Part 3:Practicing Politics....315

Chapter 12: Case Studies and Experience Reports....316

RCRCRC at scale....317

The 1-day test plan....318

RCRCRC in the enterprise....318

A coverage dashboard....320

Test coverage blinders....322

Pair and tri-programming....323

Discovering the expertise paradox....324

Making expertise transparent....326

The evolution of the test strategy....326

The alternative to evolution – information hiding....328

Professional pushback – dealing with bullies....329

Power in the workplace....330

One way to say it....331

Boundaries in the enterprise....337

Narcissistic communication....339

Boundary enforcement – the power move....341

If you choose to stay....341

Summary....342

Chapter 13: Testing Activities or a Testing Role?....343

Technical requirements....343

The cultural conflict with a testing role....343

How we got here – the bad news....344

How we got here – developing at internet speed....346

Building a risk mitigation team....348

The purpose of the risk exercise....349

Faith-based versus empirical test automation....352

The math behind faith-based test automation....353

Possible outcomes....353

Shift left and shift right....354

(Actually) continuous testing....355

Summary....356

Chapter 14: Philosophy and Ethics in Software Testing....358

Philosophy and why it matters in testing....358

Sprint Length: it depends....359

Shu Ha Ri: learning by osmosis....360

A tough question: what do you want?....361

Ethics and ethical reasoning in testing....363

Ethical frameworks....364

Classic ethical challenges....366

Practical ethical issues in testing....366

Skipping test steps and the good news....367

Decisions are not made in meetings....368

Scientific thinking and logical fallacies....369

Logical fallacies....370

How we wind up in hell and how to escape....371

Put the responsibility in the right place....372

Summary....373

Additional readings for the serious philosopher....373

Chapter 15: Words and Language About Work....375

Context-driven testing and the other schools....375

The Agile school....376

The DevOps or CD school....376

The analytical school....376

The factory school....377

The quality school....377

The context-driven school....377

Precise language....379

Wordsmatter....380

The benefits of being imprecise....381

Process versus skill....384

Testing and checking....386

Yes, we can assure quality....386

Summary....389

Further reading....389

Chapter 16: Testing Strategy Applied....390

A mobile test strategy example....390

Our sample mobile app....391

Designing a test program....392

Mobile system coordination and deployment....394

The human element....395

AI in software testing....396

The state of AI tools....397

Possible ways to use AI for testing....398

Other forms of AI in testing....401

The bottom line for AI testing....402

A few thoughts to leave with....402

Thoughts on documentation....403

Summary....404

Index....405

Why subscribe?....427

Other Books You May Enjoy....427

Packt is searching for authors like you....431

Share your thoughts....431

Download a free PDF copy of this book....431

Software Testing Strategies covers a wide range of topics in the field of software testing, providing practical insights and strategies for professionals at every level. With equal emphasis on theoretical knowledge and practical application, this book is a valuable resource for programmers, testers, and anyone involved in software development.

The first part delves into the fundamentals of software testing, teaching you about test design, tooling, and automation. The chapters help you get to grips with specialized testing areas, including security, internationalization, accessibility, and performance. The second part focuses on the integration of testing into the broader software delivery process, exploring different delivery models and puzzle pieces contributing to effective testing. You’ll discover how to craft your own test strategies and learn about lean approaches to software testing for optimizing processes. The final part goes beyond technicalities, addressing the broader context of testing. The chapters cover case studies, experience reports, and testing responsibilities, and discuss the philosophy and ethics of software testing.

By the end of this book, you’ll be equipped to elevate your testing game and ensure software quality, and have an indispensable guide to the ever-evolving landscape of software quality assurance.

What you will learn

• Explore accessibility, functional testing, performance testing, and more as an integral part of testing
• Find out how to implement a wide range of testing approaches
• Develop the skills needed to create effective testing strategies tailored to your project's needs
• Discover how to prioritize and execute the most impactful test ideas
• Gain insight into when and how to apply different testing elements
• Defend your chosen testing strategy with a comprehensive understanding of its components

Who this book is for

This book is for a broad spectrum of professionals engaged in software development, including programmers, testers, and DevOps specialists. Tailored to those who aspire to elevate their testing practices beyond the basics, the book caters to anyone seeking practical insights and strategies to master the nuanced interplay between human intuition and automation. Whether you are a seasoned developer, meticulous tester, or DevOps professional, this comprehensive guide offers a transformative roadmap to become an adept strategist in the dynamic realm of software quality assurance.