Node.js Design Patterns: Level up your Node.js skills and design production-grade applications using proven techniques. 4 Ed

Cover....1

Title Page....2

Copyright and Credits....3

Forewords....4

Contributors....8

Table of Contents....12

Preface....26

Chapter 1: The Node.js Platform....38

The Node.js philosophy....39

Small core....40

Small modules....40

Small surface area....42

Simplicity and pragmatism....42

How Node.js works....43

I/O is often the bottleneck....44

Blocking I/O....44

Non-blocking I/O....45

Event demultiplexing....46

The reactor pattern....48

libuv, the I/O engine of Node.js....50

The complete recipe for Node.js....51

JavaScript in Node.js....52

Run the latest JavaScript with confidence....52

The module system....53

Full access to operating system services....53

Running native code....54

Node.js and TypeScript....55

Using TypeScript with Node.js....55

The @types/node package....56

Summary....57

Chapter 2: The Module System....58

The need for modules....59

Module systems in JavaScript and Node.js....59

The revealing module pattern....60

ES modules....62

Using ES modules in Node.js....62

The ES module syntax....64

Named exports and imports....64

Default exports and imports....66

Mixed exports....67

Module identifiers....69

Static and dynamic imports....70

The module resolution algorithm....72

Module loading in depth....74

Loading phases....74

Read-only live bindings....76

Circular dependencies....77

Modules that modify other modules....83

How monkey patching affects type safety in TypeScript projects....89

CommonJS modules....90

ES modules and CommonJS—differences and interoperability....91

Strict mode....91

Top-level await....92

Behavior of `this`....92

Missing references in ES modules....93

Import interoperability....94

Import CommonJS modules from ES modules....94

Import ES modules from CommonJS....96

Importing JSON files....97

Using modules in TypeScript....99

The role of the TypeScript compiler....99

Configuring the module output format....101

Input module syntax and output emission....101

Module resolution....101

Summary....102

Chapter 3: Callbacks and Events....104

The Callback pattern....105

The continuation-passing style....105

Synchronous CPS....106

Asynchronous CPS....107

Non-CPS callbacks....110

Synchronous or asynchronous?....110

Writing an inconsistent function....110

Unleashing Zalgo....111

Using synchronous APIs....113

Guaranteeing asynchronicity with deferred execution....114

Node.js callback conventions....117

The callback is the last argument....117

Any error always comes first....117

Propagating errors....118

Avoiding uncaught exceptions....119

The Observer pattern....121

The EventEmitter....121

Creating and using the EventEmitter....122

Propagating errors....123

Making any object observable....124

The risk of memory leaks....125

Synchronous and asynchronous events....127

EventEmitter versus callbacks....128

Combining callbacks and events....129

Summary....133

Exercises....134

Chapter 4: Asynchronous Control Flow Patterns with Callbacks....136

The challenges of asynchronous programming....137

Creating a simple web spider....137

Callback hell....140

Callback best practices....142

Callback discipline....143

Applying the callback discipline....143

Control flow patterns....146

Sequential execution....146

Executing a known set of tasks in sequence....146

Sequential iteration....147

Concurrent execution....152

Web spider version 3....154

The pattern....155

Fixing race conditions with concurrent tasks....156

Limited concurrent execution....158

Limiting concurrency....159

Globally limiting concurrency....161

Summary....167

Exercises....167

Chapter 5: Asynchronous Control Flow Patterns with Promises and Async/Await....170

Promises....171

What is a promise?....171

Promises/A+ and thenables....174

The promise API....175

Creating a promise....177

Promisification....178

Sequential execution and iteration....180

Concurrent execution....184

Limited concurrent execution....185

Implementing the TaskQueue class with promises....185

Updating the web spider....187

Lazy promises....190

Async/await....194

Async functions and the await expression....194

Top-level await....195

Error handling with async/await....196

A unified try...catch experience....197

The “return” versus “return await” trap....198

Sequential execution and iteration....199

Antipattern – using async/await with Array.forEach for serial execution....201

Concurrent execution....201

Limited concurrent execution....203

The problem with infinite recursive promise resolution chains....204

Summary....207

Exercises....208

Chapter 6: Coding with Streams....210

Discovering the importance of streams....212

Buffering versus streaming....212

Spatial efficiency....213

Gzipping using a buffered API....214

Gzipping using streams....215

Time efficiency....215

Composability....220

Adding client-side encryption....221

Adding server-side decryption....222

Getting started with streams....223

Anatomy of streams....224

Readable streams....224

Reading from a stream....224

Implementing Readable streams....227

Writable streams....232

Writing to a stream....232

Backpressure....234

Implementing Writable streams....236

Duplex streams....238

Transform streams....239

Implementing Transform streams....240

Filtering and aggregating data with Transform streams....243

PassThrough streams....247

Observability....247

Late piping....248

Lazy streams....251

Connecting streams using pipes....252

Pipes and error handling....253

Better error handling with pipeline()....254

Asynchronous control flow patterns with streams....255

Sequential execution....256

Unordered concurrent execution....258

Implementing an unordered concurrent stream....258

Implementing a URL status monitoring application....260

Unordered limited concurrent execution....262

Ordered concurrent execution....264

Piping patterns....266

Combining streams....266

Implementing a combined stream....268

Forking streams....270

Implementing a multiple checksum generator....271

Merging streams....272

Merging text files....272

Multiplexing and demultiplexing....274

Building a remote logger....275

Multiplexing and demultiplexing object streams....279

Readable stream utilities....280

Mapping and transformation....280

Filtering and iteration....281

Searching and evaluation....281

Limiting and reducing....281

Web Streams....283

Converting Node.js streams to Web Streams....284

Converting Web Streams to Node.js streams....284

Stream consumer utilities....286

Summary....288

Exercises....288

Chapter 7: Creational Design Patterns....290

Factory....291

Decoupling object creation and implementation....292

A mechanism to enforce encapsulation....293

Building a simple code profiler....294

In the wild....297

Builder....297

Implementing a URL object builder....302

In the wild....305

Revealing Constructor....308

Building an immutable buffer....309

In the wild....311

Singleton....312

Wiring modules....315

Singleton dependencies....316

Dependency Injection....319

Summary....323

Exercises....324

Chapter 8: Structural Design Patterns....326

Proxy....326

Techniques for implementing proxies....327

Object composition....329

Object augmentation....332

The built-in Proxy object....333

A comparison of the different proxying techniques....336

Creating a logging Writable stream....337

Change Observer with Proxy....338

In the wild....341

Decorator....341

Techniques for implementing decorators....341

Composition....342

Object decoration....343

Decorating with the Proxy object....344

Decorating a Level database....346

Introducing Level and LevelDB....346

Implementing a Level plugin....346

In the wild....348

The decorator proposal for ECMAScript....349

The line between Proxy and Decorator....350

Adapter....351

Using Level through the filesystem API....351

In the wild....354

Summary....355

Exercises....355

Chapter 9: Behavioral Design Patterns....358

Strategy....359

Multi-format configuration objects....362

In the wild....367

State....368

Implementing a basic failsafe socket....369

In the wild....377

Template....378

A configuration manager template....379

In the wild....381

Iterator....381

The iterator protocol....382

The iterable protocol....385

Iterators and iterables as a native JavaScript interface....387

Implementing the iterable protocol on iterators....389

Iterator utilities....390

Generators....396

Generators in theory....396

A simple generator function....397

Controlling a generator iterator....398

How to use generators in place of iterators....399

Async iterators....401

Async generators....405

Async iterators and Node.js streams....406

Async iterator utilities....407

In the wild....409

Middleware....410

Middleware in Express....410

Middleware as a pattern....411

Creating a middleware framework for ZeroMQ....412

The middleware manager....412

Implementing the middleware to process messages....415

Using the ZeroMQ middleware framework....416

In the wild....418

Command....419

The Task pattern....420

A more complex command....420

In the wild....424

Summary....425

Exercises....426

Chapter 10: Testing: Patterns and Best Practices....428

An introduction to software testing....429

Definitions....430

System under test....430

Arrange, Act, Assert....430

Code coverage....431

Test doubles: Stubs, spies, and mocks....431

Test-driven development....432

Behavior-driven development....433

Continuous integration....434

Continuous delivery and continuous deployment....434

Types of tests....435

Unit tests....436

Integration tests....437

End-to-end tests....437

Other types of tests....439

The testing pyramid....440

Writing tests with Node.js....441

Our first unit test....441

The Node.js test runner....444

Our first test with the Node.js test runner....445

Organizing tests....446

Subtests....448

Subtest concurrency....449

Parametrized test cases....450

Test suites....452

Test runner tips and tricks....453

Watch mode....453

Targeted test execution with custom glob patterns....454

Test reporters....456

Collecting code coverage....457

Using the test runner with TypeScript....460

Writing unit tests....460

Testing asynchronous code....461

Mocking....467

Creating spies with mock.fn()....467

Mocking HTTP requests with the built-in test mock....469

Mocking HTTP requests with Undici....472

Mocking Node.js core modules....475

Mocking other dependencies....478

Problems with mocking imports....482

Mocking imports versus dependency injection....483

Writing integration tests....486

Testing with a local database....486

Testing a web application....494

Setting up the project....495

Writing E2E tests....504

The application structure....505

Home page....505

Sign-in and sign-up forms....506

Event Page....506

My reservations....507

The user flow....507

Browser automation....509

Writing an E2E test with Playwright....510

Setting up a new Playwright project....510

Understanding the Playwright API....512

Understanding timeouts....516

Testing our user flow....516

Summary....521

Exercises....521

Chapter 11: Advanced Recipes....524

Dealing with asynchronously initialized components....525

The issue with asynchronously initialized components....525

Local initialization check....527

Delayed startup....528

Pre-initialization queues....529

Using the State pattern....532

In the wild....535

Asynchronous request batching and caching....535

What’s asynchronous request batching?....535

Optimal asynchronous request caching....537

An API server without caching or batching....539

Batching and caching with promises....542

Batching requests....542

Batching and caching requests....544

Canceling asynchronous operations....546

A basic recipe for creating cancelable functions....546

Wrapping asynchronous invocations....548

Cancelable async functions with AbortController....549

Running CPU-bound tasks....553

Solving the subset sum problem....553

Interleaving with setImmediate....557

Interleaving the steps of the subset sum algorithm....557

Considerations on the interleaving approach....559

Using external processes....559

Delegating the subset sum task to an external process....560

Considerations for the multi-process approach....566

Using worker threads....567

Running the subset sum task in a worker thread....568

Running CPU-bound tasks in production....571

Summary....571

Exercises....571

Chapter 12: Scalability and Architectural Patterns....574

An introduction to application scaling....575

Scaling Node.js applications....575

The three dimensions of scalability....575

X-axis: cloning....576

Y-axis: decomposing by service or functionality....577

Z-axis: splitting by data partition....577

Combining strategies across the scale cube....577

Cloning and load balancing....578

The cluster module....579

Notes on the behavior of the cluster module....580

Building a simple HTTP server....581

Scaling with the cluster module....582

Resiliency and availability with the cluster module....584

Zero-downtime restart....586

Dealing with stateful communications....589

Sharing the state across multiple instances....590

Sticky load balancing....591

Scaling with a reverse proxy....592

Load balancing with Nginx....594

Dynamic horizontal scaling....598

Using a service registry....599

Implementing a dynamic load balancer....600

Peer-to-peer load balancing....607

Implementing an HTTP client that can balance requests across multiple servers....609

Scaling applications using containers....611

What is a container?....611

Creating and running a container with Docker....612

What is Kubernetes?....615

Deploying and scaling an application on Kubernetes....617

Decomposing complex applications....620

Monolithic architecture....620

The microservice architecture....622

An example of a microservice architecture....623

Microservices: advantages and disadvantages....624

Integration patterns in a microservice architecture....625

The API proxy....626

API orchestration....627

Integration with a message broker....630

Summary....632

Exercises....632

Chapter 13: Messaging and Integration Patterns....634

Fundamentals of a messaging system....635

One way versus request/reply patterns....635

Message types....636

Command messages....636

Event messages....637

Document messages....637

Push versus pull delivery semantics....638

Pull delivery (consumer-initiated)....638

Push delivery (producer-initiated)....638

Choosing between pull and push delivery....638

Asynchronous messaging, queues, and streams....639

Peer-to-peer or broker-based messaging....641

Publish/Subscribe pattern....642

Building a minimalist real-time chat application....643

Implementing the server side....643

Implementing the client side....644

Running and scaling the chat application....647

Using Redis as a simple message broker....648

Peer-to-peer Publish/Subscribe with ZeroMQ....652

Introducing ZeroMQ....652

Designing a peer-to-peer architecture for the chat server....652

Using the ZeroMQ PUB/SUB sockets....653

Reliable message delivery with queues....656

Introducing AMQP....658

Durable subscribers with AMQP and RabbitMQ....660

Reliable messaging with streams....666

Characteristics of a streaming platform....667

Streams versus message queues....668

Implementing the chat application using Redis streams....668

Task distribution patterns....672

The ZeroMQ fan-out/fan-in pattern....674

PUSH/PULL sockets....675

Building a distributed hashsum cracker with ZeroMQ....675

Pipelines and competing consumers in AMQP....683

Point-to-point communications and competing consumers....683

Implementing the hashsum cracker using AMQP....684

Distributing tasks with Redis streams....687

Redis consumer groups....687

Implementing the hashsum cracker using Redis streams....688

Request/Reply patterns....693

Correlation Identifier....693

Implementing a request/reply abstraction using correlation identifiers....694

Return address....699

Implementing the Return Address pattern in AMQP....699

Summary....705

Exercises....705

Other Books You May Enjoy....710

Index....714

An essential read for any JavaScript developer - take full advantage of the Node.js platform and build reliable, scalable web applications using design patterns

Purchase of the print or Kindle book includes a free eBook in PDF format

Key Features

• Gain a deep understanding of the Node.js philosophy, its core components, and the solutions in its ecosystem
• Avoid common pitfalls in applying proven patterns to create robust, maintainable Node.js applications
• Enhance your development skills through a wealth of real-world examples and case studies

Book Description

Node.js underpins much of modern web development, reliably powering APIs and full-stack apps across all industries. Authors Luciano Mammino and Mario Casciaro offer a practical guide that unpacks the JavaScript runtime so you can write reliable, high-performance Node.js apps.

Building on the highly rated third edition, this new edition adds fresh case studies and the latest Node.js developments: newer APIs and libraries, ESM improvements, practical security and production tips, and guidance on using Node.js with TypeScript. It also introduces a new chapter on testing that gives you a full introduction to testing philosophy and practical guidance on writing unit, integration, and end-to-end tests, giving you the confidence to write functional, stable, and reliable code.

Real-world, end-to-end examples throughout the book show how to build microservices and distributed systems with Node.js, integrating production-proven technologies such as Redis, RabbitMQ, LevelDB, and ZeroMQ, the same components you’ll find in scalable deployments at companies of all sizes. End-of-chapter exercises consolidate your understanding.

By the end of this Node.js book, you’ll have the design patterns, mindset, and hands-on skills every serious Node.js professional needs to confidently architect robust, efficient, and maintainable applications.

What you will learn

• Understand Node.js basics and its async event-driven architecture
• Write correct async code using callbacks, promises, and async/await
• Harness Node.js streams to create data-driven processing pipelines
• Implement trusted software design patterns for production-grade applications
• Write testable code and automated tests (unit, integration, E2E)
• Use advanced recipes: caching, batching, async init, offload CPU-bound work
• Build and scale microservices and distributed systems powered by Node.js

Who this book is for

This book is for you if you’re a developer or software architect with basic knowledge of JavaScript and Node.js and want to get the most out of these technologies to maximize productivity, design quality, and scalability. It’ll help you level up from junior to senior roles.

This book is a tried-and-tested reference guide for readers at all levels. Even those with more experience will find value in the more advanced patterns and techniques presented.

You’re expected to have an intermediate understanding of web application development, databases, and software design principles.