Microservices with Go: Building scalable and reliable microservices with Go

 Cover....1

Title Page....2

Copyright and Credits....3

Contributors....5

Table of Contents....8

Preface....14

Part 1: Introduction....22

Chapter 1: Introduction to Microservices....24

What is a microservice?....24

Motivation to use microservices....25

Pros and cons of microservices....27

Benefits of microservices....27

Common issues of microservices....29

When to use microservice architecture....30

Role of Go in microservice development....31

Summary....32

Further reading....32

Part 2: Foundation....34

Chapter 2: Scaffolding a Go Microservice....36

Technical requirements....36

Go basics....37

Core principles....37

Writing idiomatic Go code....38

Interfaces....40

Tests....41

Context....41

Project structure....42

Private packages....42

Public packages....43

Executable packages....43

Other commonly used directories....43

Common files....43

Best practices....44

Scaffolding an example application....44

Movie application....45

Application code structure....48

Movie metadata service....50

Movie service....65

Summary....75

Further reading....76

Chapter 3: Service Discovery....78

Technical requirements....78

Service discovery overview....78

Registry....80

Service discovery models....81

Service health monitoring....82

Service discovery solutions....82

HashiCorp Consul....83

Kubernetes....83

Adopting service discovery....83

Preparing the application....84

Implementing the discovery logic....87

Using the discovery logic....93

Summary....99

Further reading....99

Chapter 4: Serialization....100

Technical requirements....100

The basics of serialization....101

Popular serialization formats....103

Using Protocol Buffers....107

Best practices for serialization....113

Summary....114

Further reading....114

Chapter 5: Synchronous Communication....116

Technical requirements....116

Introduction to synchronous communication....117

Go RPC frameworks and libraries....118

Defining a service API using Protocol Buffers....119

Implementing gateways and clients....123

Metadata service....123

Rating service....128

Movie service....130

Summary....136

Further reading....136

Chapter 6: Asynchronous Communication....138

Technical requirements....138

Asynchronous communication basics....138

Benefits and challenges of asynchronous communication....139

Techniques and patterns of asynchronous communication....141

Using Apache Kafka for messaging....143

Adopting Kafka for our microservices....145

Asynchronous communication best practices....153

Versioning....153

Leveraging partitioning....154

Summary....155

Further reading....155

Chapter 7: Storing Service Data....156

Technical requirements....156

Introduction to database....157

Using MySQL to store our service data....160

Summary....168

Further reading....169

Chapter 8: Deployment with Kubernetes....170

Technical requirements....170

Preparing application code for deployments....171

Deployment basics....171

Application configuration....172

Deploying via Kubernetes....174

Introduction to Kubernetes....175

Setting up our microservices for Kubernetes deployments....176

Deployment best practices....181

Automated rollbacks....182

Canary deployments....182

Replace with Continuous Deployment (CD)....183

Summary....183

Further reading....184

Chapter 9: Unit and Integration Testing....186

Technical requirements....186

Go testing overview....187

Unit tests....191

Mocking....193

Implementing unit tests....196

Integration tests....198

Testing best practices....210

Using helpful messages....210

Avoiding the use of Fatal in your logs....211

Making a comparison using a cmp library....212

Summary....213

Further reading....213

Part 3: Maintenance....214

Chapter 10: Reliability Overview....216

Technical requirements....216

Reliability basics....217

Achieving reliability through automation....218

Communication error handling....218

Graceful shutdown....228

Achieving reliability through development processes and culture....231

On-call process....231

Incident management....233

Reliability drills....235

Summary....236

Further reading....236

Chapter 11: Collecting Service Telemetry Data....238

Technical requirements....238

Telemetry overview....239

Collecting service logs....240

Choosing the logging library....243

Using logging features....244

Storing microservice logs....247

Logging best practices....248

Collecting service metrics....252

Storing metrics....254

Popular Go metrics libraries....256

Emitting service metrics....256

Metrics best practices....259

Tracing....260

Tracing tools....262

Collecting tracing data with the OpenTelemetry SDK....263

Summary....272

Further reading....273

Chapter 12: Setting Up Service Alerting....274

Technical requirements....274

Alerting basics....275

Alerting use cases....276

Introduction to Prometheus....277

Setting up Prometheus alerting for our microservices....280

Alerting best practices....288

Summary....289

Further reading....289

Chapter 13: Advanced Topics....290

Technical requirements....290

Profiling Go services....291

Creating microservice dashboards....295

Frameworks....301

Storing microservice ownership data....303

Securing microservice communication with JWT....305

JWT basics....306

Implementing authentication and authorization with JWTs....307

Summary....312

Further reading....312

Index....314

Other Books You May Enjoy....324

This book covers the key benefits and common issues of microservices, helping you understand the problems microservice architecture helps to solve, the issues it usually introduces, and the ways to tackle them.

You'll start by learning about the importance of using the right principles and standards in order to achieve the key benefits of microservice architecture. The following chapters will explain why the Go programming language is one of the most popular languages for microservice development and lay down the foundations for the next chapters of the book. You'll explore the foundational aspects of Go microservice development including service scaffolding, service discovery, data serialization, synchronous and asynchronous communication, deployment, and testing. After covering the development aspects, you'll progress to maintenance and reliability topics. The last part focuses on more advanced topics of Go microservice development including system reliability, observability, maintainability, and scalability. In this part, you'll dive into the best practices and examples which illustrate how to apply the key ideas to existing applications, using the services scaffolded in the previous part as examples.

By the end of this book, you'll have gained hands-on experience with everything you need to develop scalable, reliable and performant microservices using Go.

What you will learn

• Get familiar with the industry's best practices and solutions in microservice development
• Understand service discovery in the microservices environment
• Explore reliability and observability principles
• Discover best practices for asynchronous communication
• Focus on how to write high-quality unit and integration tests in Go applications
• Understand how to profile Go microservices

Who this book is for

This book is for all types of developers, from people interested in learning how to write microservices in Go to seasoned professionals who want to take the next step in mastering the art of writing scalable and reliable microservice-based systems. A basic understanding of Go will come in handy.