Terraform Cookbook: Provision, run, and scale cloud architecture with real-world examples using Terraform. 2 Ed

Cover....1

Copyright....3

Contributors....7

Table of Contents....10

Preface....36

Chapter 1: Setting Up the Terraform Environment....44

Technical requirements....45

Overviewing Terraform best practices....45

Downloading and installing Terraform on Windows manually....47

Getting ready....47

How to do it…....47

How it works…....52

Installing Terraform using Chocolatey on Windows....54

Getting ready....54

How to do it…....55

How it works…....55

There’s more…....56

See also....56

Installing Terraform on Linux using the APT package manager....56

Getting ready....56

How to do it…....56

How it works…....57

See also....57

Installing Terraform using a script on Linux....57

Getting ready....57

How to do it…....58

How it works…....58

There’s more…....59

See also....60

Executing Terraform in a Docker container....60

Getting ready....60

How to do it…....61

How it works…....63

There’s more…....63

See also....63

Switching between multiple Terraform versions....64

Getting ready....65

How to do it…....65

How it works…....68

There’s more…....68

See also....70

Upgrading Terraform providers....70

Getting ready....70

How to do it…....71

How it works…....74

There’s more…....74

See also....75

Chapter 2: Writing Terraform Configurations....76

Technical requirements....77

Configuring Terraform and the provider version to use....77

Getting ready....78

How to do it…....80

How it works…....80

There’s more…....82

See also....83

Adding alias to a provider to use multiple instances of the same provider....83

Getting ready....83

How to do it…....84

How it works…....85

See also....85

Manipulating variables....86

Getting ready....86

How to do it…....86

How it works…....87

There’s more…....88

See also....89

Keeping sensitive variables safe....90

Getting ready....90

How to do it…....90

How it works…....91

There’s more…....91

See also....92

Using local variables for custom functions....92

Getting ready....92

How to do it…....93

How it works…....94

There’s more…....94

See also....94

Using outputs to expose Terraform provisioned data....95

Getting ready....95

How to do it…....96

How it works…....96

There’s more…....96

See also....97

Calling Terraform’s built-in functions....97

Getting ready....97

How to do it…....98

How it works…....98

See also....99

Using YAML files in Terraform configuration....100

Getting ready....100

How to do it…....100

How it works…....102

There’s more…....102

See also....102

Writing conditional expressions....102

Getting ready....103

How to do it…....103

How it works…....103

There’s more…....104

See also....104

Generating passwords with Terraform....104

Getting ready....105

How to do it…....105

How it works…....106

See also....106

Managing Terraform resource dependencies....107

Getting ready....107

How to do it…....108

How it works…....108

There’s more…....109

See also....109

Adding custom pre and postconditions....109

Getting ready....110

How to do it…....110

How it works…....111

See also....113

Using checks for infrastructure validation....113

Getting ready....113

How to do it…....114

How it works…....114

There’s more…....115

See also....115

Chapter 3: Scaling Your Infrastructure with Terraform....118

Technical requirements....119

Provisioning infrastructure in multiple environments....119

Getting ready....119

How to do it…....120

How it works…....121

See also....124

Provisioning multiple resources with the count meta-argument....124

Getting ready....124

How to do it…....125

How it works…....125

There’s more…....127

See also....128

Using maps....128

Getting ready....129

How to do it…....129

How it works…....130

There’s more…....131

See also....132

Looping over a map of objects....132

Getting ready....133

How to do it…....133

How it works…....134

There’s more…....135

See also....136

Generating multiple blocks with the dynamic block....136

Getting ready....137

How to do it…....137

How it works…....139

There’s more…....140

See also....140

Filtering maps....140

Getting ready....141

How to do it…....141

How it works…....142

There’s more…....143

See also....143

Chapter 4: Using Terraform with External Data....146

Technical requirements....147

Obtaining external data with data sources....147

Getting ready....148

How to do it…....148

How it works…....148

There’s more…....149

See also....150

Querying external data with Terraform....150

Getting ready....151

How to do it…....151

How it works…....152

There’s more…....154

See also....154

Manipulating local files with Terraform....154

Getting ready....155

How to do it…....155

How it works…....156

There’s more…....156

See also....157

Executing local programs with Terraform....157

Getting ready....157

How to do it…....157

How it works…....158

There’s more…....159

See also....160

Executing shell scripts using the Terraform Shell provider....160

Getting ready....160

How to do it…....160

How it works…....162

There’s more…....164

See also....164

Chapter 5: Managing Terraform State....166

Using the local Terraform state....167

Getting ready....167

How to do it…....167

How it works…....168

There’s more…....168

See also....170

Managing resources in Terraform state....170

Getting ready....170

How to do it…....171

Displaying the content of state....171

Listing Terraform resource names within state....173

Showing detailed resource properties in state....173

Deleting resources from state....174

There’s more…....175

See also....175

Synchronizing Terraform state....175

Getting ready....176

How to do it…....177

How it works…....178

There’s more…....180

See also....180

Importing existing resources....180

Getting ready....181

How to do it…....182

How it works…....183

There’s more…....183

See also....186

Using external resources from other Terraform state files....186

Getting ready....187

How to do it…....188

How it works…....188

There’s more…....189

See also....189

Refactoring resources in configuration....189

Getting ready....190

How to do it…....192

How it works…....193

There’s more…....196

See also....197

Chapter 6: Applying a Basic Terraform Workflow....200

Technical requirements....201

Keeping your Terraform configuration clean....201

Getting ready....202

How to do it…....202

How it works…....202

There’s more…....203

See also....205

Validating the code syntax....205

Getting ready....205

How to do it…....206

How it works…....207

There’s more…....207

See also....209

Destroying infrastructure resources....209

Getting ready....209

How to do it…....210

How it works…....210

There’s more…....210

See also....211

Displaying a list of providers used in a configuration....212

Getting ready....212

How to do it…....213

How it works…....213

There’s more…....213

See also....214

Generating one Terraform lock file with Windows and Linux compatibility....214

Getting ready....215

How to do it…....216

How it works…....216

See also....217

Copying a Terraform module configuration....218

Getting ready....218

How to do it…....218

How it works…....218

There’s more…....219

See also....219

Using workspaces to manage environments....220

Getting ready....220

How to do it…....220

How it works…....221

There’s more…....223

See also....224

Exporting the output in JSON....224

Getting ready....224

How to do it…....225

How it works…....225

There’s more…....227

See also....228

Tainting resources....228

Getting ready....228

How to do it…....228

How it works…....229

There’s more…....230

See also....231

Generating the dependency graph....231

Getting ready....232

How to do it…....232

How it works…....232

See also....233

Using different Terraform configuration directories....233

Getting ready....234

How to do it…....235

How it works…....235

There’s more…....235

See also....236

Testing and evaluating a Terraform expression....236

Getting ready....236

How to do it…....237

How it works…....237

There’s more…....238

See also....239

Debugging the Terraform execution....239

Getting ready....239

How to do it…....239

How it works…....240

There’s more…....241

See also....241

Chapter 7: Sharing Terraform Configuration with Modules....242

Technical requirements....243

Creating a Terraform module and using it locally....243

Getting ready....244

How to do it…....244

How it works…....247

There’s more…....247

See also....249

Provisioning multiple instances of a Terraform module....249

Getting ready....249

How to do it…....250

How it works…....251

There’s more…....251

See also....252

Using modules from the public registry....252

Getting ready....252

How to do it…....253

How it works…....254

There’s more…....255

See also....256

Sharing a Terraform module in the public registry using GitHub....256

Getting ready....256

How to do it…....256

How it works…....259

There’s more…....260

See also....261

Using another file inside a custom module....261

Getting ready....261

How to do it…....262

How it works…....262

There’s more…....263

See also....264

Using the Terraform module generator....264

Getting ready....264

How to do it…....265

How it works…....265

There’s more…....267

See also....267

Generating module documentation....268

Getting ready....268

How to do it…....269

How it works…....269

There’s more…....271

See also....272

Using a private Git repository for sharing a Terraform module....272

Getting ready....273

How to do it…....274

How it works…....276

There’s more…....277

See also....278

Applying a Terrafile pattern for using modules....278

Getting ready....279

How to do it…....279

How it works…....281

There’s more…....281

See also....282

Chapter 8: Provisioning Azure Infrastructure with Terraform....284

Technical requirements....285

Using Terraform in Azure Cloud Shell....285

Getting ready....286

How to do it…....286

How it works…....287

There’s more…....288

See also....289

Protecting the Azure credential provider....289

Getting ready....290

How to do it…....290

How it works…....292

There’s more.......293

See also....294

Protecting the state file in the Azure remote backend....294

Getting ready....295

How to do it…....295

How it works…....296

There’s more…....297

See also....299

Executing ARM templates in Terraform....299

Getting ready....300

How to do it…....300

How it works…....302

There’s more…....303

See also....303

Executing Azure CLI commands in Terraform....304

Getting ready....304

How to do it…....305

How it works…....306

There’s more…....306

See also....307

Using Azure Key Vault with Terraform to protect secrets....308

Getting ready....308

How to do it…....309

How it works…....311

There’s more…....312

See also....314

Provisioning and configuring an Azure VM with Terraform....314

Getting ready....314

How to do it…....315

How it works…....318

There’s more…....319

See also....320

Building Azure serverless infrastructure with Terraform....320

Getting ready....320

How to do it…....321

How it works…....322

There’s more…....322

See also....323

Generating a Terraform configuration for existing Azure infrastructure....323

Getting ready....324

How to do it…....326

How it works…....327

There’s more…....329

See also....330

Enabling optional Azure features....330

Getting ready....330

How to do it…....331

How it works…....331

There’s more…....331

Estimating Azure cost of infrastructure using Infracost....331

Getting ready....332

How to do it…....334

How it works…....336

There’s more…....336

See also....337

Using the AzApi Terraform provider....337

Getting ready....338

How to do it…....338

How it works…....340

There’s more…....341

See also....342

Chapter 9: Provisioning Simple AWS and GCP Infrastructure Using Terraform....344

Technical requirements....345

Getting started using Terraform for AWS....345

Getting ready....345

How to do it…....347

How it works…....348

There’s more…....349

See also....350

Using the S3 backend in AWS....350

Getting ready....350

How to do it…....353

How it works…....353

There’s more…....354

See also....354

Getting started using Terraform for GCP....355

Getting ready....355

How to do it…....359

How it works…....360

There’s more…....362

See also....362

Using a GCS backend in GCP....362

Getting ready....362

How to do it…....365

How it works…....365

See also....366

Executing Terraform in GCP Cloud Shell....366

Getting ready....367

How to do it…....367

How it works…....370

There’s more…....370

See also....371

Chapter 10: Using Terraform for Docker and Kubernetes Deployment....372

Technical requirements....373

Creating a Docker container using Terraform....374

Getting ready....374

How to do it…....374

How it works…....375

There’s more…....377

See also....377

Deploying Kubernetes resources using Terraform....377

Getting ready....378

How to do it…....378

How it works…....381

There’s more…....382

See also....383

Deploying a Helm chart in Kubernetes using Terraform....383

Getting ready....383

How to do it…....384

How it works…....385

There’s more…....386

See also....387

Using a Kubernetes controller as a Terraform reconciliation loop....387

Getting ready....388

How to do it…....390

How it works…....391

There’s more…....393

See also....394

Chapter 11: Running Test and Compliance Security on Terraform Configuration....396

Technical requirements....398

Using PowerShell’s Pester framework to perform Terraform testing....398

Getting ready....399

How to do it…....400

How it works…....403

There’s more…....404

See also....405

Testing the Terraform configuration using Python....405

Getting ready....405

How to do it…....406

How it works…....408

There’s more…....409

See also....409

Using OPA to check the Terraform configuration....409

Getting ready....410

How to do it…....412

How it works…....414

There’s more…....415

See also....416

Using tfsec to analyze the compliance of Terraform configuration....416

Getting ready....417

How to do it…....418

How it works…....419

There’s more…....419

See also....420

Applying Terraform compliance using terraform-compliance....420

Getting ready....420

How to do it…....421

How it works…....423

There’s more…....425

See also....425

Testing Terraform module code with Terratest....425

Getting ready....425

How to do it…....426

How it works…....429

There’s more…....430

See also....431

Testing the Terraform configuration using Kitchen-Terraform....431

Getting ready....431

How to do it…....432

How it works…....435

There’s more…....437

See also....437

Using the new integrated Terraform module integration test....438

Getting ready....438

How to do it…....439

How it works…....441

There’s more…....442

See also....443

Chapter 12: Deep-Diving into Terraform....444

Technical requirements....445

Preventing resources from being destroyed....445

Getting ready....445

How to do it…....446

How it works…....447

There’s more…....447

See also....449

Ignoring manual changes....449

Getting ready....450

How to do it…....451

How it works…....452

There’s more…....452

See also....453

Using Terraform’s templating feature....453

Getting ready....453

How to do it…....455

How it works…....456

There’s more…....457

See also....457

Zero-downtime deployment with Terraform....458

Getting ready....458

How to do it…....459

How it works…....460

There’s more…....460

See also....461

Managing Terraform configuration dependencies using Terragrunt....461

Getting ready....461

How to do it…....463

How it works…....463

There’s more…....464

See also....464

Using Terragrunt as a wrapper for Terraform....465

Getting ready....465

How to do it…....466

How it works…....467

See also....467

Generating a self-signed SSL certificate using Terraform....468

Getting ready....468

How to do it…....468

How it works…....469

There’s more…....469

See also....471

Checking the configuration before committing code using Git hooks....472

Getting ready....472

How to do it…....473

How it works…....474

There’s more…....475

See also....475

Visualizing Terraform resource dependencies with Rover....476

Getting ready....476

How to do it…....476

How it works…....477

See also....478

Using the Terraform CDK for developers....478

Getting ready....479

How to do it…....480

How it works…....484

There’s more…....486

See also....487

Chapter 13: Automating Terraform Execution in a CI/CD Pipeline....488

Running Terraform in automation mode....489

Getting ready....489

How to do it…....490

How it works…....491

There’s more…....491

See also....491

Displaying a summary of the execution of terraform plan....492

Getting ready....492

How to do it…....492

There’s more…....494

See also....494

Building CI/CD pipelines to apply Terraform configurations in Azure Pipelines....494

Getting ready....495

How to do it…....497

How it works…....501

There’s more…....502

See also....503

Automating Terraform execution in GitHub Actions....503

Getting ready....504

How to do it…....504

How it works…....506

There’s more…....506

See also....508

Working with workspaces in CI/CD....508

Getting ready....508

How to do it…....509

How it works…....511

There’s more…....512

See also....512

Building CI/CD for Terraform modules in Azure Pipelines....512

Getting ready....513

How to do it…....513

How it works…....518

There’s more…....518

See also....518

Building a workflow for publishing Terraform modules using GitHub Actions....519

Getting ready....519

How to do it…....519

How it works…....521

There’s more…....524

See also....524

Chapter 14: Using Terraform Cloud to Improve Team Collaboration....526

Technical requirements....528

Authenticating Terraform to Terraform Cloud....529

Getting ready....529

How to do it…....530

There’s more…....532

See also....533

Managing workspaces in Terraform Cloud....533

Getting ready....534

How to do it…....534

How it works…....541

There’s more…....542

See also....543

Using the remote backend in Terraform Cloud....543

Getting ready....544

How to do it…....545

How it works…....548

There’s more…....549

See also....549

Migrating Terraform State to Terraform Cloud....549

Getting ready....550

How to do it…....551

How it works…....553

See also....553

Using Terraform Cloud as a private module registry....553

Getting ready....554

How to do it…....556

How it works…....558

There’s more…....558

See also....558

Executing Terraform configuration remotely in Terraform Cloud....559

Getting ready....559

How to do it…....560

How it works…....564

There’s more…....564

See also....569

Checking the compliance of Terraform configurations using OPA in Terraform Cloud....569

Getting ready....569

How to do it…....571

There’s more…....578

See also....578

Using integrated cost estimation for cloud resources....578

Getting ready....579

How to do it…....579

How it works…....582

There’s more…....583

See also....583

Integrating the Infracost run task during the Terraform Cloud run....583

Getting ready....584

How to do it…....586

How it works…....590

There’s more…....590

See also....590

Configuring Terraform Cloud with the Terraform TFE provider....590

Getting ready....591

How to do it…....592

There’s more…....594

See also....595

Chapter 15: Troubleshooting Terraform Errors....596

Fixing interpolation errors....597

Getting ready....597

How to do it…....598

How it works…....599

See also....599

Fixing cycle errors....599

Getting ready....599

How to do it…....600

How it works…....601

There’s more…....601

Fixing for_each errors....602

Getting ready....602

How to do it…....604

There’s more…....605

Fixing output errors....605

Getting ready....605

How to do it…....606

How it works…....607

There’s more…....607

Appendix A: Terraform Cheat Sheet....608

Basic commands....608

Format Terraform configuration....609

Terraform providers management....609

Terraform dependency file....609

Basic workflow commands....610

Backend configuration....610

Validate configuration....611

Get outputs....611

Import resources....611

Terraform workspaces....611

Terraform debug....612

State management....612

Display Terraform graph dependencies....613

Taint/untaint resources....613

Terraform Cloud/Enterprise....613

Appendix B:Terraform Resources....614

Terraform official resources....614

Documentation....614

Registry....614

Providers development....614

Terraform community resources....615

Terraform news feed....615

Terraform certifications and certification preparation....615

Terraform certification program pages....615

Terraform certification preparation....615

PacktPage....616

Other Books You May Enjoy....618

Index....2

Explore how to provision, manage, and scale your infrastructure using Infrastructure as Code (IaC) with Terraform

Purchase of the print or Kindle book includes a free PDF eBook

Key Features

• Get up and running with the latest version of Terraform (v1+) CLI
• Discover how to deploy Kubernetes resources with Terraform
• Learn how to troubleshoot common Terraform issues

Book Description

HashiCorp Configuration Language (HCL) has changed how we define and provision data center infrastructure with the launch of Terraform, a top-tier product for building Infrastructure as Code (IaC). Terraform Cookbook shows you how to leverage Terraform to manage complex infrastructure with ease.

This new edition has been updated to include real-world examples for provisioning Azure, AWS and GCP infrastructure with Terraform. You'll delve into manual and automated testing with Terraform configurations, creating and managing a balanced, efficient, and reusable infrastructure with Terraform modules. You'll learn how to automate the deployment of Terraform configuration with continuous integration and continuous delivery (CI/CD).

Besides that, several new chapters have been added that describe the use of Terraform for Docker and Kubernetes, examine advanced topics on GitOps practices, and explain how to test Terraform configurations using different tools to check code and security compliance. The final chapter covers troubleshooting common Terraform issues and provides solutions for frequently encountered errors.

By the end of this book, you'll have developed the skills needed to get the most value out of Terraform and to effectively manage your infrastructure.

What you will learn

• Use Terraform to build and run cloud and Kubernetes infrastructure using IaC best practices
• Adapt the Terraform command line adapted to appropriate use cases
• Automate the deployment of Terraform confi guration with CI/CD
• Discover manipulation of the Terraform state by adding or removing resources
• Explore Terraform for Docker and Kubernetes deployment, advanced topics on GitOps practices, and Cloud Development Kit (CDK)
• Add and apply test code and compliance security in Terraform configuration
• Debug and troubleshoot common Terraform errors

Who this book is for

This book is for developers, operators, and DevOps engineers looking to improve their workflow and use Infrastructure as Code. Experience with Microsoft Azure, Jenkins, shell scripting, and DevOps practices is required to get the most out of this Terraform book.