Explore Go: Cryptography

Praise for Explore Go: Cryptography....14

Introduction....15

Cryptography....15

Why learn about cryptography?....16

What do you need to know?....17

About the book....18

What you’ll learn....18

How to use this book....19

1. Ciphers....20

Codes and ciphers....20

A shift cipher....22

Key points....22

Wheels within wheels....23

Enciphering....23

One byte at a time....23

Assume a spherical cow....24

From behaviour to test....25

Choose your own adventure....26

A first test....27

Anatomy of a test....28

When should the test fail?....28

The world’s greatest bug detector....29

A function about nothing....30

Cutting code....31

The smarty-pants answer....33

We’re gonna need a bigger test....33

A table of cases....34

Combining cases....34

A suspiciously similar test....35

A test case struct....36

Looping over the cases....36

Lighting a fire under the test....37

Introducing subtests....38

Subtests and t.Run....38

A litany of failure....40

Adding more cases....40

A working table test for Encipher....42

2. Enciphering....45

A simple enciphering filter....45

A command package....46

Garbage in, garbage out....46

All about Eve....48

Reverse engineering....48

Variable keys....49

Adding a keyhole....49

New keys, new cases....50

Compiler complaints....54

Stochastic debugging....54

Getting back to green....56

Getting the key....57

Defining a key flag....58

Getting the flag value....59

Enciphering with arbitrary keys....59

3. Deciphering....61

Brute force and ignorance....61

A statistical vulnerability....62

How many possible keys are there?....62

How safe is your safe?....63

Doing it for the crack....64

Cribs....64

Designing a cracking function....65

First, we need Decipher....66

Do we even need a test, then?....68

New test, same old table....68

Look before you loop....70

A deciphering tool....71

Let’s not complicate encipher....72

4. Cracking....74

Testing a Crack function....74

Failure is an option....75

A key-guessing function....76

A command-line cracker....77

What do users want?....78

Crib constraints....79

Cracking a real ciphertext....80

Waiting for the tiger....80

The devil in the details....81

0x89 is the magic number....82

Printing the unprintable....83

5. Keys....85

Lost in keyspace....85

The space of all possible keys....86

How long should the key be?....86

A keyspace the size of the universe....87

Multi-byte keys....88

Embiggening the key....89

Updating the test cases....89

Bytes in, bytes out....90

Fixing Encipher....91

A magical solution....91

Constraining the key index....92

Modulate your enthusiasm....92

The modulus operator....93

Refactoring will continue until morale improves....94

Testing longer keys....95

Designing the test cases....95

Sharpening the tools....97

When the right answer is wrong....98

What kind of flag do we need?....99

The joy of hex(adecimal)....100

Binary notation....100

A string flag....101

Where’s the BEEF?....102

Changing frequencies....103

6. Cribs....104

Cracking long keys....104

What needs to change?....105

A byte at the problem....105

Length limitations....106

Checking our guesses....107

Knowing when to stop....108

A proper little cracker....108

Updating the crack tool....111

Applying brute force....111

Crib considerations....112

A false deciphering....112

On being the right size....113

7. Passwords....114

Security....114

Password spaces....115

Dictionary words....115

Extremely guessable passwords....116

Passphrases....117

Rules....118

Character sets....118

Password policies....119

You’re really not helping....119

Maximising the keyspace....120

Unicode....121

Generation....122

What does “random” mean?....122

Predictability....123

Binary choices....124

Information....124

Knowledge....125

8. Blocks....127

Block ciphers....127

Why blocks?....128

The cipher.Block interface....129

Plugs and sockets....130

Adapting the shift cipher....130

The Encrypt method....131

Where’s the key?....131

Fixing the key size....132

A cipher constructor....133

The NewCipher function....133

If the key fits....134

A polite refusal....135

A special error value....135

Wrapping errors....136

Writing NewCipher....137

Implementing the cipher.Block interface....138

Testing Encrypt....138

Circumstances alter cases....139

Creating the cipher object....140

A test for Encrypt....141

Writing Encrypt and Decrypt....142

Forgotten something?....144

9. Modes....146

Handling data in blocks....146

Updating the encipher tool....147

Block by block....148

The cipher.BlockMode interface....148

A simple block mode....149

Testing CryptBlocks....150

Creating the encrypter object....152

Implementing CryptBlocks....152

Using a BlockMode in encipher....155

Block alignment....156

When misaligned data attacks....157

Checking our assumptions....157

The long and short of it....158

A test about nothing....159

A panic-proof CryptBlocks....160

10. Padding....162

A padding scheme....162

Making ends meet....163

The “illegal pixel” problem....163

An unambigous padding scheme....163

Testing Pad....164

Implementing Pad....167

Writing Unpad....168

Padding input to encipher....170

Adding the padding....170

Checking the results....171

Deciphering....172

Adding the unpadding....173

A decrypter that implements BlockMode....174

Waiting for the tiger....176

11. Enumeration....178

Adventures in keyspace....178

A new test for cracking long keys....178

A block is a black box....180

Enumerating long keys....181

Designing a Next function....182

Testing Next....184

Big endians and little endians....184

A simple implementation....185

Checking our key guesses....186

The soul of a new cracker....187

We apologise for the delay....188

Benchmarking key cracking....188

Pick an easier problem....189

Understanding benchmark output....189

Cranking up the difficulty....190

Ballparking a realistic crack time....191

A test we might live to see pass....192

Updating the crack tool....193

Putting it all together....193

Will it crack?....195

Parallelisation....195

Big computers are too slow....196

Dense computers become black holes....196

Parallel cracking is still useful....197

But we won’t implement it here....197

Strong keys can’t be cracked....198

12. Entropy....199

Information....199

Entropy: a measure of our ignorance....200

Entropy of digital messages....201

Compression....202

Enciphered data....203

Identifying ciphertexts....203

Complexity....204

Describing sequences....204

A generating algorithm....204

The shortest program that describes a sequence....206

Kolmogorov complexity....206

Complexity from simplicity....207

Security....208

Randomness is high entropy....209

Key generation....209

Pragmatic non-determinism....210

A little entropy goes a long way....211

Being unpredictable....211

13. Randomness....213

Pseudo-random generation....213

Randomness in games....214

The Fibonacci sequence....214

A simple random generator....215

A Fibonacci generator in Go....215

Repeatability....216

Seeding the RNG....217

Security problems....217

Periodicity....218

Distribution....218

Uniformity....219

“Secure enough” random generators....219

Environmental noise....220

The system entropy pool....220

Security is relative....221

You don’t need to outrun the bear....221

Keeping secrets from the gods....222

Hardware entropy sources....222

Defeating “God Emperor Eve”....223

Quantum measurements....223

A quantum randomness source....224

Generating quantum keys....225

14. Chains....226

Visualising the problem....226

A completely random image....227

Separating metadata and pixel data....229

Hidden figures....230

Muddying the waters....231

What’s wrong with this picture?....233

Mallory in the middle....234

ECB: a block-headed operating mode....234

Introducing Mallory....235

Block replay....235

Dropping and modifying blocks....236

More sophisticated modes....236

Counter mode (CTR)....237

Nonces....237

Cipher Block Chain (CBC)....237

Initialization Vector (IV)....238

Generating a secure IV....239

Implementing CBC mode....239

Enciphering in CBC mode....240

The updated encipher program....242

Updating the decipher program....244

The devil in disguise....246

15. Hashing....248

Message integrity....248

The night is dark, and full of errors....248

Bit-flipping and block-dropping....249

Integrity checking....250

Digests and hashing....250

Hash tables....251

Buckets and distribution....252

Collisions....252

Cryptographic hashing....253

Simple hash functions....254

A naïve algorithm....254

Implementing LenHash....255

Problems with LenHash....256

Preimage attacks....257

A slight improvement....258

Implementing SumHash....258

Testing SumHash....259

A preimage attack on SumHash....260

The avalanche effect....260

Real hash algorithms....261

MD5....261

SHA-1....262

SHA-256....262

Password hashing....263

Zero-knowledge secret storage....263

Password hashing requirements....264

Rainbow tables....264

Salting....265

Slow down, you move too fast....266

16. Coins....268

Cryptocurrency....268

Let there be “Bobcoin”....269

The problem of trust....269

A distributed digital ledger....270

Security....271

The double-spending problem....271

Ordering transactions....271

The blockchain....272

Doomed stubs....273

Integrity....274

Consensus....274

Proof of work....275

Up in smoke....276

Proof of stake....276

17. Authentication....278

Message integrity....278

Hash preimage attacks....279

Chosen ciphertext attacks....279

MAC....280

Length extension attacks....281

HMAC....281

Key exchange....282

The problem....283

Key splitting....283

Asymmetric encryption....284

Public and private keys....285

The Diffie-Hellman-Merkle protocol....285

A one-way function for key exchange....286

A DHM worked example....287

The reveal....288

Public-key cryptography....289

RSA....290

Signing....291

Authentication....292

Verification....292

The chain of trust....293

18. Cryptography....295

A little history....296

The origins of DES....296

Tripling down on DES....297

AES....297

Rijndael....297

The starting grid....298

Round and round....299

Confusion and diffusion....300

Putting it all together....301

Implementing AES....302

Getting ready to rumble....302

Ding ding, round one....303

The diffusion loop....304

The last round....305

AES encryption in practice....306

Enciphering with AES-CBC....306

Updating encipher and decipher....307

Encryption plus authentication....311

Enciphering with AES-GCM....312

The final final version....313

Weaknesses....317

Implementation fails....317

Mashing the keys....318

Tomorrow....319

The future is quantum....319

Quantum parallelism....320

The catch....321

Why your computer isn’t quantum… yet....322

Post-quantum cryptography....322

Afterword....323

About this book....325

Who wrote this?....325

Feedback....325

Get my free newsletter....325

Free updates to future editions....326

The Deeper Love of Go....326

The Power of Go: Tools....327

Further reading....327

Credits....328

Acknowledgements....329

Can you keep a secret? I hope so, because much of the modern world is built on cryptography: the art of secret messages. This book will show you what it’s all about and how it really works, with dozens of example programs in Go. Have you ever wondered how passwords are stored securely? What makes a good password? How codes and ciphers are designed–and broken? Where random numbers come from, and what makes them random? What are the connections between lava lamps, space games, digital signatures, black holes, and Bitcoin? Let’s find out. Join Alice, Bob, Eve, and Mallory as we learn about the fundamental principles of cryptography and digital security, from brute force and blockchains to keyspaces and hashing. We’ll build a cipher system in Go from scratch, with step-by-step instructions and code examples at each stage (also available on GitHub). Starting with the simplest cipher imaginable, we’ll gradually improve the system by attacking it, adding sophisticated features like block chaining, padding, digests, and authentication. Along the way, you’ll develop a powerful intuitive understanding of ciphers and keys, what makes them strong (or weak), and how to use them securely. We’ll see how state-of-the-art modern algorithms like AES, SHA-256, Diffie-Hellman, and RSA work under the hood, and how to integrate them into real-world Go tools. This book is essential reading for all Go programmers who have to deal with encryption, authentication, and security… in other words, all of us!