Learn LLVM 17: A beginner's guide to learning LLVM compiler tools and core libraries with C++. 2 Ed

Learn LLVM 17....2

Contributors....10

About the authors....10

About the reviewers....11

Preface....31

What’s new in this edition....32

Who this book is for....33

What this book covers....34

To get the most out of this book....37

Download the example code files....39

Conventions used....39

Get in touch....40

Share Your Thoughts....41

Download a free PDF copy of this book....41

Part 1: The Basics of Compiler Construction with LLVM....44

Chapter 1: Installing LLVM....45

Compiling LLVM versus installing binaries....45

Getting the prerequisites ready....45

Ubuntu....47

Fedora and RedHat....47

FreeBSD....47

OS X....47

Windows....48

Cloning the repository and building from source....49

Configuring Git....49

Cloning the repository....50

Creating a build directory....51

Generating the build system files....51

Compiling and installing LLVM....52

Customizing the build process....53

Variables defined by CMake....54

Using LLVM-defined build configuration variables....56

Summary....60

Chapter 2: The Structure of a Compiler....61

Building blocks of a compiler....62

An arithmetic expression language....64

Formalism for specifying the syntax of a programming language....64

How does grammar help the compiler writer?....66

Lexical analysis....67

A hand-written lexer....67

Syntactical analysis....75

A hand-written parser....75

The abstract syntax tree....85

Semantic analysis....90

Generating code with the LLVM backend....93

Textual representation of LLVM IR....94

Generating the IR from the AST....96

The missing pieces – the driver and the runtime library....103

Summary....109

Part 2: From Source to Machine Code Generation....111

Chapter 3: Turning the Source File into an Abstract Syntax Tree....112

Defining a real programming language....113

Creating the project layout....119

Managing the input files for the compiler....120

Handling messages for the user....121

Structuring the lexer....126

Constructing a recursive descent parser....135

Performing semantic analysis....146

Handling the scope of names....148

Using an LLVM-style RTTI for the AST....152

Creating the semantic analyzer....154

Summary....165

Chapter 4: Basics of IR Code Generation....167

Generating IR from the AST....168

Understanding the IR code....168

Learning about the load-and-store approach....177

Mapping the control flow to basic blocks....179

Using AST numbering to generate IR code in SSA form....184

Defining the data structure to hold values....185

Reading and writing values local to a basic block....186

Searching the predecessor blocks for a value....187

Optimizing the generated phi instructions....191

Sealing a block....194

Creating the IR code for expressions....194

Emitting the IR code for a function....197

Controlling visibility with linkage and name mangling....197

Converting a type from an AST description into LLVM types....200

Creating the LLVM IR function....201

Emitting the function body....205

Setting up the module and the driver....206

Wrapping all in the code generator....207

Initializing the target machine class....209

Emitting assembler text and object code....212

Summary....218

Chapter 5: IR Generation for High-Level Language Constructs....220

Technical requirements....221

Working with arrays, structs, and pointers....221

Getting the application binary interface right....229

Creating IR code for classes and virtual functions....234

Implementing single inheritance....234

Extending single inheritance with interfaces....242

Adding support for multiple inheritance....245

Summary....250

Chapter 6: Advanced IR Generation....252

Throwing and catching exceptions....253

Raising an exception....263

Catching an exception....268

Integrating the exception handling code into the application....273

Generating metadata for type-based alias analysis....276

Understanding the need for additional metadata....276

Creating TBAA metadata in LLVM....280

Adding TBAA metadata to tinylang....282

Adding debug metadata....290

Understanding the general structure of debug metadata....290

Tracking variables and their values....299

Adding line numbers....306

Adding debug support to tinylang....308

Summary....319

Chapter 7: Optimizing IR....321

Technical requirements....322

The LLVM pass manager....322

Implementing a new pass....325

Developing the ppprofiler pass as a plugin....326

Adding the pass to the LLVM source tree....334

Using the ppprofiler pass with LLVM tools....340

Adding an optimization pipeline to your compiler....354

Creating an optimization pipeline....354

Extending the pass pipeline....364

Summary....370

Part 3: Taking LLVM to the Next Level....371

Chapter 8: The TableGen Language....372

Technical requirements....373

Understanding the TableGen language....373

Experimenting with the TableGen language....375

Defining records and classes....376

Creating multiple records at once with multiclasses....381

Simulating function calls....385

Generating C++ code from a TableGen file....390

Defining data in the TableGen language....392

Implementing a TableGen backend....396

Drawbacks of TableGen....413

Summary....414

Chapter 9: JIT Compilation....415

Technical requirements....416

LLVM’s overall JIT implementation and use cases....416

Using JIT compilation for direct execution....419

Exploring the lli tool....420

Implementing our own JIT compiler with LLJIT....423

Integrating the LLJIT engine into the calculator....426

Code generation changes to support JIT compilation via LLJIT....433

Building an LLJIT-based calculator....439

Building a JIT compiler class from scratch....444

Creating a JIT compiler class....446

Using our new JIT compiler class....460

Summary....466

Chapter 10: Debugging Using LLVM Tools....467

Technical requirements....468

Instrumenting an application with sanitizers....469

Detecting memory access problems with the address sanitizer....469

Finding uninitialized memory accesses with the memory sanitizer....473

Pointing out data races with the thread sanitizer....475

Finding bugs with libFuzzer....479

Limitations and alternatives....485

Performance profiling with XRay....486

Checking the source with the clang static analyzer....495

Adding a new checker to the clang static analyzer....500

Creating your own clang-based tool....518

Summary....531

Part 4: Roll Your Own Backend....533

Chapter 11: The Target Description....534

Setting the stage for a new backend....535

Adding the new architecture to the Triple class....536

Extending the ELF file format definition in LLVM....538

Creating the target description....542

Adding the register definition....543

Defining the instruction formats and the instruction information....548

Creating the top-level file for the target description....555

Adding the M88k backend to LLVM....557

Implementing the assembler parser....564

Creating the disassembler....587

Summary....591

Chapter 12: Instruction Selection....593

Defining the rules of the calling convention....594

Implementing the rules of the calling convention....596

Instruction selection via the selection DAG....598

Implementing DAG lowering – handling legal types and setting operations....601

Implementing DAG lowering – lowering formal arguments....605

Implementing DAG lowering – lowering return values....609

Implementing DAG-to-DAG transformations within instruction selection....613

Adding register and instruction information....615

Putting an empty frame lowering in place....621

Emitting machine instructions....623

Creating the target machine and the sub-target....628

Implementing M88kSubtarget....629

Implementing M88kTargetMachine – defining the definitions....632

Implementing M88kTargetMachine – adding the implementation....634

Global instruction selection....641

Lowering arguments and return values....643

Legalizing the generic machine instructions....648

Selecting a register bank for operands....650

Translating generic machine instructions....654

Running an example....655

How to further evolve the backend....656

Summary....657

Chapter 13: Beyond Instruction Selection....659

Adding a new machine function pass to LLVM....659

Implementing the top-level interface for the M88k target....661

Adding the TargetMachine implementation for machine function passes....662

Developing the specifics of the machine function pass....664

Building newly implemented machine function passes....676

A glimpse of running a machine function pass with llc....677

Integrating a new target into the clang frontend....679

Implementing the driver integration within clang....679

Implementing ABI support for M88k within clang....691

Implementing the toolchain support for M88k within clang....695

Building the M88k target with clang integration....702

Targeting a different CPU architecture....704

Summary....711

Index....713

Why subscribe?....740

Other Books You May Enjoy....741

Packt is searching for authors like you....745

Share Your Thoughts....745

Download a free PDF copy of this book....746

LLVM was built to bridge the gap between the theoretical knowledge found in compiler textbooks and the practical demands of compiler development. With a modular codebase and advanced tools, LLVM empowers developers to build compilers with ease. This book serves as a practical introduction to LLVM, guiding you progressively through complex scenarios and ensuring that you navigate the challenges of building and working with compilers like a pro.

The book starts by showing you how to configure, build, and install LLVM libraries, tools, and external projects. You’ll then be introduced to LLVM's design, unraveling its applications in each compiler stage: frontend, optimizer, and backend. Using a real programming language subset, you'll build a frontend, generate LLVM IR, optimize it through the pipeline, and generate machine code. Advanced chapters extend your expertise, covering topics such as extending LLVM with a new pass, using LLVM tools for debugging, and enhancing the quality of your code. You'll also focus on just-in-time compilation issues and the current state of JIT-compilation support with LLVM. Finally, you’ll develop a new backend for LLVM, gaining insights into target description and how instruction selection works.

By the end of this book, you'll have hands-on experience with the LLVM compiler development framework through real-world examples and source code snippets.

What you will learn

• Configure, compile, and install the LLVM framework
• Understand how the LLVM source is organized
• Discover what you need to do to use LLVM in your own projects
• Explore how a compiler is structured, and implement a tiny compiler
• Generate LLVM IR for common source language constructs
• Set up an optimization pipeline and tailor it for your own needs
• Extend LLVM with transformation passes and clang tooling
• Add new machine instructions and a complete backend

Who this book is for

This book is for compiler developers, enthusiasts, and engineers new to LLVM. C++ software engineers looking to use compiler-based tools for code analysis and improvement, as well as casual users of LLVM libraries who want to gain more knowledge of LLVM essentials will also find this book useful. Intermediate-level experience with C++ programming is necessary to understand the concepts covered in this book.