Skip to content

Compiler

November 22, 2023
April 14, 2023

How source code is turned into machine code

Compiler - Wikiwand

  1. Tokenize source into tokens
  2. Parse tokens into AST
  3. Generate Intermediate Representation (IR) from AST nodes
  4. Optimization on IR
  5. Render IR into object files (machine code)
  6. Linking of object files

KEYNOTE: What Everyone Should Know About How Amazing Compilers Are - Matt Godbolt [C++ on Sea 2019] - YouTube
How A Compiler Works: GNU Toolchain
I wrote a programming language. Here’s how you can, too.
Oops, I Wrote a C++ Compiler C/C++ compiler/interpreter in .NET

Create Your Own Programming Language Series' Articles - DEV Community

When Zig Outshines Rust - Memory Efficient Enum Arrays Zig comptime for staged compilation
Modernizing Compiler Design for Carbon Toolchain - Chandler Carruth - CppNow 2023 - YouTube parsed AST consumes 50x more memory than the source code

ThePrimeagen/ts-rust-zig-deez interpreter in different languages

Just In Time Compiler

Just-in-time compilation - Wikiwand

Just In Time (JIT) Compilers - Computerphile - YouTube

Cross Compiler

Cross compiler - Wikiwand
Introduction to cross-compiling for Linux

Many modern compiled languages (Go, Rust, Zig) have cross compilation built-in.

Target Triplet

compiler construction - Does a list of all known target triplets in use exist? - Stack Overflow

Target Triplet - OSDev Wiki
machine-vendor-operatingsystem:

Cross-compilation using Clang — Clang documentation
The triple has the general format <arch><sub>-<vendor>-<sys>-<env>, where:

$ zig version
0.10.1

$ zig targets | jq -c ". | keys"
["abi","arch","cpuFeatures","cpus","glibc","libc","native","os"]

$ zig targets | jq  -c ".arch"
["arm","armeb","aarch64","aarch64_be","aarch64_32","arc","avr","bpfel","bpfeb","csky","dxil","hexagon","loongarch32","loongarch64","m68k","mips","mipsel","mips64","mips64el","msp430","powerpc","powerpcle","powerpc64","powerpc64le","r600","amdgcn","riscv32","riscv64","sparc","sparc64","sparcel","s390x","tce","tcele","thumb","thumbeb","i386","x86_64","xcore","nvptx","nvptx64","le32","le64","amdil","amdil64","hsail","hsail64","spir","spir64","spirv32","spirv64","kalimba","shave","lanai","wasm32","wasm64","renderscript32","renderscript64","ve","spu_2"]

$ zig targets | jq  -c ".libc"
["aarch64_be-linux-gnu","aarch64_be-linux-musl","aarch64_be-windows-gnu","aarch64-linux-gnu","aarch64-linux-musl","aarch64-windows-gnu","aarch64-macos-none","aarch64-macos-none","aarch64-macos-none","armeb-linux-gnueabi","armeb-linux-gnueabihf","armeb-linux-musleabi","armeb-linux-musleabihf","armeb-windows-gnu","arm-linux-gnueabi","arm-linux-gnueabihf","arm-linux-musleabi","arm-linux-musleabihf","thumb-linux-gnueabi","thumb-linux-gnueabihf","thumb-linux-musleabi","thumb-linux-musleabihf","arm-windows-gnu","csky-linux-gnueabi","csky-linux-gnueabihf","i386-linux-gnu","i386-linux-musl","i386-windows-gnu","m68k-linux-gnu","m68k-linux-musl","mips64el-linux-gnuabi64","mips64el-linux-gnuabin32","mips64el-linux-musl","mips64-linux-gnuabi64","mips64-linux-gnuabin32","mips64-linux-musl","mipsel-linux-gnueabi","mipsel-linux-gnueabihf","mipsel-linux-musl","mips-linux-gnueabi","mips-linux-gnueabihf","mips-linux-musl","powerpc64le-linux-gnu","powerpc64le-linux-musl","powerpc64-linux-gnu","powerpc64-linux-musl","powerpc-linux-gnueabi","powerpc-linux-gnueabihf","powerpc-linux-musl","riscv64-linux-gnu","riscv64-linux-musl","s390x-linux-gnu","s390x-linux-musl","sparc-linux-gnu","sparc64-linux-gnu","wasm32-freestanding-musl","wasm32-wasi-musl","x86_64-linux-gnu","x86_64-linux-gnux32","x86_64-linux-musl","x86_64-windows-gnu","x86_64-macos-none","x86_64-macos-none","x86_64-macos-none"]

# mapping of features for specific CPU, not listed here
$ zig targets | jq  -c ".cpus"
$ zig targets | jq ".cpus | map_values(keys)"

Cross compile for Windows

MXE (M cross environment)
MXE - build-pkg by starius

Install The Mingw Cross-Compiler - WxWiki
The Beez' speaks..: Cross compilers, the new wave

Bootstrapping

Bootstrapping (compilers) - Wikiwand
Bootstrapping in Compiler Design - GeeksforGeeks

From a compiler that I trust, to a compiler (or toolchain) that can compile source code that I have.

Bootstrapping EDSAC: Initial Orders - Computerphile - YouTube
What is Bootstrapping? - Computerphile - YouTube
Compilers with Professor Brailsford - YouTube

trust_stack_slides.pdf
Countering "Trusting Trust" - Schneier on Security
Reflections on Trusting Trust
Backdoor (computing) - Wikiwand

C

ccache — Compiler cache
2 tips to make your C++ projects compile 3 times faster - Red Hat Developer Blog

mozilla/sccache: sccache is ccache with cloud storage

distcc: a fast, free distributed C/C++ compiler

How the GNU C Library handles backward compatibility - Red Hat Developer Blog

C 语言全局变量那些事儿 | 酷 壳 - CoolShell

GCC optimization - Gentoo Wiki
i386 and x86-64 Options - Using the GNU Compiler Collection (GCC)
CFLAG="-O3 -march=native"

gcc --help=target
# -c: no linkage
# -Q: shows options
gcc -c -Q -march=native --help=target
# show compile flags
gcc -### -march=native /usr/include/stdlib.h

LLVM

LLVM defines an intermediate representation (IR) and a backend that targets and optimizes for any instruction set architecture (ISA)

LLVM - Wikiwand
The LLVM Compiler Infrastructure Project
LLVM | Everything I Know

The Architecture of Open Source Applications (Volume 1) LLVM
A Brief Introduction to LLVM - YouTube
LLVM, in Greater Detail PDF
SE-Radio Episode 291: Morgan Wilde on LLVM : Software Engineering Radio

LLVM Tutorial
My First Language Frontend with LLVM Tutorial — LLVM documentation

LLVM Language Reference Manual — LLVM documentation
llir/grammar: EBNF grammar of LLVM IR assembly.

Tools

see reverse-engineering#Lifting Bits

LLVM IR and Go | Gopher Academy Blog ❗!important
Welcome to llir/llvm · llir/llvm better than go-llvm as each node has corresponding Go struct, rather than a single catchall struct
llir/llvm: Library for interacting with LLVM IR in pure Go.

tinygo-org/go-llvm: Go bindings to a system-installed LLVM. Used as part of TinyGo. CGO, originally from LLVM source tree

rsms/llvmbox: Self contained, fully static llvm tools & libs

llvm-as: .ll to .bc
llvm-dis: .bc to .ll

Clang

Clang is the C frontend for LLVM

Clang C Language Family Frontend for LLVM
Clang - Wikiwand

c++ - LLVM vs clang on OS X - Stack Overflow
GCC 11 vs. LLVM Clang 12 Compilers On The AMD EPYC 7763 - Phoronix

Dynamic Dispatch

Two Ways To Do Dynamic Dispatch - YouTube Rust and C++
Better Code: Runtime Polymorphism - Sean Parent - YouTube C++