Performance Begins in the Hardware

Performance on modern x86-64 Linux systems is not determined by individual instructions in isolation. It emerges from the coordinated behavior of multiple microarchitectural subsystems, including:

  • Speculative execution
  • Deep out-of-order pipelines
  • Multi-level cache hierarchies
  • Branch predictors
  • Memory-disambiguation logic
  • SIMD execution units
  • Load and store subsystems
  • High-resolution timing facilities

Together, these components define the practical performance limits of real software.

Writing fast programs therefore requires more than selecting instructions that appear efficient. It requires understanding how algorithms interact with the processor as a complete execution system.

An algorithm may achieve high throughput when its data remains in registers and caches, yet stall severely when memory access becomes irregular, branches become unpredictable, or instruction dependencies prevent parallel execution.

This booklet examines performance from the perspective of the hardware itself.

Objective of This Book

The primary goal is to move beyond surface-level optimization techniques and establish the principles that govern high-performance execution on Linux.

Rather than focusing only on compiler behavior or algorithmic complexity, the processor is treated as what it truly is:

A parallel, speculative machine capable of issuing multiple instructions per cycle and executing wide vector operations concurrently.

The book examines how this performance degrades when:

  • Memory access patterns lack locality
  • Branches are difficult to predict
  • Loops contain dependency chains
  • Insufficient independent work is available
  • Execution units remain underutilized
  • Data repeatedly moves between memory and registers
  • Loop overhead consumes unnecessary cycles

The objective is not merely to write fast assembly, but to understand why it is fast.

Core Performance Concepts

The booklet introduces several essential principles of modern performance engineering:

  • Latency versus throughput
  • Instruction-level parallelism
  • Data dependencies
  • Out-of-order execution
  • Branch prediction
  • Memory locality
  • Cache behavior
  • Loop unrolling
  • Register residency
  • SIMD vectorization
  • Execution-resource saturation

These concepts are demonstrated through complete, hand-written assembly routines rather than presented only as theory.

Latency and Throughput

Latency and throughput describe different aspects of instruction performance.

Latency is the number of cycles required before an instruction's result becomes available to a dependent instruction.

Throughput is the rate at which the processor can begin or complete multiple independent instances of an instruction.

An instruction may have relatively high latency while still supporting high throughput when independent operations are available.

For example, a chain of dependent additions exposes instruction latency:

TEXT
a = a + b
a = a + c
a = a + d

Each operation must wait for the preceding result.

Independent accumulators expose parallelism:

TEXT
a0 = a0 + b0
a1 = a1 + b1
a2 = a2 + b2
a3 = a3 + b3

The processor may execute several of these operations concurrently.

Understanding this distinction is essential for designing loops that make effective use of modern execution pipelines.

Instruction-Level Parallelism

Modern x86-64 processors can decode, issue, and execute multiple instructions during the same cycle.

This capability is effective only when instructions are sufficiently independent.

Instruction-level parallelism is reduced by:

  • Long dependency chains
  • Repeated use of a single accumulator
  • Serialization instructions
  • Register reuse
  • Load dependencies
  • Unpredictable branches
  • Insufficient loop unrolling

The examples demonstrate how to expose more independent work through:

  • Multiple accumulators
  • Loop unrolling
  • Register blocking
  • Independent memory loads
  • Parallel SIMD operations
  • Careful instruction scheduling

The goal is to keep execution units occupied rather than allowing the pipeline to wait for dependent results.

Out-of-Order and Speculative Execution

Modern processors do not necessarily execute instructions strictly in program order.

Instead, they may:

  • Decode instructions in order
  • Schedule independent operations early
  • Execute instructions out of order
  • Predict branch outcomes
  • Speculatively execute future instructions
  • Retire completed instructions in architectural order

This design allows the processor to hide latency and maintain throughput.

However, performance falls when:

  • Dependencies prevent reordering
  • Loads wait for unresolved addresses
  • Branch predictions fail
  • Cache misses delay required data
  • Execution resources become oversubscribed

The book explains how instruction structure influences the scheduler's ability to find useful parallel work.

Branch Prediction

Conditional branches can disrupt execution when their outcomes are difficult to predict.

A correctly predicted branch usually has low overhead because the processor continues executing along the predicted path.

A mispredicted branch may force the processor to:

  • Discard speculative work
  • Flush part of the pipeline
  • Restore the correct execution state
  • Fetch instructions from the correct path

This penalty can be substantial inside frequently executed loops.

The book examines when to use:

  • Predictable branches
  • Branch-free arithmetic
  • Conditional moves
  • Masked vector operations
  • Fixed iteration counts
  • Data-independent control flow

Branch removal is not automatically beneficial. The correct choice depends on predictability, instruction cost, and measurable hardware behavior.

Data Locality and the Memory Hierarchy

Processor performance is strongly influenced by where data resides.

The memory hierarchy typically includes:

  1. Registers
  2. L1 cache
  3. L2 cache
  4. Shared last-level cache
  5. Main memory

Each level provides greater capacity but higher access latency.

Efficient code attempts to:

  • Keep frequently used values in registers
  • Reuse data while it remains in cache
  • Access memory sequentially
  • Reduce unnecessary loads and stores
  • Avoid large working sets
  • Minimize cache-line transfers
  • Prevent avoidable cache misses

A mathematically efficient algorithm may perform poorly when its memory-access pattern does not align with the cache hierarchy.

Register Residency

Register-resident data can be reused without repeated memory access.

High-performance kernels therefore attempt to keep:

  • Accumulators
  • Loop counters
  • Constants
  • Partial results
  • Frequently accessed operands

inside registers for as long as possible.

Unnecessary spilling to the stack or repeated loading from memory increases traffic and may create additional dependencies.

The examples demonstrate how register blocking reduces memory operations and improves arithmetic intensity.

SIMD Execution

Single Instruction, Multiple Data (SIMD) instructions perform the same operation across multiple elements simultaneously.

On modern x86-64 systems, SIMD computation commonly uses:

  • XMM registers for 128-bit operations
  • YMM registers for 256-bit AVX and AVX2 operations
  • ZMM registers for 512-bit AVX-512 operations

Depending on the data type, one vector instruction may process:

  • Multiple bytes
  • Multiple integers
  • Multiple single-precision values
  • Multiple double-precision values

SIMD improves performance only when the surrounding algorithm can supply data efficiently and expose enough parallel work.

Vectorization may be limited by:

  • Misaligned or irregular memory access
  • Scalar dependencies
  • Horizontal reductions
  • Branch-heavy loops
  • Data-layout problems
  • Insufficient workload size
  • Memory bandwidth

The book therefore treats SIMD as part of a complete optimization strategy rather than an isolated instruction feature.

YMM and ZMM Registers

YMM and ZMM registers enable wide vector computation.

YMM registers support 256-bit operations, while ZMM registers support 512-bit operations on processors with AVX-512.

Wide vectors can increase arithmetic throughput, but wider instructions do not always guarantee faster execution.

Performance also depends on:

  • Processor implementation
  • Instruction throughput
  • Frequency behavior
  • Memory bandwidth
  • Data alignment
  • Register pressure
  • Port utilization
  • Masking overhead
  • Workload size

The reader learns to evaluate vector width through measurement rather than assumption.

Hand-Written Assembly Examples

The concepts are demonstrated through complete assembly kernels that reveal the actual cost of operations.

Examples include:

  • Introductory vector addition
  • SIMD load and store loops
  • AVX2 reductions
  • Dependency-chain benchmarks
  • Unrolled arithmetic loops
  • Branch-free kernels
  • Fused multiply-add routines
  • An eight-element matrix-multiplication microkernel

These examples illustrate how performance depends on:

  • Loop structure
  • Register allocation
  • Instruction ordering
  • Load and store patterns
  • Data alignment
  • Accumulator independence
  • Branch elimination
  • Cache reuse
  • SIMD width

Each routine is designed to expose a specific microarchitectural principle.

Measuring Performance Precisely

Optimization must be validated through measurement.

The booklet uses techniques such as:

  • Serialized rdtsc
  • Dependency-chain microbenchmarks
  • Controlled iteration counts
  • Loop unrolling
  • Warm-up execution
  • Repeated measurement
  • Cycle-per-element calculations
  • Carefully isolated kernels

These methods allow execution behavior to be quantified rather than estimated.

Performance may be expressed using metrics such as:

  • Total cycles
  • Cycles per iteration
  • Cycles per element
  • Instructions per cycle
  • Bytes processed per cycle
  • Operations completed per cycle

Small changes in instruction order, memory layout, or unrolling strategy can produce measurable differences.

This reinforces the central principle of performance engineering:

The correct optimization is the one validated by hardware measurement, not the one that merely appears optimal in theory.

Serialized Timing with rdtsc

The rdtsc instruction reads the processor's timestamp counter.

Because surrounding instructions may be reordered, reliable timing requires careful serialization.

A correct measurement strategy must ensure that:

  • Earlier operations complete before the start timestamp
  • The measured region executes without unintended reordering
  • The final timestamp is not collected too early
  • Timing overhead is measured and considered
  • The workload is large enough to reduce noise

The book demonstrates controlled timestamp measurement for small assembly kernels while explaining the limitations of cycle-level benchmarking.

Dependency-Chain Microbenchmarks

Dependency chains are useful for measuring instruction latency.

A benchmark repeatedly feeds one result into the next operation:

TEXT
result₁ → result₂ → result₃ → result₄

Because each instruction depends on the previous one, the processor cannot execute them in parallel.

This exposes the latency of the tested operation more clearly.

To examine throughput, the benchmark instead uses several independent dependency chains, allowing the processor to issue multiple operations concurrently.

These techniques help distinguish between:

  • Latency limits
  • Throughput limits
  • Front-end limits
  • Execution-port limits
  • Memory limits

Loop Unrolling

Loop unrolling reduces loop-control overhead and exposes more independent instructions to the processor.

It may improve performance by:

  • Reducing branch frequency
  • Reducing counter updates
  • Increasing instruction-level parallelism
  • Supporting multiple accumulators
  • Improving scheduling opportunities

However, excessive unrolling can increase:

  • Code size
  • Instruction-cache pressure
  • Register pressure
  • Complexity
  • Maintenance cost

The effective unrolling factor must therefore be determined through measurement.

Load and Store Patterns

Memory operations often dominate performance.

Efficient kernels use predictable access patterns such as:

  • Sequential loads
  • Sequential stores
  • Cache-friendly strides
  • Reused operands
  • Aligned accesses where beneficial
  • Reduced temporary storage

Poor access patterns may cause:

  • Cache misses
  • Translation lookaside buffer pressure
  • Split cache-line accesses
  • Store-forwarding delays
  • Memory-bandwidth saturation
  • Hardware prefetcher failure

The book demonstrates how small changes in layout and access order can significantly alter performance.

Arithmetic Intensity

Arithmetic intensity describes the amount of computation performed for each unit of data transferred from memory.

Low arithmetic intensity makes a routine more likely to be limited by memory bandwidth.

High arithmetic intensity allows more work to be completed while data remains in registers or cache.

Optimization techniques that improve arithmetic intensity include:

  • Data reuse
  • Register blocking
  • Cache tiling
  • Loop fusion
  • Reduced intermediate storage
  • Fused multiply-add operations

Matrix multiplication is a particularly important example because effective blocking allows loaded data to participate in many arithmetic operations before being evicted.

Fused Multiply-Add

Fused multiply-add (FMA) instructions combine multiplication and addition into a single operation:

TEXT
destination = destination + source1 × source2

FMA can provide:

  • Higher arithmetic throughput
  • Reduced instruction count
  • Improved register efficiency
  • A single rounding step for floating-point computation

These instructions are central to optimized numerical kernels such as:

  • Matrix multiplication
  • Convolution
  • Signal processing
  • Scientific simulation
  • Machine-learning operations

The book demonstrates how FMA works together with SIMD registers, register blocking, and loop unrolling.

Tiling and Register Blocking

Tiling divides a larger computation into smaller blocks that fit more effectively within the cache hierarchy.

Register blocking goes further by keeping a small working set directly in registers.

These techniques reduce:

  • Repeated memory access
  • Cache misses
  • Temporary storage
  • Loop overhead

They increase:

  • Data reuse
  • Arithmetic intensity
  • SIMD utilization
  • Instruction-level parallelism

The final matrix-multiplication microkernel illustrates how carefully chosen block sizes and register allocation can transform a simple algorithm into a high-throughput numerical kernel.

The Matrix-Multiplication Microkernel

The final microkernel integrates the book's major ideas:

  • SIMD vectorization
  • Register blocking
  • Fused multiply-add
  • Loop unrolling
  • Data reuse
  • Reduced memory traffic
  • Branch minimization
  • Instruction scheduling

The microkernel keeps partial results in vector registers while repeatedly reusing loaded input values.

This structure demonstrates why optimized mathematical libraries divide large problems into small architecture-specific kernels.

The microkernel is not intended to replace a complete production matrix library. Its purpose is to reveal the low-level principles that such libraries use internally.

Understanding High-Performance Libraries

Production-grade numerical libraries rely on the same principles explored throughout this booklet:

  • Cache tiling
  • Register blocking
  • SIMD arithmetic
  • Fused operations
  • Branch-free inner loops
  • Architecture-specific kernels
  • Careful memory layouts
  • Runtime feature selection

By studying these techniques directly in assembly, the reader gains insight into how optimized implementations achieve performance.

The same methods can be applied to:

  • Custom numerical software
  • Image processing
  • Signal processing
  • Simulation engines
  • Compression
  • Cryptography
  • Database operations
  • Machine-learning kernels

A Measurement-Driven Methodology

The broader objective is to develop a repeatable optimization process.

A disciplined workflow includes:

  1. Identify the performance-critical region.
  2. Establish a reliable baseline.
  3. Measure cycles and throughput.
  4. Determine whether the bottleneck is computation, memory, or control flow.
  5. Inspect dependencies and register usage.
  6. Improve locality and reduce memory traffic.
  7. Expose more parallel work.
  8. Apply SIMD where appropriate.
  9. Measure again.
  10. Retain only validated improvements.

This methodology is more valuable than memorizing processor-specific tricks because it can be applied to future architectures.

Book 9 in the Series

Book 9 concludes the SIMD-focused portion of the Low-Level Programming on Linux (x86-64) series.

It integrates concepts from:

  • Processor architecture
  • Assembly programming
  • Instruction scheduling
  • Cache hierarchy
  • Memory access
  • SIMD execution
  • Cycle-accurate benchmarking
  • Numerical-kernel design

The objective is not merely to present fast routines, but to provide a transferable framework for analyzing and improving software performance.

Principles That Remain Constant

Hardware continues to evolve, but the central principles of performance engineering remain stable:

  • Measure precisely
  • Understand the pipeline
  • Reduce memory traffic
  • Improve data locality
  • Expose independent work
  • Exploit SIMD parallelism
  • Minimize unnecessary branches
  • Keep critical data in registers
  • Align algorithms with the processor's structure
  • Validate every optimization on real hardware

These principles remain useful across future x86-64 processors and other modern architectures.

Applications Beyond Numerical Kernels

The techniques developed in this volume form a foundation for later low-level work, including:

  • Memory allocators
  • Cryptographic primitives
  • Runtime systems
  • Compression engines
  • Parsers
  • Database kernels
  • Multimedia processing
  • Operating-system components
  • Specialized numerical routines
  • High-performance libraries

In each case, understanding the processor's execution model allows the programmer to make informed decisions about data layout, control flow, register use, and memory access.

Final Objective

A program that produces correct results is not necessarily an efficient program.

To use the full capabilities of modern hardware, the programmer must understand how instructions interact with pipelines, caches, branch predictors, SIMD units, and memory systems.

By the end of this booklet, the reader will understand how to:

  • Distinguish latency from throughput
  • Analyze instruction dependencies
  • Expose instruction-level parallelism
  • Measure execution using timestamp counters
  • Design controlled microbenchmarks
  • Improve loop structure
  • Use SIMD registers effectively
  • Reduce unnecessary memory traffic
  • Apply loop unrolling
  • Use fused multiply-add instructions
  • Build register-blocked kernels
  • Validate optimizations through hardware measurements

A solid understanding of SIMD execution and low-level performance engineering is indispensable for anyone who intends to write software that does not merely execute correctly, but operates near the full capability of the hardware.