How Should x86-64 Performance Be Measured?

These blog posts are written as lessons. CPU performance depends on microarchitecture, power and thermal state, memory hierarchy, OS, and input. Do not memorize one fixed cycle count as a universal fact.

Fewer assembly instructions do not guarantee faster code. One instruction may decode to several micro-operations, while independent instructions may overlap. A single cache miss or long dependency can dominate the entire loop.

Optimization is the process of measuring a bottleneck, forming a causal hypothesis, and trying to disprove it under controlled conditions.

Latency and throughput

• Latency: time until a result is available to a dependent operation.
• Throughput: how often independent operations can begin.

; dependent chain
add rax, rbx
add rax, rcx
add rax, rdx

; independent work
add r8,  rbx
add r9,  rcx
add r10, rdx

Both contain three adds, but their critical paths differ. State whether a benchmark measures a dependency chain or independent throughput.

Instructions and uops are not the same unit

x86 front ends decode instructions into internal uops. An instruction can produce one or several; fusion and microcode can change behavior. Source lines, instruction count, byte size, and jmp count are useful clues but none directly proves execution time.

Bottlenecks exist at several levels

Removing ten instructions cannot matter when page faults or I/O dominate. Confirm a hot path with wall time and profiling first.

Data locality can outweigh instruction choice

Sequential arrays use cache lines and hardware prefetching well. Reading one field from a large structure wastes much of each line, while pointer chasing serializes future addresses behind current loads. Moving from AoS to SoA or reducing the working set can beat handwritten assembly by a wide margin.

Branch prediction is about real input distribution

A branch that almost always goes one way can be cheap. Random-looking directions may mispredict. cmov or masks replace a control dependency with data dependencies and can execute work from both paths. A microbenchmark with artificial random input may misrepresent a product workload.

Register and memory dependencies

A 32-bit write clears the upper half and can break dependencies; processors often recognize zeroing idioms such as xor eax,eax. Partial-register behavior and exact costs vary by microarchitecture, so “X is always faster than Y” folklore is unsafe.

Store-to-load forwarding can avoid a cache round trip, but mismatched sizes or offsets can stall:

mov qword ptr [rsp], rax
mov eax, dword ptr [rsp + 1]

Compiler alias uncertainty can also prevent reordering. Improving a truthful source-level non-alias contract can improve code generation.

A benchmark answers one question

Specify the hypothesis, input sizes and distribution, prevention of dead-code elimination, inclusion or exclusion of setup, cold or warm cache state, and independence between repetitions. Do not use one number to claim latency, throughput, and end-to-end performance simultaneously.

Record CPU model, microcode, compiler and flags, ISA target, LTO/PGO, input seed, affinity or migration, power mode, temperature, run count, distribution, outlier handling, OS, and background load. Prefer distributions across independent runs to one best result.

rdtsc is not automatically a function timer

Out-of-order execution, serialization, migration, invariant TSC behavior, and frequency interpretation matter. Start with a trusted benchmark framework and monotonic high-resolution clock. A custom TSC harness must first quantify empty-measurement and serialization overhead.

Profile location, then narrow the cause

Sampling profilers identify where time is spent. Hardware counters help test hypotheses about cycles, instructions, branches, caches, and TLBs. Event semantics vary by CPU and may include multiplexing, skid, and speculative events. IPC alone is not a verdict: high IPC can be wasted work, and low IPC can be normal for memory-bound code.

Keep compiler output as the baseline

1. Write a clear tested C/C++ reference.
2. Inspect optimized compiler output.
3. Improve algorithm and data layout.
4. Consider target options, PGO, and intrinsics.
5. Experiment with assembly only in a narrow hot loop.
6. Compare correctness and speed under the same contract.

A separate assembly function is opaque to many whole-program optimizations and can lose inlining, constant propagation, scheduling, or vectorization opportunities.

Fast but incorrect is rejected

Test zero, one, vector boundaries, large sizes, integer overflow contracts, aligned and unaligned input, aliasing and overlap, FP special values, feature fallback, sanitizers, exceptions, and stack unwinding. If performance requires narrower preconditions, document them and make callers prove them.

Adoption includes maintenance

Evaluate end-to-end impact, consistency across supported CPUs, dispatch and fallback cost, revalidation after compiler updates, debugging and sanitizer compatibility, security mitigations, and team maintainability. If compiler code or intrinsics match the result, the more transparent implementation may be the better long-term optimization.

Report template

hot location:
hypothesis:
baseline:
candidate:
correctness contract/tests:
CPU/OS/compiler/options:
input sizes/distribution:
measurement method/repetitions:
result distribution:
counter observations:
alternative explanations:
adoption/removal condition:

Summary

x86-64 performance combines latency, throughput, uops, execution resources, dependencies, cache/TLB behavior, branch prediction, and system noise. Find the hot path with profiling, test a precise hypothesis with a controlled benchmark, and retain handwritten assembly only where its benefit and maintenance conditions remain reproducible.

Official references

• Intel Optimization Reference Manual
• Intel Software Developer's Manual
• AMD Software Optimization resources