Introduction to x86-64 Floating Point and SIMD

These blog posts are written as lessons. Examples begin with SSE2-capable x86-64 user mode. AVX, AVX2, and AVX-512 require both processor support and enabled operating-system state.

Seeing an XMM register does not prove that code is vectorized. addss adds one low float; addps adds four packed floats. The same register file supports scalar and packed operations.

x87, SSE, and AVX

• x87 uses an older stack-register model and extended precision.
• SSE/SSE2 uses XMM registers and mainly two-operand legacy encodings.
• AVX adds YMM registers and VEX-encoded three-operand forms.
• AVX-512 adds ZMM, masks, and EVEX features.

Modern x86-64 compilers normally use SSE-family instructions for ordinary scalar float and double. x87 can indicate compatibility code, special long double behavior, or old options; check the target ABI.

XMM is 128-bit storage, not a type

The opcode defines the interpretation:

packed float32 : 4 × 32-bit lanes
packed float64 : 2 × 64-bit lanes
scalar float32 : low 32 bits
scalar float64 : low 64 bits
packed int32   : 4 × 32-bit lanes

addss xmm0, xmm1     ; one float32
addsd xmm0, xmm1     ; one float64
addps xmm0, xmm1     ; four float32 lanes
addpd xmm0, xmm1     ; two float64 lanes

With AVX, separate sources are possible:

vaddps ymm0, ymm1, ymm2

Packed operations work lane by lane

movups xmm0, [rcx]
movups xmm1, [rdx]
addps  xmm0, xmm1
movups [r8], xmm0

This computes [a0+b0, a1+b1, a2+b2, a3+b3]. It is not one 128-bit integer addition with carry between lanes. Integer packed opcodes such as paddb, paddw, paddd, and paddq also state their lane width.

Aligned and unaligned memory forms

movaps xmm0, [rcx]    ; aligned packed float
movups xmm0, [rcx]    ; unaligned packed float
movdqa xmm0, [rcx]    ; aligned integer data
movdqu xmm0, [rcx]    ; unaligned integer data

An aligned form can fault on a misaligned address. Modern CPUs may execute an unaligned form efficiently when the address is aligned, but cache-line and page crossings still matter. Alignment of a base allocation does not automatically align every field offset.

Reassociation changes floating-point results

IEEE 754 addition is not associative:

(a + b) + c != a + (b + c)

Vector reductions change summation order and may not be bit-identical to a scalar reference. Define acceptable error and test NaN, infinity, signed zero, extreme magnitudes, and rounding behavior. Fast-math compiler options allow stronger assumptions and must be part of the build record.

Comparisons can be unordered

NaN makes an FP comparison unordered.

ucomiss xmm0, xmm1
jp unordered_case
ja above_case

ucomiss/ucomisd and comiss/comisd differ in exception details. Find the actual comparison opcode that produced flags. Packed comparisons often create all-ones or zero masks per lane for later AND, blend, or movemask operations.

Shuffles and reductions

shufps xmm0, xmm1, imm8
pshufd xmm0, xmm1, imm8
unpcklps xmm0, xmm1

Decode the immediate using the instruction reference. Many shuffles can indicate AoS/SoA conversion, transpose, broadcast, or horizontal reduction.

For a four-float dot product:

movups xmm0, [rcx]
movups xmm1, [rdx]
mulps xmm0, xmm1
; shuffle/add the four lanes to one result

Specialized haddps or dpps forms are not automatically optimal; compare throughput, dependencies, accuracy, and compiler output on the target CPU.

AVX state and vzeroupper

AVX legacy-SSE transitions had penalties on some processors when upper YMM state remained live. This explains vzeroupper at some AVX boundaries. The exact need and cost depend on the processor and compiler convention; do not add or remove it by folklore.

Feature detection includes the OS

Before executing AVX-family instructions, distinguish:

1. CPUID hardware support;
2. operating-system support for extended register state;
3. XGETBV enabling the required XMM/YMM state;
4. the feature level guaranteed on every deployment CPU.

AVX does not imply AVX2 or AVX-512. Unsupported execution causes an illegal-instruction fault. Use dispatch, multiple targets, or compiler multiversioning.

ABI preservation differs

System V treats XMM registers as caller-saved in its normal convention. Microsoft x64 preserves the low 128 bits of xmm6–xmm15. Handwritten Windows assembly using them needs save/restore and matching unwind metadata. Do not assume that the upper YMM/ZMM portions follow the same rule without checking current target documentation.

Tails and overreads

Lengths not divisible by vector width need a scalar tail or mask. Reading beyond the logical end because “the allocation probably has padding” can violate object, page, sanitizer, or concurrency contracts. Only overread when the API explicitly guarantees it.

Auto-vectorization may fail because of aliasing, loop-carried dependency, calls, unknown trip count, cost model, strict FP order, or missing target ISA. Check vectorization reports and compiler intrinsics before handwritten assembly.

SIMD checklist

1. Scalar or packed, and what lane width?
2. Does alignment hold for every pointer plus offset?
3. Are zero, sub-vector, exact-width, and remainder inputs tested?
4. What are NaN, infinity, signed-zero, and error contracts?
5. Does feature detection include hardware and OS state?
6. Is nonvolatile vector state preserved for the ABI?
7. Is the code faster than compiler or intrinsics in the real bottleneck?

Summary

XMM is 128-bit storage whose opcode selects scalar or packed and floating or integer interpretation. SIMD processes several elements at once but adds contracts for alignment, tails, FP reassociation, feature detection, and ABI preservation. Prove lane semantics and correctness before measuring real workload speed.

Official references

• Intel Software Developer's Manual
• Intel Intrinsics Guide
• AMD64 Architecture Programmer's Manual
• Microsoft x64 register usage