SIMD and Vector Instructions

Analogy categoryAnalogies

Single Instruction, Multiple Data

SIMD (Single Instruction, Multiple Data) lets one instruction perform the same operation on several data elements packed into a single wide register, instead of issuing separate scalar instructions for each element. x86 exposes this through SSE, which added 128-bit XMM0–XMM15 registers holding, for example, four 32-bit floats or two 64-bit doubles, and AVX/AVX2, which extended these to 256-bit YMM registers, and AVX-512, which extended further to 512-bit ZMM registers with eight mask registers (K0–K7) for per-lane predication. The core win is throughput: a single VADDPS on a 256-bit YMM register adds eight 32-bit floats in roughly the same time a single scalar ADDSS adds one, so vectorized code can process array-heavy workloads like image filters or numerical simulations many times faster than scalar loops.

🏏

Cricket analogy: It is like a bowling machine that can fire eight balls simultaneously down eight parallel lanes for a squad net session instead of one coach feeding balls to one batter at a time — the same delivery motion, applied to eight targets at once.

Packed Data Types and Aligned vs. Unaligned Moves

SIMD registers are typeless containers whose interpretation is set by the instruction: MOVAPS/MOVUPS treat 128 bits as four packed single-precision floats, PADDD treats them as four packed 32-bit integers to add, and PSHUFB treats them as sixteen packed bytes for arbitrary byte-lane shuffling. The 'A' in instructions like MOVAPS (Move Aligned Packed Single) means the memory operand must be aligned to the register width — 16 bytes for XMM, 32 for YMM, 64 for ZMM — and using it on unaligned memory raises a #GP general protection fault; MOVUPS (Move Unaligned Packed Single) relaxes that requirement at a historically small performance cost that is largely negligible on modern CPUs with unified load ports. Compilers typically align heap and stack buffers intended for SIMD work using aligned allocation functions or the alignas specifier, and hand-written assembly must do the same when declaring static SIMD buffers, typically via an align directive in the data section.

🏏

Cricket analogy: It is like a strict pitch-marking rule where the aligned bowling crease (MOVAPS) demands the bowler's front foot land exactly on the marked line or the delivery is called a no-ball, while a relaxed practice-net rule (MOVUPS) tolerates a slightly off-line foot landing without penalty.

AVX/SSE Transition Penalties

Mixing legacy 128-bit SSE instructions with VEX-encoded AVX instructions on the same YMM register without a VZEROUPPER in between triggers a costly AVX/SSE transition penalty on many microarchitectures: the CPU must save the full, potentially dirty upper 128 bits of the YMM state before it can safely execute the narrower SSE instruction, because SSE instructions are architecturally defined to leave the upper bits of the corresponding YMM register unmodified, and the hardware has to reconcile that guarantee. The standard fix is to execute VZEROUPPER — which zeroes the upper 128 bits of all YMM registers — at the boundary between AVX-heavy and SSE-heavy code, and this is exactly why compilers automatically insert it at the end of AVX-using functions and why hand-written mixed SSE/AVX assembly must do it explicitly.

🏏

Cricket analogy: It is like switching from a day-night pink-ball Test straight into a T20 without recalibrating the sight screens and ball-tracking cameras first — the system has to pause and reset its full calibration state before it can trust the new format's readings, exactly as the CPU must reconcile YMM state before trusting SSE.

nasm

; AVX2: c[i] = a[i] + b[i] for 8 floats at a time, then VZEROUPPER before
; returning to code that may use legacy SSE instructions.
section .text
global vector_add_f32           ; void vector_add_f32(float *a, float *b, float *c, size_t n)
                                  ; rdi=a, rsi=b, rdx=c, rcx=n (n assumed multiple of 8)

vector_add_f32:
    xor rax, rax                 ; i = 0
.loop:
    cmp rax, rcx
    jge .done
    vmovups ymm0, [rdi + rax*4]   ; load 8 floats from a (unaligned-safe)
    vaddps  ymm0, ymm0, [rsi + rax*4]  ; ymm0 = a[i..i+7] + b[i..i+7]
    vmovups [rdx + rax*4], ymm0    ; store result into c
    add rax, 8
    jmp .loop
.done:
    vzeroupper                    ; clear upper YMM bits to avoid SSE transition penalty
    ret

AVX-512's mask registers (K0-K7) allow per-lane predication directly in the instruction encoding, so a single vectorized instruction can skip updating specific lanes (for example, to handle an array whose length is not a multiple of the vector width) without needing a separate scalar cleanup loop, unlike SSE/AVX2 which typically require manual tail handling.

AVX-512 is not universally available even on recent CPUs — some Intel consumer chips have disabled it entirely, and enabling wide 512-bit execution units can trigger CPU frequency downclocking on certain server chips under sustained load. Always feature-detect with CPUID before dispatching to AVX-512 code paths rather than assuming availability from the CPU generation alone.

SIMD applies one instruction to multiple packed data elements at once, using XMM (128-bit), YMM (256-bit), and ZMM (512-bit) registers for SSE, AVX/AVX2, and AVX-512 respectively.
Instructions like MOVAPS/PADDD/PSHUFB reinterpret the same register bits as floats, integers, or bytes depending on the operation.
Aligned moves (MOVAPS) require memory aligned to the register width and fault on unaligned data; unaligned moves (MOVUPS) relax that at minimal modern cost.
Mixing legacy SSE and VEX-encoded AVX instructions without VZEROUPPER causes a costly state-reconciliation transition penalty.
VZEROUPPER zeroes the upper 128 bits of all YMM registers and should be used at SSE/AVX code boundaries.
AVX-512 adds mask registers (K0-K7) enabling per-lane predication without separate scalar tail loops.
AVX-512 availability and thermal/frequency behavior vary by chip, so runtime CPUID feature detection is essential.