The Stack in Assembly

Analogy categoryAnalogies

The stack is a region of memory managed via the ESP register (RSP in 64-bit mode) that follows a strict last-in-first-out (LIFO) discipline: the most recently pushed value is always the first one popped. Unlike a heap, which grows upward from low addresses, the x86 stack conventionally grows downward -- each PUSH decrements ESP before storing a value, and each POP reads the value at [ESP] before incrementing it back up, so ESP always points to whatever was pushed most recently.

🏏

Cricket analogy: A stack of players waiting to bat in the pavilion, where the next man in is always the one who arrived most recently, mirrors how the stack is LIFO and ESP always points to the most recently pushed value.

Function Calls: CALL, RET, and Return Addresses

CALL and RET rely entirely on the stack to remember where execution should resume. CALL pushes the address of the instruction immediately following itself onto the stack and then jumps to the target label; RET pops that saved return address off the stack and jumps back to it. Functions commonly begin with PUSH ebp followed by MOV ebp, esp to establish a stable frame pointer, since ESP itself will shift as the function pushes further data, while EBP stays fixed for reliably addressing parameters and locals throughout the function body.

🏏

Cricket analogy: When a substitute fielder comes on, the ground marks exactly where the injured player will return to resume mirrors how CALL automatically pushes the return address onto the stack so RET knows exactly where to resume execution.

nasm

; A simple function using a standard stack frame
section .text
global _start

add_numbers:
    push ebp                ; save caller's base pointer
    mov  ebp, esp            ; establish new frame base
    sub  esp, 4               ; reserve 4 bytes for a local variable

    mov  eax, [ebp+8]         ; first argument
    add  eax, [ebp+12]        ; second argument
    mov  [ebp-4], eax         ; store result in local variable
    mov  eax, [ebp-4]         ; return value in eax

    mov  esp, ebp              ; deallocate locals
    pop  ebp                    ; restore caller's base pointer
    ret                          ; pop return address into EIP

_start:
    push 20                    ; push second argument
    push 10                    ; push first argument
    call add_numbers
    add  esp, 8                 ; caller cleans up (cdecl convention)
    mov  ebx, eax
    mov  eax, 1
    int  0x80

Local Variables and the Stack Frame

Local variables that a function needs during its execution but doesn't want to keep in registers are allocated by subtracting their combined size from ESP -- for example, SUB esp, 16 carves out 16 bytes of scratch space -- and accessed via negative offsets from EBP, such as [ebp-4]. This space is deallocated instantly when the function's epilogue runs MOV esp, ebp (restoring ESP to EBP's value) followed by POP ebp; NASM's ENTER and LEAVE instructions can perform the equivalent prologue and epilogue in a single instruction each, though discrete instructions are often faster on modern CPUs.

🏏

Cricket analogy: Reserving a specific patch of the outfield exclusively for a fielding drill during practice, distinct from the main pitch, mirrors SUB esp, 16 carving out 16 bytes of stack space for a function's local variables.

The prologue push ebp / mov ebp, esp and epilogue mov esp, ebp / pop ebp can be replaced by the single instructions ENTER n, 0 and LEAVE respectively, which many compilers avoid in optimized code because the discrete instructions are typically faster on modern CPUs despite ENTER/LEAVE being more compact.

Calling Conventions and Stack Cleanup

Calling conventions define who is responsible for cleaning up the stack after a function call. Under cdecl, the caller pushes arguments and is responsible for restoring ESP afterward, typically with ADD esp, N -- this flexibility is why cdecl supports variable-argument functions like printf, since the caller always knows exactly how many bytes it pushed. Under stdcall, the callee instead cleans up its own arguments as part of its RET N instruction. On modern x86-64 ABIs, the stack must additionally be kept 16-byte aligned before a CALL, since SIMD instructions operating on stack-resident data require that alignment.

🏏

Cricket analogy: Whether the fielding side or the batting side resets the field markers after an over depends on the match's specific rules, just as cdecl requires the caller to clean up pushed arguments with ADD esp, N after CALL, while stdcall makes the callee clean up via RET N.

Mixing calling conventions -- for example, calling a stdcall function (which cleans up its own arguments via RET N) as if it were cdecl (where the caller cleans up with ADD esp, N) -- cleans the stack twice, leaving ESP pointing at the wrong location for the next instruction and typically crashing the program soon after, often far from the actual mismatched call site, which makes it hard to debug.

The stack grows toward lower memory addresses; PUSH decrements ESP, POP increments it.
CALL automatically pushes the return address before jumping; RET pops it back into EIP.
A standard stack frame is set up with PUSH ebp; MOV ebp, esp and torn down with MOV esp, ebp; POP ebp.
Local variables are allocated by subtracting from ESP and accessed via negative offsets from EBP.
cdecl requires the caller to clean up pushed arguments after CALL; stdcall makes the callee clean up via RET N.
ENTER and LEAVE are compact alternatives to the manual prologue/epilogue sequence.
Mismatched calling conventions or unbalanced PUSH/POP corrupt the stack pointer and typically crash the program.

Practice what you learned

1. What does the CALL instruction push onto the stack before jumping?

2. In the standard stack frame setup, what does MOV ebp, esp accomplish?

3. Under the cdecl calling convention, who is responsible for removing pushed arguments from the stack after a call?

4. What is a likely consequence of an unmatched PUSH inside a function body (one PUSH with no corresponding POP before RET)?

5. Why do modern x86-64 calling conventions often require 16-byte stack alignment before a CALL?

Was this page helpful?

Data Types and Directives

Related Study Notes in Programming

Browse all study notes

Programming

Apache Spark Study Notes

Programming · 30 topics

Programming

Apache Flink Study Notes

Programming · 30 topics

Programming

Hadoop Study Notes

Programming · 30 topics

Programming

Snowflake Study Notes

Programming · 30 topics

Programming

Apache Airflow Study Notes

Programming · 30 topics

Programming

dbt (Data Build Tool) Study Notes

Programming · 30 topics

Topics covered

#Programming#AssemblyLanguageStudyNotes#TheStackInAssembly#Stack#Assembly#Function#Calls#DataStructures#StudyNotes#SkillVeris

The Stack in Assembly