Reverse Engineering with Assembly

Analogy categoryAnalogies

Disassembly vs Decompilation

Disassembly converts machine bytes into a linear listing of assembly mnemonics — tools like objdump -d or IDA Pro's disassembly view show exactly what the CPU executes but preserve none of the original variable names, types, or control-flow structure. Decompilation, performed by tools like Ghidra or the Hex-Rays decompiler, attempts to reconstruct higher-level pseudo-C from that assembly by recognizing patterns like loop structures and stack frame layouts, but this reconstruction is heuristic and can misinterpret optimized or obfuscated code, so cross-checking decompiler output against the raw disassembly remains essential.

🏏

Cricket analogy: Disassembly is like watching raw ball-by-ball footage of an innings with no commentary, while decompilation is like a commentator's post-match analysis reconstructing the game plan — useful, but occasionally misreading intent behind a shot.

asm

; Disassembly view (objdump-style) - raw and literal
0x0000: 55                push   rbp
0x0001: 48 89 e5          mov    rbp, rsp
0x0004: 89 7d fc          mov    DWORD PTR [rbp-0x4], edi
0x0007: 8b 45 fc          mov    eax, DWORD PTR [rbp-0x4]
0x000a: 83 c0 01          add    eax, 0x1
0x000d: 5d                pop    rbp
0x000e: c3                ret

; Decompiler reconstruction (Ghidra/Hex-Rays style pseudo-C)
int func(int param_1) {
    return param_1 + 1;
}

Recognizing Compiler-Generated Patterns

A standard x86-64 function prologue push rbp; mov rbp, rsp; sub rsp, N establishes a stack frame, and its matching epilogue mov rsp, rbp; pop rbp; ret (or the shorter leave; ret) tears it down — recognizing this pair instantly tells a reverse engineer where a function begins and ends even without symbol information. Compilers also emit recognizable idioms for common operations: a division by a power of two often compiles to a right shift (sar), and switch statements with dense case values frequently compile to a jump table accessed via jmp [rax*8 + table], which is a strong signal to look for a .rodata array of code addresses nearby.

🏏

Cricket analogy: Recognizing a function prologue/epilogue is like recognizing a bowler's consistent run-up and follow-through — even without seeing the scoreboard, the shape alone tells you exactly when a delivery starts and ends.

Symbol stripping removes function names but not code structure: prologue/epilogue patterns, calling conventions, and idiom recognition let experienced reverse engineers recover function boundaries and even rough purpose from stripped binaries.

Tracing Control Flow and Identifying Data Structures

Following control flow in a disassembler means building a mental (or tool-generated) control-flow graph: conditional jumps like jz/jnz following a cmp or test mark decision points, and back-edges (a jump targeting an earlier address) mark loops. Struct field access shows up as a fixed offset from a base pointer — mov eax, [rdi+0x10] repeated across a function strongly suggests rdi points to a structure with a field at offset 0x10, and cross-referencing that offset across multiple functions in tools like IDA's 'Xrefs' view often reveals the structure's full layout even without original source.

🏏

Cricket analogy: Tracing conditional jumps is like reading a match's DRS review pattern — each cmp is the on-field decision and each conditional jump is whether the review upholds or overturns it, forming a branching narrative of the innings.

Obfuscated or packed binaries can defeat naive static disassembly entirely — opaque predicates, control-flow flattening, and self-modifying code can cause a disassembler to mis-decode instruction boundaries. In these cases, dynamic analysis (stepping through the code in a debugger like x64dbg or using a sandboxed tracer) is often more reliable than static disassembly alone.

Disassembly is a literal, structure-free listing of instructions; decompilation heuristically reconstructs higher-level pseudo-code and can misinterpret optimized code.
Standard prologue (push rbp; mov rbp,rsp) and epilogue (leave; ret) patterns mark function boundaries even in stripped binaries.
Division by powers of two often compiles to shift instructions; dense switch statements often compile to jump tables in .rodata.
Conditional jumps following cmp/test mark branch points; back-edges (jumps to earlier addresses) indicate loops.
Repeated fixed-offset memory accesses from a base register (e.g. [rdi+0x10]) suggest structure field access, recoverable via cross-referencing.
Symbol stripping removes names but not structural patterns that experienced analysts can still recognize.
Obfuscated/packed binaries often require dynamic analysis (debuggers, tracers) rather than static disassembly alone.