Binary Pattern Matching

Analogy categoryAnalogies

The Bit Syntax

Erlang represents raw binary data as a bitstring written between << and >> delimiters, such as <<1, 2, 3>> for three 8-bit bytes or <<"hello">> for the UTF-8 bytes of a string literal. The same << >> syntax is used both to construct binaries and, critically, to pattern-match against them, so a function clause like parse(<<Version:8, Rest/binary>>) -> ... simultaneously validates that a binary is at least one byte long and extracts that first byte as an 8-bit unsigned integer while binding everything after it to Rest.

🏏

Cricket analogy: A TV broadcast's on-screen scorecard graphic parses a fixed-width data feed byte by byte to pull out the current over number and score, the same declarative extraction <<Version:8, Rest/binary>> performs on a raw binary stream in one pattern match.

Matching Segments by Size and Type

Each segment in a bit-syntax pattern can specify its own size and type: <<A:8, B:16, C:4, D:4>> matches a leading byte into A, a 16-bit value into B, then splits the following byte into two 4-bit nibbles C and D. Type specifiers after the slash control interpretation, integer (the default), float, binary, or bitstring, and a segment's size is measured in units (1 bit for integer/float by default, 8 bits for binary), so <<Len:16, Body:Len/binary>> is the idiomatic pattern for a length-prefixed field, where the value just matched (Len) is used as the size of the very next segment.

🏏

Cricket analogy: A DRS ball-tracking overlay splits a single data frame into separate fields for pitch location, impact point, and projected trajectory, each with its own fixed width, the same way <<A:8, B:16, C:4, D:4>> carves one binary into differently-sized typed segments.

The Len:16, Body:Len/binary trick, using a value bound earlier in the same pattern as the size of a later segment, only works left to right; you cannot reference a field that appears later in the same pattern, so protocols are always parsed in the order their fields are laid out on the wire.

Building Binaries and Handling the Remainder

Building a binary uses the identical syntax in the opposite direction, Packet = <<16#01:8, Length:16, Payload/binary>> assembles a header byte, a 16-bit length, and the payload bytes into one new binary, which means the encoder and decoder for a protocol can, and often should, be written as near-mirror-image bit-syntax expressions. The catch-all /binary specifier on the final segment of a match, as in <<Header:8, Rest/binary>>, is what lets you peel off a fixed-size header and capture 'everything else, whatever length it is' without knowing that length in advance.

🏏

Cricket analogy: A television graphics engine both builds the on-screen scorecard packet and, on the broadcast truck's end, parses it back apart using mirror-image logic, the same symmetry Erlang encourages by using identical bit-syntax on the construct and match sides.

erlang

-module(packet_demo).
-export([encode/2, decode/1, decode_ipv4/1]).

%% Build a simple length-prefixed packet: <<Type:8, Length:16, Payload/binary>>
encode(Type, Payload) when is_binary(Payload) ->
    Length = byte_size(Payload),
    <<Type:8, Length:16, Payload/binary>>.

%% Decode the same packet format, peeling off header fields one at a time.
decode(<<Type:8, Length:16, Payload:Length/binary, Rest/binary>>) ->
    {ok, Type, Payload, Rest};
decode(_TooShort) ->
    {error, incomplete_packet}.

%% Decode the first 20 bytes of an IPv4 header's bitfields in one match.
decode_ipv4(<<4:4, IHL:4, _DSCP:6, _ECN:2, TotalLength:16,
              _Id:16, _Flags:3, _FragOff:13, TTL:8, Proto:8,
              _Checksum:16, Src:32, Dst:32, _Rest/binary>>) ->
    {ok, #{ihl => IHL, total_length => TotalLength, ttl => TTL,
           protocol => Proto, src => Src, dst => Dst}}.

Endianness and Real-World Protocol Parsing

Multi-byte integer and float segments accept an explicit endianness specifier, big (the default), little, or native, so a field arriving from a network protocol like TCP, which is conventionally big-endian, is matched with <<Port:16/big>> (or just <<Port:16>>, since big is the default), while a field from an x86 file format is often <<Value:32/little>>. This precision is exactly why bit syntax is the standard tool for parsing real-world binary protocols in Erlang: a single pattern like <<4:4, IHL:4, DSCP:6, ECN:2, TotalLength:16, Rest/binary>> can decode an IPv4 header's tightly packed bitfields in one match, something that would require substantial manual bit-shifting and masking in most other languages.

🏏

Cricket analogy: Reading a scoreboard feed correctly requires knowing whether the host broadcaster sends the most significant digit of the score first or last, the same big-endian-versus-little-endian distinction <<Port:16/big>> versus <<Value:32/little>> makes explicit in Erlang's bit syntax.

A pattern match against a binary that is too short, or whose fixed literal segments (like the 4:4 IP-version nibble in an IPv4 header) don't match, fails with a function_clause or badmatch rather than silently returning a partial result, always pair binary parsing with a catch-all clause or a try...catch when the input may be malformed or truncated.

Binaries and bitstrings are written <<...>> and the same syntax both constructs and pattern-matches binary data.
Each segment in a bit-syntax pattern can declare its own size and type: <<A:8, B:16/integer, C/binary>>.
A length-prefixed field is decoded idiomatically as <<Len:16, Body:Len/binary>>, using an earlier-matched value to size a later segment.
The final /binary segment in a pattern, e.g. Rest/binary, captures the remaining bytes regardless of length.
Multi-byte segments accept /big (default), /little, or /native endianness specifiers to match real-world protocol byte orders.
Bit syntax can decode tightly packed sub-byte bitfields, like an IPv4 header's 4-bit version and IHL nibbles, in a single declarative pattern match.