Protobuf Data Types

Analogy categoryAnalogies

What Data Types Does Protobuf Offer?

Protocol Buffers gives you a compact set of scalar types (int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, float, double, bool, string, bytes) plus composite constructs like repeated fields, maps, and nested messages. Choosing the right scalar type matters because it directly affects wire size and encoding speed, unlike JSON where every number looks the same on the wire.

🏏

Cricket analogy: Just as a scorer chooses between recording a delivery as a dot, a single, or a boundary rather than writing a full sentence each time, protobuf picks the tightest numeric type (int32 vs sint32) to keep the record compact.

Scalar Types and Wire Encoding

Protobuf's varint encoding (used by int32, int64, uint32, uint64, bool, and enum) stores small numbers in fewer bytes, but it encodes negative int32/int64 values as ten-byte sequences because two's-complement representation sets the high bit. sint32 and sint64 fix this with zigzag encoding, which maps signed values to unsigned ones so small negative numbers stay small on the wire. fixed32, fixed64, float, and double always take a constant number of bytes (4 or 8), which is faster to decode and often smaller for large or already-large-magnitude numbers.

🏏

Cricket analogy: A team's over-by-over run rate is easy to compress when scores stay low, but if you always reserved ten digits for every score just in case of a super over, you would waste space, exactly why plain varint wastes bytes on negative int32s and sint32 fixes it.

Composite Types: repeated, map, bytes, and string

The repeated keyword marks a field as a list, and for scalar numeric types it is packed by default in proto3, meaning all the values are concatenated into a single length-delimited block rather than each getting its own tag-length-value entry. The map<K, V> type is syntactic sugar for a repeated message with key and value fields, so maps cannot be repeated themselves and their key type is restricted to integral or string types. bytes stores raw binary data without any encoding validation, while string must contain valid UTF-8, which the runtime enforces at parse time in many language implementations.

🏏

Cricket analogy: A packed repeated field is like a single ball-by-ball commentary feed concatenated into one transcript, whereas an unpacked encoding would be like issuing a separate press release for every single ball.

protobuf

syntax = "proto3";

package catalog.v1;

message Product {
  int64 id = 1;
  string name = 2;
  double price = 3;
  bool in_stock = 4;
  repeated string tags = 5;          // packed-by-default for scalars
  map<string, string> attributes = 6; // e.g. {"color": "red"}
  bytes thumbnail = 7;               // raw binary, no UTF-8 validation
  sint32 stock_delta = 8;            // efficient for negative adjustments
  fixed64 last_updated_epoch_ms = 9; // constant width, fast to decode
}

As a rule of thumb: use int32/int64 for counters and IDs that are usually non-negative, sint32/sint64 for fields that are frequently negative (deltas, offsets), and fixed32/fixed64 for values that are often larger than 2^28 (~268M) where the fixed width actually beats varint's overhead.

Choosing the Right Type in Practice

Because proto3 field presence for scalar types is based on the default value (0, empty string, false) rather than an explicit 'set' flag, you cannot distinguish 'not sent' from 'sent as zero' unless you wrap the field in the optional keyword (proto3.15+) or use a well-known wrapper type like google.protobuf.Int32Value. This matters for fields like a price of 0 or a boolean flag that legitimately needs a false-vs-unset distinction, such as a partial PATCH-style update.

🏏

Cricket analogy: A batter with zero runs on the board looks identical to a batter who hasn't come in to bat yet unless the scoreboard explicitly marks 'yet to bat', just as an unset int32 field is indistinguishable from an explicit zero unless you use optional.

Never use 'required' fields (proto2 legacy) or rely on implicit zero-as-unset semantics for fields you intend to evolve later; a missing distinction between 'unset' and 'default' has broken production PATCH APIs where clients could not clear a field back to zero.

Protobuf scalar types include varint-based (int32/int64/uint32/uint64/bool/enum), zigzag varint (sint32/sint64), and fixed-width (fixed32/fixed64/float/double) encodings.
sint32/sint64 are more efficient than int32/int64 for fields that are frequently negative, due to zigzag encoding.
fixed32/fixed64 beat varint encoding once values regularly exceed roughly 2^28 in magnitude.
repeated scalar fields are packed by default in proto3, concatenating values into one length-delimited block.
map<K,V> is sugar for a repeated message of key/value pairs and cannot itself be repeated.
bytes holds raw binary without validation; string must be valid UTF-8.
Use the optional keyword or well-known wrapper types when you need to distinguish 'unset' from 'explicit zero/false/empty'.