Designing a URL Shortener

A URL shortener (think bit.ly or TinyURL) takes a long URL and maps it to a short, unique alias that redirects users to the original destination. It looks trivial at first glance, but a production system must handle billions of stored mappings, extremely read-heavy traffic (redirects vastly outnumber creations, often 100:1 or more), collision-free key generation across many machines, and low-latency redirects — typically a single-digit-millisecond lookup plus an HTTP 301/302 response. The interesting design decisions are how short codes are generated without central bottlenecks, how the data is stored and partitioned, and how the read path is kept fast and highly available.

Functional requirements: given a long URL, generate a short alias (optionally custom); given a short alias, redirect to the long URL; support link expiration and basic analytics (click counts). Non-functional requirements: high availability, low read latency, and uniqueness of generated keys even with many app servers writing concurrently. A rough capacity estimate matters: if the service accepts 100 new URLs per second and reads are 100x that, you need to plan for roughly 10,000 redirects per second at peak, and with 7-character base62 codes you get 62^7 (~3.5 trillion) possible codes — comfortably enough for years of writes without reuse.

Three common approaches exist. Hashing the long URL (e.g., MD5/SHA-256 truncated to 7 base62 characters) is simple but risks collisions, requiring a check-and-retry loop against the datastore. A counter-based approach assigns each URL a unique monotonically increasing ID (from a database sequence or a distributed ID generator like Snowflake) and encodes that ID in base62 — this guarantees uniqueness with no collision checks, at the cost of needing a coordinated ID source. A pre-generated key pool has a background service produce batches of unused random codes and store them in a 'key database'; app servers simply claim an unused key, avoiding any collision-check latency on the write path entirely. Most production designs favor the pre-generated pool or a Snowflake-style ID generator because both scale writes horizontally without a shared counter becoming a bottleneck.

The core mapping (short_code -> long_url, created_at, expiry, owner) is simple key-value data, so it fits well in either a relational table with an index on short_code or a NoSQL key-value/wide-column store — the choice usually comes down to whether you need strong relational features elsewhere. As the dataset grows past what a single node can serve, shard by a hash of the short code so lookups route deterministically to the owning shard without a lookup service. Because reads dominate so heavily, an in-memory cache (Redis/Memcached) sitting in front of the database absorbs the vast majority of redirect traffic; popular links follow a Zipfian access pattern, so even a modest cache achieves a high hit ratio. A CDN or edge cache can serve extremely hot redirects without hitting the origin at all.