Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced

Four real answers, and why three of them lose

"Generate a unique ID" sounds trivial until you add the constraints that actually appear: 64 bits (not 128 — it has to fit a BIGINT primary key), numeric, sortable by time, and generated at high rate without coordination between servers. Those four together eliminate most of the obvious options, and knowing which constraint kills which option is the real content here.

Option 1 — Multi-master replication

Use the database's own auto_increment, but instead of stepping by 1, step by k, where k is the number of database servers. Server A yields 1, 3, 5…; server B yields 2, 4, 6…. IDs now scale with the number of servers. Three drawbacks sink it:

• Hard to scale across multiple data centers.
• IDs do not go up with time across servers — server A's ID 101 may be minutes newer than server B's 102, so you cannot sort by ID.
• It does not scale well when a server is added or removed — k changes, and the whole stepping scheme has to be re-planned without ever colliding with IDs already issued.

Option 2 — UUID

A 128-bit value generated independently on every server, with no coordination at all. The collision probability is famously negligible: generating 1 billion UUIDs per second for about 100 years gives roughly a 50% chance of a single duplicate. Each web server just holds its own generator, which is genuinely elegant to operate — nothing to synchronize, nothing to fail.

It loses on the constraints, not on the idea: 128 bits when the requirement is 64; not time-ordered; and not numeric (09c93e62-50b4-468d-bf8a-c07e1040bfb2). Worth knowing the practical cost of that second point: random IDs as a primary key scatter inserts across a B-tree index instead of appending to the end, which fragments pages and hurts write throughput on exactly the table you were trying to scale.

Option 3 — Ticket server

Flickr's approach: a single database server with a centralized auto_increment hands out IDs to everyone. IDs are numeric and monotonic, and it is genuinely easy to implement — a good answer for small-to-medium scale, and worth saying so rather than dismissing it.

The cost is a single point of failure: if the ticket server is down, every system that needs an ID stops. You can run multiple ticket servers, but then you have reintroduced the data-synchronization problem you were avoiding — two ticket servers must not issue the same number, which is the original problem again.

Option 4 — Twitter Snowflake

Rather than generating an ID as one opaque number, divide the 64 bits into sections that each carry different information. That is the whole trick, and it works because uniqueness stops being a global property that needs coordination and becomes a composition of things each machine already knows locally: what time it is, who it is, and how many IDs it has issued this millisecond.

The bit layout, section by section

• Sign bit — 1 bit. Always 0, reserved. It exists so the value is always a positive signed 64-bit integer, which is what makes it safe in languages and databases that have no unsigned type.
• Timestamp — 41 bits. Milliseconds since a custom epoch. Twitter's default epoch is 1288834974657 (Nov 4, 2010, 01:42:54 UTC). Because this section sits in the most significant position, larger timestamps produce larger IDs — which is exactly what makes the IDs sortable by time.
• Datacenter ID — 5 bits → 2⁵ = 32 datacenters.
• Machine ID — 5 bits → 32 machines per datacenter.
• Sequence number — 12 bits → 2¹² = 4,096 values. Incremented per ID generated on that machine, and reset to 0 every millisecond. It is 0 unless more than one ID is generated within the same millisecond on the same server.

Datacenter and machine IDs are chosen at startup and fixed thereafter. This deserves emphasis because it is the design's one operational landmine: an accidental change — or worse, two machines configured with the same pair — produces duplicate IDs, silently, for as long as both are running. The uniqueness argument depends entirely on those 10 bits being genuinely unique across the fleet, so assigning them by hand does not scale and assigning them from a config template is how collisions happen.

The arithmetic worth being able to do live

Lifespan: 2⁴¹ − 1 = 2,199,023,255,551 ms. Divide by 1000, then 3600, then 24, then 365 and you get about 69 years. So the generator works for 69 years from its epoch — which is why choosing a custom epoch near your launch date matters: it buys back every year since 1970 for free. After 69 years you need a new epoch or a migration.

Throughput: 4,096 IDs per millisecond per machine = ~4.1 million IDs/second per machine. With 32 × 32 = 1,024 machines, the theoretical ceiling is about 4.2 billion IDs/second — far beyond almost any real requirement, which is a hint that the default split allocates more bits to concurrency than most systems need.

Which ID scheme, when

Scheme	Bits	Time-sortable	Coordination	Choose it when	Fails when
DB auto_increment (single)	64	Yes	Centralized	One database, moderate rate — the boring right answer	The DB is sharded, or write rate exceeds one node
Multi-master step-k	64	No	Config-level	You already run k masters and only need uniqueness	You need ordering, or the fleet size changes
UUID (v4)	128	No	None	Client-side generation, offline creation, no size limit	64-bit keys required; index locality matters
Ticket server	64	Yes	Centralized	Small/medium scale wanting clean numeric keys	You cannot tolerate a single point of failure
Snowflake	64	Yes	None at runtime	Sharded, high-rate, needs sortable numeric IDs	You cannot guarantee unique machine IDs or monotonic clocks

Note the honest shape of this table: Snowflake is not strictly best. It moves the coordination problem from runtime (a ticket server you must keep up) to deployment time (machine IDs you must assign uniquely) and adds a dependency on clocks. If you run one database and a few thousand writes per second, auto_increment is the correct answer and Snowflake is over-engineering.

The clock problem, stated properly

The design assumes ID-generating servers share a clock. That assumption is not reliably true — not just across machines but even across cores on one machine. NTP is the standard mitigation, and it is a mitigation rather than a fix: NTP can step a clock backwards. If the clock moves back, the timestamp section repeats, and combined with a reset sequence counter you can re-issue an ID that already exists. Production generators therefore refuse to issue IDs while the clock is behind the last-seen timestamp — they block or error rather than emit a possible duplicate. That is a deliberate availability-for-correctness trade, and it is the part most candidates never mention.

Pitfalls

• Duplicate machine IDs from cloned VMs, container images, or copy-pasted config. The most common real-world Snowflake failure, and it is silent.
• Assuming IDs are gapless. They are sortable, not sequential — you cannot infer counts, and exposing them still leaks approximate creation time and volume.
• Clock rollback from NTP correction or a VM snapshot restore, producing duplicates unless the generator explicitly guards against it.
• Using the default 1970 epoch and throwing away decades of the 69-year budget.
• Sorting by ID across datacenters and calling it a global order. Two IDs from different DCs in the same millisecond order by DC bits, not by real time — the ordering is approximate, not authoritative.
• UUIDs as clustered primary keys at scale, then blaming the database for write amplification.

Cost model — what dominates the bill

An ID generator is one of the few components whose direct cost is essentially zero and whose indirect cost can be large, so the interesting analysis is entirely about which option shifts cost elsewhere.

Snowflake generation is arithmetic on local state — no network call, no storage, so it can run inside the application process and costs nothing beyond the CPU it already has. A ticket server, by contrast, adds a network round trip on the critical path of every insert. At 10,000 inserts/second with a 1 ms in-datacenter round trip, that is 10 seconds of aggregate added latency per second of traffic — requiring roughly 10 concurrent in-flight requests continuously, plus a highly-available database you now must run, patch and monitor purely to hand out integers. Call it a small always-on instance pair, on the order of $100–200/month, plus the operational attention of a tier-1 dependency.

The genuinely expensive choice is UUID as a clustered primary key. Random keys scatter inserts across the index, so pages split and the working set stops fitting in RAM. Doubling the index size and losing insert locality can mean provisioning a materially larger database instance and more IOPS for the same workload — and 128-bit keys also double the size of every foreign key and index that references them, across every table.

Dominant line item: none for Snowflake itself. For the alternatives, the ticket server's availability infrastructure, or the database over-provisioning caused by non-sequential keys.

Lever: if you need UUID-style independence and 64-bit index locality, a time-ordered ID (Snowflake, or a time-prefixed UUID variant) restores insert locality — the cheapest optimization here is choosing the ID shape correctly once, because changing a primary key later is a full-table migration.

Operability: the fingerprints of a broken ID generator

The failure modes are unusually nasty because the symptom appears far from the cause. Primary-key violations appearing in bursts on unrelated tables is the signature of duplicate machine IDs — two hosts issuing the same ID space, so collisions cluster in the same millisecond windows and stop when one host is drained. Correlate the colliding IDs' decoded machine bits and the answer is immediate, which is why being able to decode your own IDs is an operational requirement, not a party trick.

ID generation stalling or erroring while CPU is idle is the clock-rollback guard doing its job: NTP stepped the clock back and the generator is refusing to emit until real time catches up. It looks like an outage and it is actually correctness being defended, so the runbook must say so or someone will "fix" it by removing the guard. Sequence-number exhaustion within a millisecond shows up as brief spin-waits under burst load — the generator waiting for the next millisecond; if that is frequent, you have outgrown 12 sequence bits on one machine and should add machines rather than bits.

IDs that suddenly jump far into the future mean a clock skewed forward, and unlike a rollback this one is unrecoverable in a subtle way: you have permanently consumed timestamp space, and IDs issued now will sort after legitimately-later records forever. Signals worth having: decoded machine-ID distribution across the fleet (each should be unique and stable), clock offset per host from NTP, sequence-exhaustion counter, and generator error rate split by cause (rollback guard versus everything else).

---

Re-authored for this guide from the Alex Xu Vol. 1 unique-ID chapter (ticket server after Flickr's engineering blog; Snowflake after Twitter); bit-layout diagram hand-authored to scale as SVG. Complements the existing "Designing Unique ID Generator" problem page by comparing all four candidate schemes and their failure modes.

I just read a lesson on **Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced** (System Design) and want to truly understand it. Explain Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced from first principles using ONE vivid real-world analogy and a visual mental model — draw it as ASCII art or a clear step-by-step diagram — with a concrete example using real numbers. Then ask me one question to check I got the mental picture, and wait for my reply. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Teach me **Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced** interactively. Ask me ONE guiding question at a time, wait for my answer, and adapt to my confusion — build the idea with me step by step instead of explaining it all at once. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Quiz me on **Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced** with 5 questions, easy to tricky, ONE at a time. Tell me if each answer is right; at the end, explain clearly what I got wrong and why. If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.

Help me remember **Distributed Unique IDs — Ticket Server, Multi-Master & Snowflake Trade-offs, Traced** for the long term: give the one-sentence intuition, a memorable hook/mnemonic, a tiny worked example, and 3 active-recall flashcards (Q -> A). If you're unsure or a claim isn't standard, say so and reason from first principles instead of guessing.