📦Key Takeaways
- 1Dynamo (Amazon): leaderless, eventual consistency, consistent hashing, vector clocks — inspired DynamoDB, Riak, Cassandra
- 2Bigtable (Google): sorted string table, column-family store — inspired HBase, Cassandra's data model
- 3Spanner (Google): globally distributed, strongly consistent via TrueTime — first 'NewSQL' database
- 4Each system makes different trade-offs on the CAP spectrum based on its primary use case
Papers That Shaped Modern Databases
Three seminal papers from Amazon and Google defined the design space for distributed storage: Dynamo (2007), Bigtable (2006), and Spanner (2012). Understanding these systems means understanding the trade-offs behind DynamoDB, Cassandra, HBase, CockroachDB, and most modern distributed databases.
Landmark Systems Compared
| System | Consistency | Model | Key Innovation | Inspired |
|---|---|---|---|---|
| Dynamo | Eventual (tunable) | Key-value, leaderless | Consistent hashing, vector clocks, sloppy quorum | DynamoDB, Riak, Cassandra |
| Bigtable | Strong (single-row) | Column-family, sorted | SSTable/LSM tree, tablet servers | HBase, Cassandra data model |
| Spanner | Strong (global) | Relational, distributed | TrueTime (GPS + atomic clocks), 2PC at global scale | CockroachDB, YugabyteDB |
System Deep Dives
Key insight: for Amazon's shopping cart, availability > consistency. It's better to show a stale cart than an error.
Techniques: Consistent hashing (data partitioning), vector clocks (conflict detection), sloppy quorum (R+W>N for tunable consistency), hinted handoff (availability during failures).
DynamoDB is the commercial evolution — managed, with stronger consistency options, but same philosophical DNA.
Designed for petabyte-scale structured data at Google (web indexing, Google Earth, Google Finance).
Architecture: Tablet servers store row ranges. Data stored as SSTables (sorted string tables) in GFS. Master assigns tablets to servers.
Influenced the entire NoSQL movement: HBase is an open-source clone; Cassandra borrowed the data model.
The 'impossible' database: globally distributed AND strongly consistent.
Key innovation: TrueTime API provides bounded clock uncertainty (~7ms) using GPS and atomic clocks in every datacenter.
Commit protocol: acquire locks, get TrueTime timestamp, wait out the uncertainty interval, then release. Guarantees linearizability.
CockroachDB and YugabyteDB are open-source Spanner-inspired databases (using HLC instead of TrueTime).
Advantages
- •Dynamo's techniques are widely applicable
- •Spanner proved global strong consistency is possible
- •Understanding these papers explains all modern distributed DBs
Disadvantages
- •TrueTime requires specialized hardware
- •Dynamo's eventual consistency creates application complexity
- •Each system optimizes for specific workloads
🧪 Test Your Understanding
What is Spanner's key innovation for achieving global strong consistency?