Disaggregation & Elasticity

• Every database holds data pages and the redo log.
• They are informationally redundant — the log rebuilds every page.
• For forty years we replicated both.
• Cloud datacenter: fast network, storage that runs code.
• Aurora: ship only the log; materialize pages in storage.

• Redo log record — plus the full 16 KB page.
• Second page copy: the double-write buffer.
• Binlog for replication, plus metadata.
• Networked block storage replicates each write again.
• A hot standby repeats the entire set — synchronously.

reduction in write traffic when only redo records cross the network — a few hundred bytes of intent, not megabytes of pages.

more transactions than mirrored MySQL over a 30-minute SysBench write-only run — with 7.7× fewer I/Os per transaction.

• Primary keeps InnoDB’s parser, optimizer, transactions, buffer pool.
• But it only ever writes redo records, tagged by LSN.
• Records stream to six replicas per 10 GB segment.
• Commit: volume durable LSN passes commit LSN on quorum.
• No checkpointing — storage applies the log continuously.

• Quorum rules: V_r + V_w > V, and V_w > V/2.
• 2/3 satisfies both — most systems used it.
• But AWS failures are correlated: a whole AZ can vanish.
• Node failures continue at background rates regardless.
• Design target: survive AZ+1 — an AZ plus one node.

to re-replicate a lost 10 GB segment over a 10 Gbps link. A second failure only hurts inside that window, in the same protection group. Big segments would stretch it to hours.

• Traditional: replay log from checkpoint before opening.
• Downtime proportional to log backlog — minutes.
• Aurora: storage is always applying the log.
• Re-establish durable LSN by quorum reads; open immediately.
• Pages reach consistency lazily, on first touch.

• Cloud services: catalog, transactions, optimizer, access control.
• Virtual warehouses: stateless EC2 clusters; SSD is just cache.
• Storage: S3 — slow first byte, unbounded bandwidth, eleven nines.
• Each layer scales independently; nothing ever stresses S3.
• Kill a warehouse mid-query: lose cache, never data.

• Files of 50–500 MB uncompressed data each.
• Internally columnar — a PAX-style hybrid layout.
• Writes never modify files: write new, swap metadata.
• Updating one row rewrites that row’s partition.
• Sounds wasteful — until you see what immutability buys.

• Pruning: per-column min/max zone maps skip partitions.
• 99%+ of I/O eliminated before bytes leave S3.
• No B-tree indexes — pruning over sorted-ish data is the index.
• Time travel: “as of” = resolve an older file list.
• Zero-copy clone: 50 TB table = one metadata entry.

• 100 machines × 1 hour = 1 machine × 100 hours.
• Only one of them returns your answer today.
• Coupled systems can’t exploit it — re-sharding outlasts the query.
• Stateless compute: resize = boot nodes, warm caches.
• 64 nodes × 9 min ≈ one node × 9.6 hours.

• Production telemetry confirms the 2016 design.
• Real workloads are violently bursty.
• SSD caches hit high rates holding a sliver of data — skew helps.
• Idle 95% of the day → 5% of list price, via auto-suspend.
• Unsolved: intermediate data — shuffle and spill in big joins.

• Unmodified Postgres, stateless — no data directory that matters.
• WAL streams to three safekeepers: Paxos-style 2/3 quorum.
• Commit = quorum of safekeepers has the bytes.
• Pageservers build delta + image layers, cache on NVMe.
• Everything tiers to S3 as immutable layer files.

GetPage@LSN(tenant, timeline, rel, blkno, lsn)
  → 8 KB page image

• A page requested at a point in logical time.
• Newest image layer ≤ LSN, plus deltas between.
• Pages addressable by (block, time) → branches are trivial.

to branch a 2 TB database — a branch is just (parent timeline, LSN) plus its own WAL after. A hundred branches diverging by megabytes cost a few hundred megabytes total.

• Human OLTP: a diurnal sine wave. Provision for the peak.
• Agents: flat zero, then 400 sessions in two seconds.
• Hot for ninety seconds, then gone. Duty cycle under 1%.
• Provisioned-for-peak at 1% duty = 100× cost penalty.
• Only scale-to-zero compute prices that curve honestly.

• Never rehearse ALTER TABLE on the production timeline.
• Branch → apply → check → promote or delete.
• Human scale: a weekend restoring dumps to staging.
• Agent scale: hundreds of experiments a day — one second each.
• Isolation for a client that ships code faster than review.

• 1981: buffer pool = megabytes of RAM before a local disk.
• Now: shared buffers ~100 ns → NVMe ~100 µs → S3 (ms).
• S3: ~$0.023/GB·month, request-priced GETs.
• Every design choice is a buffer-pool policy at datacenter size.
• Week 2’s five-minute rule got promoted to chief architect.

• 200 bytes across 5 pages: count network bytes, mirrored MySQL vs Aurora. Which writes are on the synchronous commit path?
• Show 3/5 and 2/3 quorums each violate AZ+1. How does the vulnerability window change with 100 GB segments?
• 10,000 branches/day on a 2 TB parent: what invariant must safekeepers preserve when a pageserver dies?

• Amazon Aurora — Verbitski et al., SIGMOD 2017. §2–3: Figure 2’s write accounting; derive 4/6–3/6 from AZ+1 yourself first.
• The Snowflake Elastic Data Warehouse — Dageville et al., SIGMOD 2016. Three layers; §3.3 on micro-partitions and pruning.
• Building an Elastic Query Engine on Disaggregated Storage — Vuppalapati et al., NSDI 2020. Skew, cache hit rates, intermediate data.