One Size Fits None

• Commercial RDBMSs: System R descendants, tuned for 1985 OLTP.
• By 2005: streams, scientific arrays, text, warehousing.
• Move: design for one workload class alone, then measure.
• For warehousing, that engine was C-Store.
• Agents are exactly the new workload class this method demands.

C-Store’s scan speedup over the general-purpose incumbent — one to two orders of magnitude. The generalist isn’t a compromise; it’s a category error.

• Row store: tuple’s attributes contiguous — optimal per-tuple work.
• AVG(price) WHERE shipdate > … touches 2 of 16 attributes.
• 200-byte tuples, 12 useful bytes: 94% waste per cache line.
• Column store: each column contiguous; scan reads only named columns.
• 2-column query on 16 columns: 8× I/O reduction before compression.

• A column is same-typed, often-sorted, low-entropy data.
• Compresses dramatically better than interleaved rows.
• Three encodings carry most of the weight: RLE, dictionary, bit-packing.
• Worked example: a 1-billion-row lineitem table.

• shipdate, table sorted on it: ~2,500 distinct days.
• Each date repeats ~400,000 times consecutively.
• Store (value, run_length): 2,500 pairs × 8 B ≈ 20 KB.
• Raw: 4 GB (4-byte dates × 10⁹).
• Unsorted column: runs of length 1 → 2× expansion.

4 GB → 20 KB. Absurd, and real — but only on the sort column. That’s why C-Store makes sort order a first-class physical-design decision.

• Gluing tuples at the bottom rebuilds a row store, badly.
• Carry only position lists between operators.
• Predicate on an RLE run of 400,000 values: one comparison.
• AND the bitmaps; fetch price only at surviving positions.
• Stitch tuples at the top, for the few survivors.

• Stores projections: overlapping column subsets, each sorted differently.
• Queries pick the projection with the friendliest sort order.
• Redundant storage buying read speed — the RUM triangle again.
• Writes hit a small row-oriented Writeable Store, merged in batches.
• Squint: it’s the LSM shape from Week 2.

Same 40-line revenue query: row-store warehouse vs. an early Vertica cluster on cheaper hardware. Not the cache — 14 columns never read, RLE-run arithmetic on the date predicate, tuples materialized only for the final 900 rows.

• Every operator: open(), next(), close() — composes like Unix pipes.
• Each next() pulls one tuple from below.
• Beautiful, modular, memory-frugal: one tuple in flight.
• Mattered enormously when RAM was megabytes.
• On modern CPUs: catastrophically slow for analytics.

Share of CPU cycles commercial DBMSs spent on actual query work running TPC-H Q1 — ~3× lower IPC and an order of magnitude more cycles per tuple than hand-written C.

• Each next(): a virtual call, ~20–40 cycles with pipeline flush.
• Three operators → three calls per tuple, plus attribute extraction.
• Type dispatch decided per value in the interpreter.
• Hundreds of cycles of ceremony around ~5 cycles of work.
• At 10⁹ tuples: ~5 B useful cycles, 100+ B on ceremony.

• next() returns a vector of ~1000 values per column.
• Virtual-call overhead amortized: 30 ÷ 1000 = 0.03 cycles/value.
• Work done by primitives: precompiled, type-specialized tight loops.
• Type dispatch once per vector, not once per value.
• Result: TPC-H Q1 limited by memory bandwidth, not interpretation.

/* one virtual call delivered n ≈ 1024 values;
 * below: branch-free, type-specialized, cache-resident,
 * auto-vectorized (AVX2: 8 lanes per instruction) */
size_t sel_lt_int32(const int32_t *col, int32_t bound,
                    uint32_t *sel_out, size_t n)
{
    size_t k = 0;
    for (size_t i = 0; i < n; i++) {
        sel_out[k] = (uint32_t)i;     /* write position…        */
        k += (col[i] < bound);        /* …keep it only on match */
    }
    return k;   /* positions, not copies */
}

• Primitive returns qualifying positions, not copied survivors.
• Downstream primitives take (data, sel, n).
• Tuesday’s late materialization, recurring at micro-scale.
• Branch-free k += (cond) sidesteps the branch predictor.
• A 50%-selective predicate is the predictor’s worst nightmare.

• HyPer (Neumann, 2011): JIT-compile each query to LLVM machine code.
• Fuses a whole pipeline into one loop; tuple stays in registers.
• Kersten et al. 2018: built both in one codebase — neither dominates.
• Compilation wins computation-heavy, long pipelines; vectorization wins memory-bound scans.
• Usually within 2× of each other; both ~100× over Volcano.

• For short queries, LLVM passes can cost more than execution.
• HyPer’s fix: run quick-and-dirty code, swap in optimized mid-flight.
• Umbra: a custom IR to make codegen itself near-instant.
• When queries are numerous, short, speculative — fixed startup cost is the bill.

• Analyst: ~a dozen considered queries per hour. Agent: hundreds.
• Scan-dominant: profiling and hypothesis tests are wide aggregations.
• Speculative: most queries are dead ends — demand cheap, not fast-but-expensive.
• Impatient in aggregate: per-query overhead multiplies by query count.
• Fit: vectorized engines, precompiled primitives, near-zero startup (DuckDB embedded).

• Why does RLE on an unsorted column expand instead of compress?
• What bounds the vector-size sweet spot from each side?
• For bursts of 200–500 speculative agent queries, what metric do you optimize? (Hint: not single-query latency.)

• “One Size Fits All”: An Idea Whose Time Has Come and Gone — Stonebraker & Çetintemel, ICDE 2005. Read for the method, not the predictions.
• C-Store: A Column-oriented DBMS — Stonebraker et al., VLDB 2005. Focus on §3–4: projections, sort-order-dependent compression, WS/RS split.
• MonetDB/X100: Hyper-Pipelining Query Execution — Boncz, Zukowski & Nes, CIDR 2005. The U-shaped vector-size curve is the whole lecture in one figure.