Claims-to-Experiments Matrix

Copy to notes/claims-matrix.md. Fill this in during Phase 0, before you build anything.

Two rules, and the second is the one that saves time:

1. Every claim maps to at least one experiment.
2. Every experiment supports at least one claim. An experiment supporting no claim is an experiment you should not run.

Fill in the falsification column before running anything. Deciding in advance what counts as success is what separates evaluation from rationalization — when a result comes in ambiguous, you want to have already decided.

Review with your advisor in Phase 0 and again at the week-24 mid-point.

---

Thesis statement

By [mechanism], [system] achieves [quantified improvement] in [metric] on [workload class] compared to [named baseline], at a cost of [honest overhead].

Claims

Status values: not started / harness ready / data collected / figure done / written up

Baselines

Four kinds; the production default and the strongest prior work are mandatory.

If the strongest prior work is not obtainable, write here how you will approximate it and what caveat you will state in the thesis. Silently omitting this comparison reads as avoidance.

Workloads

Two or three real, one or two synthetic. One sentence of justification each — readers should never wonder why a benchmark is present.

Workload where I expect to do badly: (You must have one, and you must report it.)

Overhead accounting

Every one of these needs a number in the thesis, not a hand-wave.

• Inference latency in the critical path, at the tail not the mean
• Feature collection cost (often larger than inference, often forgotten)
• Memory footprint of model and features
• Training / retraining cost and frequency
• Cold-start period and fallback behavior
• Cache pollution

Ablations planned

Sensitivity sweeps planned

Fallback plan

If the main idea does not work, what do I submit? Write this in week four.