Candidate Topics

Twelve topics at the OS/AI intersection, tiered by whether a masters-scale contribution still exists. Literature verified 26 August 2026; see 02-crowdedness-evidence.md for who occupies each area.

Every topic below is stated in its reframed form. The obvious framing of each one is taken, and in several cases the reframing is the entire contribution — so read the "how it goes wrong" notes before you fall in love with a title.

Thirteen further topics from a second literature pass are in 04-new-candidates.md, covering three areas this file does not reach: the memory and storage hierarchy for local inference, agents as an OS workload beyond sandboxing and rollback, and AI-for-systems outside the fashionable subsystems.

Corrections from the second pass

Made the same day, from the reports added in reports/. One entry below changed tier and one was weakened.

#3, rollback of escaped effects, moved to Tier 3. The exception this entry was built on is now claimed by name: Externalization Barriers: An OS Abstraction for Untrusted Agent Exploration (Hu, Mohammadi, Goel, Bindschaedler — MPI-SWS and Toronto) appears on the AgenticOS @ SOSP 2026 program for 29 September, alongside The Irreversibility Budget from the same group. See the entry for what, if anything, is left.

#2, sandbox policy for agent code, weakened but still Tier 1. Non-stationarity survives as a gap. But the surrounding lane closed: OS-level information-flow control is taken by ActPlane, sandbox density by GKE and AgentENV, and idle-window scheduling by six separate systems. See 04-new-candidates.md for the evidence.

The cause is structural and worth understanding rather than just noting. A dedicated venue — AgenticOS, the Workshop on OS Design for AI Agents — has now run twice in six months, with three well-resourced groups systematically enumerating the same abstraction gaps a student would. The lane did not get incrementally busier; it acquired a conference.

The three lanes

Work here splits into three research programmes that look similar from outside and are completely different to execute. This is a finer cut of the A/B/C angles in guide/01-scoping.md.

Systems for AI. Classic OS problems — memory, scheduling, isolation — re-asked for AI workloads. Highest impact, most competitive, needs GPUs. You are racing well-funded industry labs, and in several areas they have already won.

AI for Systems. Replace or assist an OS heuristic with a learned model. Cheap to run: traces and simulators, not clusters. The hard part is beating a well-tuned heuristic once you charge yourself for inference cost — see the learned-cache entry for what happens when somebody finally does that accounting properly.

Agents as a workload. AI agents execute untrusted generated code, backtrack constantly, and run for hours. The least settled lane, the least infrastructure required, and the only one where the fundamental abstractions are still in dispute.

Requirement scales

Used in the requirement lines below.

---

Tier 1 — Live options

A gap survives, it is reachable on hardware a student can get, and it is not already somebody's published contribution. Start here.

1. Learned thread placement on heterogeneous CPUs via sched_ext

AI for Systems · Compute 0 · Kernel 3 · ML 2 · 12 months · top-tier plausible · least crowded topic in this set

The question. Linux sched_ext lets you ship a full CPU scheduler as a BPF program without patching the kernel. The academic literature on learned placement there is nearly empty — but production engineering has taken the easy version, so beating EEVDF is no longer a contribution. The real question: can a PMU-derived online predictor of a thread's core-type sensitivity — its IPC and memory-boundedness ratio between P- and E-cores — outperform scx_lavd's hand-tuned capacity-invariant load balancing? Or do those invariants already capture all the available signal?

Smallest defensible result. A learned per-task core-sensitivity predictor inside a BPF scheduler, benchmarked against scx_lavd and scx_p2dq rather than the in-tree default, reporting joules alongside latency.

How you measure it. Interactive-job latency percentiles at fixed batch throughput; joules per unit of work; migration count; scheduling overhead in cycles.

What you build on. sched_ext (SCX) on kernel 6.12+, on any Meteor Lake or newer laptop or Arm big.LITTLE board. This is the only topic here where a laptop puts you at parity with the people doing the work.

How it goes wrong. Your baseline is not EEVDF. Ching-Chun Huang's ML load balancer already beat it by about 10% on kernel compile with 77% fewer migrations, and Meta shipped a 16-hidden-unit network predicting CPU yield. scx_lavd already builds per-core-type scheduling domains with an energy model, in production on Steam Deck. Benchmark against those or the result is meaningless. Accept up front that hand-tuned invariants may simply win — that negative result is publishable but you must be willing to write it. Kernel debugging will eat months and a bad scheduler hangs the machine; develop in a VM.

2. Learning least-privilege sandbox policies for agent-generated code

Agents · Compute 0 · Kernel 2 · ML 1 · 9 months · top-tier plausible

The question. Learning a seccomp profile from a representative run is productized for microservices, and every such tool assumes a stationary workload. An agent writes new code each run, so its syscall footprint is drawn from an open-ended distribution. How do you build a least-privilege policy for a workload whose behaviour you have not seen yet?

Smallest defensible result. A measurement of how much an agent's syscall and file-access footprint varies across runs of the same task, then a policy that adapts or escalates rather than failing closed — routing unseen calls to a supervisor instead of killing the process.

How you measure it. Footprint variance across runs; false-denial rate on held-out runs; attack-blocking rate on a red-team set; supervisor escalation frequency and its latency cost.

What you build on. Landlock plus seccomp-bpf with SECCOMP_RET_USER_NOTIF for the escalation path; Sandlock as the enforcement substrate; eBPF tracing to record behaviour.

How it goes wrong. Do not pitch this as "generate a seccomp profile from traces" — Santa, sandprint, and the Kubernetes Security Profiles Operator already do that, and sandprint's own README concedes the stationarity limitation outright. Non-stationarity is the open part. You must build your own corpus of agent runs; start in week one, because the corpus gates everything downstream.

Second-pass note, 26 August 2026. Non-stationarity survives, but the surrounding lane closed, so the enforcement half of this topic is gone and only the measurement half is yours. ActPlane (arXiv 2606.25189, code at eunomia-bpf/actplane) now does eBPF-hooked multi-label information flow with a policy DSL compiled to BPF, semantic feedback to the agent instead of silent blocking, and ships a benchmark testing whether enforcement survives subprocess and shell-out evasion — pre-empting the critique you would have levelled at framework-level defences. Note also that AgentCgroup (arXiv 2602.09345) reports agent resource demand varying 20× across tasks and 1.8× across runs of the same task, which is your non-stationarity observation arriving from the resource-management side. Confidence that the syscall-drift measurement is unclaimed is low — it is genuinely hard to search for.

3. Rollback when effects have already escaped the sandbox

Moved to Tier 3 on 26 August 2026. See below.

4. Fair-share GPU scheduling over KV cache capacity

Systems for AI · Compute 1 · Kernel 1 · ML 1 · 9 months · top-tier plausible · strongest single-GPU option

The question. Every published GPU fairness mechanism meters the wrong resource. VTC and the deficit-round-robin schedulers meter tokens or compute time; none meters KV cache capacity. Can a scheduler enforce proportional-share fairness over cache footprint — and does a tenant issuing a few long-context requests defeat token-based fairness by monopolizing cache while barely touching its token budget?

Smallest defensible result. A demonstration that token-fair schedulers are cache-unfair — one tenant starving another while staying inside its token allocation — then a scheduler that meters cache footprint instead.

How you measure it. Cache-share deviation from target; p99 TTFT inflation of the starved tenant; aggregate goodput against a token-fair baseline.

What you build on. vLLM with per-tenant cache accounting you add yourself, since vLLM currently exposes cache usage per model rather than per tenant. MPS or SM masking for the compute side.

How it goes wrong. Do not build an argument on GPUs being unable to preempt — REEF demonstrated microsecond-scale GPU preemption in 2022 and that premise is dead. The compute-fairness and interference-matrix work is also finished, by VTC, Orion, USHER, and Bullet. Cache-capacity fairness is the only part nobody has taken. Note that Bullet needed libsmctrl, an out-of-tree SM-masking library, so the compute side may cost you more plumbing than expected.

5. Compute-normalized re-evaluation of LLM-guided kernel fuzzers

AI for Systems · Compute 0 · Kernel 2 · ML 1 · 9 months · mid-tier

The question. At least six groups shipped LLM-driven kernel fuzzers in eighteen months and crash deduplication is separately taken, so building another is out. What nobody has asked: do the reported coverage gains survive when the LLM's own inference compute is converted into CPU-hours and handed to plain syzkaller instead?

Smallest defensible result. A compute-normalized re-evaluation of two or three published LLM fuzzers against a syzkaller baseline given the same total compute including inference. KernelGPT charges 192 CPU-hours of fuzzing and nothing for inference.

How you measure it. Unique kernel coverage per total CPU-hour including inference cost; the crossover point at which extra plain fuzzing wins.

What you build on. syzkaller, KCOV, a KVM box, and the public artifacts of the fuzzers you re-evaluate.

How it goes wrong. This is a critique paper and you must be willing to publish one. You also cannot match the field's statistical power — KnitFuzz averaged ten runs across four fuzzers on a 475 GB host, on the order of ten thousand CPU-hours, while one 16-core box gets you a single arm. That constraint is precisely why the narrower accounting claim is the only defensible framing left. Note the pleasing asymmetry: this exact accounting trick is settled in the caching literature and unpublished in fuzzing.

6. Where prefill/decode disaggregation stops paying

Systems for AI · Compute 2 · Kernel 0 · ML 1 · 12 months · top-tier plausible

The question. The general disaggregation question was settled in 2024 and it is now the production default. What survives is the inverse regime: IBM's EuroSys 2026 revisit finds the benefit is not guaranteed and depends on load and KV transfer medium, while every industry result assumes NVLink or InfiniBand. At what request rate, context length, and interconnect bandwidth does disaggregation become a net loss on commodity PCIe-only hardware — and can a controller detect that crossover online and fall back to colocation?

Smallest defensible result. A characterization of the PCIe-only crossover point on two commodity GPUs, then a controller that switches modes when it detects the regime change.

How you measure it. SLO attainment on each side of the crossover; KV transfer time as a fraction of TTFT; regret against an oracle that always picks the better mode.

What you build on. vLLM or SGLang, which support disaggregation natively now, plus Vidur for simulation instead of writing your own simulator.

How it goes wrong. Do not propose the original framing; it is a 2024 replication. Vidur already provides a profiling-calibrated simulator at under 9% error. The low-bandwidth regime is unstudied only because nobody with a cluster cares about it, which is exactly why it is available to you — but it also means you must argue that the regime matters, not just that it is unmeasured.

---

Tier 2 — Narrow or contested survivors

Something survives, but it is thin, or it changes what kind of thesis you are writing, or somebody well-resourced is actively closing it. Viable with clear eyes.

7. Tiered memory placement for mixture-of-experts models

Systems for AI · Compute 1 · Kernel 2 · ML 1 · 12 months · mid-tier

The premise is taken: that LLM weight and KV access is deterministic enough for proactive placement to beat reactive kernel tiering is the headline claim of both ITME and HybridGen. What survives is the case where the premise fails. For MoE models, per-token expert routing is data-dependent rather than deterministic — so does offline layer profiling still beat PACT-style online criticality tiering, and at what fast-tier capacity ratio and contention level does it stop winning? A measurement study that tolerates a negative result, which is what makes it survivable.

Benchmark against PACT and Memtis, not default Linux: TPP has been in mainline since 5.18, so beating the default proves nothing. Two warnings. The MVP in the obvious framing is a completed 2025 Virginia Tech masters thesis — same tool, same NUMA emulation, same baseline. And on hardware, emulation is still accepted for OS-tiering papers, but LLM-tiering papers now use real CXL silicon and reviewers will cite "Demystifying CXL Memory" at you, which argues emulation differences are large enough to force revisiting prior conclusions.

8. Multi-GPU snapshot and restore for serverless inference

Systems for AI · Compute 1 · Kernel 2 · ML 0 · 9 months · mid-tier

Single-GPU cold start is closed: gCROP hit sub-100ms restore of a 3.1 GB model including the snapshot storage-cost question back in 2024, and GPU state capture has since become vendor tooling in cuda-checkpoint, CRIU 4.0, and NVIDIA Dynamo Snapshot. The one unowned piece is multi-GPU — can a tensor-parallel worker group be snapshot and restored coherently, rebuilding NCCL communicators and IPC-shared KV buffers, and how does restore latency scale with tensor-parallel degree?

Two hard problems. NVIDIA is actively working on exactly this, so your window may close mid-thesis. And the binding resource is storage rather than GPUs: restore time is snapshot size divided by storage bandwidth, so shared NFS or object storage destroys the result. Driver 550+ is required, 580+ to restore onto a different physical GPU. Do not reimplement the mechanism; there is no contribution left in it.

9. Learned cache replacement inside the Linux page cache

AI for Systems · Compute 0 · Kernel 2 · ML 2 · 9 months · mid-tier

The accounting study — charge the learned policy for its own CPU cost — is published and settled. 3L-Cache compared twelve policies across 4,855 traces with overhead measured relative to LRU, and S4-FIFO then used that accounting to defeat learned caches outright by keeping the model off the data path. SIEVE had already shown a non-learned policy matching learned ones.

One slice survives, and it changes the thesis type: does any learned eviction advantage hold up inside the Linux page cache, where decisions cost nanoseconds rather than CDN control-plane microseconds? Replicating LeCaR, LRB, and 3L-Cache under cache_ext with charged overhead is unclaimed. Note what changed — this is now a kernel-engineering thesis, not the light trace-driven measurement study it looks like. Resources are not the constraint; traces are public and one large-memory machine suffices. Novelty is.

---

Tier 3 — Recommend dropping

The contribution is taken, the hardware bar has risen out of reach, or the remainder is not an OS thesis. Listed so you can defend the decision to your advisor in one sentence rather than rediscovering it in month six.

3. Rollback when effects have already escaped the sandbox

Agents · Compute 0 · Kernel 2 · ML 1 · demoted from Tier 1, 26 August 2026

The original framing: local rollback for agent sandboxes is solved, every system that solved it explicitly gives up on effects that left the sandbox, so what should rollback mean when part of the state cannot be un-written?

What changed. That exception is now claimed by name. Externalization Barriers: An OS Abstraction for Untrusted Agent Exploration (Hu, Mohammadi, Goel, Bindschaedler — MPI-SWS and University of Toronto) is on the AgenticOS @ SOSP 2026 program for 29 September 2026. The title states the claim precisely: an OS abstraction for the barrier between reversible in-sandbox exploration and irreversible external effect. The same group is presenting The Irreversibility Budget: Fleet-Level Risk Accounting and Admission Control for Agent Operating Systems, and Bindschaedler is giving the workshop keynote. Independently, the Fork, Explore, Commit slides (AgenticOS @ ASPLOS 2026) list "external side effects (network, IPC) not rolled back" as their explicit limitation — so two groups converged on this boundary as the next thing.

Why this is worse than an ordinary scoop. The PDF is not public and will not be until the workshop runs. You would be writing against a paper you cannot read, from a group with more resources, in a lane where they hold the keynote.

Is anything left? Possibly the characterization half — how often real agent runs produce irreversible external effects, and of what kind — since a measurement is harder to scoop than a mechanism. But you would be betting that a paper you cannot read did not measure the obvious thing. Re-read the program in October, when the PDFs exist, before reconsidering.

The original vocabulary warning stands regardless and applies to anything in this space: compensating transactions, sagas, workflow recovery, exactly-once semantics, and the outbox pattern have addressed undoing committed effects since the late 1980s.

10. KV-cache eviction and offload for long-context serving

Compute 3 · very crowded

DirectKV sets the bar using NVLink-C2C on GH200/GB200, OrbitFlow solves SLO-aware placement with an online ILP solver, and KVDrive covers GPU/DRAM/SSD tiering. Competing requires Grace Hopper class hardware. Drop unless you have it.

11. Power-capped inference scheduling

Compute 3 · very crowded

Four or more groups published on this exact knob set in eighteen months, and PALS matches the obvious contribution almost line for line. The competitive hardware bar is 4–8 A100s or DGX nodes. A thin slice remains — nobody treats the model variant as a third knob alongside frequency and batch size, and all published work is GPU-only with CPU-side energy ignored — but it is a weak thesis. Single-consumer-GPU energy work does get published, as measurement studies rather than systems papers, so this is only viable reframed as Angle C with the lower ceiling accepted.

12. LLM synthesis of verified eBPF policies

Compute 0 · very crowded

Kgent synthesizes eBPF from natural language, verifies with symbolic execution over LLM-generated Hoare annotations, reports 80% correctness, and released the benchmark corpus. The obvious framing is done. One seam remains: synthesis and comprehension come from the same model, so their errors may correlate, and constructing prompts where a model's synthesis and its own specification fail together would attack that soundness assumption. But that is a verification-soundness question that reads as programming languages rather than operating systems — wrong department unless your advisor sits across both.