Masters portal

Topic selectiontopic-selection/reports/local-inference-memory-hierarchy

Report — OS × local LLM inference, memory and storage hierarchy

Second pass, 26 August 2026. Verbatim, including its own verification caveats.

Areas checked: model weight loading through the storage stack; page-cache pollution from model loading; memory pressure, swap and the OOM killer under local inference; transparent hugepages and promotion/demotion; NUMA for CPU-only inference; unified memory on integrated-GPU and Apple Silicon systems; anything else reachable without a cluster.

Headline: areas 1 and 5 are dead. Areas 4 and 6 have their obvious contribution taken. Areas 2 and 3 have survivable but narrow gaps, and in both cases the surviving gap is on the victim side (what inference does to the rest of the machine) rather than the inference side (making inference faster) — a much less fashionable framing and therefore genuinely less occupied.

Two methodological warnings first. Several of the strongest incumbents here are not papers, they are merged upstream code. A reviewer will say "llama.cpp already does this" and that is fatal even though there is no citation for it. And a large fraction of 2026 work in this space is arXiv-only with no confirmable review, while some of it (Zenodo, MDPI) is refereed at a tier you cannot rely on. Peer-review status could not be confirmed for anything marked [preprint].

Area 1 — Weight loading through the OS storage stack

Rating: crowded. Dead. Do not pursue.

The exact contribution — a kernel readahead/eviction policy tuned for weight loading, evaluated without GPUDirect Storage — was published at a top-tier refereed storage venue four months ago.

  • Accelerating Model Loading in LLM Inference by Programmable Page Cache. Yubo Liu et al., USENIX FAST '26 (refereed). page · PDF. Builds PPC, a programmable page-cache framework, and MAIO, a loading policy on top of it: interruptible prefetching (prefetch from the miss position to the end of an I/O group, cancel and re-issue when front-end I/O overtakes), XPU-affinity placement, and a Burn-after-Reading eviction policy. Its motivating measurement is that kernel readahead achieves ~17% of peak SSD bandwidth during loading, because the 128 KB contiguous-segment heuristic mismatches loader access patterns — precisely the observation you would have opened with. Up to 79% loading-latency reduction. Its explicit selling point is compatibility, no application changes, which forecloses the "but I'll do it in the kernel so it's general" move.

The userspace side is equally taken, by shipped code rather than papers.

  • llama.cpp PR #18012, async DirectIO model loading on Linux (PR) — O_DIRECT with async reads, benchmarked across PCIe 4.0 and 5.0 SSDs with RTX 5080/5090 and a DGX Spark. GPT-OSS-120B-MXFP4 goes from ~110 s cold / ~67 s warm under mmap to a consistent ~10.5 s.
  • llama.cpp PR #18166, --direct-io (PR) — makes Direct I/O the default on Linux and Windows, keeping mmap default only on macOS. "mmap is the default and nobody has tried O_DIRECT" is factually false on current master.
  • llama.cpp PR #7420, Direct I/O and Transparent HugePages (PR) — the older attempt, whose abstract is literally "up to 3-6x faster uncached loading, fewer pageouts, no page cache pollution." Kills area 1 and dents area 2 simultaneously.
  • HuggingFace safetensors PR #692 (PR) — io_uring plus O_DIRECT fast path, 3.6–3.8× faster than mmap on cold NVMe, adaptive chunk sizing 64 KB → 16 MB, THP-requested buffers, defer_taskrun on kernel 6.1+.
  • InstantTensor (scitix) has both URING (O_DIRECT) and URING_BUFFERED backends, the latter using POSIX_FADV_SEQUENTIAL plus IOSQE_ASYNC to dodge inline page-cache memcpy: loader_io_uring.cpp. Parasail's writeup of the cold/warm tradeoff: blog.
  • A community fork already combines io_uring, O_DIRECT and madvise(MADV_DONTNEED) for tiered MoE expert caching on a single consumer GPU: Lidenburg/llama.cpp.

Surviving gap: only a trivial one. llama.cpp assumes a static per-invocation choice between mmap and O_DIRECT, and that choice does not hold when free memory and page-cache residency change between runs. Everyone in PR #18012's thread observes the cold/warm crossover and nobody has built an adaptive policy — but that is a heuristic knob, not a thesis, and MAIO's PPC arguably subsumes it in the kernel where it belongs.

Rejection vocabulary: "prefetch accuracy," "I/O templates," "burn-after-reading," "already upstream."

Area 2 — Page-cache pollution and eviction damage from model loading

Rating: active, closer to crowded than you would like — but a real gap survives on the victim side.

  • MAIO's Burn-after-Reading policy (FAST '26, above) evicts cold model data immediately after the accelerator reads it, specifically to preserve cache space. The mitigation for pollution during loading is published at a top venue.
  • Rethinking I/O Caching for Large Language Model Inference on Resource-Constrained Mobile Platforms. Heejin Kim, Jeongha Lee, Hyokyung Bahn. Mathematics (MDPI) 13(22):3689, 2025. doi:10.3390/math13223689. Refereed journal, but MDPI and not a systems venue — treat its prestige as low and its priority claim as real. File-level trace analysis of mobile LLM apps decomposing access into one-time sequential init scans, persistent small hot sets (tokenizer, metadata, index), and looped weight accesses, stating that one-time scans "provide little benefit and should be evicted quickly to avoid polluting the cache," with cache-sizing guidelines relating loop size, hot-set coverage and storage bandwidth. This is the characterization half of area 2, already published.
  • Beyond System Calls: Uncovering Hidden I/O Behavior of Mmap-Based On-Device LLMs. ICAIIC 2026. doi:10.1109/icaiic68212.2026.11454159. Refereed IEEE conference, minor venue. Uses ftrace to capture page-fault-driven page-cache loading for mmap'd models across four workloads (text, VLM, ASR, T2I), finds 500 MB–3 GB effective working sets, concludes the bottleneck is DRAM available to hold model pages rather than the I/O interface, and notes eviction causing repeated major faults. This is the kernel-level tracing methodology, already published.
  • Who Should Own the Expert Cache? Kernel-Managed Tiering for Trillion-Parameter MoE Inference. arXiv 2608.12103 [preprint, Aug 2026]. The most dangerous paper here: it runs the comparison directly, page cache as expert tier versus a userspace frequency-ranked arena, on a 1.45 TB expert pool, GH200. The oracle arena is only 1.09× faster at C=256 GB; plain kernel recency serves 75.3% of demand without touching the device versus the oracle arena's 74.6%; and POSIX_FADV_RANDOM hurts (1.13 s → 1.80 s with unchanged device bytes). It also reports that a previous draft's 1.50× number was an artifact of a dead balloon — self-correction suggesting it will survive review. If your thesis is "the page cache is bad at this, I will replace it," this paper says the page cache is roughly as good as an unattainable oracle and better off-domain.
  • Application-level pollution avoidance is upstream already: llama.cpp #7420 claims "no page cache pollution"; safetensors #692 lists "avoiding page cache pollution for large infrequent reads" as a design goal.

Surviving gap — the inverse framing. Every incumbent measures pollution from the model's point of view. None measures the damage to co-resident interactive applications on a single-user desktop:

MAIO (FAST '26) and Kim et al. (2025) both state that one-time sequential init scans should be evicted early to avoid polluting the cache, and both evaluate the policy by its effect on the loading process itself. That objective function does not hold on a single-user desktop, where the page cache is shared with a browser, an IDE and a language server, and where the cost of pollution is paid by a different process than the one the policy optimizes. Neither work reports any metric of the victim workload.

Smallest defensible result. Instrument a laptop (/proc/vmstat, per-cgroup pgmajfault, mincore() residency maps, fincore on a fixed victim file set), define a reproducible interactive victim workload (cold cargo/tsc build, browser tab restore, git status on a large repo, sqlite query set), and quantify eviction damage from loading a 4–8 GB GGUF under each of the four loading modes now shipping in llama.cpp. Report victim-side major faults and completion-time regression, plus recovery time. Then show a one-knob mitigation — POSIX_FADV_DONTNEED after residency stabilizes, or a cgroup v2 memory.high on the loader, or memory.low protection on the victim — and measure the Pareto frontier between loader throughput and victim damage. Hardware: one laptop with NVMe and 16–32 GB, bare-metal boot. No GPU strictly required.

How it goes wrong. Your strongest baseline is not a research system, it is cgroup v2 memory.low/memory.high plus POSIX_FADV_DONTNEED, correctly configured. If well-tuned cgroup confinement removes most of the victim damage, you have written a documentation page. Run it in week two. Second baseline: --direct-io, now the Linux default, which by construction does not pollute — so your contribution must be about the warm/repeat-load regime where users deliberately want mmap. Rejection vocabulary: "this is a cgroups configuration problem," "MGLRU already handles this," "burn-after-reading solved it," "no novelty over Kim et al.," "your victim workload is a microbenchmark."

Area 3 — Memory pressure, swap/zswap/zram, and the OOM killer

Rating: active. The most promising area on the list, and the one with the shortest shelf life.

The occupants are kernel patches and blog posts, not papers — good for novelty, bad for stability.

  • oom_badness() in mm/oom_kill.c scores get_mm_rss_sum() + MM_SWAPENTS + mm_pgtables_bytes/PAGE_SIZE, then adds oom_score_adj × totalpages/1000 (source). RSS includes file-backed mapped pages. For an mmap'd GGUF those pages are clean, file-backed and trivially reclaimable — dropping them costs an NVMe re-read, not data loss — yet they are charged at full weight, exactly as anonymous heap. A llama-server holding a 7 GB model resident on a 16 GB laptop is scored as the fattest process and selected first, even though reclaiming its pages is nearly free. A clean, concrete, defensible pathology claim, and not stated in the literature for this workload.
  • mm: Reduce direct reclaim stalls with RAM-backed swap. Matt Fleming (Cloudflare), LWN, March 2026. LWN 1061060. Production machines with zram-only swap spinning in direct reclaim for 20–30 minutes without ever invoking the OOM killer, because should_reclaim_retry() reads zram's thin-provisioned free-slot count as reclaimable capacity. Introduces BLK_FEAT_RAM_BACKED / SWP_RAM_BACKED. Argues explicitly for OOM-killing over brownout on MTTR grounds.
  • Debunking zswap and zram myths. Chris Down, March 2026. post · LWN discussion. States that swap-on-zram "is increasingly unsupported upstream" and that zram without a userspace OOM daemon leaves the machine hung for minutes. In the thread, Shakeel Butt states the MM community's position outright: the kernel OOM killer is deliberately conservative and aggressiveness is punted to userspace policy (systemd-oomd, Android LMKD, fb-oomd). This is your incumbent's stated assumption, and it is quotable.
  • mm: BPF OOM, v3, Roman Gushchin, 26 Jan 2026. LWN 1056177 · patch 07/17. bpf_handle_out_of_memory() struct_ops attachable system-wide or per-memcg, traversed up the cgroup tree, with bpf_oom_kill_process() and bpf_out_of_memory() kfuncs, falling back to the in-kernel killer if no memory is freed. Simultaneously your enabling mechanism and your biggest threat — an OOM policy for inference is now a BPF program a student can write in a week.

The mobile side is genuinely crowded and must be read first.

  • An Efficient Context Management System for On-Device LLMaaS (Libra). Wangsong Yin, Mengwei Xu, Yuanchun Li, Xuanzhe Liu. SenSys 2026, pp. 377–391. doi:10.1145/3774906.3800479; earlier arXiv version "LLM as a System Service on Mobile Devices", arXiv 2403.11805. Their Observation #3 is exactly area 3 for mobile: "conventional app-level memory management is not satisfactory," LMKD treats LLM context memory as ordinary app memory, "a lightweight app could easily get killed by OS's LMK if it possesses an active LLM context," and recomputing a Llama2-7B context on a Xiaomi 14 costs 22.92 s and 94.57 J. The argument that the low-memory killer's heuristic is pathological for inference state is already published at a top mobile venue. Your defensible territory is the Linux desktop, with weights rather than KV cache, and with file-backed-RSS mis-scoring as the mechanism.
  • KVSwap. Hao Zhang, Chenyang Xia, Zheng Wang (Leeds). ACM MobiSys 2026 (accepted). PDF · arXiv 2511.11907. Storage rather than CPU RAM as KV backing store on unified-memory mobile systems.
  • Android 17 is adding per-app memory limits; LMKD already uses PSI monitors with ro.lmk.psi_partial_stall_ms defaults of 200 ms (low-RAM) and 70 ms (high-end) (docs). Framed as mobile, you compete with Google's roadmap.

Surviving gap, two ways.

Linux's oom_badness() charges every resident page of an mmap'd model file to the inference process at the same weight as anonymous memory, on the assumption that a task's RSS approximates the memory that killing it would recover. That assumption does not hold for a process whose RSS is dominated by clean, file-backed, read-only weight pages, which reclaim would recover for the price of an NVMe re-read. Consequently the process the kernel is most eager to kill is the one whose memory is cheapest to reclaim without killing anything.

Chris Down (2026) and the LWN discussion state that the kernel OOM killer is deliberately conservative and that aggressiveness belongs in userspace daemons — an argument calibrated on datacenter fleets where the workload is restartable and MTTR is the objective. That calibration does not hold on a single-user laptop running local inference, where the brownout the daemons are tuned to avoid is a 30-second stall in an interactive session, and where killing the inference process discards minutes of prefill work that no supervisor will restart.

Smallest defensible result, two shapes.

  1. Characterization only. A reproducible pressure ladder on a laptop with a fixed memory budget, measuring what happens as a model is loaded and decoded past capacity across the swap configurations users really run: no swap, disk swap, zswap, zram, zram plus systemd-oomd. Report time-to-first-OOM, direct-reclaim stall duration, pgmajfault and thrashing rates, decode-throughput collapse, and which process the kernel selected. Show the mis-scoring concretely: log /proc/<pid>/oom_score alongside smaps_rollup for the weight mapping. Confirm or refute the Fleming zram pathology at laptop scale. Defensible on its own.
  2. Plus a BPF OOM policy that discounts clean file-backed weight pages when scoring, or prefers trimming the KV cache over killing, showing kills converted into graceful degradation.

Hardware: one laptop or workstation, 16 GB (deliberately tight), NVMe, bare metal. Not negotiable. The linux-inference-memory-hints repo — someone else's attempt at this — documents that WSL2 validates scripts but produces no reclaim signal at all, and that "real-world reclaim bias evaluation requires a patched bare-metal kernel." Budget a scratch machine you will hard-lock, and repeated hard reboots.

How it goes wrong. (i) Overtaken mid-thesis — BPF OOM at v3 in January and Fleming's fix in flight; if both land, half the pathology disappears and the other half becomes "write a BPF program." Pin your kernel, state it, frame as characterization-plus-policy. (ii) The oom_score_adj objection — you must show these knobs are insufficient, not merely unset: that protecting the inference process just relocates the kill to the browser, so the problem is not which process dies but that the scoring model has no notion of reclamation cost. (iii) "This is Libra, on a different OS" — cite Yin et al. early and prominently, and state the difference in one sentence.

Rejection vocabulary: "userspace OOM policy is the accepted design," "just use cgroups," "MGLRU," "already known to the MM community," "no mechanism, only measurement."

Adjacent and mostly unoccupied, as a second front: PSI-driven admission control for local inference. /proc/pressure/memory triggers with poll() are documented (PSI docs), and only hobby implementations exist — infiniteregrets/kv-psi, gtrak/tinyllb. No refereed paper. But absence of hits is weak evidence, PSI-based load shedding is a well-worn datacenter idea, and "apply PSI to a new workload" is thin novelty. Treat it as a mechanism inside area 3, not a thesis.

Area 4 — Transparent hugepages and promotion/demotion

Rating: crowded for the obvious result; a narrow file-backed opening remains.

The obvious contribution — apply THP to the weight mapping, measure the TLB and minor-fault win — is merged upstream code with the result already reported.

  • llama.cpp PR #22022, MADV_HUGEPAGE hint for THP on Linux (PR, merged as dd17481, April 2026). A 4–5 GB weight map drops from ~1M 4 KB pages to ~2K 2 MB pages, and critically: "Neutral on an unloaded machine (pages stay resident); reduces re-fault latency spikes under memory pressure." The author has already told the community both the mechanism and the shape of the result.
  • llama.cpp PR #21821, --hugepages (PR) — anonymous 2 MiB HugeTLB backing, motivated not by TLB speedup but by HugeTLB Vmemmap Optimization recovering ~1.75 GiB of struct page metadata on a 128 GiB system, with a worked example of that being the difference between OOM and fitting a MiniMax m2.5 IQ4_XS on a Strix APU. Notes correctly that MADV_HUGEPAGE does not deliver this, since the struct page array survives. Cross-references issues #2251, #12444 and PR #7420 — the design space is enumerated in public.

Research-side:

  • Towards Segmentation-Based Address Translation for LLM Inference. Youngjoon Cheon, Yunho Oh, Jeongseob Ahn (Korea University). IEEE Computer Architecture Letters, 2026. doi:10.1109/lca.2026.3693796. Refereed letter. Observes the growing KV-cache footprint contending for TLB capacity and starving weight translations, then bypasses paging entirely for weights — physically and virtually contiguous, segment-translated, since weights are read-only and never reclaimed for the instance lifetime. 2.51× IPC over the paging baseline. This is the ceiling on any area-4 result: a 2 MB-page improvement is a weak approximation of a published stronger idea.
  • xHeap: Transparent Hugepage Optimizations for Memory Offloading. Malliotakis, Papagiannis, Marazakis, Bilas (Crete / FORTH). CHEOPS '26 (refereed EuroSys-affiliated workshop). PDF. Not LLM-specific, but lands the general critique: Linux THP with memory offloading is "overly aggressive in committing memory, and too coarse-grained," and promotions "lack the responsiveness and concurrency necessary." Their file-backed mmio path with concurrent asynchronous promotions cuts dTLB miss cycles up to 15× and improves performance up to 76% when offloading 15–25% of the heap. If your pitch is "THP promotion is too slow and coarse for a swapping workload," xHeap said it in 2026 with a working kernel module.
  • The kernel is moving into the adjacent space: Usama Arif's large-folio readahead for exec memory, v7 by June 2026 (LWN 1064021, LWN 1066175, v7 posting) — bypassing mmap_miss for VM_EXEC, using mapping_max_folio_order(), capping at 2 M, fixing ASLR-induced misalignment that defeats contpte. Plus readahead folio-insertion batching (LWN 1055007) and "no PG_readahead on EOF" (patch). The machinery for "large folios for a big read-only file mapping, adapting to memory pressure" is being built right now for executables, and generalizing it to weight files is an obvious follow-up for someone with kernel standing.

Surviving gap — narrow, and only file-backed rather than anonymous:

llama.cpp PR #22022 hints MADV_HUGEPAGE on the weight mapping on the assumption that promotion is cheap and one-directional, reporting the result as neutral when pages stay resident. That assumption does not hold under memory pressure on a constrained laptop, where the mapping is repeatedly demoted to 4 K for reclaim and re-promoted by khugepaged, and where compaction and khugepaged CPU time are charged against a foreground interactive session rather than a batch fleet.

Smallest defensible result. Promotion/demotion churn cost on the weight mapping under a controlled pressure ladder: khugepaged CPU time, thp_split_pmd / thp_collapse_alloc / compact_stall, decode-latency tail, across enabled=always|madvise|never × defrag=always|defer|defer+madvise|madvise|never, with and without #22022's hint, plus #21821's HugeTLB path as an upper bound. Then a policy: suppress promotion on the weight mapping under pressure, or use mTHP (16 K/64 K) rather than PMD-size to cut demotion cost. One laptop, bare metal, THP sysfs. Very cheap.

How it goes wrong. Your baseline is a merged PR whose author already told you the answer is "neutral unless under pressure," so your whole contribution lives inside the pressure regime; if the effect there is small you have nothing. "THP causes latency spikes, use defrag=defer" is a 2016 answer a reviewer will call configuration advice (thread). Rejection vocabulary: "known THP fragmentation behavior," "just use defer+madvise," "mTHP already addresses this," "Ingens," "HawkEye," "Temeraire" (Hunter et al., OSDI '21, cited as evidence the hugepage-policy space is thoroughly worked).

Area 5 — NUMA and thread/memory placement for CPU-only inference

Rating: crowded, and separately disqualified by hardware. Drop it.

The hardware point is decisive: a laptop has one NUMA node. There is no effect to measure. Every result here requires a dual-socket or many-core server.

It is also not open.

  • --numa mirror is upstream: llama.cpp PR #16000 (PR) mirrors weights to every NUMA node with a thread-local selector in the OMP threadpool, plus first-touch allocation and thread binding; the documentation commit claims up to 147% text-generation improvement (commit).
  • NUMA-aware KV cache buffers: PR #11580 (PR), with a concrete diagnosis — a ~20 GB KV cache landing entirely on node 5, exhausting it, forcing foreign accesses for the weights that should have lived there.
  • Arm's Neoverse N2 analysis (post) identifies both root causes — malloc() is not NUMA-aware, and the per-op barrier degrades once threads exceed a node — and fixes both with NUMA-local atomic barriers, 53–55% uplift. Note the tell: the patch "was reviewed by a llama.cpp author, but was not merged as interest in server and cloud use cases is low." The engineering is done; the demand is absent.
  • ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs. arXiv 2603.07770 [preprint] — NUMA-aware memory and thread management plus cross-NUMA tensor parallelism, benchmarked against llama.cpp on 2- and 4-node configurations, correctly identifying llama.cpp's residual weakness ("does not bind tensors to specific NUMA nodes").
  • Community measurement is extensive and quantitative: discussions #12289 (dual 9275F, DDR5-6000, QwQ-32B FP16: 6.66 → 10.80 tok/s with mirroring) and #19102 (dual 5th-gen Xeon, full tuning matrix plus an mbind(2) patch for interleaving mmap'd pages).

No gap reachable by a masters student on a laptop. Say no and move on.

Area 6 — Unified and shared memory on integrated-GPU and Apple Silicon systems

Rating: active on Apple (crowded for the obvious result); thinner on Linux iGPU, but the reachable part is characterization and the mechanism work is being done by vendor kernel engineers.

Apple side — the characterization contribution is taken, mostly by preprints.

  • Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective. Afsara Benazir, Felix Xiaozhu Lin. arXiv 2508.08531 [preprint] — per-stage latency and cost, kernel- and hardware-level utilization, bottleneck analysis, recommendations for practitioners and vendors. Broad and careful; hard to differentiate against.
  • vllm-mlx: Native LLM and MLLM Inference at Scale on Apple Silicon. arXiv 2601.19139 [preprint] — 21–87% higher throughput than llama.cpp, continuous batching to 4.3× aggregate.
  • FusionML. arXiv 2607.22785 [preprint] — per-layer contention-aware CPU+GPU row splitting for prefill across five chips and three generations, 1.15–1.38% on decoder-block prefill, 1.18–1.25× TTFT. Its negative result matters most: intra-operator splitting offers nothing for decode, because CPU and GPU share the same memory bus and using both does not increase bandwidth. Any "exploit unified memory by using both processors" idea for decode is pre-refuted.
  • A five-framework comparative study on an M2 Ultra also exists (arXiv 2511.5502) [preprint; quality looks poor — mangled text, thin methodology — but it establishes priority on "compare local runtimes on Apple Silicon"].
  • The one genuinely OS-level knob, iogpu.wired_limit_mb versus Metal's recommendedMaxWorkingSetSize (~75% of RAM), is documented in blogs, not papers (stencel.io). You cannot instrument the XNU reclaim path, which caps how deep an OS contribution can go — a structural reason to avoid Apple as your platform.

Linux iGPU side — thinner in the literature, but the work is happening in the driver tree.

  • amdgpu SVM VRAM migration via drm_pagemap, RFC April 2026 and v4 May 2026 (v4 thread, RFC) — ZONE_DEVICE registration, SDMA copies through a GART window, eviction fences for overcommit, migration policy driven by SVM range attributes. Explicitly POC/RFC, single-GPU, "eviction fence path is functional but not stress-tested under heavy memory pressure." That last clause is an invitation to AMD's own engineers, and competing with a vendor driver team on their own subsystem is not an MSc-scale bet.
  • Dissecting CPU-GPU Unified Physical Memory on AMD MI300A APUs. arXiv 2508.12743 [preprint] — the characterization template: latency, bandwidth, coherence overhead, allocator comparison, page-fault handling, TLB management, Infinity Cache. HPC workloads and datacenter APU, but it is the paper a reviewer will say you replicated.
  • The consumer-iGPU reality is entirely grey literature: GTT versus BIOS carveout, amdgpu.gttsize, ttm.pages_limit, GGML_CUDA_ENABLE_UNIFIED_MEMORY (ROCm RDNA3.5 guide, Framework thread, Strix Halo guide). One community stack reports that dropping the BIOS carveout from 96 GiB to 512 MB raised usable GPU memory to ~124 GiB and improved prefill 19–94% (strix-halo-llm-stack) — and, tellingly, that it was OOM-killed twice when free host RAM dropped below 10 GiB.

Surviving gap — characterization, and hardware-gated:

The AMD ROCm RDNA3.5 guidance states that GTT allocations are dynamic and "not permanently reserved, allowing the operating system to reclaim memory when the GPU isn't actively using it." That reclaim path has no published characterization under a workload that holds tens of gigabytes of GTT-backed weights resident for the lifetime of a server process while a general-purpose desktop competes for the same physical pages — and the community evidence is that the outcome is an OOM kill rather than reclaim.

Smallest defensible result. On an iGPU system with a large unified pool (Strix Halo / Ryzen AI Max class, 64–128 GB), characterize GTT-backed weight residency against host reclaim: what the kernel can actually reclaim from a GTT allocation under pressure, at what latency, whether TTM eviction or the OOM killer fires first, and how BIOS carveout size shifts the boundary. Cross-reference the Apple wired_limit behavior for contrast. If you do not have this class of machine, the area is closed — a discrete-GPU laptop cannot substitute.

How it goes wrong. The dominant failure is writing a benchmark report. Many already exist, some maintained continuously with raw logs (slb350.github.io/strix-benchmarks) — a moving target you cannot beat on breadth. Your baseline is "set amdgpu.gttsize and ttm.pages_limit per the vendor guide," and if correct configuration removes the pathology you have written a wiki page. Rejection vocabulary: "driver configuration issue," "vendor-specific," "no OS mechanism," "already characterized on MI300A," "not generalizable."

Area 7 — Other OS-level gaps found in 2025–2026

Ranked by how much room remains.

(a) Semantic reclaim hints for inference memory — occupied by grey literature, which is a specific hazard. The idea of madvise-style tensor-semantic hints is claimed, but only in venues you cannot cite and cannot dismiss.

  • Semantic Tensor-Aware Paging (STAP): Re-Engineering the Linux Kernel for High-Efficiency Distributed LLM Inference. Zenodo, January 2026. doi:10.5281/zenodo.20112034. Extends vma_struct with a semantic madvise interface so the runtime can flag tensor types, claiming 7.5× P99 latency reduction at 90% memory saturation. Not peer-reviewed. Zero citations. All three authors have h-index 0. I would not trust the numbers. But its abstract states your thesis premise almost verbatim — "traditional kernel memory management... remains demand-agnostic, treating high-priority model weights and volatile KV caches as uniform anonymous pages" — so a reviewer who finds it will ask you to differentiate against work you cannot evaluate.
  • Hobby projects have enumerated the same design space in public: linux-kernel-inference-fastpath lists MADV_INFER_MODEL, MADV_KV_HOT, MADV_KV_COLD, MADV_KV_PREFIX_SHARED, an inference cgroup controller, a sched_ext inference scheduler, an inference-aware hugepage policy, and a KV-aware reclaim/demotion order — essentially every idea in this candidate list, with an explicit "kernel patches should come after measurement" discipline, and a stated intent to post an RFC to linux-mm (issue).

The practical consequence: the "semantic hints to the kernel" framing is spent as a novelty claim, but nobody has produced trustworthy evidence. That is an unusual and exploitable position — a careful, honest, reproducible evaluation of how much semantic hinting actually buys on a laptop would be a genuine contribution, including as a negative result. It is also a thesis whose contribution is rigor rather than novelty, which some examiners reward and others do not. Discuss with your supervisor before committing.

(b) Inference as a noisy neighbour to the interactive desktop — the most under-occupied framing found. Every system above optimizes inference; almost nothing measures what inference does to the machine it runs on. The one exception is CPU-side and in a toy kernel: Elastic Gang: Per-Token Membership Change for a Hard-Barriered LLM Inference Gang Co-Scheduled with OS Processes, arXiv 2607.04668 [preprint; single-author bare-metal x86_64 Rust kernel, ~232 kLOC; venue prospects uncertain]. Makes the inference gang a first-class schedulable entity whose core membership changes between tokens, reporting 1.75×/1.52×/1.28× general-process throughput over static core partitions at 25/50/75% inference duty cycle on an AMD Zen 5 8C/16T box. It does not address the page cache, reclaim, or the memory hierarchy. That half of "local inference is a bad citizen" is open, and it is the same gap as area 2 — the strongest reason to treat areas 2 and 3 as one thesis rather than two.

(c) MoE expert streaming on consumer hardware — crowded, including by hobbyists, and the negative result is already known. DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs, arXiv 2602.03495 [preprint] — 0-1 integer expert assignment across CPU/GPU, residual-based prefetching, workload-aware cache replacement, 3.43×/1.87×/1.32× decode speedup over llama.cpp, KTransformers and HybriMoE. Beneath that a large hobbyist layer: qwen35-moe-offload (RTX 3070 8 GB + 16 GB RAM + NVMe, windowing and bundling per "LLM in a Flash"), moe-ssd-streaming-windows (32 GB model on 28 GB of memory at 2.5–4.3 tok/s, relying on the page cache as a free LRU). Most usefully, the vLLM RFC thread (issue #38256) contains measured NVMe numbers pre-empting the naive project: ~6.9 GB/s ceiling, a single thread reaching 93% of it at expert-sized 22 MB reads, 16 threads only adding latency (p99 4 ms → 128 ms), sub-1 MB granularity two orders of magnitude off the ceiling, and a DeepSeek-R1-class model over NVMe penciling out to single-digit tok/s. Combined with the "Who Should Own the Expert Cache?" finding, the headline conclusion — streaming experts from NVMe buys capacity, not speed, and the page cache is already near-optimal — is established. Do not go here expecting a speedup.

(d) Multi-model residency and unload/switch cost on one machine — possibly open, weak evidence. Ollama-style hot-swapping between models, and what the page cache and reclaim path do across a switch, was not found characterized. Nearest: a community router-mode stack keeping a small always-resident aux model plus one swappable heavy model on Strix Halo (strix-halo-llm-stack, ~30 s to ~3 min load times, OOM kills below 10 GiB free). This is absence of hits, which is weak — model-swapping cold start is heavily worked in the serving literature (it is MAIO's own headline application, at 36% inference-throughput improvement in "the elastic deployment scenario"), so the datacenter version is certainly taken and the single-machine version may be too. Unverified.

(e) Things that look like these areas but are not. Several 2026 papers use OS vocabulary for non-OS work; recognize them so you neither cite them as incumbents nor mistake them for gaps. The Missing Memory Hierarchy: Demand Paging for LLM Context Windows (arXiv 2603.09023) [preprint] applies Denning working-set language to token context management via an API proxy — no kernel, no pages. ProbeLogits (arXiv 2604.11943) [preprint] is a safety classifier framed as a kernel primitive. IsotopeOS (Zenodo, Feb 2026) [preprint, 0 citations, author h-index 0] is "semantic paging" of the context window in a Rust wrapper around llama-cpp-2. Symphony / Serve Programs, Not Prompts (Gim et al., HotOS 2025 — refereed workshop, PDF) is real and well-executed but borrows OS abstractions for a serving system rather than changing the OS. AIOS (arXiv 2403.16971) is agent orchestration. The "LLM × OS" title space is saturated with work that does not touch the kernel; that saturation makes venue positioning harder without actually occupying your technical ground.

Bottom line

Merge areas 2 and 3 into a single thesis about local inference as a bad citizen of the memory hierarchy on a single-user Linux machine — page-cache eviction damage to co-resident interactive workloads, plus the reclaim-and-OOM pathology, with clean-file-backed-RSS mis-scoring in oom_badness() as the concrete mechanism and a BPF OOM or cgroup-v2 policy as the fix. It is the only framing where (i) the incumbents' objective function is demonstrably different from yours, (ii) a laptop is the correct platform rather than a compromise, and (iii) the strongest baseline is a configuration you can implement and beat, or honestly fail to beat, in a few weeks.

Two things before committing. Run the cgroup-v2 baseline (memory.low on the victim, memory.high on the loader, POSIX_FADV_DONTNEED after load) in week two — if it closes the gap, the thesis is dead and you want to know immediately. And pin your kernel version and state it prominently, because BPF OOM (v3, January 2026) and the RAM-backed-swap reclaim fix (March 2026) are both in flight, and a chunk of the pathology has roughly a one-year shelf life.

Areas 1 and 5 are not worth revisiting. Areas 4 and 6 are viable only as narrow measurement studies, and area 6 only if you already own a large-unified-memory APU machine.