Research
We filed 34 papers before writing the compiler and read the 2025 and 2026 work since. Sending work to a cheaper model saves money, and that is measured per request on public benchmarks. A gated, per-task assignment on a company's own fixtures is not published anywhere.
What is already measured
Preprints are marked. Most of this evidence comes from academic benchmarks, not enterprise fixtures, and we say so wherever it matters.
| Question | What the work shows | For sllim |
|---|---|---|
| Does a cheaper model save money? | Yes, measured per request: FrugalGPT matched the best single model at up to 98% lower cost, and AutoMix cut cost by more than half at comparable quality. FrugalGPT; AutoMix (NeurIPS 2024) | Measured |
| Does per-request routing beat one model per task? | Barely. Against a frozen best model per task, the best router gains 1.4 to 2.6 points and recovers 7.5% to 14.4% of the oracle headroom; the lower confidence bound on the gain is zero. Opportunity Is Not Realizability (preprint, 2026) | Supports task-level assignment |
| Do the routing methods differ much? | Little. The top 5 of 21 methods differ by 0.22 points on average, and a nearest-neighbour baseline is competitive. The Routing Plateau (preprint, 2026); RouterArena (ICLR 2026) | Routing is commoditising |
| When does a cascade pay? | When the check is accurate. With repository tests as the check, cascade routing scored 54.12 against 40.47 for routing on SWE-Bench, counting the cost of every call. A Unified Approach to Routing and Cascading (ICML 2025) | Keep cascades check-based |
| Does a leaderboard predict my task? | Weakly. Models move about six to seven rank positions between benchmark and downstream task, and top-three overlap is at most 0.25. Benchmarking the Benchmarks (preprint, 2026) | Measure on your own fixtures |
| How sure is "cleared the floor"? | A normal 95% interval at 100 items covered the true pass rate only 92.5% of the time in simulation; Wilson and Beta-Bernoulli intervals held their coverage. Don't Use the CLT in LLM Evals (ICML 2025) | Why the gate uses Wilson |
| Do hosted models change silently? | Yes. 11 of 31 third-party endpoints serving a Llama model returned something other than the reference weights. Model Equality Testing (ICLR 2025) | Why policies carry a fingerprint |
| Are cheap checks trustworthy? | Not by themselves. LLM-judge verifiers missed more wrong answers as the cheap model grew, and valid JSON did not mean correct fields. Cheap Verifiers, Large Blind Spots (preprint, 2026); ExtractBench (KDD 2026) | Program checks and a gold slice |
Still open
None of them needs a new model or a new public benchmark. Each is a measurement on a real contract, and each becomes data nobody else has.
Every study above uses academic benchmarks. Nobody has published the comparison on a fixed enterprise task at a fixed quality bar. Our design partners' tasks are where it gets measured.
Textbook power: about 80 fixtures show a 10-point margin over the floor, and catching a 2-point drop needs more than 1,500. Real contracts have their own variance.
Audits show it happens. No one has published how often it changes a task's ranking. Every gate run on a customer's sealed set is a data point.
The notes, the market research, and the run's evidence are shared with design partners and investors on request.
Ask for it