Llama 3.2 1B
The same local model and deterministic prompts were used in each paired lane.
Benchmark methodology
A reproducible description of what FlowMemory measured, how the baselines were selected, and what the results do—and do not—establish.
Claim under test
FlowMemory can restore valuable repeated-prefix model state faster than recomputing it when cache capacity is constrained and competing prompts would otherwise evict that state. The benchmark measures the final hot-prefix request, not general inference throughput.
96 MiB cache
The deliberately constrained cache forces the replacement policy to choose which reusable state survives.
Promote, compete, revisit
Hot A, short displacement, hot A again, cold B, C, D, E, then hot A for the measured final request.
Three paired runs
Lane order alternated to reduce thermal and execution-order bias. The reported headline is the median speedup.
External baselines
Clean upstream build
Commit cb295bf59663cd3577389315636772f4060bd1f5 was checked out separately and compiled without FlowMemory modifications.
Official 0.34.0
The official installed Ollama service was compared with FlowMemory Ollama 0.34.0 through /api/generate.
Controlled inputs
Model, prompt order, context settings, cache budget, temperature, seed, and generation length were held constant within each paired comparison.
One-token TTFT suite
Generation was capped at one deterministic token to isolate cache restoration and time to first token across distinct official runner builds.
Interpretation
Cache-pressure advantage
The results support a substantial latency advantage for this repeated-prefix, eviction-sensitive workload.
Universal acceleration
FlowMemory does not accelerate raw token generation. Low-pressure workloads were approximately neutral.
Output equivalence
All generated outputs matched in both published external-baseline suites.
Sanitized JSON
Download results, individual timings, configuration, and caveats.