50.31×
MEDIAN.
Against a clean build of upstream llama.cpp at the exact source commit used by FlowMemory. Three paired runs: 48.36×, 50.31×, and 56.39×.
REUSE MORE. RECOMPUTE LESS. FLOWMEMORY HOLDS.
FlowMemory is a reuse-aware inference cache policy for repeated-prefix workloads. In controlled 96 MiB cache-pressure tests, FlowMemory retained valuable model state that the baselines evicted.
CACHE PRESSURE
EXPOSES
THE DIFFERENCE.
EXPOSES
THE DIFFERENCE.
HOT PREFIXES.
RETAINED.
RESTORED.
RETAINED.
RESTORED.
50.31×
20.76×
MEDIAN.
End-to-end through the Ollama API against the official Ollama 0.34.0 installation. Three paired runs: 24.24×, 20.76×, and 17.49×.
OUTPUTS
MATCHED.
Every generated output matched its baseline in the published one-token time-to-first-token suite.
ONE
NEW
TOKEN.
RAVEN restored the reusable prefix and evaluated only the new suffix instead of recomputing the full prompt.
KEEP THE
VALUABLE
CONTEXT.
FlowMemory tracks confirmed reuse, promotes hot entries, and uses bounded value-aware eviction. It targets agent loops, shared system prompts, repeated documents, and competing conversations.