Bigger than your memory
Experts are read from SSD on demand into a smart RAM cache — run 763B and 816B models on 48 GB machines.
Coming soon
Huge models. Your hardware.
Huge models. Your hardware.
Powered by your SSD.
Inferondo streams mixture-of-experts weights straight from NVMe, so 800B-parameter models run on a laptop, a gaming GPU or a desk-side box — with browser chat and an OpenAI-compatible API.
0B
parameters on one machine
0×
faster prefill vs llama.cpp
0×
faster decode vs llama.cpp
0
memory settings to tune
Built for models that don't fit
Only the experts each token needs leave the disk. Everything else stays put.
Standard GGUF
Use the community GGUF files you already have. Indexing reads headers only — no conversion, no copies.
Learns your machine
Measures your drive, CPU and GPU on first run and sizes caches and threads automatically.
OpenAI-compatible API
Point any client at 127.0.0.1:8080/v1, or chat right in your browser.
Every platform
macOS with Metal, Linux with CUDA, Windows on the way. Same models, same commands.
Open source
Built on llama.cpp and ggml, with thanks to their community.
How it works
The router picks experts. Inferondo finds, streams, caches and runs them.
- Route. The model's router chooses the experts each token needs.
- Locate. A source index maps those experts to byte ranges in your GGUF files.
- Stream. io_uring, IoRing or uncached reader threads pull them into a RAM cache.
- Compute. CPU, the CUDA tier or the Metal/Vulkan GPU lane runs them — whichever is fastest, measured live.
Benchmarks
DeepSeek-V4.1-Flash-1017 · prefill tokens/s · 4K prompt · higher is better.
llama.cpp Inferondo
Flagship models
Inferondo is built around the largest open mixture-of-experts models.
Mistral Large 4
~1T total · 52B active
Mistral's flagship MoE. 128 routed experts per layer with 4 active per token, streamed from SSD while attention and shared experts stay resident.
GLM-5.3 by Z.AI
~801 GB at Q8_0
Z.AI's frontier GLM. Run the full 8-bit GGUF on a machine with a fraction of the RAM or VRAM it would normally demand.
Also supported
Muse-Spark-Open
816B total · 42B active
DeepSeek-V4.1-Flash
763B total · 8–16B active
Qwen3.8 Flash Next
177B total · A3B
Qwen3.6-35B-A3B
35B total · 3B active
Gemma4 E4B
7.46B · in-RAM
+ More planned
Tell us what to run next
One command. No tuning.
Install without sudo, start a model, and open your browser. Inferondo uses the memory you have.
Join the waitlist# coming soon
$ curl -fsSL https://inferondo.com/install.sh | sh
$ inferondo serve -hf mrfakename/DeepSeek-V4.1-Flash-1017-GGUF
→ web UI + API on http://127.0.0.1:8080