github.com/magnitudedev/magnitude @0.0.11
Chat with this repoBrowse by type
What it actually does AI analysis from the code graph
README
<img alt="Magnitude icon" src="https://github.com/magnitudedev/magnitude/raw/0.0.11/assets/brand/icon-light.svg" width="120">
Magnitude
Easy local inference for agents
An open source inference server that profiles your hardware, recommends the best local models for it, then downloads, tunes, and runs them. Use the built-in harness or connect OpenCode, Pi, Hermes, or whatever you already use.
⭐ Help us reach more developers and grow the Magnitude community. Star this repo!
Get started
npm i -g @magnitudedev/cli && magnitude setup
Magnitude profiles your machine and shows you the best local models for it. Choose one and Magnitude downloads it, then connects it to your harness of choice.
Magnitude supports macOS and Linux. Windows is supported through WSL.
Why Magnitude?
- Set up in one command: pick a recommended model and it launches your harness
- Knows your hardware: profiles your chip, memory, and bandwidth
- Recommends what fits: the best models for your machine, with estimated tok/s
- Tuned end to end: speculative decoding, concurrency, all set for your machine
- Models on demand: loaded on request, unloaded when idle or memory fills
- Fully private and offline: models, prompts, and files stay on your machine
- Free to run: no token costs, API keys, or rate limits
- Open source: Apache 2.0, yours to modify
FAQ
What is Magnitude?
An open source inference server your agent sets up and operates through the Magnitude CLI. It profiles your hardware, recommends the best models for it, and runs them tuned to your machine.
What hardware do I need?
There's no fixed minimum. Magnitude profiles your hardware and recommends the best models for your machine. More memory lets you run larger models.
Why not just have my agent set up Ollama?
Your agent can install Ollama and pull a model, but it's guessing. It doesn't know your hardware, which quant fits, or how fast it'll run. Magnitude gives it a curated catalog with recommendations computed for your machine. Setup is agent-first, through a headless CLI that connects the model to your harness. Inference is built for agents. Models load just in time, unload when idle or memory gets tight, and concurrency is handled for you.
Which harnesses work with it?
Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick. Or use Magnitude's built-in harness.
Do I need to manage Magnitude after setup?
No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the Magnitude CLI anytime.
Does my data go to the cloud?
No. Prompts, files, and models stay on your machine.
Can it run completely offline?
Yes. Once Magnitude and a model are downloaded, no internet connection needed.
Can I use models outside the catalog?
Yes. You can download compatible GGUF models from Hugging Face and use them in Magnitude.
Learn more
License
Magnitude is licensed under the Apache License 2.0.
Extension points exported contracts — how you extend this code
browse all types & interfaces →
Core symbols most depended-on inside this repo
Shape
Languages
Modules by API surface
Dependencies from manifests, versioned
Datastores touched
For agents
$ claude mcp add magnitude \
-- python -m otcore.mcp_server <graph>