LLMForge is an AI API gateway. Route between local GPUs, machines on your network, and cloud providers. Strip secrets from prompts. Audit every call. Build your own harness with plugins. OpenAI-compatible, one env var.
Control where requests go. Secure every prompt. Extend with plugins. All in a single Rust binary that sits between your tools and your models.
Multi-tier failover: local GPU → machines on your network → cloud providers. Each route profile (/v1/code, /v1/fast, /v1/reasoning) has its own model, pattern, plugin chain, and budget. Cloud is opt-in, not the default.
PII redaction strips API keys, tokens, emails before they reach any model. Audit logging records every call. Budget caps per API key. Circuit breaker kills runaway agents — semantic loop detection catches retries with different wording, not just exact matches.
4 built-in plugins (Rust, zero network overhead): PII redact, prompt harness, code formatter, audit log. HTTP webhooks at 3 extension points for any language. Fusion is a plugin pattern, not a core feature — route to multiple models, vote, cascade with tests.
Every prompt your team sends to GPT-5.2 leaves your network — source code, internal docs, customer data. You can't control the flow. You can't inspect what's sent. You can't extend the pipeline. LLMForge puts you in control.
LLMForge runs on a single GPU server you already own. Route, cache, and cap spend for every engineer. Cloud fallback only kicks in for the ~5% of hard problems.
| Team size | Requests / month | Current cost / mo | LLMForge / mo | Annual savings |
|---|---|---|---|---|
| 5 engineers | 2,000 | $20 | $0.10 | $239 |
| 50 engineers | 50,000 | $500 | $2.10 | $5,975 |
| 200 engineers | 500,000 | $6,750 | $11.70 | $80,860 |
Based on GPT-5.2 pricing ($1.62/1K calls) vs LLMForge on existing hardware. LLMForge cost = electricity + ~5% cloud fallback. Does not include GPU purchase — uses hardware you already have.
Route between local and cloud. Secure every request. Build your own harness with plugins. Fusion is a plugin pattern, not the core product.
Multi-tier: local GPU → machines on your network → cloud. Per-request routing. Multi-endpoint profiles with fixed model, pattern, and budget.
100% on your server by default. PII redaction strips secrets. Audit logging records every call. Cloud is opt-in, per-request.
Semantic loop detection — exact hash + embedding similarity. Kills runaway agents before they drain budget. 5 similar in 60s → 120s block.
PII redact, prompt harness, code formatter, audit log. Rust code — zero network overhead. Enable per route profile.
HTTP hooks at 3 extension points: pre_request, prompt_filter, post_response. Any language, any service. Build your own harness.
Plugin patterns, not core features: Validated Fallback, Validated Consensus, Self-Consistency, Stream Race. Route to multiple models, vote, cascade.
Per-API-key spend limits, rate limiting, Bearer token auth. Built for teams sharing infrastructure.
Exact match (instant) + semantic match (embedding similarity). $0 on cached calls. 54x faster on hits.
SSE passthrough + stream race. Race models, first response wins. Works with Cursor, Claude Code, OpenCode.
Fusion is a plugin pattern — route to multiple local models, vote on results. +10pp on HumanEval when you need it. Off by default for speed. The core product is flow control and security. Fusion is one thing the plugin system enables.
| Configuration | HumanEval Pass Rate | Cost / 1K calls |
|---|---|---|
| LLMForge — Local (single best model) | 81.7% | $0.00 |
| LLMForge — Local + Fusion (5 models) | 92.1% | $0.00 |
| LLMForge — Auto-Routing | 82.3% | $0.00 |
| GPT-5.2 (cloud, for comparison) | ~95% | $1.62 |
3pp accuracy gap. 100% cost gap. Local handles the 95%, cloud fills the 5%. Run the benchmarks yourself: benchmarks/
You run Ollama on your Mac Studio for side projects but pay OpenAI for the hard stuff. There's no way to route easy prompts to local and hard prompts to cloud. Your local GPUs sit idle 90% of the time. You have no control over the flow.
/v1/fast hits local, /v1/reasoning cascades to cloud if local can't answer. Your local GPUs handle 95% of traffic. Cloud fills the gap. You control exactly where each request goes.
Every prompt your engineers send leaves your network — source code, internal docs, customer data. No PII redaction. No audit trail. No way to extend the pipeline. You're paying $6,750/mo for the privilege of losing control over your data.
Free, open source, MIT licensed. No signup, no cloud, no lock-in.