Open source · MIT licensed

Control where your AI calls go.
Secure every request. Extend with plugins.

LLMForge is an AI API gateway. Route between local GPUs, machines on your network, and cloud providers. Strip secrets from prompts. Audit every call. Build your own harness with plugins. OpenAI-compatible, one env var.

Cursor Claude Code OpenCode Continue Any OpenAI client
3
Infrastructure tiers (local → network → cloud)
4
Built-in plugins + custom webhooks
100%
Prompts stay on your server by default
$0
Cost on local models + cache hits

Three pillars. One gateway.

Control where requests go. Secure every prompt. Extend with plugins. All in a single Rust binary that sits between your tools and your models.

1

Flow Control — route where you want

Multi-tier failover: local GPU → machines on your network → cloud providers. Each route profile (/v1/code, /v1/fast, /v1/reasoning) has its own model, pattern, plugin chain, and budget. Cloud is opt-in, not the default.

2

Security — every prompt protected

PII redaction strips API keys, tokens, emails before they reach any model. Audit logging records every call. Budget caps per API key. Circuit breaker kills runaway agents — semantic loop detection catches retries with different wording, not just exact matches.

3

Plugins — build your own harness

4 built-in plugins (Rust, zero network overhead): PII redact, prompt harness, code formatter, audit log. HTTP webhooks at 3 extension points for any language. Fusion is a plugin pattern, not a core feature — route to multiple models, vote, cascade with tests.

# 1. One-click install (builds from source, installs to ~/.local/bin) curl -sSf https://llmforge.pages.dev/install.sh | bash # 2. Generate config & start the proxy llmforge init llmforge serve # 3. Point your tools at LLMForge instead of OpenAI OPENAI_BASE_URL=http://localhost:8787/v1

You're sending every prompt to someone else's server.

Every prompt your team sends to GPT-5.2 leaves your network — source code, internal docs, customer data. You can't control the flow. You can't inspect what's sent. You can't extend the pipeline. LLMForge puts you in control.

Your current setup

$6,750/mo for 200 engineers on GPT-5.2
  • Every prompt leaves your network — no control over the flow
  • No PII redaction — API keys and secrets sent to cloud
  • No audit trail of what was sent where
  • Agent loops burn $30K before anyone notices
  • One endpoint, one model, no extensibility
  • Vendor lock-in — your whole stack depends on one API

With LLMForge

$12/mo electricity + optional cloud fallback
  • Control the flow — local GPU, network machines, cloud, per request
  • PII redaction strips secrets before they reach any model
  • Audit logging records every call — model, cost, latency, key
  • Circuit breaker kills runaway agents before they drain budget
  • Plugin system — built-in or HTTP webhooks in any language
  • Any OpenAI-compatible client, zero code changes

One GPU server serves your whole team.

LLMForge runs on a single GPU server you already own. Route, cache, and cap spend for every engineer. Cloud fallback only kicks in for the ~5% of hard problems.

Team size Requests / month Current cost / mo LLMForge / mo Annual savings
5 engineers 2,000 $20 $0.10 $239
50 engineers 50,000 $500 $2.10 $5,975
200 engineers 500,000 $6,750 $11.70 $80,860

Based on GPT-5.2 pricing ($1.62/1K calls) vs LLMForge on existing hardware. LLMForge cost = electricity + ~5% cloud fallback. Does not include GPU purchase — uses hardware you already have.

One gateway. Three pillars. Full control.

Route between local and cloud. Secure every request. Build your own harness with plugins. Fusion is a plugin pattern, not the core product.

🔀

Flow Control

Multi-tier: local GPU → machines on your network → cloud. Per-request routing. Multi-endpoint profiles with fixed model, pattern, and budget.

🔒

Security & Privacy

100% on your server by default. PII redaction strips secrets. Audit logging records every call. Cloud is opt-in, per-request.

🛡️

Circuit Breaker

Semantic loop detection — exact hash + embedding similarity. Kills runaway agents before they drain budget. 5 similar in 60s → 120s block.

🧩

Built-in Plugins

PII redact, prompt harness, code formatter, audit log. Rust code — zero network overhead. Enable per route profile.

🔌

Webhook Plugins

HTTP hooks at 3 extension points: pre_request, prompt_filter, post_response. Any language, any service. Build your own harness.

Fusion Patterns

Plugin patterns, not core features: Validated Fallback, Validated Consensus, Self-Consistency, Stream Race. Route to multiple models, vote, cascade.

🔐

Budget Caps + Auth

Per-API-key spend limits, rate limiting, Bearer token auth. Built for teams sharing infrastructure.

📦

Two-Layer Cache

Exact match (instant) + semantic match (embedding similarity). $0 on cached calls. 54x faster on hits.

📡

Streaming

SSE passthrough + stream race. Race models, first response wins. Works with Cursor, Claude Code, OpenCode.

Fusion plugin patterns work. Local models handle the rest.

Fusion is a plugin pattern — route to multiple local models, vote on results. +10pp on HumanEval when you need it. Off by default for speed. The core product is flow control and security. Fusion is one thing the plugin system enables.

Configuration HumanEval Pass Rate Cost / 1K calls
LLMForge — Local (single best model) 81.7% $0.00
LLMForge — Local + Fusion (5 models) 92.1% $0.00
LLMForge — Auto-Routing 82.3% $0.00
GPT-5.2 (cloud, for comparison) ~95% $1.62

3pp accuracy gap. 100% cost gap. Local handles the 95%, cloud fills the 5%. Run the benchmarks yourself: benchmarks/

If you want control over your AI infrastructure, yes.

You have local GPUs and cloud accounts

Stop choosing between local and cloud.

You run Ollama on your Mac Studio for side projects but pay OpenAI for the hard stuff. There's no way to route easy prompts to local and hard prompts to cloud. Your local GPUs sit idle 90% of the time. You have no control over the flow.

With LLMForge: Route by profile — /v1/fast hits local, /v1/reasoning cascades to cloud if local can't answer. Your local GPUs handle 95% of traffic. Cloud fills the gap. You control exactly where each request goes.
You manage an engineering team

Your source code is on OpenAI's servers right now.

Every prompt your engineers send leaves your network — source code, internal docs, customer data. No PII redaction. No audit trail. No way to extend the pipeline. You're paying $6,750/mo for the privilege of losing control over your data.

With LLMForge: Control the flow — local, network, or cloud per request. PII redaction strips secrets. Audit logging records every call. Plugin system lets you build custom harnesses. $80,860/year saved — and your IP stays yours.

Deploy in 5 minutes.

Free, open source, MIT licensed. No signup, no cloud, no lock-in.

curl -sSf https://llmforge.pages.dev/install.sh | bash llmforge init && llmforge serve