open source · apache-2.0 · zero-dependency core

There are many models. Prove the swap before you ship it.

SwapGate turns your production traces into evals, replays them against a candidate model, and blocks the swap on regression — so moving from rented frontier APIs to self-hosted open weights is a receipt, not a hope.

pip install swapgate
swapgate compare --suite evals/suite.json \
  --baseline openai-compat:https://api.provider.com/v1|frontier-large \
  --candidate openai-compat:http://localhost:8000/v1|sarvam-30b \
  --fail-under 0.95 --max-regression 0.02

Get started in 60 seconds GitHub

Terminal recording: pip install swapgate, then swapgate compare with a regressing candidate prints baseline 0.5 candidate 0.0 gap 0.5 -> FAIL — $ swapgate compare … → FAIL — the gate blocks the swap on regression

How it works

1 · capture

Ingest LiteLLM callbacks, OpenTelemetry GenAI spans, or sec-MCP tool-call logs. PII is pseudonymised before anything is stored.

2 · build-evals

Traces cluster into eval suites offline. A human reviews and approves every rubric before it can run the gate.

3 · compare

Baseline and candidate replay every case. Deterministic, LLM, or typed-decision judges score both. Regression fails the build.

4 · receipt

Every run emits a signed swap receipt — JSON verdict plus Markdown/HTML report — your audit trail for the model you now own.

Reports

Public swap reports, generated by the eval harness: 001 · support-desk rehearsal (mock providers).

Why SwapGate

Concern	What SwapGate does
own your intelligence	Rented models today, owned models tomorrow — carry your evals across every base-model upgrade.
runs anywhere	Stdlib-only Python core. No network, no keys, no phone-home — in CI, on a laptop, or fully air-gapped.
Bharat pack	Hinglish templates, DPDP redaction profiles, and a FREE-AI control annex for teams evaluating self-hosted Indic models.