Quickstart

Fully offline. No API keys. Clone the repo for the fixtures, or use your own traces.

1 · Install

pip install swapgate
git clone https://github.com/nitishagar/swapgate.git
cd swapgate

2 · Capture traces

swapgate capture --from litellm \
  --in fixtures/traces-litellm.jsonl \
  --out /tmp/events.jsonl --redact bharat

Reads LiteLLM payloads (--from otel and --from secmcp also work), pseudonymises PII, and writes canonical events. Corrupt lines are counted and skipped.

3 · Build evals (human-approved)

swapgate build-evals --in /tmp/events.jsonl --out /tmp/suite.json --name demo
# inspect /tmp/suite.pending.json — every rubric is yours to review
swapgate build-evals --out /tmp/suite.json --approve

4 · Compare and gate

swapgate compare --suite /tmp/suite.json \
  --baseline mock:parity --candidate mock:regress \
  --fail-under 0.5 --out /tmp/receipt.json --report /tmp/report.md
echo "exit: $?"   # 2 — the candidate regressed

Exit 0 passes, 2 fails the gate, 1 is a usage error. Swap the mocks for openai-compat:<base-url>|<model> with SWAPGATE_API_KEY set to gate real models.

5 · Verify the receipt

swapgate compare --suite /tmp/suite.json --baseline mock:parity \
  --candidate mock:parity --out /tmp/receipt.json --key /tmp/demo.key
swapgate verify --receipt /tmp/receipt.json --key /tmp/demo.key
# receipt VALID

Read concepts for judges, the regression rule, and receipts; CLI reference for every flag; and the Sarvam swap example.