API Reference¶
Auto-generated from the lexvault source via mkdocstrings.
Public API¶
lexvault.LexVaultGuardrail¶
lexvault.LexVaultGuardrail ¶
Bases: CustomGuardrail
Reversible proprietary-term pseudonymization guardrail for LiteLLM.
Constructed by LiteLLM's file-mount loader with litellm_params minus
{guardrail, mode, default_on} as kwargs. Unknown kwargs are accepted
(extra="allow" on LexVaultConfig mirrors the loader's forwarding).
lexvault.LexVaultConfig¶
lexvault.config.LexVaultConfig ¶
Bases: BaseModel
Public configuration contract for LexVaultGuardrail.
These keys are the litellm_params a user puts in their config.yaml;
extra="allow" mirrors LiteLLM's own Guardrail model
(types/guardrails.py:660) so unknown keys are forwarded harmlessly.
placeholder_namespace_pattern ¶
Regex matching placeholders generated by this config.
The {code} slot in placeholder_format is replaced by a charset
class accepting the base32 alphabet (upper A–Z, digits 2–7) plus an
optional -N collision disambiguator suffix. Used by the detector
to pre-exclude placeholder spans (idempotency, invariant 16) and by
unmask to find restorable spans (invariant 17).
lexvault.MappingVault¶
lexvault.vault.MappingVault ¶
SQLite mapping vault: placeholder → original, keyed by scope.
assign records a deterministic placeholder for an original term. If the
placeholder is already taken by a different original (a truncated-HMAC
collision), a -2, -3 … disambiguator is appended and the resulting
placeholder is returned. lookup retrieves the original for a placeholder
string. Both are idempotent under repeated identical input.
assign
async
¶
assign(scope: str, placeholder: str, original: str, *, request_id: str | None, term_type: str | None = None) -> str
Record placeholder → original for scope. Returns the final placeholder.
If placeholder is already mapped to original (idempotent re-mask),
it's a no-op and the same placeholder is returned. If it's mapped to a
different original (HMAC collision), a -N disambiguator is appended
until a free slot is found (fail-closed if exhausted). The write is
serialized under the vault asyncio.Lock (invariant 13).
lookup
async
¶
Return the original for placeholder (lock-free WAL read).
Engine¶
The pure masking engine (no LiteLLM dependency).
derive_placeholder¶
lexvault.engine.derive_placeholder ¶
derive_placeholder(term: str, org_key: str, scope: str, placeholder_format: str = '[LEX-{code}]') -> str
Derive a deterministic placeholder for term under (org_key, scope).
HMAC-SHA256(org_key, scope\x1fterm) → first 5 bytes → base32 → first 8
chars. The scope and term are separated by a unit-separator (\x1f) so a
scope/term boundary can't be ambiguous. The result is formatted with
placeholder_format (which must contain {code}).
mask¶
lexvault.engine.mask
async
¶
mask(text: str, *, detector: Detector, vault: MappingVault, org_key: str, scope: str, placeholder_format: str, request_id: str | None) -> str
Mask dictionary/regex terms in text → text with placeholders.
Single rebuild from detector spans (invariant 10). Each placeholder is recorded in the vault (collision-resolved). Already-present placeholders are pre-excluded by the detector (idempotency, invariant 16). Fail-closed: a vault error propagates (invariant 11).
unmask¶
lexvault.engine.unmask ¶
Replace placeholder-namespace spans in text with their originals.
Only spans matching placeholder_namespace_re are considered, and only
those that resolve to a stored mapping are replaced (invariant 17: a
user-typed lookalike with no mapping is left untouched). Synchronous — uses
the vault's lock-free WAL read. Fails closed: a vault error propagates.
Detector¶
Detector¶
lexvault.detector.Detector ¶
Detect dictionary + regex spans in text.
Constructed ONCE per guardrail (the Aho-Corasick automata and compiled
regexes are reused across requests). placeholder_namespace is the
regex string used to pre-exclude already-present placeholders.
Case sensitivity: case-insensitive terms are collected in a lowercased
automaton and matched against a lowercased copy of the text; case-sensitive
terms are collected in an exact automaton and matched against the original
text. str.lower is length-preserving for ASCII (the realistic domain for
enterprise dictionary terms); Unicode case-folding that changes length is
out of scope for v0.1 and documented as such.
find_matches ¶
Find all non-overlapping dictionary + regex spans in text.
Overlaps are resolved longest-then-leftmost. Spans that overlap a placeholder-namespace region are pre-excluded (invariant 16: an already-present placeholder in conversation history is left untouched). Returns matches sorted by start position.
Streaming¶
PlaceholderBuffer¶
lexvault.streaming.buffer.PlaceholderBuffer ¶
Accumulate text, restore placeholders, emit only text that has provably left any placeholder boundary.
Usage (streaming restore)::
buf = PlaceholderBuffer(window, namespace_re)
for chunk in chunks:
buf.feed(chunk_text)
yield buf.drain_restored_with_vault(vault._lookup_sync) # safe text
tail, partial = buf.flush_restored_with_vault(vault._lookup_sync)
if partial: # dangling partial → fail closed
...
drain_restored_with_vault ¶
Emit text that has provably left any placeholder boundary, restored.
We hold back the trailing <= window chars PLUS any partial
placeholder opener that straddles the window cut (an opener in the ready
portion whose closing ] would fall in the held window). This is the
RC4 correctness core: a placeholder is never split across an emit, so
_restore_inplace always sees complete placeholders.
We ALSO hold back a trailing proper prefix of the opener (e.g. [,
[L, [LE, [LEX) at the end of the ready slice — if the held
window completes it into a full placeholder, emitting the prefix now
would split the placeholder across an emit (CS-E1: the straddle leak).
flush ¶
Drain the buffer on stream end.
Returns (remaining, partial_in_namespace). partial_in_namespace is
True if the held tail contains an unclosed placeholder opener (the caller
should fail-closed / sanitize rather than emit a partial that could be a
real leaked placeholder).
flush_restored_with_vault ¶
Like :meth:flush but restores complete placeholders in the tail.
SseReframer¶
lexvault.streaming.reframer.SseReframer ¶
Accumulate raw SSE bytes; yield complete frames one at a time.
Usage::
reframer = SseReframer()
reframer.feed(b"...raw bytes...")
for frame in reframer.drain_complete():
...
# On stream end:
leftover = reframer.remaining()
drain_complete ¶
Pop and return all fully-terminated frames currently buffered.
Each returned Frame's raw includes its terminating blank line. The
buffer keeps any trailing partial frame.