Agent Infrastructure Framework
The resilience, telemetry, and auth layer under every agent product I ship.
Client work — case study anonymized; architecture and metrics are real.
Why a framework
Every agent product needs the same unglamorous things: retries that don't melt providers, telemetry that knows what a token costs, API keys that can be revoked, caching that can be invalidated. This library gives any agent application those for free, so the application stays focused on its domain. Four production systems run on it.
Fail closed, degrade gracefully
The resilience core is a composable runner: a circuit breaker as the outer gate wrapping a retry loop as the inner one, safe across event loops and threads. Configuration is fail-closed — a misconfigured auth setup raises at bootstrap instead of silently running open.
The API-key subsystem stores only SHA-256 hashes, enforces sliding-window rate limits made atomic through a Postgres RPC, caches validations for 30 seconds, and ships as ASGI middleware that attaches proper rate-limit headers.
Telemetry that knows the bill
Metrics export is buffered and batched, with a per-model cost table so every LLM call, tool call, and HTTP request carries its price. Collectors cover sessions, circuit-breaker state, and media handling. The RAG pipeline runs cache → vector retrieve → rerank with O(1) partition invalidation.
879 test functions across 39 files run on every push, alongside ruff and mypy.
Built with
Every technology this project runs on, and the job it does here. Each one links to the other projects it appears in.