One API for every
AI provider.
Loom sits between your app and every LLM provider. Tell it what matters most, and it handles routing, failovers, and provider selection automatically.
Everything you need to run AI in production.
A full stack of infrastructure primitives in one Python package — from intelligent routing to structured outputs. Backward-compatible in v2.0, semver-stable, observable end-to-end.
Intelligent routing
State intent, not a vendor. router="cheapest" | "fastest" | "highest_quality" | "balanced", providers=[...] in order, or just a prompt — Loom picks the provider and model.
Automatic fallback
FallbackPolicy(retries=3, providers=[...]) walks the chain on timeouts, rate limits, and 5xx — each attempt under your retry policy. Cross-vendor equivalence map included.
Health monitoring
Per-provider circuit breaker with EWMA latency, rolling failure counts, and rate-limit cooldown. Routing skips open circuits and deprioritizes recovering ones.
Load balancing
Spread traffic across a pool: round_robin, weighted, least_latency, or least_failures. Wired once via Loom(balancer=...).
Provider benchmarking
compare(providers=[...], prompt=...) runs candidates concurrently and returns latency, tokens, cost, and output side by side — cheapest, fastest, and best named.
Structured outputs
generate(schema=User) returns a validated Pydantic model, not a dict. Native response_format, tool-JSON, and response_schema per provider, with prompt-driven fallback.
Analytics & observability
client.analytics() — summary, by-provider, by-model, recent — over a zero-config in-memory sink. Plus a SQLite sink and read-only Flask dashboard for history.
Unified AI API
One stable contract — generate(provider, model, prompt) — across every vendor and modality. The explicit path is unchanged and fully supported in v2.
Native SDK preservation
Each vendor integrated with its native SDK. Prompt caching, grounding, streaming — preserved, not flattened.
Centralized retries
Exponential backoff with jitter, configurable per client. RetryPolicy(max_attempts=3, base_delay=0.5).
Prompt caching
Vendor-native prompt caching for OpenAI, Anthropic, DeepSeek, and Gemini — discounts already applied in cost.
Batch processing
OpenAI and Anthropic batch endpoints behind a single submit_batch() with poll and wait primitives.
Request deduplication
Single-flight coalescing collapses identical concurrent calls into one upstream request.
Cost tracking
Every result carries cost.usd and cost.local computed from the catalog. Pricing tracked per model.
API key centralization
AWS Secrets Manager, GCP Secret Manager, and HashiCorp Vault backends in the box. Keys live in one place.
OpenAI-compatible adapters
Register any OpenAI-shape vendor in ~10 lines with register_openai_compatible(key, base_url, env).
One control plane for every AI call.
Cost, latency, retries, cache hits, and provider health — captured for every request, surfaced in a single dashboard.
The path of a single generate() call.
Every request flows the same way — through cache, router, and adapter to the upstream vendor, then back with cost and a structured log. Optimization happens once, in the middle. New providers plug in without touching consuming code.
Upstream AI providers
Designed for engineers who ship.
State intent, not a vendor. The same generate() call routes to the right provider, falls back on failure, and can return a validated object — with usage and cost on every response.
import loom
# State intent — Loom picks the provider and model, health-aware.
result = loom.generate(router="cheapest", prompt="Summarize this transcript.")
print(result["text"])
print(result["cost"]["usd"]) # 0.0000041
print(result["_router"]["used"]) # "gemini:text:gemini-2.5-flash"
# A provider order, a named strategy, or nothing at all:
loom.generate(providers=["gemini", "openai"], prompt="...")
loom.generate(router="fastest", prompt="...")
loom.generate(prompt="...") # fully automatic — balanced
from fastapi import FastAPI
from loom import AsyncLoom
app = FastAPI()
aclient = AsyncLoom.from_env()
@app.get("/answer")
async def answer(q: str):
# Same intent-based routing, fully async.
result = await aclient.generate(router="fastest", prompt=q)
return {"text": result["text"], "cost": result["cost"]["usd"]}
from pydantic import BaseModel
from loom import Loom
class User(BaseModel):
name: str
age: int
client = Loom.from_env()
# Validated object out, provider-agnostic. pip install loom[structured]
user = client.generate(prompt="Extract the user from: ...", schema=User)
assert isinstance(user, User)
print(user.name, user.age)
from loom import Loom
client = Loom.from_env()
report = client.compare(
providers=["openai", "anthropic", "gemini"],
prompt="Explain quantum entanglement.",
)
for row in report:
print(row.provider, row.model, row.latency_ms, row.cost_usd)
report.summary.cheapest.provider # lowest cost row
report.summary.fastest.provider # lowest latency row
from loom import Loom, FallbackPolicy
client = Loom.from_env()
# Survive an outage — walk the chain on retryable failures.
result = client.generate(
prompt="Draft a launch announcement.",
router="balanced",
fallback=FallbackPolicy(retries=3,
providers=["gemini", "openai", "anthropic"]),
)
result["_router"]["used"] # provider that answered
result["_router"]["tried"] # every provider attempted
Built-in savings on every call.
Response cache, vendor prompt cache, smart routing, and batch APIs — orchestrated automatically. Targets validated in production.
Build AI infrastructure once.
Unify providers, optimize costs, and ship AI products faster. One install. Every vendor. Production from day one.