ecko
Multi-tenant AI features
Per-tenant keys, per-request cost(), and JSONL traces that carry a tenant id - so inference spend is attributable without a separate metering microservice.
The problem
SaaS products add summarisation, classification, or generation per customer account. Finance needs to know what each tenant cost you this month; support needs to replay what the model returned for a specific ticket. The usual split is application code plus a wrapper SDK plus a logging pipeline plus a spreadsheet - none of which agree on token counts.
Why Ecko
- Built-in metering
cost()andtokens()after everyaicall. Log them with your tenant id in the same process - no sidecar counting HTTP responses from the provider.- Trace per request
ECKO_TRACEappends one JSON line per inference call. Ship traces to object storage partitioned by tenant, or attach the relevant lines to a support export.- Provider config per env
- Point staging at mock mode and production at your vendor via environment variables. The feature code path is identical; only the backend changes.
In practice
fn summarise(tenant_id: str, text: str) -> str {
let out = ai "Summarise for the user" from text
log.info("ai", tenant=tenant_id, cost=cost(), tokens=tokens())
out
}
Try it on your workload.