Token budgeting

n   = tokens("some prompt text")       # Int
usd = cost(n, 500, 0.15, 0.6)          # Float, USD at the prices you pass

if tokens(prompt) < 100000 { ai prompt }

tokens

Counts with the cl100k_base tokenizer - exact for OpenAI models, close enough elsewhere for budgeting. Use it as a guard before spending, and to decide when a session needs trimming.

cost

Arithmetic over prices you supply, in USD per million tokens:

cost(in_tokens, out_tokens, in_per_1m, out_per_1m)

Use your provider's current rates. Until 0.59 cost also took a model name and priced it from a built-in table; vendors reprice often enough that the table was wrong more than it was right, so it was removed, and cost(model, in, out) now fails with this form in its message. --trace and ecko profile report token counts, and no estimated dollar figure.

The hard budget

ECKO_AI_MAX_CALLS caps paid provider calls. For a script, that means every call the run makes; in a server, every call one request makes (below). The call that would exceed it fails rather than spending. Set it in anything unattended.

Everything counts: every vote, every retry, every tool round, every streamed call, every embedding, and mock calls too - so a budget is testable offline. Cache replays do not count, because they do not call anything.

Embeddings are worth naming, because they are easy to forget and they arrive in loops. embed and embed_all are billed calls on a billed endpoint, one per text: indexing a thousand documents for retrieval is a thousand calls, and the budget is what stops that running away unattended.

In a server

Inside http.serve the budget is per request. Each request starts at zero and can spend up to ECKO_AI_MAX_CALLS. Work the handler hands off counts against that request too: pmap workers, tasks, tool rounds and the feeder of a streamed response. A server can run for months, and the cap stops one runaway request without ever cutting off the next.

A WebSocket handler (on_ws:) gets the same budget for each message: every message ws.recv returns starts a fresh count, so a socket held open all day never runs out.

The error says which budget ran out:

try { answer = ai "..." } catch (e) {
    if get(e, "kind") == "budget" {
        print("over budget: {get(e, "calls")} calls in this {get(e, "scope")}")
    }
}

scope is "request", "message", or "process" for a script.

Why the ceiling is higher than it looks

The dials multiply:

So ai[T] "..." votes 5 can spend 20 calls, and a typed ai body inside a failing string contract can reach 16 calls plus verification for one invocation.

None of this is hidden, and none of it is wrong - retrying a failed coercion is what you want. But it is why "one ai call" is not one call, and why the hard budget exists.

Seeing where it went

ECKO_AI_TRACE records every call with its tokens, latency and retry count. When a bill is surprising, the trace file is the answer, and it works in mock mode too.