ecko
Use casesClassification pipelines
Trust and safety moderation
pmap for fan-out, hard budgets via ECKO_AI_MAX_CALLS so a runaway queue cannot produce a runaway invoice.
The problem
Moderation pipelines classify posts, images' alt text, and profile fields under latency pressure. Spike traffic must not spawn unbounded parallel model calls - both for cost and for provider rate limits.
Why Ecko
- pmap
- Fan out over a batch with controlled concurrency. The same abstraction works in mock mode tests with a handful of fixtures.
- ECKO_AI_MAX_CALLS
- A process-wide ceiling on inference calls. When the queue dumps ten thousand items, the worker stops calling the model instead of silently spending ten thousand times.
- Tracing for appeals
- When a user appeals a ban, JSONL traces show exactly which policy version and model output fired.
In practice
# ECKO_AI_MAX_CALLS=5000
fn moderate_batch(items: list(str)) {
pmap(moderate, items, workers=8)
}
Try it on your workload.