ecko
ETL with an AI enrichment stage
Pipelines with |>, plus SQL and CSV in the box. The AI step is a stage, not a separate service.
The problem
Classic ETL extracts and loads fine until someone adds a column that only a model can fill: product description cleanup, address normalisation, entity resolution. The quick fix is a Lambda that calls OpenAI, which splits observability and types across two repos.
Why Ecko
- Pipeline operator
|>chains pure functions and AI steps in one file. Read CSV, enrich, write SQL - one process, one log stream.- SQL and CSV stdlib
- No JDBC driver archaeology for batch jobs. Connect, query, and map rows to structs.
- Tracing across stages
- JSONL traces cover both SQL pull and model enrich in one correlated run.
In practice
rows
|> read_csv("in.csv")
|> pmap(enrich_row, workers=4)
|> write_sql("staging.enriched")
Try it on your workload.