Embeddings & RAG

An embedding turns text into a vector, so that similarity becomes arithmetic.

v = embed("some text")  # [Float]
vs = embed_all(["a", "b", "c"])  # [[Float]]
cosine(v, v)  # 1.0, to float precision

embed, embed_all and cosine are global built-ins - no import.

Retrieval

There is no vector-store module to import: an index is a list of embeddings, and retrieval is cosine and a sort. Embed each document once, then rank them against the question:

docs = [
    { id: "moon", text: "The moon is made of basalt rock." },
    { id: "cheese", text: "Gouda is a mild Dutch cheese." },
]
index = map(docs, fn(d) ({ id: d.id, text: d.text, vector: embed(d.text) }))

fn search(question, k) {
    q = embed(question)
    scored = map(index, fn(d) ({ id: d.id, text: d.text, score: cosine(q, d.vector) }))
    sort_by(scored, fn(d) 0 - d.score)[..k]
}

print(map(search("what is the moon made of?", 1), fn(d) d.id))

To answer from what you retrieved, put the passages into a prompt with {input ...} so they cannot pass for instructions, and call ai - the model answers from what you gave it rather than from what it recalls. Save the index with json.write if computing it again would be slow or costly; every embed is a billed call.

std.db and std.rag packaged this until 0.59 and were removed in 0.61.

Offline

Mock embeddings are deterministic vectors, so similarity search, clustering and the whole retrieval path work with no key and give the same answer every run. That makes a RAG pipeline testable in CI, which is unusual and deliberate.

They are not semantically meaningful. Offline, "the moon is rock" and "cheese is dairy" have a stable but arbitrary similarity. Structure is testable; retrieval quality needs a real model.

Configuration

ECKO_AI_EMBED_MODEL picks the model, defaulting per provider - text-embedding-3-small for OpenAI, nomic-embed-text for Ollama.

Chunking, briefly

Embed passages, not whole documents. Split long text into pieces of 100 to 200 words, overlapping by 10 to 20% so a sentence that straddles a boundary is still findable from either side - str.split_whitespace and a slice are enough. Pieces that are too large dilute the match; too small and they lose the context that made them meaningful. Tune against your own retrieval results.