Multimodal / vision

import std.image
img = image.load("chart.png")

ai "what does this chart show?" on img

The on clause attaches image input. The image is a std.image handle, so a resize or crop pipeline flows straight into the call:

small = image.resize(image.load("photo.jpg"), 512, 512)
ai "describe this" on small

Typed and multiple

type Kind = Chart | Photo | Diagram | Screenshot

ai[Kind] "classify this image" on img            # typed vision output
ai "spot the differences" on [before, after]     # several images

on composes with [T], so vision output is coerced and retried like any other typed call.

An ordinary call that carries images

Attaching an image does not put the call on a side path. It counts against ECKO_AI_MAX_CALLS, a typed answer that fails to coerce is retried with the failure fed back to the model, the response is cached, and --trace records it with the tokens the provider charged - the same as any text call.

The cache keys on the image as well as the prompt, so asking the same question about a different picture is a different call and never replays the first picture's answer.

One program, either provider

The handle serializes into whatever the configured provider expects - OpenAI image_url data URLs, Ollama base64 image arrays. The same source runs on both; switching is an environment variable.

Offline

A vision call in mock mode echoes the prompt plus each image's real dimensions:

[AI Mock] describe [image 4x2]

The dimensions are read from the actual file, so the mock proves the image was loaded, decoded and passed correctly - the part of the wiring that actually breaks

while staying deterministic. Typed vision calls return their schema-valid mock as usual.

Restrictions

on composes with [T]. Combining it with using (tools), with (a session), voting, or -> stream is not supported yet and is a clear parse error rather than a runtime surprise.

Size and cost

Images are tokens, often a lot of them. Resize before sending: a 4000-pixel photograph rarely answers a question better than a 1000-pixel one, and costs several times more.

ai "read the label" on image.resize(photo, 1024, 1024)