ecko test

ecko test                  # discover and run everything
ecko test tests/           # an explicit path
ecko test --jobs 4         # run up to four files at once
ecko test --generate file.ecko   # draft tests for a file with AI

Discovery

Two conventions, both honoured:

• anything under tests/, at any depth
• *_test.ecko anywhere in the tree

Both walk the whole tree, so a suite split across subdirectories is found. Four kinds of directory are skipped: vendor/, because a dependency's tests belong to its maintainer and would fail for reasons you cannot fix; target/ and node_modules/, which are build output; and anything beginning with a dot. Symlinked directories are not followed.

A package keeps its tests in tests/. The release workflow's archive step takes only the top-level *.ecko files, so tests/ stays out of the published zip, which is what you want; a root-level <name>_test.ecko would ship with it.

Writing a test

import std.test

test.case("splits on commas", fn() {
    test.eq(split("a,b", ","), ["a", "b"])
})

test.case("rejects an empty name", fn() {
    test.err(fn() validate(""))
})

test.eq, test.ok, test.err and test.fail are the assertions. See std.test.

Each file runs in its own process

State in one test file - module-level variables, a port a fake server bound, os.pid() - never reaches another. Files run one after another by default and their output streams as they go. --jobs N (or -j N) runs up to N files at once; each file's output is buffered and printed in file order, so the report reads the same. Use it when files do not share a port, a database or a fixture directory.

Mock mode is forced

ecko test ignores your API key. Every ai call in a test run answers from the deterministic mock, whether or not a provider is configured.

This is not a convenience, it is the point. It means a test suite over an AI pipeline is fast, free, offline, and identical on every machine - so the suite can run on every commit, and a failure is a real regression rather than a model having a different day. What mock mode cannot assert is answer quality; that is what contracts are for, and they are evaluated in tests too.

Output and exit status

Each file's path, one line per case under it, a summary at the end, and a non-zero exit if anything failed:

tests/parse_test.ecko
  ok splits on commas
  ok rejects an empty name
1 file(s), 2 case(s): 2 passed, 0 failed (0.01s)

A failing case reads FAIL <name> - <why>, for example FAIL totals - expected 2, got 1. A case that called test.skip reads SKIP <name> - <reason>, and the summary counts it apart: 3 case(s): 2 passed, 0 failed, 1 skipped. Skips do not change the exit status. The summary mentions skips only when there are some.

--generate

ecko test --generate parser.ecko

Reads the file and drafts a test file for it. It is a starting point, not a substitute for deciding what the behaviour should be - read what it writes before trusting it. With no provider configured this produces mock output, so it is one of the few commands that genuinely needs a key.