std.io

Streams, and the console.

import std.fs
import std.io

s = fs.open("data.bin")     # a file, a socket, a pipe, an HTTP body: one type
piece = io.read(s, 4096)    # at most 4096 bytes -> bytes; null at the end
io.close(s)

io.print("hello")           # identical to the global print("hello")
line = io.read_line()       # standard input; null at end of input

One verb set reads everything. A file from fs.open, a socket from net.connect, a child's pipe from proc.stdout, an HTTP body from http.get(url, stream: true), standard input, and the result of ai "..." -> stream are one type. Code that reads a stream does not have to know which one it was handed.

The verbs

Where streams come from

The constructor is the gate; the verbs are not. Holding a stream is the grant, the same way holding a file handle is in any other language. That is why a package can be given fs:read:./data and hand the stream it opened to a helper that has no capabilities at all.

The end is null, a timeout is an error

loop {
    piece = io.read(s)
    if is_null(piece) { break }
    handle(piece)
}

An empty line is "" and the end of the stream is null, so a loop terminates on the null and never on a legitimate empty read. The exception is an end that arrives mid-answer, which raises rather than answering short: io.read_exact under its count, io.read_until with bytes read and no delimiter, and io.read_text or io.read_line on a character the stream cut in half. Asking for exactly eight bytes and getting three is not a shorter answer, it is a broken frame. read_exact and read_until hand back what they consumed, so the bytes are there for the next read; half a character is not recoverable and the read reports it instead.

A read that runs out of time raises an error rather than returning null. If it returned null the loop above would treat a slow peer as a finished one, which is the bug the old proc.read_line timeout could not avoid. Sockets start with the deadline ECKO_NET_TIMEOUT_MS gives them, 30 seconds by default. Pipes and HTTP bodies start with none.

A file and the console take a deadline and ignore it. A file read does not wait on anything, so the deadline has nothing to bound; standard input is not wired to a clock either, so io.timeout(io.stdin(), 300) is accepted and does nothing. The deadline is enforced where a read can genuinely stall: sockets, child pipes and HTTP bodies.

io.timeout(s, 5000)     # five seconds per read
io.timeout(s, null)     # wait as long as it takes

The deadline bounds each read the stream makes, not the verb you called. io.read_line on a peer that sends a byte every second finishes the line rather than timing out at five, because no single read waited that long. The deadline is there to stop a stream that has gone silent, and that is what it does.

Closing while a read is in flight

io.close interrupts a read that is already in flight, rather than queueing behind it. A task blocked in io.read (or io.read_line, io.read_exact - any verb that reads from the stream) holds the stream for the whole wait, so closing it raises a flag the waiting read checks: the read gives up with an error of kind io saying the stream is closed, and the close returns. That is what makes closing a usable way to stop a reader.

The pipe and the HTTP body check that flag while they wait. A socket, a file and the console sit inside a blocking system call instead, where there is nothing to check, so a close on one of those still waits for the read to finish on its own - bounded by ECKO_NET_TIMEOUT_MS for a socket, and by the data arriving for the rest.

cancel(task) interrupts a reader the same way and reaches the same streams: a child's pipe and an HTTP body poll for it, a socket, a file and the console cannot.

A piece is bounded, the stream is not

ECKO_MAX_ALLOC (256 MiB) caps what one read may produce, and nothing caps the stream. That is the point: a program handles input larger than memory as long as it never asks for all of it at once.

A read_until or a line whose delimiter never arrives stops at the cap with an error, and the bytes it read are still there for io.read. fs.read on a whole file, io.read_all on standard input and a buffered HTTP body are all refused above the same cap, and each error names the streaming form to use instead.

Bytes or text

Sockets, files, pipes and HTTP bodies are byte streams; io.lines and an ai stream are text. for over a stream binds its own unit, so for piece in s gives bytes and for line in io.lines(s) gives strings.

for line in io.lines(fs.open("access.log")) {
    if str.contains(line, " 500 ") { print(line) }
}

io.read_text decodes on the way out and errors on invalid UTF-8 rather than producing replacement characters. The byte verbs refuse a line stream from io.lines, because its terminators are already gone and reading it as bytes would hand back something that was never on the wire.

Reading standard input

This is what lets an Ecko program be a pipeline stage:

import std.io

mut n = 0
for line in io.lines() {
    n = n + 1
    print("{n}: {line}")
}

cat access.log | ecko number.ecko

io.read_line() is the same thing one line at a time, and io.read_all() reads the lot as one string. A trailing \r is stripped along with the \n, so a file written on Windows compares equal to what you expect rather than failing every match by one invisible character. A final line with no terminator is still returned.

io.stdin() hands standard input over as a stream, for code that takes one and should not have to care. It returns the same stream every call and shares its buffer with io.read_line, so the two mix freely. std.term reads the same descriptor in raw mode, so a program should drive standard input through one of them or the other.

Do not close a console stream. There is one per process and it is kept for the program's life, so io.close(io.stdin()) costs you standard input for good, and the same goes for io.stdout(). Closing a line stream you derived from one does it too, since an explicit close reaches through. The descriptor stays open and print keeps working, but the stream does not come back. Closing is for things you opened.

Capability

Reading standard input needs no grant. A pipeline stage is handed its input by whoever started it, exactly the way it is handed its arguments - there is no authority here to withhold, which is why std.cli is ungated too.

Reading a file is std.fs and does need fs:read.

Errors

Failures raise { kind: "io" } with the verb and what went wrong in the message:

io.read_exact: stream ended after 3 of 8 bytes

What lives elsewhere