std.fs

Files and directories. Gated on fs:read and fs:write for package code; root code has full authority. Both capabilities take an optional path scope (fs:write:./logs), so a package can be confined to one directory.

import std.fs

fs.read("notes.txt")            # string
fs.read_bytes("logo.png")       # bytes
fs.write("out.txt", text)
fs.append("log.txt", line)

fs.open("huge.csv")             # a stream, read or written a piece at a time
fs.create_new("app.lock", pid)  # true if this call created it, false if taken

fs.exists(p)  fs.is_file(p)  fs.is_dir(p)  fs.size(p)
fs.exists(p, follow: false)     # the link itself, even a dangling one
fs.is_symlink(p)  fs.modified(p)  fs.canonical(p)  fs.symlink(target, link)
fs.readlink(p)                  # the link's literal target
fs.set_modified(p, ms)          # set the mtime, unix millis
fs.list_dir(dir)                # names
fs.walk(dir)                    # recursive entries
fs.walk(dir, on_error: fn(e) ...)   # skip and report what can't be read
fs.glob("src/**/*.ecko")

fs.mkdir(p)  fs.copy(a, b)  fs.rename(a, b)  fs.remove(p)
fs.mode(p)   fs.chmod(p, "755")
fs.owner(p)  fs.chown(p, uid, gid)    # { uid, gid }; follow: false for a link itself

copy reads its source and writes its target, so for package code it needs fs:read on the first path and fs:write on the second. rename needs fs:write on both.

Lock files and exclusive create

fs.create_new(path, data) creates the file and writes data only if nothing is at the path, and says which happened: true when this call created it, false when it was already there. Of two processes racing for the same path, exactly one gets true, which is what a lock file needs; checking with fs.exists and then writing lets both through. A symlink at the path, even a dangling one, counts as taken. Any other failure, such as a missing directory, throws { kind: "fs" }.

if not fs.create_new("backup.lock", "{os.pid()}") {
    error({ kind: "busy", message: "another backup is running" })
}

For a stream, fs.open(path, "x") does the same and throws { kind: "fs" } when the path is taken. Both need fs:write.

Owners

fs.owner(p) returns { uid, gid }, and fs.chown(p, uid, gid) sets them. Both follow a symlink by default; follow: false reads or sets the owner of the link itself, which restoring a link needs. Changing an owner usually needs root. Owners are numbers on Unix only: on Windows both calls throw { kind: "fs" }. owner needs fs:read and chown needs fs:write.

Making a file executable

fs.write("./deploy.sh", "#!/bin/sh\necho deploying\n")
fs.chmod("./deploy.sh", "755")

Without this the only way to mark a script runnable was os.exec("chmod", ..), which needs the exec capability - so a package allowed to write a hook had to be trusted to run any program on the machine to finish the job. chmod needs fs:write, because setting the executable bit is a write to that file; fs.mode needs only fs:read.

The mode is a string. "755", not 755. Ecko has no octal literal, so the readable spelling is not available as a number, and the number you would reach for means something else entirely: 755 in decimal is 0o1363 - setuid, and readable by nobody. A numeric mode is refused rather than silently applied, and the error shows the string to write instead.

fs.mode gives back the same spelling, so the two round-trip:

fs.mode("./deploy.sh")     # "755"

Unix only. Windows decides what is executable from the file extension, so there is no mode to set and both calls say so rather than pretending.

Text or bytes

modified is unix milliseconds, the same clock as time.now(), so the two compare without converting anything.

is_symlink is the one that is easy to miss: exists, is_file and is_dir all follow a link, so a symlink - including a dangling one, which exists reports as absent - is invisible to them. To ask whether anything is at a path, link or not, write fs.exists(p, follow: false): it looks at the link itself, so a dangling link or a lock held as a symlink counts as there.

canonical is a realpath: absolute, with symlinks and .. resolved. The path has to exist, because resolving a link means reading it.

size is a stat, not a read: it answers in bytes without opening the file, so it works on something far larger than memory. len(fs.read_bytes(p)) gives the same number by loading the whole file, which is what you are avoiding. A directory is refused rather than answered with the size of its entry.

read returns a string and errors on invalid UTF-8. read_bytes returns bytes and always works. Use read_bytes for anything that is not certainly text - the strict default is what stops a binary file from becoming a string full of replacement characters.

Reading in pieces

fs.open hands back a stream rather than the file's contents, so the memory a program spends is a piece rather than the file:

import std.fs
import std.io

s = fs.open("huge.csv")
for line in io.lines(s) {
    process(line)
}
io.close(s)

"r" is the default. "w" truncates an existing file or creates one, and "a" appends, so fs.open(p, "w") and fs.open(p, "a") are the streaming forms of fs.write and fs.append. Reading needs fs:read, the two writing modes need fs:write, and both are checked against the same path scopes as the rest of the module. Anything else, including an unknown mode string, is an error that says which modes exist.

fs.read and fs.read_bytes refuse a file larger than ECKO_LIMIT_ALLOC (256 MiB by default, see limits) rather than trying to hold it, and the error names fs.open as the way through. A stream has no such ceiling: the cap applies to each piece you ask for, not to the file.

The cap is not this module's alone. csv.read, json.read, config.load and the ecko.json loader all read a whole file to parse it, and all go through the same reader - so a file too big to hold is the same refusal whichever module asks for it. http.serve's static handler holds a response body in memory too, and answers 500 rather than reading past the cap. The refusal is kind: "limit", carrying limit, env and path.

hash.sha256(fs.open(path)) digests a file without loading it - see std.hash.

Paths

fs.join(a, b, ...) builds a path, and fs.basename(p), fs.dirname(p) and fs.extension(p) take one apart. Use these rather than concatenating with "/", so the code works on Windows too.

Walking and matching

list_dir is one level; walk recurses and yields entries shaped {path, is_dir, kind, size, mtime}. kind is "file", "dir", "symlink" or "other"; size is in bytes (0 for a directory); mtime is unix milliseconds, the same unit as fs.modified. walk does not follow symlinks: a link to a directory is one "symlink" entry and is not entered.

walk on a directory that does not exist or cannot be read - the one you named, or any below it - is an fs error naming the path, not an empty list. To list everything that can be read and hear about the rest, pass on_error:. Each unreadable directory goes to it as the same {kind: "fs", message, path} the walk would have thrown, its contents are left out, and the walk carries on:

mut skipped = []
entries = fs.walk(home, on_error: fn(e) { skipped = push(skipped, e.path) })

glob takes a pattern with * and ** and skips what it cannot read. A malformed glob pattern such as "[" is a parse error.

fs.match(path, pattern) tests one path against a pattern, path first. If only the first argument has wildcards, the call is swapped and match throws rather than answering false.

for entry in fs.walk("src") {
    if entry.kind == "file" and fs.extension(entry.path) == "ecko" { check(entry.path) }
}

Names that are not valid UTF-8 are listed with each bad byte shown as a private-use character (U+F780 to U+F7FF); pass the name back to any fs function and it opens the original file.

Links and timestamps

fs.readlink(p) returns what a symlink points at, exactly as stored, even if the target is missing. fs.set_modified(p, ms) sets a file's modification time to ms unix milliseconds, so a copy or restore can keep the original time. It needs write permission like fs.write; readlink needs read permission.

Temporary files

dir = fs.temp_dir()
f = fs.temp_file()               # or fs.temp_file(".json") for a suffix

temp_file gives a unique path in the system temporary directory. Nothing cleans up after you - remove what you create.

Both write into the system temp directory, so a package on a scoped fs:write needs that directory covered by its scope.

Errors

Everything raises { kind: "fs", path, message } on failure; copy and rename add to, the destination. The path field is there so a handler can report which file, which is the thing you always want and often lose:

try { data = fs.read(p) } catch (e) {
    match get(e, "kind") { "fs" => print("cannot read {e.path}") _ => error(e) }
}

Paths from untrusted input

A path built from user input can escape the directory you meant. ../../etc/passwd is a path. If a request names a file, validate it - or better, do not use fs for it at all: web.static resolves safely and is the right tool for serving files.

Capabilities do confine a package to a directory, if you grant them that way: fs:read:./uploads covers ./uploads and nothing else, and neither .. nor a symlink gets a path out of it. See capabilities.

That protects you from a package. It does not protect a package from its own caller - your top-level code is ungated, so validating a path you built from a request is still your job.