Keep File-Shaped Evidence

The Thing That Is Too Big For the Journal

Chapter 8 said it and every chapter since has kept it: bulk bytes never enter the journal. A screen recording, a multi-gigabyte log, a rendered report — none of these is a transaction, and none of them belongs in an order that is replayed on every boot.

But they are still evidence. Something has to say a recording exists, how long it is, which run it belongs to, and that the bytes on disk are the bytes that were written. And that "something" is exactly the thing a journal is good at.

Evidence is that split done once, honestly: the bytes live in a vault, and one journal fact says what they are. The whole facility exists to make those two halves impossible to separate.

The design behind it — the ordering, the keel evidence, and the decisions that were closed — is The Evidence Facility.

Declaring One

An evidence kind is a type that names itself and says what finished evidence of that kind records about itself. Identified is the trait you already met in chapter 2; EvidenceKind is the second half:

use harmos::{DefinitionId, EvidenceKind, Identified};
use serde::{Deserialize, Serialize};

/// Your account of one recording. Harmos declares no field of it.
#[derive(Serialize, Deserialize)]
struct Upload {
    filename: String,
    content_type: String,
    taken_at: u64,
}

struct Recording;

impl Identified for Recording {
    fn definition() -> (DefinitionId, u32) {
        (DefinitionId::new("recording"), 1)
    }
}

impl EvidenceKind for Recording {
    type Details = Upload;
}

Details is the typed payload the finalize fact carries — the producer's own filename, the content type it arrived as, the id it has in your application, whatever the kind is about. It is declared here, on the kind, for the same reason every other fact in this runtime declares its shape: a payload nobody declared is a payload two ends decode differently one release apart. A kind with nothing of its own to say declares type Details = ();.

Declare where the bytes live and which kinds this build keeps, in one line each, and the catalog is frozen at boot exactly as your transactions and streams are:

use jsonl_adapter::EvidenceDirectory;

let runtime = Runtime::<Editor>::builder(origin, ())
    .register(Transactions::catalog())
    .evidence(
        EvidenceDirectory::at("/var/lib/acme/evidence")?,
        [Recording::definition()],
    )
    .runtime()
    .await?;

Leave the declaration out and nothing breaks: every verb refuses EvidenceError::Unserved on the same code path a declared vault takes, so a memory-only test and a persisted deployment differ in what they answer rather than in how they are written.

Writing One

Evidence is addressed by three things you choose — its kind, a scope, and a key — and there is no fourth. There is no evidence id to keep, to match up, or to lose:

let draft = runtime.evidence.open::<Recording>("session-7", "capture.log", Upload {
    filename: "camera-3.mp4".to_owned(),
    content_type: "video/mp4".to_owned(),
    taken_at: 1_754_000_000,
}).await?;
draft.write(b"the first line\n").await?;
draft.write(b"the second line\n").await?;

let recording = runtime.evidence.finalize(&draft).await?;
assert_eq!(recording.details::<Recording>()?.content_type, "video/mp4");

Your account is given at open, because that is the one call that names the kind — which is also what lets finalize_scope end drafts of several kinds at once, and what makes an unfinished draft describe itself already.

That finalize line is the whole facility. finalize makes the staged bytes durable, reads them back to recompute their digest and length — never trusting what the writer said it wrote — records one journal fact carrying both — and your account beside them, in the same entry — waits for that fact to be durable, and only then gives the evidence its finished name.

Nothing in that order is decorative:

Why the Order Is the Feature

A crash before the fact leaves a draft nothing names. list will not show it, a read will not find it, and the next boot meets it as an unfinished draft you can resume or abandon. A partial capture is never mistaken for evidence.

A crash after the fact leaves evidence whose bytes have not been renamed yet. The fact is already history, so the evidence is already true, and the first read completes the rename on its way in.

What cannot happen is finished evidence your application state never heard about. That failure — a durable, verified, invisible evidence that nothing will ever find — is what this facility was built to make unrepresentable.

Reading One, at Any Stage

Reads are bounded, and the same bounded read serves a draft and a finished evidence. That is what the shared key buys: a viewer tailing a capture does not change code when the capture finalizes underneath it.

let mut at = 0;
loop {
    let window = runtime.evidence.read::<Recording>("session-7", "capture.log", at, 64 * 1024).await?;
    if window.bytes.is_empty() {
        break;
    }
    sink.write_all(&window.bytes)?;
    at = window.next();
}

Reading past the end is not an error — offset clamps, bytes comes back short, and requested tells you it happened. Asking for more than the vault's ceiling is an error, because a bound you got wrong is worth hearing about.

list folds a whole scope from the journal, drafts and finished evidence together, one entry per key:

for evidence in runtime.evidence.list("session-7") {
    let upload = evidence.details::<Recording>()?;
    match evidence.finalized {
        Some(at) => println!("{} · {} bytes · digest {:x}", upload.filename, evidence.length, at.value),
        None => println!("{} · {} bytes so far", upload.filename, evidence.length),
    }
}

This scope holds one kind, so one details::<Recording>() serves the whole listing; a scope holding several matches on evidence.kind first. details decodes through the declaring kind and nothing else: name a kind the evidence was not opened under and it refuses EvidenceError::Mismatched rather than handing you a shape that happened to parse. Nothing here is a second registry — the account came out of the same fact the evidence did, folded at boot from the same order.

A Recorder: Handing a Child Its Output Path

Chapter 12's sidecar had one loose end. Bulk data never rides the protocol, so the child writes "to paths the embedder gave it" — and where those paths came from was left to you. An external draft is the answer.

Evidence::external opens a draft the runtime does not write, and hands back the path to give a child before it is launched:

struct Capture;

impl Resident<Editor> for Capture {
    type Error = String;

    async fn attempt(&mut self, mut context: Service<Editor>) -> Result<(), Fault<String>> {
        let evidence = context.evidence().clone();
        let draft = evidence
            .external::<Recording>("session-7", "screen.mkv", Upload {
                filename: "screen.mkv".to_owned(),
                content_type: "video/x-matroska".to_owned(),
                taken_at: 1_754_000_000,
            })
            .await
            .map_err(|why| Fault::Environmental(why.to_string()))?;

        // The path exists before the child does.
        recorder.call(&Start { output: draft.path().expect("a filesystem vault").to_owned() }).await?;

        context.stopping().await;
        evidence
            .finalize(&draft)
            .await
            .map_err(|why| Fault::Environmental(why.to_string()))?;
        Ok(())
    }
}

The digest and length recorded are measured from what the child actually wrote, because they are recomputed from the stored bytes. The child is never asked, and never believed.

Writing to an external draft from your own process refuses EvidenceError::Foreign. Two writers appending to one capture interleave, and the runtime does not pretend otherwise.

A Vault May Refuse a Path

external is a capability, not a guarantee. A backend with no filesystem behind it answers EvidenceError::Unsupported rather than inventing a path, and the refusal lands at the one line that asked for it.

A Tail: Watching Something Grow

There is no log substrate here, deliberately. A tail is an ordinary Resident that appends into a draft, and watch is how anything else follows it:

let draft = runtime.evidence.open::<Recording>("run-3", "device.log", Upload {
    filename: "device.log".to_owned(),
    content_type: "text/plain".to_owned(),
    taken_at: 1_754_000_000,
}).await?;
let mut growth = runtime.evidence.watch(&draft)?;

while growth.changed().await.is_ok() {
    let length = *growth.borrow_and_update();
    view.set_length(length);
}
// The channel closed: the last length you saw is the length it ended at.

Watching is opt-in all the way down. A draft nobody watches costs nothing — a draft a foreign process is writing is not even looked at until somebody subscribes, and the looking stops when the last watcher goes. There is no changed() call for a producer to remember, because a producer that has to remember will one day forget.

The final revision always precedes the close: the last value a watcher sees is the length the evidence ended at, and the channel closing after it is how the watcher learns there will be no more.

Ending a Draft Without Finishing It

runtime.evidence.abandon(&draft).await?;
runtime.evidence.abandon_scope("run-3").await?;
runtime.evidence.finalize_scope("session-7").await?;

abandon moves the bytes into the vault's quarantine rather than deleting them. A partial capture is the only account of an interrupted write, and a store that tidied it away would be destroying evidence in exchange for a tidier directory. No fact ever named it, so nothing that folds the journal ever saw it.

finalize_scope finalizes every open draft in one scope, each on its own terms. There is no rollback: a fact is truth the moment it lands, so a refusal partway leaves what was already finalized finalized and visible, and the rest still open.

What You Have Now

One home for file-shaped evidence, where the bytes and the fact about them cannot come apart. A typed account of each piece of evidence, declared by its kind and carried in that same fact. A key that means the same thing before and after finalization. A bounded read that serves both. A watch that polls nothing nobody is watching. And a boot that learns what exists by folding facts, never by listing a directory.

Chapter 18 counts the surface. This facility adds six names to it — Evidence, EvidenceRecord, EvidenceKind, Draft, Excerpt, and EvidenceError — plus the Vault port for whoever writes a backend. There is no evidence id among them, and that absence is the design.