Data-schema validation

Every persistent store nubos-pilot reads back from disk is validated against a versioned JSON schema before the data is trusted. The validator is homegrown (lib/validate.cjs) — no ajv, no runtime dependency, per ADR-0002. Schemas live in lib/schemas/data/<name>.v<n>.json.

Why

The stores started out guarded by hand-written typeof walls, one per file. Two of those walls drift apart the moment a field is added to one and not the other, and the error messages never match. A schema is one source of truth: it describes the shape, the validator reports the exact failing path, and a new field is covered the day it enters the schema.

Two contracts

A store picks one of two behaviours on a violation. The choice follows the store's read contract, not preference.

Hard-throw — object stores that hold a single document. A corrupt file is unrecoverable, so the read throws a NubosPilotError with the store's own code and the validator's failing paths in details. Used via assertValid(data, schema, code, details).

Validate-and-skip — append-only and self-healing stores (JSONL logs, manifests). A single bad line must not sink the whole read, so the line is dropped (and counted), never thrown. Used via validate(data, schema), which returns an errors[] array the caller checks.

Store	Schema	Contract	Code on violation
checkpoints/<task>.json	checkpoint.v1	hard-throw	checkpoint-corrupt
codebase/.hashes.json	codebase-manifest.v1	hard-throw	manifest-invalid-shape
learnings.json	learnings.v1	hard-throw	learnings-store-corrupt
messages/inbox/*.json (write)	message.v1	hard-throw	messages-invalid-record
messages/inbox/*.json (read)	message.v1	validate-and-skip	— (file skipped)
memory/records.jsonl	memory-record.v1	validate-and-skip	— (line skipped)
memory/.manifest.json	memory-manifest.v1	validate-and-skip	— (rebuilt)
metrics/*.jsonl	metrics-record.v1	validate-and-skip	— (line skipped)

Messaging is in the table twice on purpose: send() self-checks the record it just built and throws on a producer bug, while the reader skips a malformed inbox file silently.

Schema versioning and migration

A schema name carries its version (learnings.v1). When a store's shape changes, the new schema is learnings.v2 and a migrator chain moves old data forward on read. lib/migrate.cjs::runMigrators(obj, opts) applies the registered hops (capped at MAX_HOPS) and then calls assertValid against the target schema, so a migrator that produces the wrong shape fails loudly with the caller's code (default data-migration-invalid). See ADR-0013 for the learnings-store precedent.

API

lib/validate.cjs exposes three functions:

• validate(data, schemaName) — returns an errors[] array (empty when valid). Never throws on a data violation. This is the validate-and-skip primitive.
• assertValid(data, schemaName, code, details) — throws NubosPilotError(code, …) with the failing paths in details when validate returns errors. This is the hard-throw primitive.
• listSchemas() — returns the registered schema names.

A bad schema name (no matching lib/schemas/data/<name>.json, or a shipped schema file that is not valid JSON) is a code bug, not corrupt data, and surfaces as data-schema-not-found or data-schema-corrupt. Both are listed under Error Codes.