nubos-pilot

Appearance

ADR-0016: Project-Level Close + Archive Lifecycle ​

Context and Problem Statement ​

A nubos-pilot project is bottomless by construction. roadmap.yaml can hold an unbounded sequence of milestones; each milestone has its own verification (M<NNN>-VERIFICATION.md) and validation (M<NNN>-VALIDATION.md); nothing in the data model marks "the project is over" or "no more work is planned". Two practical pain points fall out of this:

  1. No project-wide verification step. Users asked "how do I check at the end of the project that every milestone was correctly implemented?" and the answer was a manual walk: open every M<NNN>-VERIFICATION.md and read it. Mistakes are silent; an entirely missing VALIDATION.md is invisible without a directory listing. The system had per-milestone gates but no aggregate.

  2. No way to start a successor project in the same workspace. /np:new-project refused with project-already-initialized whenever .nubos-pilot/PROJECT.md existed. The only escape was rm -rf .nubos-pilot/: destructive, loses history, loses learnings and solutions, breaks the institutional memory the project accumulated. The workspace effectively became single-use.

The pain points are coupled: the right time to start a successor is right after a clean close, and the right artefact to base archive decisions on is the same aggregate verification that would answer "is this project done?". Two features, one lifecycle.

The Rule ​

A nubos-pilot project occupies one of four lifecycle states (idle, active, completed, archived). /np:close-project is the only path from active to completed, writes PROJECT-SUMMARY.md, and flips roadmap.yaml.project_status = "completed". /np:new-project Phase -1 (and /np:archive-project) is the only path that moves the project content into .nubos-pilot/archive/<slug>-<YYYYMMDD>/, with learnings/ and solutions/ carried over to the successor by default.

lib/archive.cjs is the single library that implements the lifecycle. It depends only on node:fs, node:path, yaml, and existing nubos-pilot libs (core.cjs, layout.cjs, verify.cjs, frontmatter.cjs, roadmap.cjs).

Decision Drivers ​

  • One aggregate verification, not N manual walks. The Completeness Doctrine (ADR-0012) rules 5 and 11 ("aim to genuinely impress", "ship the complete thing") demand a visible whole-project signal, not a scatter of per-milestone files.
  • Non-destructive workspace reuse. Starting a successor must not require rm -rf. History (PROJECT.md, ROADMAP.md, verification reports) is the project's memory; the lifecycle has to preserve it.
  • Carry-over of institutional knowledge. learnings/ and solutions/ are accumulated patterns. A new project that ignores them re-discovers them at full cost. Default carry-over is the right ergonomic.
  • No daemon, no service. Archive is fs.renameSync plus a manifest write. Detection is fs.statSync. (ADR-0001.)
  • Zero runtime deps. The same yaml + fs toolkit the rest of the codebase uses; no new package.json entry. (ADR-0002.)
  • Three-tree orthogonality. .nubos-pilot/archive/ is a strict Project-State sub-tree. Source code is never touched. (ADR-0005.)

Considered Options ​

  • A: Status quo. Per-milestone verification only; /np:new-project refuses on an existing project; users rm -rf to reset. Reject: documented pain.
  • B: Aggregate verification only, no archive. Add /np:close-project and PROJECT-SUMMARY.md; still refuse to overwrite. Reject: solves question 1, leaves question 2 (workspace reuse) untouched.
  • C: Archive only, no project-level verification. Allow /np:new-project to archive the existing project, but compute completion ad-hoc inside the workflow. Reject: duplicates verification logic per workflow; no persisted PROJECT-SUMMARY.md for retrospectives.
  • D: Replace /np:new-project semantics with a destructive --force that deletes .nubos-pilot/. Reject: destroys learnings and history; collides with the stated need for memory carry-over.
  • E: Lifecycle: /np:close-project → roadmap.yaml.project_status → archive move to .nubos-pilot/archive/<slug>-<date>/. Chosen.

Decision Outcome ​

Chosen: Option E, the close + archive lifecycle, because it solves both pain points with one persisted data model (project_status + archive/ sub-tree), preserves history non-destructively, and integrates cleanly with the Completeness Doctrine's "the answer is the finished product, not a plan" stance.

Layout ​

.nubos-pilot/
  PROJECT.md                # active project root
  REQUIREMENTS.md
  RULES.md
  STATE.md
  ROADMAP.md
  roadmap.yaml              # project_status: active | completed (when closed)
  PROJECT-SUMMARY.md        # written by /np:close-project, regenerated on each close
  milestones/M001/ …        # active milestones
  learnings/ solutions/     # carry-over candidates
  memory/                   # vector store — preserved top-level, queryable across archives
  archive/                  # ← Project-State sub-tree (ADR-0016)
    <slug>-<YYYYMMDD>/      # one directory per archived project; suffix -2 … -99 on collisions
      ARCHIVE.json          # schema_version, archived_at, slug, project_name, completion_status,
                            #   milestones_count, blockers_at_archive, moved[], carried_over[], forced
      PROJECT.md REQUIREMENTS.md RULES.md ROADMAP.md STATE.md roadmap.yaml
      milestones/ codebase/ messages/ threads/ handoffs/ todos/ reports/ knowledge/ session/
      learnings/ solutions/ # only if carry-over included them (manifest records which)

The exact moved set is archive.cjs::ARCHIVED_ITEMS: PROJECT.md, REQUIREMENTS.md, RULES.md, ROADMAP.md, STATE.md, roadmap.yaml, and the directories milestones, codebase, messages, threads, handoffs, todos, reports, knowledge, session. Entries listed in PRESERVED_TOP_LEVEL (config.json, archive, .tmp, .gitignore, state, worktrees, memory) stay where they are.

Completion semantics ​

lib/archive.cjs::computeCompletionStatus(cwd) computes a deterministic verdict from on-disk artefacts. For each milestone in roadmap.yaml it parses M<NNN>-VERIFICATION.md (frontmatter milestone_status / sc_total / passed / failed / deferred / pending when complete; SC-block scan otherwise) and M<NNN>-VALIDATION.md (frontmatter covered / under_sampled / uncovered when complete; ## Uncovered / ## Under-Sampled H3 scan otherwise).

SignalFailure mode (blocker)
Every milestone has M<NNN>-VERIFICATION.mdM<NNN>: VERIFICATION.md missing
VERIFICATION.md has at least one parseable SC blockM<NNN>: VERIFICATION.md has 0 SC blocks (parse failure or empty)
No SC has status FailM<NNN>: N SC failed
No SC has status PendingM<NNN>: N SC pending
milestone_status: failed is consistent with the SC blocksM<NNN>: milestone status is failed but no SC marked Fail (review needed)
Every milestone has M<NNN>-VALIDATION.mdM<NNN>: VALIDATION.md missing
No requirement marked UNCOVEREDM<NNN>: N requirement(s) UNCOVERED

status: complete requires zero blockers AND phases.length > 0. roadmap.yaml.project_status = "completed" upgrades the reported status to complete for the workflow's downstream check, but the blocker list still reflects the on-disk truth; Force-complete is honoured but recorded in the archive manifest as forced: true.

Archive invariants ​

  • archiveProject refuses on incomplete projects (archive-not-complete) unless force: true is passed.
  • Active worktrees (non-.gitkeep entries in .nubos-pilot/worktrees/) refuse the archive (archive-worktrees-present) unless force: true.
  • The move runs inside a withFileLock window on .nubos-pilot/.archive-lock; each entry is fs.renameSync'd, each carry-over entry fs.cpSync'd.
  • Filename collisions on same-day archives append -2, -3, … up to -99; the 100th throws archive-collision.
  • Carry-over default is CARRY_OVER_DEFAULTS = ['learnings', 'solutions']. Both originals stay top-level AND get copied into the archive; the manifest records carried_over.
  • memory/ (vector records, ADR-0014) is neither moved nor carried over; it stays top-level and is queryable across archives via the path-traversal-safe readArchiveFile.

Workflow / CLI integration ​

  • /np:close-project: aggregate verification + PROJECT-SUMMARY.md write (writeProjectSummary) + status flip (setProjectStatus(cwd, 'completed'), which also stamps completed_at). The Definition of Done block cites Completeness Rules 1, 5, 10, 11. CLI: bin/np-tools/close-project.cjs.
  • /np:new-project Phase -1: --detect payload; askuser routes to Archive and start fresh | Close project first, then ask | Abort. The existing Re-Init Guard handles the rare race where PROJECT.md reappears between detection and --apply. CLI: bin/np-tools/new-project.cjs.
  • /np:archive-project: direct invocation for users who want to wrap up without immediately starting a successor. CLI: bin/np-tools/archive-project.cjs with verbs status, do, list, read.

Library surface ​

lib/archive.cjs exports computeCompletionStatus, archiveProject, listArchives, readArchiveFile (path-traversal-guarded via archive-path-escape), setProjectStatus, projectExists, renderProjectSummary, writeProjectSummary, plus the path helpers projectMdPath, archiveRoot, projectSummaryPath and the frozen constants ARCHIVE_DIRNAME, ARCHIVED_ITEMS, CARRY_OVER_DEFAULTS, PRESERVED_TOP_LEVEL.

Consequences ​

  • Good, because a single aggregate verification answers "is the project done?"; it replaces the manual walk of N VERIFICATION.md files.
  • Good, because /np:new-project is now safe to run repeatedly in the same workspace; history is preserved.
  • Good, because learnings + solutions carry over by default: successor projects start with the predecessor's accumulated patterns.
  • Good, because PROJECT-SUMMARY.md becomes a retrospective artefact: persisted, regenerable, no _TBD placeholders.
  • Bad, because of the new surface area (1 lib module, 3 new tools, 2 new/extended workflows, the project_status schema addition in roadmap.yaml).
  • Bad, because archiveProject is a destructive-feeling operation (it moves canonical files); the workflow MUST keep the askuser gate as a hard barrier. A silent archive would violate the "match scope to what was actually requested" rule.
  • Neutral, because roadmap.yaml.project_status is a new schema field; older roadmap.yaml files without it default to "active", but strict schema validators need to allow the field.
  • Neutral, because archive/ is git-tracked by default (no .gitignore rule added). Teams that prefer archives outside source control can add archive/ to their project's .gitignore.

Pattern Conformance ​

  • S-1 atomic write + file lock — the manifest, PROJECT-SUMMARY.md, and the roadmap.yaml rewrite all go through atomicWriteFileSync; the move runs under withFileLock.
  • S-2 NubosPilotError envelope — archive-no-project, archive-not-complete, archive-worktrees-present, archive-collision, archive-path-escape, archive-invalid-project-status.
  • S-5 sandboxed tests — every archive test creates a fresh tmp directory; teardown removes it.
  • S-6 CJS module footer — lib/archive.cjs ends with module.exports.

More Information ​

  • Library: lib/archive.cjs, tests lib/archive.test.cjs.
  • CLI verbs: bin/np-tools/close-project.cjs, bin/np-tools/archive-project.cjs, bin/np-tools/new-project.cjs (extended with --detect); each has a matching .test.cjs.
  • Workflows: workflows/close-project.md (new), workflows/new-project.md (Phase -1 extension).
  • Related ADRs:
    • ADR-0001: archive is fs.renameSync, not a service.
    • ADR-0002: preserved; reuses the existing yaml + fs toolkit.
    • ADR-0005: .nubos-pilot/archive/ is a strict Project-State sub-tree.
    • ADR-0012: the aggregate verification is the mechanical proxy for Rules 5 and 11.

This ADR specifies the project-level close + archive lifecycle. Cross-archive querying beyond readArchiveFile (for example archive search, archive diff) is out of scope and remains a future concern.