diff --git a/docs/production-readiness-roadmap.md b/docs/production-readiness-roadmap.md new file mode 100644 index 0000000..3660403 --- /dev/null +++ b/docs/production-readiness-roadmap.md @@ -0,0 +1,340 @@ +# Production Readiness Roadmap + +This roadmap tracks the work required before `geth` becomes a dependable base +layer for personal scripts, infrastructure automation, and other projects. +Deployment has not started, so large refactors are in scope when they reduce +long-term risk. + +Status markers: + +- `[ ]` Not started +- `[~]` In progress +- `[x]` Done + +## Release Principle + +Do not widen the feature surface until the existing daemon, storage, protocol, +authorization, and operational contracts are boring to test, upgrade, and +debug. The project already has enough prototype capability to validate the +architecture; the next milestone is making those capabilities stable. + +## Phase 0: Current Quality Gate + +Goal: make the documented local quality gate pass before deeper refactors. + +- `[~]` Restore the full workspace quality gate. + Acceptance criteria: + - `[x]` `cargo fmt --all -- --check` passes. + - `[x]` `cargo check --workspace --all-targets` passes. + - `[x]` `cargo test --workspace` passes. + - `[ ]` `cargo clippy --workspace --all-targets -- -D warnings` passes. + - `[ ]` CI enforces the same required checks. + +## Phase 1: Daemon Subsystem Refactor + +Goal: split `geth-node` into reviewable daemon subsystems without changing +behavior. + +- `[ ]` Extract daemon startup and runtime ownership. + Acceptance criteria: + - `[ ]` Daemon startup, shutdown, signal handling, socket setup, Iroh + endpoint ownership, and background task spawning live outside the main + feature handler module. + - `[ ]` Runtime state is represented by narrow structs with documented + ownership and locking rules. + - `[ ]` Existing daemon startup and status tests pass unchanged. + +- `[ ]` Extract local control routing. + Acceptance criteria: + - `[ ]` Local `ControlRequest` dispatch is a routing layer, not the home of + every feature implementation. + - `[ ]` Each command family has a small handler module or function group. + - `[ ]` Local-only behavior remains covered by existing integration tests. + +- `[ ]` Extract protected peer-control routing. + Acceptance criteria: + - `[ ]` Iroh control ALPN handling, nonce checks, peer-card validation, and + endpoint-binding validation are centralized. + - `[ ]` Feature handlers receive authenticated caller context rather than + repeating peer-card boilerplate. + - `[ ]` Remote request tests still prove discovery alone grants no access. + +- `[ ]` Extract resource module handlers. + Acceptance criteria: + - `[ ]` CAS/file-root, KV, DB, document, pubsub, pipe, SSH, and overlay + handlers are separated enough that each can be reviewed independently. + - `[ ]` Each module documents its resource IDs, capabilities, and mutation + points. + - `[ ]` No generic `geth-common` crate is introduced. + +- `[ ]` Extract live-sync engine. + Acceptance criteria: + - `[ ]` Sync stream selection, watermarks, cursors, run results, and health + recording live in a sync-focused module. + - `[ ]` Per-module sync handlers have consistent interfaces. + - `[ ]` `geth sync status --json` output remains stable. + +## Phase 2: Stable Automation Contracts + +Goal: make command, JSON, and protocol contracts explicit enough for scripts +and downstream projects. + +- `[ ]` Define compatibility policy. + Acceptance criteria: + - `[ ]` CLI command compatibility is documented. + - `[ ]` `--json` output compatibility is documented. + - `[ ]` Local control JSONL compatibility is documented. + - `[ ]` Peer wire protocol compatibility is documented. + - `[ ]` SQLite and signed-operation compatibility are documented. + +- `[ ]` Add golden JSON tests. + Acceptance criteria: + - `[ ]` Important script-facing commands have stable JSON fixture tests. + - `[ ]` Error JSON includes stable codes for common operator and automation + failures. + - `[ ]` Fixture updates require intentional review. + +- `[ ]` Expand protocol roundtrip tests. + Acceptance criteria: + - `[ ]` Every `ControlRequest` and `ControlResponse` variant roundtrips. + - `[ ]` Every `PeerControlRequest` and `PeerControlResponse` variant + roundtrips. + - `[ ]` Pipe and overlay wire protocol variants roundtrip. + - `[ ]` Unknown or malformed protocol inputs fail safely. + +- `[ ]` Classify command stability. + Acceptance criteria: + - `[ ]` Commands are marked stable, experimental, or prototype in docs. + - `[ ]` Help text avoids production claims for experimental paths. + - `[ ]` Prototype paths have explicit migration or removal expectations. + +## Phase 3: Storage And Migration Hardening + +Goal: treat local SQLite state as durable product data before users depend on +it. + +- `[ ]` Replace opportunistic schema setup with ordered migrations. + Acceptance criteria: + - `[ ]` The store tracks numeric schema versions. + - `[ ]` Each migration is transactional where SQLite supports it. + - `[ ]` Fresh database creation and repeated opens are idempotent. + - `[ ]` Old schema fixtures migrate to the current schema in tests. + +- `[ ]` Make multi-table writes transactional. + Acceptance criteria: + - `[ ]` Signed operation and signature imports commit atomically. + - `[ ]` Multi-record sync imports cannot leave partial state after a local + error. + - `[ ]` Tests cover failure injection for at least one multi-table path. + +- `[ ]` Add backup and restore workflow. + Acceptance criteria: + - `[ ]` `geth backup create` or an equivalent documented command captures + metadata, identity public material, config, and CAS metadata expectations. + - `[ ]` Restore can write to a separate target home for validation. + - `[ ]` Backup output avoids copying private SSH admin keys. + - `[ ]` Docs explain what is and is not included. + +- `[ ]` Document database durability settings. + Acceptance criteria: + - `[ ]` WAL and synchronous settings are chosen deliberately. + - `[ ]` Crash-recovery expectations are documented. + - `[ ]` `geth doctor` or status output reports obvious store issues. + +## Phase 4: Security Boundary Closure + +Goal: finish the authorization and remote-input audit before deployment. + +- `[ ]` Complete remote authorization matrix coverage. + Acceptance criteria: + - `[ ]` Denied and allowed paths are tested for CAS, KV, DB, document, + pubsub, pipe, SSH proxy/admin shell, SSH cert metadata, and revocations. + - `[ ]` Every remote mutating or service-opening operation has an explicit + resource capability check before mutation or host access. + - `[ ]` The matrix fails tests when a new remote operation lacks a guard. + +- `[ ]` Add adversarial identity and bearer tests. + Acceptance criteria: + - `[ ]` Tests cover discovered-only peers. + - `[ ]` Tests cover wrong endpoint bindings. + - `[ ]` Tests cover revoked nodes. + - `[ ]` Tests cover stale or mismatched peer cards. + - `[ ]` Tests cover bearer secrets with wrong capabilities. + - `[ ]` Tests cover bearer attempts to mutate trust graph state. + +- `[ ]` Bound all remote input paths. + Acceptance criteria: + - `[ ]` Remote request payloads have documented size limits. + - `[ ]` Remote stream reads have timeouts or bounded behavior. + - `[ ]` Oversized messages are rejected without state mutation. + - `[ ]` Tests cover oversized payload denial for representative protocols. + +- `[ ]` Audit host-opening paths. + Acceptance criteria: + - `[ ]` TCP forwarding remains loopback-only unless a later ADR expands it. + - `[ ]` Unix forwarding rejects unsafe paths. + - `[ ]` SSH proxy connects only to the intended local SSH endpoint. + - `[ ]` Overlay TUN setup remains explicit opt-in. + - `[ ]` Docs state host access risks and recovery commands. + +## Phase 5: Production Cryptography Boundary + +Goal: remove prototype cryptography from paths users may treat as real +confidential storage. + +- `[ ]` Replace prototype private CAS envelope. + Acceptance criteria: + - `[ ]` The BLAKE3-XOR prototype envelope is not used for new private CAS + writes. + - `[ ]` New private CAS writes use a reviewed AEAD construction or an + established envelope format such as age. + - `[ ]` Key derivation, nonce generation, and envelope versioning are + documented. + - `[ ]` Tests cover tamper detection, wrong resource, wrong key, and nonce + uniqueness behavior. + +- `[ ]` Define pre-release encrypted blob migration behavior. + Acceptance criteria: + - `[ ]` Existing prototype envelopes are either rejected with a clear error + or migrated through an explicit command. + - `[ ]` Docs state that prototype envelopes made before deployment are not a + durable security format. + +## Phase 6: Sync Correctness And Fault Testing + +Goal: make convergence and failure behavior predictable enough for automation. + +- `[ ]` Build deterministic multi-daemon fault tests. + Acceptance criteria: + - `[ ]` Tests cover peer restart. + - `[ ]` Tests cover endpoint rotation. + - `[ ]` Tests cover temporary peer unavailability. + - `[ ]` Tests cover partial stream failure. + - `[ ]` Tests cover duplicate records and stale cursors. + +- `[ ]` Define per-resource conflict semantics. + Acceptance criteria: + - `[ ]` CAS/file-root conflict semantics are documented and tested. + - `[ ]` KV conflict semantics are documented and tested. + - `[ ]` Document merge semantics are documented and tested. + - `[ ]` DB change application limits are documented and tested. + +- `[ ]` Add live-sync retry and backoff policy. + Acceptance criteria: + - `[ ]` Failed peer streams back off without starving healthy streams. + - `[ ]` Retry state appears in `geth sync status --json`. + - `[ ]` Operators can trigger immediate retry with `geth sync now`. + +## Phase 7: Script And Automation UX + +Goal: make `geth` ergonomic and stable as a base layer for custom automation. + +- `[ ]` Stabilize JSON errors. + Acceptance criteria: + - `[ ]` Common failures include stable machine-readable error codes. + - `[ ]` Human errors still include next-step recovery hints. + - `[ ]` Tests assert both code and operator-facing hint for representative + failures. + +- `[ ]` Add wait commands for automation. + Acceptance criteria: + - `[ ]` `geth wait daemon` can block until local control is ready. + - `[ ]` `geth wait peer ` can block until peer control succeeds or + times out. + - `[ ]` `geth wait sync ` can block until required streams are healthy + or stale. + - `[ ]` Wait commands support JSON output and timeout flags. + +- `[ ]` Make common commands idempotent. + Acceptance criteria: + - `[ ]` Resource creation commands can be safely repeated where practical. + - `[ ]` Grant creation supports deterministic caller-provided IDs. + - `[ ]` Repeated sync and import commands report no-op state clearly. + +- `[ ]` Publish automation examples. + Acceptance criteria: + - `[ ]` Docs include shell examples. + - `[ ]` Docs include Python examples using CLI JSON. + - `[ ]` Docs include user-service automation examples. + - `[ ]` Examples avoid private-key copying and privileged service mutation. + +## Phase 8: Observability And Operations + +Goal: make production failures diagnosable from the CLI and logs. + +- `[ ]` Improve structured tracing. + Acceptance criteria: + - `[ ]` Logs include fields for command, peer node, resource, capability, + stream, cursor, and stable error code where applicable. + - `[ ]` Sensitive bearer tokens and private material are not logged. + +- `[ ]` Expand health status. + Acceptance criteria: + - `[ ]` `geth status --json` reports daemon uptime. + - `[ ]` Store schema version is visible. + - `[ ]` Native backend health is visible. + - `[ ]` Iroh relay/local-discovery state is visible without exposing + unrelated config secrets. + +- `[ ]` Add `geth doctor`. + Acceptance criteria: + - `[ ]` Doctor detects daemon not running. + - `[ ]` Doctor detects stale local socket. + - `[ ]` Doctor detects bad config. + - `[ ]` Doctor detects missing `ssh-keygen`. + - `[ ]` Doctor explains missing grants and endpoint mismatches when enough + local metadata exists. + - `[ ]` Doctor has JSON output for scripts. + +## Phase 9: Packaging, Upgrade, And Release Discipline + +Goal: make first deployment the start of a controlled compatibility story. + +- `[ ]` Harden release artifacts. + Acceptance criteria: + - `[ ]` Linux, macOS, and Windows archives are built from tags. + - `[ ]` Release archives are smoke-tested directly, not only through + `cargo run`. + - `[ ]` Archives include relevant docs and license files. + +- `[ ]` Add upgrade tests. + Acceptance criteria: + - `[ ]` Fixture homes from previous tagged pre-releases migrate forward. + - `[ ]` Upgrade tests include store schema, config, keychain/auth logs, CAS + metadata, and peer cards. + - `[ ]` Rollback expectations are documented. + +- `[ ]` Publish release and support policy. + Acceptance criteria: + - `[ ]` Supported platforms are listed. + - `[ ]` Breaking-change policy is documented. + - `[ ]` Security update expectations are documented. + - `[ ]` User-service installation remains user-level only. + +## Phase 10: Pre-Deployment Dogfood Gate + +Goal: prove the system works as an actual base layer before broader use. + +- `[ ]` Complete two-machine dogfood checklist. + Acceptance criteria: + - `[ ]` Fresh install works on at least two real machines. + - `[ ]` Owner init, enrollment, peer exchange, grant, sync status, CAS, KV, + document, pipe, SSH proxy, and service restart work end to end. + - `[ ]` Denied remote operations do not mutate serving node state. + - `[ ]` Backup and restore work. + - `[ ]` Upgrade from one tagged pre-release to the next works. + - `[ ]` `geth doctor` gives actionable output for intentionally broken + setups. + - `[ ]` Docs match actual commands. + +## Working Order + +1. `[~]` Finish Phase 0. +2. `[ ]` Refactor `geth-node` into daemon subsystems. +3. `[ ]` Add stable contract and golden JSON tests. +4. `[ ]` Harden store migrations and backup. +5. `[ ]` Complete security-boundary test coverage. +6. `[ ]` Replace prototype private CAS cryptography. +7. `[ ]` Add fault-injection sync tests. +8. `[ ]` Improve automation commands and JSON errors. +9. `[ ]` Add operational health, doctor, and release gates. diff --git a/docs/roadmap.md b/docs/roadmap.md index 973740f..d8b9634 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -9,6 +9,9 @@ Status markers: - `[~]` In progress - `[x]` Done +For deployment-readiness work that cuts across feature areas, see +[`docs/production-readiness-roadmap.md`](production-readiness-roadmap.md). + ## Long-Term Goal: Distributed Homelab Overlay Goal: evolve geth into a distributed, fault-tolerant homelab overlay runtime in