Document DB sync prototype scope

This commit is contained in:
Eric Wendland 2026-05-22 14:55:31 +02:00
commit 59c86aba6a
4 changed files with 40 additions and 14 deletions

View file

@ -292,7 +292,11 @@ background live-sync require `db.sync` on `resource:db:<name>`, exchange typed
schema metadata against the local DB before applying and advancing the per-peer schema metadata against the local DB before applying and advancing the per-peer
cursor. Compatible batches are inserted into the local `crsql_changes` table or cursor. Compatible batches are inserted into the local `crsql_changes` table or
view; for real cr-sqlite databases, loading/configuring cr-sqlite remains the view; for real cr-sqlite databases, loading/configuring cr-sqlite remains the
database owner's responsibility. database owner's responsibility. The prototype does not use CAS-backed DB
snapshots or change-batch blobs; that is deferred until large initial catch-up
needs it. The automated tests use deterministic `crsql_changes` fixtures because
this dev environment does not provide a `sqlite3` CLI or cr-sqlite extension
artifact for a real extension-backed integration test.
Importing or pinging a peer card never grants capabilities by itself. Importing or pinging a peer card never grants capabilities by itself.
When `[iroh].local_discovery = true`, the daemon also advertises and discovers When `[iroh].local_discovery = true`, the daemon also advertises and discovers
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is

View file

@ -6,11 +6,25 @@ Accepted.
## Decision ## Decision
Database sync will use local SQLite databases with cr-sqlite. Changes from Database sync uses local SQLite databases with cr-sqlite-compatible
`crsql_changes` will be exchanged over Iroh and applied through cr-sqlite, with `crsql_changes` tables or views. Geth extracts typed change batches, exchanges
schema hash gating and resource authorization. them over the protected Iroh control path, checks schema metadata before
application, and applies compatible batches through the local `crsql_changes`
interface.
The prototype does not add a second snapshot transport for DBs. CAS-backed DB
snapshots or change-batch blobs are deferred until the change path proves too
large for the control path or until initial peer catch-up needs snapshots.
## Consequences ## Consequences
The MVP stores DB resource metadata and leaves synchronization as explicit future The prototype has deterministic fixture coverage for extraction, application,
work. two-node Iroh exchange, authorization, schema mismatch detection, and live-sync
watermarks.
A real cr-sqlite extension integration test is intentionally not marked done in
this environment because there is no `sqlite3` CLI or cr-sqlite extension
artifact available to load. The blocker is environmental, not a protocol
decision. When the extension is available, add a two-node test that creates a
real cr-sqlite-enabled table, mutates one node, runs `geth db sync`, and proves
the second node applies the change through cr-sqlite.

View file

@ -194,7 +194,10 @@ advancing its per-peer/per-DB cursor. Compatible remote batches are inserted
into the local `crsql_changes` table or view before the cursor advances. into the local `crsql_changes` table or view before the cursor advances.
Loading/configuring the cr-sqlite extension for real application databases Loading/configuring the cr-sqlite extension for real application databases
remains the database owner's responsibility; the bootstrap tests use remains the database owner's responsibility; the bootstrap tests use
deterministic fixture tables. deterministic fixture tables because this dev environment has no `sqlite3` CLI
or cr-sqlite extension artifact. DB sync intentionally does not use CAS-backed
snapshots or batch blobs in the prototype. Those become useful when initial
catch-up or large batches outgrow the protected control path.
`geth-kv` currently provides a SQLite-backed local fallback for named KV stores `geth-kv` currently provides a SQLite-backed local fallback for named KV stores
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth

View file

@ -53,11 +53,11 @@ Implementation order:
- `[x]` Sync Automerge changes over Iroh. - `[x]` Sync Automerge changes over Iroh.
- `[x]` Gate document sync with resource authorization. - `[x]` Gate document sync with resource authorization.
5. `[ ]` Harden DB sync for real cr-sqlite usage. 5. `[x]` Harden DB sync for real cr-sqlite usage.
Acceptance criteria: Acceptance criteria:
- `[ ]` Add a real cr-sqlite-enabled two-node integration test, or document - `[x]` Add a real cr-sqlite-enabled two-node integration test, or document
a precise blocker if the extension is unavailable in CI/dev. a precise blocker if the extension is unavailable in CI/dev.
- `[ ]` Decide and document whether CAS-backed DB snapshots/batches are part - `[x]` Decide and document whether CAS-backed DB snapshots/batches are part
of the prototype. of the prototype.
6. `[ ]` Finish operational first-run polish. 6. `[ ]` Finish operational first-run polish.
@ -639,7 +639,7 @@ Automerge documents.
- `[x]` Schema hash/version metadata is included in extracted change batches. - `[x]` Schema hash/version metadata is included in extracted change batches.
- `[x]` Extracted batches are exposed through `geth db changes`. - `[x]` Extracted batches are exposed through `geth db changes`.
- `[~]` DB sync over Iroh. - `[x]` DB sync over Iroh.
Acceptance criteria: Acceptance criteria:
- `[x]` `geth db sync <node-id> <name>` exists and talks to the daemon. - `[x]` `geth db sync <node-id> <name>` exists and talks to the daemon.
- `[x]` Remote DB sync uses the protected Iroh control ALPN. - `[x]` Remote DB sync uses the protected Iroh control ALPN.
@ -659,9 +659,14 @@ Automerge documents.
`crsql_changes` table or view before advancing the cursor. `crsql_changes` table or view before advancing the cursor.
- `[x]` Tests cover typed batch application into deterministic fixture - `[x]` Tests cover typed batch application into deterministic fixture
`crsql_changes` tables. `crsql_changes` tables.
- `[ ]` Add an integration test with a real cr-sqlite-enabled SQLite DB that - `[x]` A real cr-sqlite-enabled two-node integration test is blocked in this
proves two local test nodes exchange and apply changes. dev environment because no `sqlite3` CLI or cr-sqlite extension artifact is
- `[ ]` Optional CAS-backed snapshots or batches are documented if used. available. The current coverage uses deterministic `crsql_changes`
fixtures and exercises two-node Iroh exchange plus local application.
- `[x]` CAS-backed DB snapshots/batches are deferred beyond the prototype.
The prototype exchanges typed `crsql_changes` batches over the protected
Iroh control path and advances high-water cursors only after schema checks
and successful local application.
- `[~]` Automerge document resource. - `[~]` Automerge document resource.
Acceptance criteria: Acceptance criteria: