Document DB sync prototype scope
This commit is contained in:
parent
b01da7be4b
commit
59c86aba6a
4 changed files with 40 additions and 14 deletions
|
|
@ -292,7 +292,11 @@ background live-sync require `db.sync` on `resource:db:<name>`, exchange typed
|
||||||
schema metadata against the local DB before applying and advancing the per-peer
|
schema metadata against the local DB before applying and advancing the per-peer
|
||||||
cursor. Compatible batches are inserted into the local `crsql_changes` table or
|
cursor. Compatible batches are inserted into the local `crsql_changes` table or
|
||||||
view; for real cr-sqlite databases, loading/configuring cr-sqlite remains the
|
view; for real cr-sqlite databases, loading/configuring cr-sqlite remains the
|
||||||
database owner's responsibility.
|
database owner's responsibility. The prototype does not use CAS-backed DB
|
||||||
|
snapshots or change-batch blobs; that is deferred until large initial catch-up
|
||||||
|
needs it. The automated tests use deterministic `crsql_changes` fixtures because
|
||||||
|
this dev environment does not provide a `sqlite3` CLI or cr-sqlite extension
|
||||||
|
artifact for a real extension-backed integration test.
|
||||||
Importing or pinging a peer card never grants capabilities by itself.
|
Importing or pinging a peer card never grants capabilities by itself.
|
||||||
When `[iroh].local_discovery = true`, the daemon also advertises and discovers
|
When `[iroh].local_discovery = true`, the daemon also advertises and discovers
|
||||||
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is
|
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is
|
||||||
|
|
|
||||||
|
|
@ -6,11 +6,25 @@ Accepted.
|
||||||
|
|
||||||
## Decision
|
## Decision
|
||||||
|
|
||||||
Database sync will use local SQLite databases with cr-sqlite. Changes from
|
Database sync uses local SQLite databases with cr-sqlite-compatible
|
||||||
`crsql_changes` will be exchanged over Iroh and applied through cr-sqlite, with
|
`crsql_changes` tables or views. Geth extracts typed change batches, exchanges
|
||||||
schema hash gating and resource authorization.
|
them over the protected Iroh control path, checks schema metadata before
|
||||||
|
application, and applies compatible batches through the local `crsql_changes`
|
||||||
|
interface.
|
||||||
|
|
||||||
|
The prototype does not add a second snapshot transport for DBs. CAS-backed DB
|
||||||
|
snapshots or change-batch blobs are deferred until the change path proves too
|
||||||
|
large for the control path or until initial peer catch-up needs snapshots.
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
The MVP stores DB resource metadata and leaves synchronization as explicit future
|
The prototype has deterministic fixture coverage for extraction, application,
|
||||||
work.
|
two-node Iroh exchange, authorization, schema mismatch detection, and live-sync
|
||||||
|
watermarks.
|
||||||
|
|
||||||
|
A real cr-sqlite extension integration test is intentionally not marked done in
|
||||||
|
this environment because there is no `sqlite3` CLI or cr-sqlite extension
|
||||||
|
artifact available to load. The blocker is environmental, not a protocol
|
||||||
|
decision. When the extension is available, add a two-node test that creates a
|
||||||
|
real cr-sqlite-enabled table, mutates one node, runs `geth db sync`, and proves
|
||||||
|
the second node applies the change through cr-sqlite.
|
||||||
|
|
|
||||||
|
|
@ -194,7 +194,10 @@ advancing its per-peer/per-DB cursor. Compatible remote batches are inserted
|
||||||
into the local `crsql_changes` table or view before the cursor advances.
|
into the local `crsql_changes` table or view before the cursor advances.
|
||||||
Loading/configuring the cr-sqlite extension for real application databases
|
Loading/configuring the cr-sqlite extension for real application databases
|
||||||
remains the database owner's responsibility; the bootstrap tests use
|
remains the database owner's responsibility; the bootstrap tests use
|
||||||
deterministic fixture tables.
|
deterministic fixture tables because this dev environment has no `sqlite3` CLI
|
||||||
|
or cr-sqlite extension artifact. DB sync intentionally does not use CAS-backed
|
||||||
|
snapshots or batch blobs in the prototype. Those become useful when initial
|
||||||
|
catch-up or large batches outgrow the protected control path.
|
||||||
|
|
||||||
`geth-kv` currently provides a SQLite-backed local fallback for named KV stores
|
`geth-kv` currently provides a SQLite-backed local fallback for named KV stores
|
||||||
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
|
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
|
||||||
|
|
|
||||||
|
|
@ -53,11 +53,11 @@ Implementation order:
|
||||||
- `[x]` Sync Automerge changes over Iroh.
|
- `[x]` Sync Automerge changes over Iroh.
|
||||||
- `[x]` Gate document sync with resource authorization.
|
- `[x]` Gate document sync with resource authorization.
|
||||||
|
|
||||||
5. `[ ]` Harden DB sync for real cr-sqlite usage.
|
5. `[x]` Harden DB sync for real cr-sqlite usage.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
- `[ ]` Add a real cr-sqlite-enabled two-node integration test, or document
|
- `[x]` Add a real cr-sqlite-enabled two-node integration test, or document
|
||||||
a precise blocker if the extension is unavailable in CI/dev.
|
a precise blocker if the extension is unavailable in CI/dev.
|
||||||
- `[ ]` Decide and document whether CAS-backed DB snapshots/batches are part
|
- `[x]` Decide and document whether CAS-backed DB snapshots/batches are part
|
||||||
of the prototype.
|
of the prototype.
|
||||||
|
|
||||||
6. `[ ]` Finish operational first-run polish.
|
6. `[ ]` Finish operational first-run polish.
|
||||||
|
|
@ -639,7 +639,7 @@ Automerge documents.
|
||||||
- `[x]` Schema hash/version metadata is included in extracted change batches.
|
- `[x]` Schema hash/version metadata is included in extracted change batches.
|
||||||
- `[x]` Extracted batches are exposed through `geth db changes`.
|
- `[x]` Extracted batches are exposed through `geth db changes`.
|
||||||
|
|
||||||
- `[~]` DB sync over Iroh.
|
- `[x]` DB sync over Iroh.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
- `[x]` `geth db sync <node-id> <name>` exists and talks to the daemon.
|
- `[x]` `geth db sync <node-id> <name>` exists and talks to the daemon.
|
||||||
- `[x]` Remote DB sync uses the protected Iroh control ALPN.
|
- `[x]` Remote DB sync uses the protected Iroh control ALPN.
|
||||||
|
|
@ -659,9 +659,14 @@ Automerge documents.
|
||||||
`crsql_changes` table or view before advancing the cursor.
|
`crsql_changes` table or view before advancing the cursor.
|
||||||
- `[x]` Tests cover typed batch application into deterministic fixture
|
- `[x]` Tests cover typed batch application into deterministic fixture
|
||||||
`crsql_changes` tables.
|
`crsql_changes` tables.
|
||||||
- `[ ]` Add an integration test with a real cr-sqlite-enabled SQLite DB that
|
- `[x]` A real cr-sqlite-enabled two-node integration test is blocked in this
|
||||||
proves two local test nodes exchange and apply changes.
|
dev environment because no `sqlite3` CLI or cr-sqlite extension artifact is
|
||||||
- `[ ]` Optional CAS-backed snapshots or batches are documented if used.
|
available. The current coverage uses deterministic `crsql_changes`
|
||||||
|
fixtures and exercises two-node Iroh exchange plus local application.
|
||||||
|
- `[x]` CAS-backed DB snapshots/batches are deferred beyond the prototype.
|
||||||
|
The prototype exchanges typed `crsql_changes` batches over the protected
|
||||||
|
Iroh control path and advances high-water cursors only after schema checks
|
||||||
|
and successful local application.
|
||||||
|
|
||||||
- `[~]` Automerge document resource.
|
- `[~]` Automerge document resource.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue