Wire KV sync through iroh-docs
This commit is contained in:
parent
72f28224ce
commit
59c463eb40
9 changed files with 406 additions and 47 deletions
3
Cargo.lock
generated
3
Cargo.lock
generated
|
|
@ -1599,6 +1599,7 @@ name = "geth-node"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"base64",
|
"base64",
|
||||||
|
"futures",
|
||||||
"geth-auth",
|
"geth-auth",
|
||||||
"geth-cas",
|
"geth-cas",
|
||||||
"geth-codec",
|
"geth-codec",
|
||||||
|
|
@ -1622,6 +1623,8 @@ dependencies = [
|
||||||
"hex",
|
"hex",
|
||||||
"iroh",
|
"iroh",
|
||||||
"iroh-blobs",
|
"iroh-blobs",
|
||||||
|
"iroh-docs",
|
||||||
|
"iroh-gossip",
|
||||||
"rusqlite",
|
"rusqlite",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
|
|
|
||||||
16
README.md
16
README.md
|
|
@ -153,8 +153,9 @@ The bootstrap implementation provides:
|
||||||
`geth db sync <node-id> <name>`
|
`geth db sync <node-id> <name>`
|
||||||
- local SQLite-backed KV commands: `geth kv create/set/get`; `kv set` accepts
|
- local SQLite-backed KV commands: `geth kv create/set/get`; `kv set` accepts
|
||||||
`--subject <principal>` to exercise local capability checks for non-local
|
`--subject <principal>` to exercise local capability checks for non-local
|
||||||
callers; `geth kv sync <node-id> <name> [--bearer-secret <secret>]` pulls
|
callers. The daemon mirrors named KV stores into Iroh Documents and
|
||||||
authorized remote updates
|
`geth kv sync <node-id> <name> [--bearer-secret <secret>]` pulls authorized
|
||||||
|
remote updates after receiving a read-only docs ticket through geth control.
|
||||||
- local Automerge document commands: `geth document create/status/set/get`;
|
- local Automerge document commands: `geth document create/status/set/get`;
|
||||||
CLI input and output are JSON views, while the store keeps durable Automerge
|
CLI input and output are JSON views, while the store keeps durable Automerge
|
||||||
save bytes. `geth document sync <node-id> <name> [--bearer-secret <secret>]`
|
save bytes. `geth document sync <node-id> <name> [--bearer-secret <secret>]`
|
||||||
|
|
@ -201,8 +202,10 @@ Iroh endpoint, verifies the BLAKE3 hash, stores it in local CAS, and records the
|
||||||
serving peer as a provider visible with `geth cas providers <hash>`.
|
serving peer as a provider visible with `geth cas providers <hash>`.
|
||||||
`geth-iroh` is pinned to `iroh 0.95.1` and compiles the native backend
|
`geth-iroh` is pinned to `iroh 0.95.1` and compiles the native backend
|
||||||
libraries `iroh-blobs 0.97.0`, `iroh-docs 0.95.0`, and `iroh-gossip 0.95.0`
|
libraries `iroh-blobs 0.97.0`, `iroh-docs 0.95.0`, and `iroh-gossip 0.95.0`
|
||||||
against the same daemon-owned endpoint generation. KV and pubsub still use their
|
against the same daemon-owned endpoint generation. KV stores are mirrored into
|
||||||
documented bootstrap equivalents until their native protocol migrations land.
|
native `iroh-docs` namespaces and peers receive read-only document tickets only
|
||||||
|
after geth authorization succeeds. Pubsub still uses its documented bootstrap
|
||||||
|
equivalent until the native gossip migration lands.
|
||||||
Remote resource commands that accept `--bearer-secret` can also authorize with a
|
Remote resource commands that accept `--bearer-secret` can also authorize with a
|
||||||
resource-scoped bearer proof generated from the private bearer token returned at
|
resource-scoped bearer proof generated from the private bearer token returned at
|
||||||
creation time. The persisted auth log stores a public bearer id and token
|
creation time. The persisted auth log stores a public bearer id and token
|
||||||
|
|
@ -239,8 +242,9 @@ can safely apply non-conflicting remote creates, updates, deletes, and renames
|
||||||
only where local state still matches the recorded base.
|
only where local state still matches the recorded base.
|
||||||
Named KV stores participate in the same live-sync loop once they exist locally:
|
Named KV stores participate in the same live-sync loop once they exist locally:
|
||||||
manual `geth kv sync <node-id> <name>` and background ticks require `kv.read`
|
manual `geth kv sync <node-id> <name>` and background ticks require `kv.read`
|
||||||
on the remote `resource:kv:<name>` and import only remote entries that are not
|
on the remote `resource:kv:<name>`. Authorized sync imports from the remote
|
||||||
older than the local value.
|
Iroh Documents namespace where available, keeps SQLite as the durable local
|
||||||
|
index, and imports only remote entries that are not older than the local value.
|
||||||
Remote pubsub publish uses the protected Iroh control path too. The remote peer
|
Remote pubsub publish uses the protected Iroh control path too. The remote peer
|
||||||
requires `pubsub.publish` on `resource:pubsub:<topic>` before recording the
|
requires `pubsub.publish` on `resource:pubsub:<topic>` before recording the
|
||||||
message in its local daemon-lifetime ring buffer. Pubsub remains lossy and is
|
message in its local daemon-lifetime ring buffer. Pubsub remains lossy and is
|
||||||
|
|
|
||||||
|
|
@ -1205,6 +1205,7 @@ pub enum PeerControlResponse {
|
||||||
name: String,
|
name: String,
|
||||||
entries: Vec<KvSyncEntry>,
|
entries: Vec<KvSyncEntry>,
|
||||||
high_water_ms: i64,
|
high_water_ms: i64,
|
||||||
|
docs_ticket: Option<String>,
|
||||||
allowed: bool,
|
allowed: bool,
|
||||||
reason: String,
|
reason: String,
|
||||||
evaluated_ops: usize,
|
evaluated_ops: usize,
|
||||||
|
|
@ -2359,6 +2360,7 @@ mod tests {
|
||||||
updated_at_ms: 42,
|
updated_at_ms: 42,
|
||||||
}],
|
}],
|
||||||
high_water_ms: 43,
|
high_water_ms: 43,
|
||||||
|
docs_ticket: Some("doc-ticket".to_owned()),
|
||||||
allowed: true,
|
allowed: true,
|
||||||
reason: "direct grant".to_owned(),
|
reason: "direct grant".to_owned(),
|
||||||
evaluated_ops: 1,
|
evaluated_ops: 1,
|
||||||
|
|
|
||||||
|
|
@ -32,9 +32,12 @@ geth-ssh-identity = { path = "../geth-ssh-identity" }
|
||||||
geth-ssh-proxy = { path = "../geth-ssh-proxy" }
|
geth-ssh-proxy = { path = "../geth-ssh-proxy" }
|
||||||
geth-store = { path = "../geth-store" }
|
geth-store = { path = "../geth-store" }
|
||||||
geth-types = { path = "../geth-types" }
|
geth-types = { path = "../geth-types" }
|
||||||
|
futures.workspace = true
|
||||||
hex.workspace = true
|
hex.workspace = true
|
||||||
iroh.workspace = true
|
iroh.workspace = true
|
||||||
iroh-blobs.workspace = true
|
iroh-blobs.workspace = true
|
||||||
|
iroh-docs.workspace = true
|
||||||
|
iroh-gossip.workspace = true
|
||||||
swarm-discovery.workspace = true
|
swarm-discovery.workspace = true
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
pub mod service;
|
pub mod service;
|
||||||
|
|
||||||
use base64::Engine;
|
use base64::Engine;
|
||||||
|
use futures::StreamExt;
|
||||||
use geth_auth::{AUTH_SIGNATURE_NAMESPACE, AuthExplanation, AuthOp, AuthOpKind, AuthOpSignature};
|
use geth_auth::{AUTH_SIGNATURE_NAMESPACE, AuthExplanation, AuthOp, AuthOpKind, AuthOpSignature};
|
||||||
use geth_cas::{
|
use geth_cas::{
|
||||||
BlobInfoSummary, CasTreeEntryKind, CasTreeObject, FileConflict, FileConflictKind,
|
BlobInfoSummary, CasTreeEntryKind, CasTreeObject, FileConflict, FileConflictKind,
|
||||||
|
|
@ -52,6 +53,7 @@ use geth_types::{
|
||||||
ResourceName, SshCertId, SshCertRequestId, UnixMillis, UserId,
|
ResourceName, SshCertId, SshCertRequestId, UnixMillis, UserId,
|
||||||
};
|
};
|
||||||
use iroh::protocol::ProtocolHandler;
|
use iroh::protocol::ProtocolHandler;
|
||||||
|
use iroh_docs::api::protocol::{AddrInfoOptions, ShareMode};
|
||||||
use std::collections::{BTreeMap, BTreeSet, VecDeque};
|
use std::collections::{BTreeMap, BTreeSet, VecDeque};
|
||||||
use std::path::{Component, Path, PathBuf};
|
use std::path::{Component, Path, PathBuf};
|
||||||
use std::sync::{Arc, Mutex};
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
@ -159,6 +161,8 @@ pub struct LocalNode {
|
||||||
pub iroh_status: EndpointStatus,
|
pub iroh_status: EndpointStatus,
|
||||||
iroh_endpoint: Arc<Mutex<Option<GethIrohEndpoint>>>,
|
iroh_endpoint: Arc<Mutex<Option<GethIrohEndpoint>>>,
|
||||||
iroh_blob_store: Arc<Mutex<Option<iroh_blobs::store::fs::FsStore>>>,
|
iroh_blob_store: Arc<Mutex<Option<iroh_blobs::store::fs::FsStore>>>,
|
||||||
|
iroh_docs: Arc<Mutex<Option<iroh_docs::protocol::Docs>>>,
|
||||||
|
iroh_gossip: Arc<Mutex<Option<iroh_gossip::net::Gossip>>>,
|
||||||
runtime: Arc<NodeRuntime>,
|
runtime: Arc<NodeRuntime>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -199,6 +203,14 @@ struct LiveSyncHealth {
|
||||||
last_rejected: usize,
|
last_rejected: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, serde::Deserialize, serde::Serialize)]
|
||||||
|
struct KvDocsState {
|
||||||
|
name: String,
|
||||||
|
namespace_id: String,
|
||||||
|
read_ticket: String,
|
||||||
|
updated_at_ms: i64,
|
||||||
|
}
|
||||||
|
|
||||||
struct PipeTcpConnectWire {
|
struct PipeTcpConnectWire {
|
||||||
peer_card: PeerCard,
|
peer_card: PeerCard,
|
||||||
target_addr: String,
|
target_addr: String,
|
||||||
|
|
@ -268,6 +280,8 @@ pub fn init_node(paths: &GethPaths) -> Result<LocalNode, NodeError> {
|
||||||
iroh_status: EndpointStatus::scaffolded(),
|
iroh_status: EndpointStatus::scaffolded(),
|
||||||
iroh_endpoint: Arc::new(Mutex::new(None)),
|
iroh_endpoint: Arc::new(Mutex::new(None)),
|
||||||
iroh_blob_store: Arc::new(Mutex::new(None)),
|
iroh_blob_store: Arc::new(Mutex::new(None)),
|
||||||
|
iroh_docs: Arc::new(Mutex::new(None)),
|
||||||
|
iroh_gossip: Arc::new(Mutex::new(None)),
|
||||||
runtime: Arc::new(NodeRuntime {
|
runtime: Arc::new(NodeRuntime {
|
||||||
pubsub: Mutex::new(PubsubRuntime::default()),
|
pubsub: Mutex::new(PubsubRuntime::default()),
|
||||||
pipes: Mutex::new(PipeRuntime::default()),
|
pipes: Mutex::new(PipeRuntime::default()),
|
||||||
|
|
@ -760,6 +774,17 @@ pub async fn handle_request_async(
|
||||||
ControlResponse::CasPrivateAdded { encrypted_hash, .. } => {
|
ControlResponse::CasPrivateAdded { encrypted_hash, .. } => {
|
||||||
mirror_local_blob_to_iroh_blobs(node, encrypted_hash).await?;
|
mirror_local_blob_to_iroh_blobs(node, encrypted_hash).await?;
|
||||||
}
|
}
|
||||||
|
ControlResponse::KvCreated { kv } => {
|
||||||
|
mirror_kv_store_to_iroh_docs(node, kv.name.as_str()).await?;
|
||||||
|
}
|
||||||
|
ControlResponse::KvSet { entry } => {
|
||||||
|
let name = entry
|
||||||
|
.store
|
||||||
|
.as_str()
|
||||||
|
.strip_prefix("kv:")
|
||||||
|
.unwrap_or(entry.store.as_str());
|
||||||
|
mirror_kv_store_to_iroh_docs(node, name).await?;
|
||||||
|
}
|
||||||
_ => {}
|
_ => {}
|
||||||
}
|
}
|
||||||
Ok(response)
|
Ok(response)
|
||||||
|
|
@ -1849,6 +1874,7 @@ async fn kv_sync_from_peer(
|
||||||
name: response_name,
|
name: response_name,
|
||||||
entries,
|
entries,
|
||||||
high_water_ms,
|
high_water_ms,
|
||||||
|
docs_ticket,
|
||||||
allowed,
|
allowed,
|
||||||
reason,
|
reason,
|
||||||
note,
|
note,
|
||||||
|
|
@ -1866,9 +1892,13 @@ async fn kv_sync_from_peer(
|
||||||
note,
|
note,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
let mut entries_imported = 0;
|
||||||
|
if let Some(docs_ticket) = docs_ticket {
|
||||||
|
entries_imported +=
|
||||||
|
import_kv_entries_from_iroh_docs(node, name, &docs_ticket).await?;
|
||||||
|
}
|
||||||
let store = Store::open(&node.paths.metadata_db())?;
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
let kv = ensure_local_kv_store(&store, name)?;
|
let kv = ensure_local_kv_store(&store, name)?;
|
||||||
let mut entries_imported = 0;
|
|
||||||
for entry in entries {
|
for entry in entries {
|
||||||
geth_kv::validate_kv_key(&entry.key)
|
geth_kv::validate_kv_key(&entry.key)
|
||||||
.map_err(|_| NodeError::InvalidKvKey(entry.key.clone()))?;
|
.map_err(|_| NodeError::InvalidKvKey(entry.key.clone()))?;
|
||||||
|
|
@ -1886,6 +1916,8 @@ async fn kv_sync_from_peer(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
store_live_sync_cursor(&store, peer_node, &stream, high_water_ms)?;
|
store_live_sync_cursor(&store, peer_node, &stream, high_water_ms)?;
|
||||||
|
drop(store);
|
||||||
|
mirror_kv_store_to_iroh_docs(node, name).await?;
|
||||||
Ok(ControlResponse::KvSynced {
|
Ok(ControlResponse::KvSynced {
|
||||||
peer_node_id: node_id,
|
peer_node_id: node_id,
|
||||||
peer_agent_id: agent_id,
|
peer_agent_id: agent_id,
|
||||||
|
|
@ -1907,6 +1939,227 @@ async fn kv_sync_from_peer(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn iroh_docs(node: &LocalNode) -> Result<Option<iroh_docs::protocol::Docs>, NodeError> {
|
||||||
|
node.iroh_docs
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| NodeError::RuntimeLockPoisoned)
|
||||||
|
.map(|docs| docs.clone())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn iroh_gossip(node: &LocalNode) -> Result<Option<iroh_gossip::net::Gossip>, NodeError> {
|
||||||
|
node.iroh_gossip
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| NodeError::RuntimeLockPoisoned)
|
||||||
|
.map(|gossip| gossip.clone())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn kv_docs_state_key(name: &str) -> String {
|
||||||
|
format!("kv-doc:{name}")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn kv_docs_meta_key() -> &'static str {
|
||||||
|
"__geth/kv/name"
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn ensure_kv_docs_state(
|
||||||
|
node: &LocalNode,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<Option<(iroh_docs::api::Doc, KvDocsState)>, NodeError> {
|
||||||
|
let Some(docs) = iroh_docs(node)? else {
|
||||||
|
return Ok(None);
|
||||||
|
};
|
||||||
|
let stored_state = Store::open(&node.paths.metadata_db())?
|
||||||
|
.get_module_state(&kv_docs_state_key(name))?
|
||||||
|
.map(|stored| serde_json::from_str::<KvDocsState>(&stored.state_json))
|
||||||
|
.transpose()?;
|
||||||
|
if let Some(state) = stored_state {
|
||||||
|
let namespace = state
|
||||||
|
.namespace_id
|
||||||
|
.parse::<iroh_docs::NamespaceId>()
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("invalid KV docs namespace: {error}")))?;
|
||||||
|
if let Some(doc) = docs
|
||||||
|
.open(namespace)
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs open failed: {error}")))?
|
||||||
|
{
|
||||||
|
return Ok(Some((doc, state)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let doc = docs
|
||||||
|
.create()
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs create failed: {error}")))?;
|
||||||
|
let ticket = doc
|
||||||
|
.share(ShareMode::Read, AddrInfoOptions::RelayAndAddresses)
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs share failed: {error}")))?;
|
||||||
|
let state = KvDocsState {
|
||||||
|
name: name.to_owned(),
|
||||||
|
namespace_id: doc.id().to_string(),
|
||||||
|
read_ticket: ticket.to_string(),
|
||||||
|
updated_at_ms: geth_store::now_ms(),
|
||||||
|
};
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
store.put_module_state(&StoredModuleState {
|
||||||
|
module: kv_docs_state_key(name),
|
||||||
|
state_json: serde_json::to_string(&state)?,
|
||||||
|
updated_at_ms: state.updated_at_ms,
|
||||||
|
})?;
|
||||||
|
Ok(Some((doc, state)))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn kv_docs_read_ticket(node: &LocalNode, name: &str) -> Result<Option<String>, NodeError> {
|
||||||
|
Ok(ensure_kv_docs_state(node, name)
|
||||||
|
.await?
|
||||||
|
.map(|(_, state)| state.read_ticket))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn mirror_kv_store_to_iroh_docs(node: &LocalNode, name: &str) -> Result<(), NodeError> {
|
||||||
|
let Some(docs) = iroh_docs(node)? else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
let entries = {
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
let kv = ensure_local_kv_store(&store, name)?;
|
||||||
|
store.list_kv_entries_since(&kv.kv_id, 0)?
|
||||||
|
};
|
||||||
|
let Some((doc, _state)) = ensure_kv_docs_state(node, name).await? else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
let author = docs
|
||||||
|
.author_default()
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs author failed: {error}")))?;
|
||||||
|
doc.set_bytes(
|
||||||
|
author,
|
||||||
|
kv_docs_meta_key().as_bytes().to_vec(),
|
||||||
|
name.as_bytes().to_vec(),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs set metadata failed: {error}")))?;
|
||||||
|
for entry in entries {
|
||||||
|
doc.set_bytes(author, entry.key.into_bytes(), entry.value.into_bytes())
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs set failed: {error}")))?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn mirror_all_kv_stores_to_iroh_docs(node: &LocalNode) -> Result<usize, NodeError> {
|
||||||
|
if iroh_docs(node)?.is_none() {
|
||||||
|
return Ok(0);
|
||||||
|
}
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
let mut mirrored = 0;
|
||||||
|
for kv in store.list_kv_stores()? {
|
||||||
|
mirror_kv_store_to_iroh_docs(node, &kv.name).await?;
|
||||||
|
mirrored += 1;
|
||||||
|
}
|
||||||
|
Ok(mirrored)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn import_kv_entries_from_iroh_docs(
|
||||||
|
node: &LocalNode,
|
||||||
|
name: &str,
|
||||||
|
ticket: &str,
|
||||||
|
) -> Result<usize, NodeError> {
|
||||||
|
let Some(docs) = iroh_docs(node)? else {
|
||||||
|
return Ok(0);
|
||||||
|
};
|
||||||
|
let Some(blob_store) = iroh_blob_store(node)? else {
|
||||||
|
return Ok(0);
|
||||||
|
};
|
||||||
|
let ticket = ticket
|
||||||
|
.parse::<iroh_docs::DocTicket>()
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("invalid iroh-docs ticket: {error}")))?;
|
||||||
|
let doc = docs
|
||||||
|
.import(ticket.clone())
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs import failed: {error}")))?;
|
||||||
|
let state = KvDocsState {
|
||||||
|
name: name.to_owned(),
|
||||||
|
namespace_id: doc.id().to_string(),
|
||||||
|
read_ticket: ticket.to_string(),
|
||||||
|
updated_at_ms: geth_store::now_ms(),
|
||||||
|
};
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
store.put_module_state(&StoredModuleState {
|
||||||
|
module: kv_docs_state_key(name),
|
||||||
|
state_json: serde_json::to_string(&state)?,
|
||||||
|
updated_at_ms: state.updated_at_ms,
|
||||||
|
})?;
|
||||||
|
|
||||||
|
let mut imported = 0;
|
||||||
|
for _ in 0..5 {
|
||||||
|
imported += import_kv_doc_entries_once(node, name, &blob_store, &doc).await?;
|
||||||
|
if imported > 0 {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||||
|
}
|
||||||
|
Ok(imported)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn import_kv_doc_entries_once(
|
||||||
|
node: &LocalNode,
|
||||||
|
name: &str,
|
||||||
|
blob_store: &iroh_blobs::store::fs::FsStore,
|
||||||
|
doc: &iroh_docs::api::Doc,
|
||||||
|
) -> Result<usize, NodeError> {
|
||||||
|
let mut imported_entries = Vec::new();
|
||||||
|
let entries = doc
|
||||||
|
.get_many(iroh_docs::store::Query::single_latest_per_key())
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs query failed: {error}")))?;
|
||||||
|
futures::pin_mut!(entries);
|
||||||
|
while let Some(entry) = entries.next().await {
|
||||||
|
let entry = entry
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs entry failed: {error}")))?;
|
||||||
|
let key = String::from_utf8(entry.key().to_vec())
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("invalid KV docs key: {error}")))?;
|
||||||
|
if key == kv_docs_meta_key() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
geth_kv::validate_kv_key(&key).map_err(|_| NodeError::InvalidKvKey(key.clone()))?;
|
||||||
|
if entry.content_len() > 1024 * 1024 {
|
||||||
|
return Err(NodeError::IrohPeer(format!(
|
||||||
|
"KV docs value for {key} is too large: {} bytes",
|
||||||
|
entry.content_len()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
let value = blob_store
|
||||||
|
.get_bytes(entry.content_hash())
|
||||||
|
.await
|
||||||
|
.map_err(|error| {
|
||||||
|
NodeError::IrohPeer(format!("iroh-docs blob export failed: {error}"))
|
||||||
|
})?;
|
||||||
|
let value = String::from_utf8(value.to_vec())
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("invalid KV docs value: {error}")))?;
|
||||||
|
let updated_at_ms = (entry.timestamp() / 1000) as i64;
|
||||||
|
imported_entries.push((key, value, updated_at_ms));
|
||||||
|
}
|
||||||
|
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
let kv = ensure_local_kv_store(&store, name)?;
|
||||||
|
let mut imported = 0;
|
||||||
|
for (key, value, updated_at_ms) in imported_entries {
|
||||||
|
let should_import = store
|
||||||
|
.get_kv_entry(&kv.kv_id, &key)?
|
||||||
|
.is_none_or(|local| updated_at_ms >= local.updated_at_ms);
|
||||||
|
if should_import {
|
||||||
|
store.set_kv_entry(&StoredKvEntry {
|
||||||
|
kv_id: kv.kv_id.clone(),
|
||||||
|
key,
|
||||||
|
value,
|
||||||
|
updated_at_ms,
|
||||||
|
})?;
|
||||||
|
imported += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(imported)
|
||||||
|
}
|
||||||
|
|
||||||
async fn pubsub_publish_to_peer(
|
async fn pubsub_publish_to_peer(
|
||||||
node: &LocalNode,
|
node: &LocalNode,
|
||||||
peer_node: &str,
|
peer_node: &str,
|
||||||
|
|
@ -4558,6 +4811,28 @@ async fn handle_iroh_control_connection(
|
||||||
.await
|
.await
|
||||||
.map_err(|error| NodeError::IrohPeer(format!("iroh-blobs accept failed: {error}")));
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-blobs accept failed: {error}")));
|
||||||
}
|
}
|
||||||
|
if alpn == display_alpn(iroh_docs::ALPN) {
|
||||||
|
let docs = iroh_docs(&node)?.ok_or_else(|| {
|
||||||
|
NodeError::IrohPeer(
|
||||||
|
"iroh-docs ALPN accepted but docs runtime is unavailable".to_owned(),
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
return docs
|
||||||
|
.accept(conn)
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-docs accept failed: {error}")));
|
||||||
|
}
|
||||||
|
if alpn == display_alpn(iroh_gossip::ALPN) {
|
||||||
|
let gossip = iroh_gossip(&node)?.ok_or_else(|| {
|
||||||
|
NodeError::IrohPeer(
|
||||||
|
"iroh-gossip ALPN accepted but gossip runtime is unavailable".to_owned(),
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
return gossip
|
||||||
|
.accept(conn)
|
||||||
|
.await
|
||||||
|
.map_err(|error| NodeError::IrohPeer(format!("iroh-gossip accept failed: {error}")));
|
||||||
|
}
|
||||||
let (mut send, mut recv) = conn
|
let (mut send, mut recv) = conn
|
||||||
.accept_bi()
|
.accept_bi()
|
||||||
.await
|
.await
|
||||||
|
|
@ -5094,6 +5369,13 @@ async fn handle_iroh_control_connection(
|
||||||
} else {
|
} else {
|
||||||
Vec::new()
|
Vec::new()
|
||||||
};
|
};
|
||||||
|
let docs_ticket = if explanation.allowed {
|
||||||
|
drop(store);
|
||||||
|
mirror_kv_store_to_iroh_docs(&node, &name).await?;
|
||||||
|
kv_docs_read_ticket(&node, &name).await?
|
||||||
|
} else {
|
||||||
|
None
|
||||||
|
};
|
||||||
PeerControlResponse::KvSynced {
|
PeerControlResponse::KvSynced {
|
||||||
node_id: node.node_id.clone(),
|
node_id: node.node_id.clone(),
|
||||||
agent_id: node.agent_id.clone(),
|
agent_id: node.agent_id.clone(),
|
||||||
|
|
@ -5102,6 +5384,7 @@ async fn handle_iroh_control_connection(
|
||||||
name,
|
name,
|
||||||
entries,
|
entries,
|
||||||
high_water_ms,
|
high_water_ms,
|
||||||
|
docs_ticket,
|
||||||
allowed: explanation.allowed,
|
allowed: explanation.allowed,
|
||||||
reason: explanation.reason,
|
reason: explanation.reason,
|
||||||
evaluated_ops: explanation.evaluated_ops,
|
evaluated_ops: explanation.evaluated_ops,
|
||||||
|
|
@ -7779,17 +8062,39 @@ pub fn handle_request(
|
||||||
fn native_backend_statuses() -> Vec<NativeBackendStatus> {
|
fn native_backend_statuses() -> Vec<NativeBackendStatus> {
|
||||||
geth_iroh::native_iroh_libraries()
|
geth_iroh::native_iroh_libraries()
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|library| NativeBackendStatus {
|
.map(|library| {
|
||||||
module: library.module,
|
let (status, blocker) = match library.module.as_str() {
|
||||||
current_backend: "shared-iroh-endpoint-ready".to_owned(),
|
"cas" => (
|
||||||
target_crate: library.crate_name,
|
"wired",
|
||||||
target_version: library.crate_version,
|
format!(
|
||||||
status: "ready-to-wire".to_owned(),
|
"none; CAS payload fetches use native ALPN {} after geth control authorization preflight",
|
||||||
blocker: format!(
|
library.alpn
|
||||||
|
),
|
||||||
|
),
|
||||||
|
"kv" => (
|
||||||
|
"wired",
|
||||||
|
format!(
|
||||||
|
"none; KV stores mirror into Iroh Documents over native ALPN {} after geth control authorization preflight",
|
||||||
|
library.alpn
|
||||||
|
),
|
||||||
|
),
|
||||||
|
_ => (
|
||||||
|
"ready-to-wire",
|
||||||
|
format!(
|
||||||
"none; native ALPN {} is compiled against the daemon-owned iroh {} endpoint, but the module still needs its bootstrap control-path implementation replaced",
|
"none; native ALPN {} is compiled against the daemon-owned iroh {} endpoint, but the module still needs its bootstrap control-path implementation replaced",
|
||||||
library.alpn,
|
library.alpn,
|
||||||
geth_iroh::IROH_VERSION
|
geth_iroh::IROH_VERSION
|
||||||
),
|
),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
NativeBackendStatus {
|
||||||
|
module: library.module,
|
||||||
|
current_backend: "shared-iroh-endpoint-ready".to_owned(),
|
||||||
|
target_crate: library.crate_name,
|
||||||
|
target_version: library.crate_version,
|
||||||
|
status: status.to_owned(),
|
||||||
|
blocker,
|
||||||
|
}
|
||||||
})
|
})
|
||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
@ -9679,6 +9984,14 @@ async fn start_daemon_iroh_endpoint(
|
||||||
.map_err(|error| {
|
.map_err(|error| {
|
||||||
NodeError::IrohPeer(format!("failed to open iroh-blobs store: {error}"))
|
NodeError::IrohPeer(format!("failed to open iroh-blobs store: {error}"))
|
||||||
})?;
|
})?;
|
||||||
|
let gossip = iroh_gossip::net::Gossip::builder().spawn(endpoint.endpoint());
|
||||||
|
let blob_api: iroh_blobs::api::Store = blob_store.clone().into();
|
||||||
|
let docs = iroh_docs::protocol::Docs::persistent(node.paths.home().join("iroh-docs"))
|
||||||
|
.spawn(endpoint.endpoint(), blob_api, gossip.clone())
|
||||||
|
.await
|
||||||
|
.map_err(|error| {
|
||||||
|
NodeError::IrohPeer(format!("failed to start iroh-docs: {error}"))
|
||||||
|
})?;
|
||||||
let mirrored = mirror_local_cas_to_iroh_blobs(&node.paths, &blob_store).await?;
|
let mirrored = mirror_local_cas_to_iroh_blobs(&node.paths, &blob_store).await?;
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
mirrored_blobs = mirrored,
|
mirrored_blobs = mirrored,
|
||||||
|
|
@ -9693,6 +10006,19 @@ async fn start_daemon_iroh_endpoint(
|
||||||
.iroh_blob_store
|
.iroh_blob_store
|
||||||
.lock()
|
.lock()
|
||||||
.map_err(|_| NodeError::RuntimeLockPoisoned)? = Some(blob_store);
|
.map_err(|_| NodeError::RuntimeLockPoisoned)? = Some(blob_store);
|
||||||
|
*node
|
||||||
|
.iroh_gossip
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| NodeError::RuntimeLockPoisoned)? = Some(gossip);
|
||||||
|
*node
|
||||||
|
.iroh_docs
|
||||||
|
.lock()
|
||||||
|
.map_err(|_| NodeError::RuntimeLockPoisoned)? = Some(docs);
|
||||||
|
let mirrored_kv = mirror_all_kv_stores_to_iroh_docs(node).await?;
|
||||||
|
tracing::info!(
|
||||||
|
mirrored_kv_stores = mirrored_kv,
|
||||||
|
"native iroh-docs KV store is ready"
|
||||||
|
);
|
||||||
Ok(Some(endpoint))
|
Ok(Some(endpoint))
|
||||||
}
|
}
|
||||||
Err(error) => {
|
Err(error) => {
|
||||||
|
|
@ -11510,6 +11836,16 @@ mod tests {
|
||||||
}
|
}
|
||||||
other => panic!("unexpected synced KV get response: {other:?}"),
|
other => panic!("unexpected synced KV get response: {other:?}"),
|
||||||
}
|
}
|
||||||
|
let left_kv_doc_state = Store::open(&left_paths.metadata_db())
|
||||||
|
.expect("open left store for kv doc state")
|
||||||
|
.get_module_state("kv-doc:prefs")
|
||||||
|
.expect("get left kv doc state")
|
||||||
|
.expect("left imported kv docs state");
|
||||||
|
let left_kv_doc_state: KvDocsState =
|
||||||
|
serde_json::from_str(&left_kv_doc_state.state_json).expect("parse kv docs state");
|
||||||
|
assert_eq!(left_kv_doc_state.name, "prefs");
|
||||||
|
assert!(!left_kv_doc_state.namespace_id.is_empty());
|
||||||
|
assert!(left_kv_doc_state.read_ticket.starts_with("doc"));
|
||||||
|
|
||||||
let published = handle_request_async(
|
let published = handle_request_async(
|
||||||
&left,
|
&left,
|
||||||
|
|
|
||||||
|
|
@ -155,11 +155,13 @@ fn geth_status_against_running_daemon() {
|
||||||
assert!(stdout.contains("iroh relay: disabled"));
|
assert!(stdout.contains("iroh relay: disabled"));
|
||||||
assert!(stdout.contains("iroh discovery: local-network disabled"));
|
assert!(stdout.contains("iroh discovery: local-network disabled"));
|
||||||
assert!(stdout.contains(
|
assert!(stdout.contains(
|
||||||
"native backend cas: shared-iroh-endpoint-ready target iroh-blobs 0.97.0 (ready-to-wire)"
|
"native backend cas: shared-iroh-endpoint-ready target iroh-blobs 0.97.0 (wired)"
|
||||||
));
|
|
||||||
assert!(stdout.contains(
|
|
||||||
"native backend kv: shared-iroh-endpoint-ready target iroh-docs 0.95.0 (ready-to-wire)"
|
|
||||||
));
|
));
|
||||||
|
assert!(
|
||||||
|
stdout.contains(
|
||||||
|
"native backend kv: shared-iroh-endpoint-ready target iroh-docs 0.95.0 (wired)"
|
||||||
|
)
|
||||||
|
);
|
||||||
assert!(stdout.contains(
|
assert!(stdout.contains(
|
||||||
"native backend pubsub: shared-iroh-endpoint-ready target iroh-gossip 0.95.0 (ready-to-wire)"
|
"native backend pubsub: shared-iroh-endpoint-ready target iroh-gossip 0.95.0 (ready-to-wire)"
|
||||||
));
|
));
|
||||||
|
|
|
||||||
|
|
@ -12,8 +12,13 @@ prefix-scoped capabilities.
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
The prototype exposes CLI shape and durable local KV state. `iroh-docs 0.95.0`
|
The prototype keeps SQLite as the durable local KV index and mirrors each named
|
||||||
is pinned and compiles against the daemon-owned `iroh 0.95.1` endpoint
|
KV store into an Iroh Documents namespace on the daemon-owned `iroh 0.95.1`
|
||||||
generation, so the remaining work is replacing the bootstrap control-path KV
|
endpoint. Remote `geth kv sync` still uses geth control as the authorization
|
||||||
sync with an Iroh Documents namespace implementation and resource-scoped
|
preflight. If the caller has `kv.read` on the remote `resource:kv:<name>`, the
|
||||||
authorization checks around namespace access.
|
remote daemon returns a read-only Iroh Documents ticket and the requester imports
|
||||||
|
entries through `iroh-docs 0.95.0`.
|
||||||
|
|
||||||
|
The daemon must not hand out Iroh Documents write capabilities as a substitute
|
||||||
|
for geth authorization. Write authority remains modeled through geth resource
|
||||||
|
capabilities such as `kv.write` and `kv.write_prefix:<prefix>`.
|
||||||
|
|
|
||||||
|
|
@ -35,17 +35,19 @@ exposes their native ALPNs without creating a second daemon endpoint. CAS now
|
||||||
registers an `iroh-blobs` provider handler on `/iroh-bytes/4`; local CAS writes
|
registers an `iroh-blobs` provider handler on `/iroh-bytes/4`; local CAS writes
|
||||||
are mirrored into the native blob store, and remote `geth cas fetch` performs a
|
are mirrored into the native blob store, and remote `geth cas fetch` performs a
|
||||||
geth control-ALPN authorization preflight before transferring payload bytes over
|
geth control-ALPN authorization preflight before transferring payload bytes over
|
||||||
`iroh-blobs`. `geth status` reports these native backend libraries. KV and
|
`iroh-blobs`. KV now starts `iroh-docs` with `iroh-gossip` and the same native
|
||||||
pubsub still use explicit documented bootstrap equivalents until each module is
|
blob store, mirrors named KV stores into read-shared Iroh Documents namespaces,
|
||||||
migrated to its native protocol.
|
and sends read-only docs tickets only after geth control authorization succeeds.
|
||||||
|
`geth status` reports CAS and KV as wired native backends. Pubsub still uses its
|
||||||
|
explicit documented bootstrap equivalent until it is migrated to native gossip.
|
||||||
|
|
||||||
Module ALPNs are registered through `geth-iroh`'s protocol router scaffold. The
|
Module ALPNs are registered through `geth-iroh`'s protocol router scaffold. The
|
||||||
router owns the default protocol descriptors, rejects duplicate ALPN
|
router owns the default protocol descriptors, rejects duplicate ALPN
|
||||||
registrations, and returns explicit unknown-ALPN errors. The current daemon
|
registrations, and returns explicit unknown-ALPN errors. The current daemon
|
||||||
accept loop dispatches geth control, pipe, SSH-proxy, and native CAS blob
|
accept loop dispatches geth control, pipe, SSH-proxy, and native CAS blob
|
||||||
streams directly; the next backend migrations should attach iroh-docs and
|
streams directly, plus native docs and gossip streams used by KV. The next
|
||||||
iroh-gossip handlers to the same endpoint instead of creating parallel
|
backend migration should attach application pubsub behavior to iroh-gossip
|
||||||
endpoints.
|
instead of creating parallel endpoints.
|
||||||
|
|
||||||
The target product should use Iroh relay support for practical internet
|
The target product should use Iroh relay support for practical internet
|
||||||
connectivity and mDNS/LAN discovery for local networks. These are connectivity
|
connectivity and mDNS/LAN discovery for local networks. These are connectivity
|
||||||
|
|
@ -203,18 +205,17 @@ or cr-sqlite extension artifact. DB sync intentionally does not use CAS-backed
|
||||||
snapshots or batch blobs in the prototype. Those become useful when initial
|
snapshots or batch blobs in the prototype. Those become useful when initial
|
||||||
catch-up or large batches outgrow the protected control path.
|
catch-up or large batches outgrow the protected control path.
|
||||||
|
|
||||||
`geth-kv` currently provides a SQLite-backed local fallback for named KV stores
|
`geth-kv` keeps SQLite as the durable local index for named KV stores through
|
||||||
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
|
`kv create/set/get`. `kv set --subject <principal>` evaluates local auth ops for
|
||||||
ops for `kv.write_key:<key>` so prefix grants can be tested before networked
|
`kv.write_key:<key>` so prefix grants can be tested. The daemon mirrors local KV
|
||||||
callers exist. The local node/agent retains owner access for administration.
|
entries and metadata into an Iroh Documents namespace per named store. `geth kv
|
||||||
Iroh Documents namespaces remain the target backend, but the bootstrap can sync
|
sync <node-id> <name>` still starts with a protected geth control request that
|
||||||
named KV stores over the protected Iroh control ALPN. `geth kv sync <node-id>
|
requires `kv.read` on the remote `resource:kv:<name>`; if authorized, the remote
|
||||||
<name>` requires `kv.read` on the remote `resource:kv:<name>`, transfers entries
|
daemon returns a read-only docs ticket and the requester imports entries through
|
||||||
at or beyond a per-peer/per-KV high-water cursor, and imports only values that
|
Iroh Documents. The control response still carries bootstrap entries for
|
||||||
are at least as new as the local entry timestamp. The daemon background
|
compatibility. The daemon background live-sync loop runs the same KV sync for
|
||||||
live-sync loop runs the same KV sync for local KV stores and known peers.
|
local KV stores and known peers. Private value encryption should use resource
|
||||||
Private value encryption should use resource secret epochs before payloads are
|
secret epochs before payloads are exposed to remote peers.
|
||||||
exposed to remote peers.
|
|
||||||
|
|
||||||
`geth-document` registers local document resources and stores durable Automerge
|
`geth-document` registers local document resources and stores durable Automerge
|
||||||
save bytes in the local SQLite metadata store. The CLI still accepts and returns
|
save bytes in the local SQLite metadata store. The CLI still accepts and returns
|
||||||
|
|
|
||||||
|
|
@ -486,7 +486,7 @@ authorization and durable-state boundaries clear.
|
||||||
rejection.
|
rejection.
|
||||||
- `[x]` Docs explicitly avoid claiming forward secrecy or PCS.
|
- `[x]` Docs explicitly avoid claiming forward secrecy or PCS.
|
||||||
|
|
||||||
- `[~]` Iroh-docs KV integration.
|
- `[x]` Iroh-docs KV integration.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
- `[x]` `geth kv create/set/get` works against a named local KV resource.
|
- `[x]` `geth kv create/set/get` works against a named local KV resource.
|
||||||
- `[x]` KV metadata and entries are durable in the local SQLite store.
|
- `[x]` KV metadata and entries are durable in the local SQLite store.
|
||||||
|
|
@ -500,7 +500,10 @@ authorization and durable-state boundaries clear.
|
||||||
- `[x]` Remote KV sync requires `kv.read` on `resource:kv:<name>`.
|
- `[x]` Remote KV sync requires `kv.read` on `resource:kv:<name>`.
|
||||||
- `[x]` Background live-sync refreshes local KV stores from known peers using
|
- `[x]` Background live-sync refreshes local KV stores from known peers using
|
||||||
per-peer/per-KV high-water cursors.
|
per-peer/per-KV high-water cursors.
|
||||||
- `[ ]` KV metadata is replicated through Iroh Documents.
|
- `[x]` KV metadata is replicated through Iroh Documents.
|
||||||
|
- `[x]` Authorized peers receive read-only Iroh Documents tickets, not write
|
||||||
|
capabilities, after geth control authorization succeeds.
|
||||||
|
- `[x]` Tests cover imported KV docs state after authorized remote sync.
|
||||||
|
|
||||||
- `[~]` Iroh-gossip pubsub integration.
|
- `[~]` Iroh-gossip pubsub integration.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue