Live sync KV stores over Iroh
This commit is contained in:
parent
68153be5d5
commit
f85039367c
9 changed files with 473 additions and 13 deletions
10
AGENTS.md
10
AGENTS.md
|
|
@ -109,9 +109,9 @@ Roadmap items should be actionable and checkable:
|
||||||
and `geth peer auth-check` over Iroh, signed peer-card LAN discovery payloads,
|
and `geth peer auth-check` over Iroh, signed peer-card LAN discovery payloads,
|
||||||
authorized `geth cas fetch`, `geth ssh cert sync`, and
|
authorized `geth cas fetch`, `geth ssh cert sync`, and
|
||||||
`geth ssh revocation sync` over the Iroh control ALPN, a background SSH
|
`geth ssh revocation sync` over the Iroh control ALPN, a background SSH
|
||||||
metadata live-sync loop with per-peer cursors in `module_state`, untrusted
|
metadata and KV live-sync loop with per-peer cursors in `module_state`,
|
||||||
discovery-backend trait, custom relay-map config, and Iroh local-network
|
untrusted discovery-backend trait, custom relay-map config, and Iroh
|
||||||
discovery toggle exist.
|
local-network discovery toggle exist.
|
||||||
- Canonical signed-operation envelopes exist for keychain/auth signature
|
- Canonical signed-operation envelopes exist for keychain/auth signature
|
||||||
payloads. The keychain reducer builds an active identity view for admin keys,
|
payloads. The keychain reducer builds an active identity view for admin keys,
|
||||||
users, devices, nodes, agents, and endpoint bindings.
|
users, devices, nodes, agents, and endpoint bindings.
|
||||||
|
|
@ -138,7 +138,9 @@ Roadmap items should be actionable and checkable:
|
||||||
present. The DB crate and daemon can extract typed read-only `crsql_changes`
|
present. The DB crate and daemon can extract typed read-only `crsql_changes`
|
||||||
batches through `geth db changes` for future sync messages. cr-sqlite loading,
|
batches through `geth db changes` for future sync messages. cr-sqlite loading,
|
||||||
applying remote changes, and sync are still roadmap work.
|
applying remote changes, and sync are still roadmap work.
|
||||||
- KV stores support local SQLite-backed create/set/get. Iroh Documents
|
- KV stores support local SQLite-backed create/set/get plus authorized
|
||||||
|
`geth kv sync <node-id> <name>` over Iroh. Background live-sync refreshes
|
||||||
|
local KV stores from known peers with per-peer/per-KV cursors. Iroh Documents
|
||||||
replication and command-level prefix-capability enforcement are still roadmap
|
replication and command-level prefix-capability enforcement are still roadmap
|
||||||
work. The auth evaluator already understands `kv.write_prefix:<prefix>`
|
work. The auth evaluator already understands `kv.write_prefix:<prefix>`
|
||||||
grants for `kv.write_key:<key>` requests.
|
grants for `kv.write_key:<key>` requests.
|
||||||
|
|
|
||||||
|
|
@ -104,7 +104,7 @@ The bootstrap implementation provides:
|
||||||
`geth db changes <name>` for future sync
|
`geth db changes <name>` for future sync
|
||||||
- local SQLite-backed KV commands: `geth kv create/set/get`; `kv set` accepts
|
- local SQLite-backed KV commands: `geth kv create/set/get`; `kv set` accepts
|
||||||
`--subject <principal>` to exercise local capability checks for non-local
|
`--subject <principal>` to exercise local capability checks for non-local
|
||||||
callers
|
callers; `geth kv sync <node-id> <name>` pulls authorized remote updates
|
||||||
- local JSON document commands: `geth document create/status/set/get`
|
- local JSON document commands: `geth document create/status/set/get`
|
||||||
- local daemon-lifetime pubsub snapshots: `geth pubsub pub/sub`
|
- local daemon-lifetime pubsub snapshots: `geth pubsub pub/sub`
|
||||||
- SSH certificate flow metadata:
|
- SSH certificate flow metadata:
|
||||||
|
|
@ -142,6 +142,10 @@ approval/signing workflows. While the daemon is running, it also performs a
|
||||||
background live-sync tick for known peers every 30 seconds. Live-sync stores
|
background live-sync tick for known peers every 30 seconds. Live-sync stores
|
||||||
per-peer high-water cursors in local metadata so repeated ticks request only
|
per-peer high-water cursors in local metadata so repeated ticks request only
|
||||||
newer SSH certificate-flow and revocation records.
|
newer SSH certificate-flow and revocation records.
|
||||||
|
Named KV stores participate in the same live-sync loop once they exist locally:
|
||||||
|
manual `geth kv sync <node-id> <name>` and background ticks require `kv.read`
|
||||||
|
on the remote `resource:kv:<name>` and import only remote entries that are not
|
||||||
|
older than the local value.
|
||||||
Importing or pinging a peer card never grants capabilities by itself.
|
Importing or pinging a peer card never grants capabilities by itself.
|
||||||
When `[iroh].local_discovery = true`, the daemon also advertises and discovers
|
When `[iroh].local_discovery = true`, the daemon also advertises and discovers
|
||||||
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is
|
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is
|
||||||
|
|
|
||||||
|
|
@ -304,6 +304,10 @@ pub enum KvCommand {
|
||||||
name: String,
|
name: String,
|
||||||
key: String,
|
key: String,
|
||||||
},
|
},
|
||||||
|
Sync {
|
||||||
|
node: String,
|
||||||
|
name: String,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Subcommand)]
|
#[derive(Debug, Subcommand)]
|
||||||
|
|
@ -617,6 +621,7 @@ fn request_for_command(command: Command) -> Result<ControlRequest> {
|
||||||
subject,
|
subject,
|
||||||
},
|
},
|
||||||
KvCommand::Get { name, key } => ControlRequest::KvGet { name, key },
|
KvCommand::Get { name, key } => ControlRequest::KvGet { name, key },
|
||||||
|
KvCommand::Sync { node, name } => ControlRequest::KvSync { node, name },
|
||||||
},
|
},
|
||||||
Command::Pubsub { command } => match command {
|
Command::Pubsub { command } => match command {
|
||||||
PubsubCommand::Pub { topic, message } => ControlRequest::PubsubPub { topic, message },
|
PubsubCommand::Pub { topic, message } => ControlRequest::PubsubPub { topic, message },
|
||||||
|
|
@ -1309,6 +1314,27 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> {
|
||||||
println!("not found");
|
println!("not found");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
ControlResponse::KvSynced {
|
||||||
|
peer_node_id,
|
||||||
|
peer_agent_id,
|
||||||
|
endpoint_id,
|
||||||
|
name,
|
||||||
|
entries_imported,
|
||||||
|
allowed,
|
||||||
|
reason,
|
||||||
|
note,
|
||||||
|
} => {
|
||||||
|
if allowed {
|
||||||
|
println!("synced kv {name} from {peer_node_id}: {entries_imported} entries");
|
||||||
|
} else {
|
||||||
|
println!("kv sync denied for {name} by {peer_node_id}");
|
||||||
|
}
|
||||||
|
println!("agent: {peer_agent_id}");
|
||||||
|
println!("endpoint: {endpoint_id}");
|
||||||
|
println!("allowed: {allowed}");
|
||||||
|
println!("reason: {reason}");
|
||||||
|
println!("note: {note}");
|
||||||
|
}
|
||||||
ControlResponse::DocumentCreated { document } => {
|
ControlResponse::DocumentCreated { document } => {
|
||||||
println!("created document: {}", document.name);
|
println!("created document: {}", document.name);
|
||||||
println!("id: {}", document.id);
|
println!("id: {}", document.id);
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ use geth_db::{CrSqliteChangeBatch, DbResource};
|
||||||
use geth_discovery::{DiscoveredPeer, PeerCard};
|
use geth_discovery::{DiscoveredPeer, PeerCard};
|
||||||
use geth_document::{DocumentResource, DocumentState};
|
use geth_document::{DocumentResource, DocumentState};
|
||||||
use geth_keychain::KeychainOp;
|
use geth_keychain::KeychainOp;
|
||||||
use geth_kv::{KvEntry, KvResource};
|
use geth_kv::{KvEntry, KvResource, KvSyncEntry};
|
||||||
use geth_pipe::{PipeConnection, PipeListener};
|
use geth_pipe::{PipeConnection, PipeListener};
|
||||||
use geth_pubsub::PubsubMessage;
|
use geth_pubsub::PubsubMessage;
|
||||||
use geth_resource::ResourceDescriptor;
|
use geth_resource::ResourceDescriptor;
|
||||||
|
|
@ -196,6 +196,10 @@ pub enum ControlRequest {
|
||||||
name: String,
|
name: String,
|
||||||
key: String,
|
key: String,
|
||||||
},
|
},
|
||||||
|
KvSync {
|
||||||
|
node: String,
|
||||||
|
name: String,
|
||||||
|
},
|
||||||
DocumentCreate {
|
DocumentCreate {
|
||||||
name: String,
|
name: String,
|
||||||
},
|
},
|
||||||
|
|
@ -422,6 +426,16 @@ pub enum ControlResponse {
|
||||||
KvGet {
|
KvGet {
|
||||||
entry: Option<KvEntry>,
|
entry: Option<KvEntry>,
|
||||||
},
|
},
|
||||||
|
KvSynced {
|
||||||
|
peer_node_id: String,
|
||||||
|
peer_agent_id: String,
|
||||||
|
endpoint_id: String,
|
||||||
|
name: String,
|
||||||
|
entries_imported: usize,
|
||||||
|
allowed: bool,
|
||||||
|
reason: String,
|
||||||
|
note: String,
|
||||||
|
},
|
||||||
DocumentCreated {
|
DocumentCreated {
|
||||||
document: DocumentResource,
|
document: DocumentResource,
|
||||||
},
|
},
|
||||||
|
|
@ -521,6 +535,12 @@ pub enum PeerControlRequest {
|
||||||
since_ms: i64,
|
since_ms: i64,
|
||||||
nonce: String,
|
nonce: String,
|
||||||
},
|
},
|
||||||
|
KvSync {
|
||||||
|
peer_card: PeerCard,
|
||||||
|
name: String,
|
||||||
|
since_ms: i64,
|
||||||
|
nonce: String,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||||
|
|
@ -589,6 +609,20 @@ pub enum PeerControlResponse {
|
||||||
nonce: String,
|
nonce: String,
|
||||||
note: String,
|
note: String,
|
||||||
},
|
},
|
||||||
|
KvSynced {
|
||||||
|
node_id: String,
|
||||||
|
agent_id: String,
|
||||||
|
endpoint_id: String,
|
||||||
|
remote_endpoint_id: String,
|
||||||
|
name: String,
|
||||||
|
entries: Vec<KvSyncEntry>,
|
||||||
|
high_water_ms: i64,
|
||||||
|
allowed: bool,
|
||||||
|
reason: String,
|
||||||
|
evaluated_ops: usize,
|
||||||
|
nonce: String,
|
||||||
|
note: String,
|
||||||
|
},
|
||||||
Error {
|
Error {
|
||||||
message: String,
|
message: String,
|
||||||
},
|
},
|
||||||
|
|
@ -747,6 +781,30 @@ mod tests {
|
||||||
response
|
response
|
||||||
);
|
);
|
||||||
|
|
||||||
|
let request = ControlRequest::KvSync {
|
||||||
|
node: "node:peer".to_owned(),
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
decode_request(&encode_request(&request).expect("encode")).expect("decode"),
|
||||||
|
request
|
||||||
|
);
|
||||||
|
|
||||||
|
let response = ControlResponse::KvSynced {
|
||||||
|
peer_node_id: "node:peer".to_owned(),
|
||||||
|
peer_agent_id: "agent:peer".to_owned(),
|
||||||
|
endpoint_id: "endpoint:peer".to_owned(),
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
entries_imported: 1,
|
||||||
|
allowed: true,
|
||||||
|
reason: "direct grant".to_owned(),
|
||||||
|
note: "kv sync".to_owned(),
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
decode_response(&encode_response(&response).expect("encode")).expect("decode"),
|
||||||
|
response
|
||||||
|
);
|
||||||
|
|
||||||
let request = ControlRequest::DocumentSet {
|
let request = ControlRequest::DocumentSet {
|
||||||
name: "notes".to_owned(),
|
name: "notes".to_owned(),
|
||||||
state_json: r#"{"title":"notes"}"#.to_owned(),
|
state_json: r#"{"title":"notes"}"#.to_owned(),
|
||||||
|
|
@ -908,5 +966,29 @@ mod tests {
|
||||||
.expect("decode"),
|
.expect("decode"),
|
||||||
response
|
response
|
||||||
);
|
);
|
||||||
|
|
||||||
|
let response = PeerControlResponse::KvSynced {
|
||||||
|
node_id: "node:peer".to_owned(),
|
||||||
|
agent_id: "agent:peer".to_owned(),
|
||||||
|
endpoint_id: "endpoint:peer".to_owned(),
|
||||||
|
remote_endpoint_id: "endpoint:caller".to_owned(),
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
entries: vec![KvSyncEntry {
|
||||||
|
key: "apps/foo/theme".to_owned(),
|
||||||
|
value: "dark".to_owned(),
|
||||||
|
updated_at_ms: 42,
|
||||||
|
}],
|
||||||
|
high_water_ms: 43,
|
||||||
|
allowed: true,
|
||||||
|
reason: "direct grant".to_owned(),
|
||||||
|
evaluated_ops: 1,
|
||||||
|
nonce: "nonce".to_owned(),
|
||||||
|
note: "kv sync".to_owned(),
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
decode_peer_response(&encode_peer_response(&response).expect("encode"))
|
||||||
|
.expect("decode"),
|
||||||
|
response
|
||||||
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -16,6 +16,13 @@ pub struct KvEntry {
|
||||||
pub value: String,
|
pub value: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct KvSyncEntry {
|
||||||
|
pub key: String,
|
||||||
|
pub value: String,
|
||||||
|
pub updated_at_ms: i64,
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, thiserror::Error)]
|
#[derive(Debug, thiserror::Error)]
|
||||||
pub enum KvError {
|
pub enum KvError {
|
||||||
#[error("invalid kv store name: {0}")]
|
#[error("invalid kv store name: {0}")]
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,7 @@ use geth_discovery::{
|
||||||
use geth_document::{DocumentResource, DocumentState};
|
use geth_document::{DocumentResource, DocumentState};
|
||||||
use geth_iroh::{EndpointStatus, GethIrohConfig, GethIrohEndpoint, GethRelayMode};
|
use geth_iroh::{EndpointStatus, GethIrohConfig, GethIrohEndpoint, GethRelayMode};
|
||||||
use geth_keychain::{KeychainOp, KeychainOpKind};
|
use geth_keychain::{KeychainOp, KeychainOpKind};
|
||||||
use geth_kv::{KvEntry, KvResource};
|
use geth_kv::{KvEntry, KvResource, KvSyncEntry};
|
||||||
use geth_pipe::{PipeConnection, PipeListener};
|
use geth_pipe::{PipeConnection, PipeListener};
|
||||||
use geth_pubsub::PubsubMessage;
|
use geth_pubsub::PubsubMessage;
|
||||||
use geth_resource::ResourceDescriptor;
|
use geth_resource::ResourceDescriptor;
|
||||||
|
|
@ -259,6 +259,10 @@ pub async fn handle_request_async(
|
||||||
ControlRequest::SshRevocationSync { node: peer_node } => {
|
ControlRequest::SshRevocationSync { node: peer_node } => {
|
||||||
ssh_revocation_sync_from_peer(node, &peer_node).await
|
ssh_revocation_sync_from_peer(node, &peer_node).await
|
||||||
}
|
}
|
||||||
|
ControlRequest::KvSync {
|
||||||
|
node: peer_node,
|
||||||
|
name,
|
||||||
|
} => kv_sync_from_peer(node, &peer_node, &name).await,
|
||||||
other => handle_request(node, other),
|
other => handle_request(node, other),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -534,7 +538,8 @@ async fn peer_ping(node: &LocalNode, peer_node: &str) -> Result<ControlResponse,
|
||||||
)),
|
)),
|
||||||
PeerControlResponse::CasFetched { .. }
|
PeerControlResponse::CasFetched { .. }
|
||||||
| PeerControlResponse::SshCertSynced { .. }
|
| PeerControlResponse::SshCertSynced { .. }
|
||||||
| PeerControlResponse::SshRevocationSynced { .. } => Err(NodeError::IrohPeer(
|
| PeerControlResponse::SshRevocationSynced { .. }
|
||||||
|
| PeerControlResponse::KvSynced { .. } => Err(NodeError::IrohPeer(
|
||||||
"peer returned wrong response type to ping request".to_owned(),
|
"peer returned wrong response type to ping request".to_owned(),
|
||||||
)),
|
)),
|
||||||
}
|
}
|
||||||
|
|
@ -641,7 +646,8 @@ async fn peer_auth_check(
|
||||||
)),
|
)),
|
||||||
PeerControlResponse::CasFetched { .. }
|
PeerControlResponse::CasFetched { .. }
|
||||||
| PeerControlResponse::SshCertSynced { .. }
|
| PeerControlResponse::SshCertSynced { .. }
|
||||||
| PeerControlResponse::SshRevocationSynced { .. } => Err(NodeError::IrohPeer(
|
| PeerControlResponse::SshRevocationSynced { .. }
|
||||||
|
| PeerControlResponse::KvSynced { .. } => Err(NodeError::IrohPeer(
|
||||||
"peer returned wrong response type to auth-check request".to_owned(),
|
"peer returned wrong response type to auth-check request".to_owned(),
|
||||||
)),
|
)),
|
||||||
}
|
}
|
||||||
|
|
@ -777,7 +783,8 @@ async fn cas_fetch_from_peer(
|
||||||
PeerControlResponse::Pong { .. }
|
PeerControlResponse::Pong { .. }
|
||||||
| PeerControlResponse::AuthChecked { .. }
|
| PeerControlResponse::AuthChecked { .. }
|
||||||
| PeerControlResponse::SshCertSynced { .. }
|
| PeerControlResponse::SshCertSynced { .. }
|
||||||
| PeerControlResponse::SshRevocationSynced { .. } => Err(NodeError::IrohPeer(
|
| PeerControlResponse::SshRevocationSynced { .. }
|
||||||
|
| PeerControlResponse::KvSynced { .. } => Err(NodeError::IrohPeer(
|
||||||
"peer returned wrong response type to CAS fetch".to_owned(),
|
"peer returned wrong response type to CAS fetch".to_owned(),
|
||||||
)),
|
)),
|
||||||
}
|
}
|
||||||
|
|
@ -919,6 +926,90 @@ async fn ssh_revocation_sync_from_peer(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn kv_sync_from_peer(
|
||||||
|
node: &LocalNode,
|
||||||
|
peer_node: &str,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<ControlResponse, NodeError> {
|
||||||
|
geth_kv::validate_kv_name(name).map_err(|_| NodeError::InvalidKvName(name.to_owned()))?;
|
||||||
|
let stream = format!("kv:{name}");
|
||||||
|
let since_ms =
|
||||||
|
load_live_sync_cursor(&Store::open(&node.paths.metadata_db())?, peer_node, &stream)?;
|
||||||
|
let response = request_peer_control(node, peer_node, "kv-sync", |peer_card, nonce| {
|
||||||
|
PeerControlRequest::KvSync {
|
||||||
|
peer_card,
|
||||||
|
name: name.to_owned(),
|
||||||
|
since_ms,
|
||||||
|
nonce,
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.await?;
|
||||||
|
match response {
|
||||||
|
PeerControlResponse::KvSynced {
|
||||||
|
node_id,
|
||||||
|
agent_id,
|
||||||
|
endpoint_id,
|
||||||
|
name: response_name,
|
||||||
|
entries,
|
||||||
|
high_water_ms,
|
||||||
|
allowed,
|
||||||
|
reason,
|
||||||
|
note,
|
||||||
|
..
|
||||||
|
} if response_name == name => {
|
||||||
|
if !allowed {
|
||||||
|
return Ok(ControlResponse::KvSynced {
|
||||||
|
peer_node_id: node_id,
|
||||||
|
peer_agent_id: agent_id,
|
||||||
|
endpoint_id,
|
||||||
|
name: response_name,
|
||||||
|
entries_imported: 0,
|
||||||
|
allowed,
|
||||||
|
reason,
|
||||||
|
note,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
let kv = ensure_local_kv_store(&store, name)?;
|
||||||
|
let mut entries_imported = 0;
|
||||||
|
for entry in entries {
|
||||||
|
geth_kv::validate_kv_key(&entry.key)
|
||||||
|
.map_err(|_| NodeError::InvalidKvKey(entry.key.clone()))?;
|
||||||
|
let should_import = store
|
||||||
|
.get_kv_entry(&kv.kv_id, &entry.key)?
|
||||||
|
.is_none_or(|local| entry.updated_at_ms >= local.updated_at_ms);
|
||||||
|
if should_import {
|
||||||
|
store.set_kv_entry(&StoredKvEntry {
|
||||||
|
kv_id: kv.kv_id.clone(),
|
||||||
|
key: entry.key,
|
||||||
|
value: entry.value,
|
||||||
|
updated_at_ms: entry.updated_at_ms,
|
||||||
|
})?;
|
||||||
|
entries_imported += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
store_live_sync_cursor(&store, peer_node, &stream, high_water_ms)?;
|
||||||
|
Ok(ControlResponse::KvSynced {
|
||||||
|
peer_node_id: node_id,
|
||||||
|
peer_agent_id: agent_id,
|
||||||
|
endpoint_id,
|
||||||
|
name: response_name,
|
||||||
|
entries_imported,
|
||||||
|
allowed,
|
||||||
|
reason,
|
||||||
|
note,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
PeerControlResponse::KvSynced { .. } => Err(NodeError::IrohPeer(
|
||||||
|
"peer KV sync response did not match request".to_owned(),
|
||||||
|
)),
|
||||||
|
PeerControlResponse::Error { message } => Err(NodeError::IrohPeer(message)),
|
||||||
|
_ => Err(NodeError::IrohPeer(
|
||||||
|
"peer returned wrong response type to KV sync".to_owned(),
|
||||||
|
)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn live_sync_cursor_key(peer_node: &str, stream: &str) -> String {
|
fn live_sync_cursor_key(peer_node: &str, stream: &str) -> String {
|
||||||
format!("live-sync:{peer_node}:{stream}")
|
format!("live-sync:{peer_node}:{stream}")
|
||||||
}
|
}
|
||||||
|
|
@ -1011,6 +1102,10 @@ async fn request_peer_control(
|
||||||
| PeerControlResponse::SshRevocationSynced {
|
| PeerControlResponse::SshRevocationSynced {
|
||||||
nonce: response_nonce,
|
nonce: response_nonce,
|
||||||
..
|
..
|
||||||
|
}
|
||||||
|
| PeerControlResponse::KvSynced {
|
||||||
|
nonce: response_nonce,
|
||||||
|
..
|
||||||
} if response_nonce == &nonce => Ok(response),
|
} if response_nonce == &nonce => Ok(response),
|
||||||
PeerControlResponse::Error { .. } => Ok(response),
|
PeerControlResponse::Error { .. } => Ok(response),
|
||||||
_ => Err(NodeError::IrohPeer(format!(
|
_ => Err(NodeError::IrohPeer(format!(
|
||||||
|
|
@ -1058,6 +1153,7 @@ async fn run_live_sync_once(node: &LocalNode) -> Result<(), NodeError> {
|
||||||
return Ok(());
|
return Ok(());
|
||||||
}
|
}
|
||||||
let peers = Store::open(&node.paths.metadata_db())?.list_peer_cards()?;
|
let peers = Store::open(&node.paths.metadata_db())?.list_peer_cards()?;
|
||||||
|
let kv_stores = Store::open(&node.paths.metadata_db())?.list_kv_stores()?;
|
||||||
for peer in peers {
|
for peer in peers {
|
||||||
if let Err(error) = ssh_cert_sync_from_peer(node, &peer.peer_id).await {
|
if let Err(error) = ssh_cert_sync_from_peer(node, &peer.peer_id).await {
|
||||||
tracing::debug!(peer = %peer.peer_id, %error, "SSH cert live sync failed");
|
tracing::debug!(peer = %peer.peer_id, %error, "SSH cert live sync failed");
|
||||||
|
|
@ -1065,6 +1161,11 @@ async fn run_live_sync_once(node: &LocalNode) -> Result<(), NodeError> {
|
||||||
if let Err(error) = ssh_revocation_sync_from_peer(node, &peer.peer_id).await {
|
if let Err(error) = ssh_revocation_sync_from_peer(node, &peer.peer_id).await {
|
||||||
tracing::debug!(peer = %peer.peer_id, %error, "SSH revocation live sync failed");
|
tracing::debug!(peer = %peer.peer_id, %error, "SSH revocation live sync failed");
|
||||||
}
|
}
|
||||||
|
for kv in &kv_stores {
|
||||||
|
if let Err(error) = kv_sync_from_peer(node, &peer.peer_id, &kv.name).await {
|
||||||
|
tracing::debug!(peer = %peer.peer_id, kv = %kv.name, %error, "KV live sync failed");
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
@ -1334,6 +1435,68 @@ async fn handle_iroh_control_connection(
|
||||||
note: "SSH revocation sync authenticated endpoint/card binding and required ssh_revocation.sync on resource:ssh:revocations".to_owned(),
|
note: "SSH revocation sync authenticated endpoint/card binding and required ssh_revocation.sync on resource:ssh:revocations".to_owned(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
PeerControlRequest::KvSync {
|
||||||
|
peer_card,
|
||||||
|
name,
|
||||||
|
since_ms,
|
||||||
|
nonce,
|
||||||
|
} => {
|
||||||
|
geth_kv::validate_kv_name(&name).map_err(|_| NodeError::InvalidKvName(name.clone()))?;
|
||||||
|
peer_card.validate_candidate()?;
|
||||||
|
ensure_peer_card_matches_endpoint(&peer_card, &remote_endpoint_id)?;
|
||||||
|
let discovered = DiscoveredPeer::candidate(
|
||||||
|
peer_card.clone(),
|
||||||
|
UnixMillis(geth_store::now_ms()),
|
||||||
|
DiscoverySource::PeerExchange,
|
||||||
|
)?;
|
||||||
|
let store = Store::open(&node.paths.metadata_db())?;
|
||||||
|
store.upsert_peer_card(&StoredPeerCard {
|
||||||
|
peer_id: peer_card.node_id.to_string(),
|
||||||
|
card_json: serde_json::to_string(&peer_card)?,
|
||||||
|
updated_at_ms: discovered.discovered_at.0,
|
||||||
|
})?;
|
||||||
|
if let Some(kv) = store.get_kv_store_by_name(&name)? {
|
||||||
|
let capability = "kv.read".to_owned();
|
||||||
|
let high_water_ms = geth_store::now_ms();
|
||||||
|
let explanation = geth_auth::explain_auth_ops(
|
||||||
|
&load_auth_ops_for_resource(&store, &kv.resource_id)?,
|
||||||
|
PrincipalId::new(peer_card.node_id.to_string()),
|
||||||
|
ResourceId::new(kv.resource_id.clone()),
|
||||||
|
Capability::new(capability),
|
||||||
|
);
|
||||||
|
let entries = if explanation.allowed {
|
||||||
|
store
|
||||||
|
.list_kv_entries_since(&kv.kv_id, since_ms)?
|
||||||
|
.into_iter()
|
||||||
|
.map(|entry| KvSyncEntry {
|
||||||
|
key: entry.key,
|
||||||
|
value: entry.value,
|
||||||
|
updated_at_ms: entry.updated_at_ms,
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
} else {
|
||||||
|
Vec::new()
|
||||||
|
};
|
||||||
|
PeerControlResponse::KvSynced {
|
||||||
|
node_id: node.node_id.clone(),
|
||||||
|
agent_id: node.agent_id.clone(),
|
||||||
|
endpoint_id: node.iroh_status.endpoint_id.clone().unwrap_or_default(),
|
||||||
|
remote_endpoint_id,
|
||||||
|
name,
|
||||||
|
entries,
|
||||||
|
high_water_ms,
|
||||||
|
allowed: explanation.allowed,
|
||||||
|
reason: explanation.reason,
|
||||||
|
evaluated_ops: explanation.evaluated_ops,
|
||||||
|
nonce,
|
||||||
|
note: "KV sync authenticated endpoint/card binding and required kv.read on the remote KV resource".to_owned(),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
PeerControlResponse::Error {
|
||||||
|
message: format!("kv store not found: {name}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
};
|
};
|
||||||
send.write_all(geth_control::encode_peer_response(&response)?.as_bytes())
|
send.write_all(geth_control::encode_peer_response(&response)?.as_bytes())
|
||||||
.await
|
.await
|
||||||
|
|
@ -1472,6 +1635,7 @@ pub fn handle_request(
|
||||||
ControlRequest::CasFetch { .. } => Err(NodeError::IrohEndpointUnavailable),
|
ControlRequest::CasFetch { .. } => Err(NodeError::IrohEndpointUnavailable),
|
||||||
ControlRequest::SshCertSync { .. } => Err(NodeError::IrohEndpointUnavailable),
|
ControlRequest::SshCertSync { .. } => Err(NodeError::IrohEndpointUnavailable),
|
||||||
ControlRequest::SshRevocationSync { .. } => Err(NodeError::IrohEndpointUnavailable),
|
ControlRequest::SshRevocationSync { .. } => Err(NodeError::IrohEndpointUnavailable),
|
||||||
|
ControlRequest::KvSync { .. } => Err(NodeError::IrohEndpointUnavailable),
|
||||||
ControlRequest::ResourceList => Ok(ControlResponse::ResourceList {
|
ControlRequest::ResourceList => Ok(ControlResponse::ResourceList {
|
||||||
resources: store
|
resources: store
|
||||||
.list_resources()?
|
.list_resources()?
|
||||||
|
|
@ -2482,6 +2646,19 @@ fn kv_entry_from_stored(stored: StoredKvEntry) -> KvEntry {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn ensure_local_kv_store(store: &Store, name: &str) -> Result<StoredKvStore, NodeError> {
|
||||||
|
if let Some(kv) = store.get_kv_store_by_name(name)? {
|
||||||
|
return Ok(kv);
|
||||||
|
}
|
||||||
|
let stored = StoredKvStore {
|
||||||
|
kv_id: format!("kv:{name}"),
|
||||||
|
resource_id: format!("resource:kv:{name}"),
|
||||||
|
name: name.to_owned(),
|
||||||
|
};
|
||||||
|
store.insert_kv_store(&stored)?;
|
||||||
|
Ok(stored)
|
||||||
|
}
|
||||||
|
|
||||||
fn document_resource_from_stored(stored: &StoredDocumentResource) -> DocumentResource {
|
fn document_resource_from_stored(stored: &StoredDocumentResource) -> DocumentResource {
|
||||||
DocumentResource {
|
DocumentResource {
|
||||||
id: stored.document_id.clone().into(),
|
id: stored.document_id.clone().into(),
|
||||||
|
|
@ -2985,6 +3162,23 @@ mod tests {
|
||||||
ControlResponse::SshRevocationAdded { revocation } => revocation.id,
|
ControlResponse::SshRevocationAdded { revocation } => revocation.id,
|
||||||
other => panic!("unexpected SSH revocation response: {other:?}"),
|
other => panic!("unexpected SSH revocation response: {other:?}"),
|
||||||
};
|
};
|
||||||
|
handle_request(
|
||||||
|
&right,
|
||||||
|
ControlRequest::KvCreate {
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("right kv create");
|
||||||
|
handle_request(
|
||||||
|
&right,
|
||||||
|
ControlRequest::KvSet {
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
key: "apps/foo/theme".to_owned(),
|
||||||
|
value: "dark".to_owned(),
|
||||||
|
subject: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("right kv set");
|
||||||
|
|
||||||
let ping = handle_request_async(
|
let ping = handle_request_async(
|
||||||
&left,
|
&left,
|
||||||
|
|
@ -3063,6 +3257,29 @@ mod tests {
|
||||||
other => panic!("unexpected denied CAS fetch response: {other:?}"),
|
other => panic!("unexpected denied CAS fetch response: {other:?}"),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
let denied_kv_sync = handle_request_async(
|
||||||
|
&left,
|
||||||
|
ControlRequest::KvSync {
|
||||||
|
node: right_card.node_id.to_string(),
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("denied kv sync");
|
||||||
|
match denied_kv_sync {
|
||||||
|
ControlResponse::KvSynced {
|
||||||
|
allowed,
|
||||||
|
entries_imported,
|
||||||
|
reason,
|
||||||
|
..
|
||||||
|
} => {
|
||||||
|
assert!(!allowed);
|
||||||
|
assert_eq!(entries_imported, 0);
|
||||||
|
assert!(reason.contains("no active direct or group grant"));
|
||||||
|
}
|
||||||
|
other => panic!("unexpected denied KV sync response: {other:?}"),
|
||||||
|
}
|
||||||
|
|
||||||
handle_request(
|
handle_request(
|
||||||
&right,
|
&right,
|
||||||
ControlRequest::AuthGrant {
|
ControlRequest::AuthGrant {
|
||||||
|
|
@ -3073,6 +3290,16 @@ mod tests {
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
.expect("grant left peer");
|
.expect("grant left peer");
|
||||||
|
handle_request(
|
||||||
|
&right,
|
||||||
|
ControlRequest::AuthGrant {
|
||||||
|
subject: left.node_id.clone(),
|
||||||
|
resource: "resource:kv:prefs".to_owned(),
|
||||||
|
capability: "kv.read".to_owned(),
|
||||||
|
grant_id: Some("grant:left-kv-read".to_owned()),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("grant left kv read");
|
||||||
|
|
||||||
let allowed = handle_request_async(
|
let allowed = handle_request_async(
|
||||||
&left,
|
&left,
|
||||||
|
|
@ -3131,6 +3358,45 @@ mod tests {
|
||||||
other => panic!("unexpected allowed CAS fetch response: {other:?}"),
|
other => panic!("unexpected allowed CAS fetch response: {other:?}"),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
let kv_synced = handle_request_async(
|
||||||
|
&left,
|
||||||
|
ControlRequest::KvSync {
|
||||||
|
node: right_card.node_id.to_string(),
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("allowed kv sync");
|
||||||
|
match kv_synced {
|
||||||
|
ControlResponse::KvSynced {
|
||||||
|
allowed,
|
||||||
|
entries_imported,
|
||||||
|
reason,
|
||||||
|
note,
|
||||||
|
..
|
||||||
|
} => {
|
||||||
|
assert!(allowed);
|
||||||
|
assert_eq!(entries_imported, 1);
|
||||||
|
assert!(reason.contains("direct grant"));
|
||||||
|
assert!(note.contains("kv.read"));
|
||||||
|
}
|
||||||
|
other => panic!("unexpected allowed KV sync response: {other:?}"),
|
||||||
|
}
|
||||||
|
let synced_theme = handle_request(
|
||||||
|
&left,
|
||||||
|
ControlRequest::KvGet {
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
key: "apps/foo/theme".to_owned(),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("left kv get synced");
|
||||||
|
match synced_theme {
|
||||||
|
ControlResponse::KvGet { entry: Some(entry) } => {
|
||||||
|
assert_eq!(entry.value, "dark");
|
||||||
|
}
|
||||||
|
other => panic!("unexpected synced KV get response: {other:?}"),
|
||||||
|
}
|
||||||
|
|
||||||
let denied_cert_sync = handle_request_async(
|
let denied_cert_sync = handle_request_async(
|
||||||
&left,
|
&left,
|
||||||
ControlRequest::SshCertSync {
|
ControlRequest::SshCertSync {
|
||||||
|
|
@ -3280,6 +3546,16 @@ mod tests {
|
||||||
ControlResponse::SshRevocationAdded { revocation } => revocation.id,
|
ControlResponse::SshRevocationAdded { revocation } => revocation.id,
|
||||||
other => panic!("unexpected second SSH revocation response: {other:?}"),
|
other => panic!("unexpected second SSH revocation response: {other:?}"),
|
||||||
};
|
};
|
||||||
|
handle_request(
|
||||||
|
&right,
|
||||||
|
ControlRequest::KvSet {
|
||||||
|
name: "prefs".to_owned(),
|
||||||
|
key: "apps/foo/live".to_owned(),
|
||||||
|
value: "synced".to_owned(),
|
||||||
|
subject: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("right second kv set");
|
||||||
|
|
||||||
run_live_sync_once(&left)
|
run_live_sync_once(&left)
|
||||||
.await
|
.await
|
||||||
|
|
@ -3299,6 +3575,13 @@ mod tests {
|
||||||
.iter()
|
.iter()
|
||||||
.any(|revocation| revocation.revocation_id == second_revocation_id.as_str())
|
.any(|revocation| revocation.revocation_id == second_revocation_id.as_str())
|
||||||
);
|
);
|
||||||
|
assert_eq!(
|
||||||
|
left_store
|
||||||
|
.get_kv_entry("kv:prefs", "apps/foo/live")
|
||||||
|
.expect("get live-synced kv")
|
||||||
|
.map(|entry| entry.value),
|
||||||
|
Some("synced".to_owned())
|
||||||
|
);
|
||||||
|
|
||||||
left_endpoint.shutdown().await;
|
left_endpoint.shutdown().await;
|
||||||
right_endpoint.shutdown().await;
|
right_endpoint.shutdown().await;
|
||||||
|
|
|
||||||
|
|
@ -335,6 +335,21 @@ impl Store {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn list_kv_stores(&self) -> Result<Vec<StoredKvStore>, StoreError> {
|
||||||
|
let mut stmt = self
|
||||||
|
.conn
|
||||||
|
.prepare("SELECT kv_id, resource_id, name FROM kv_stores ORDER BY name")?;
|
||||||
|
let rows = stmt.query_map([], |row| {
|
||||||
|
Ok(StoredKvStore {
|
||||||
|
kv_id: row.get(0)?,
|
||||||
|
resource_id: row.get(1)?,
|
||||||
|
name: row.get(2)?,
|
||||||
|
})
|
||||||
|
})?;
|
||||||
|
rows.collect::<Result<Vec<_>, _>>()
|
||||||
|
.map_err(StoreError::from)
|
||||||
|
}
|
||||||
|
|
||||||
pub fn set_kv_entry(&self, entry: &StoredKvEntry) -> Result<(), StoreError> {
|
pub fn set_kv_entry(&self, entry: &StoredKvEntry) -> Result<(), StoreError> {
|
||||||
self.conn.execute(
|
self.conn.execute(
|
||||||
r#"INSERT OR REPLACE INTO kv_entries(kv_id, key, value, updated_at_ms)
|
r#"INSERT OR REPLACE INTO kv_entries(kv_id, key, value, updated_at_ms)
|
||||||
|
|
@ -366,6 +381,29 @@ impl Store {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn list_kv_entries_since(
|
||||||
|
&self,
|
||||||
|
kv_id: &str,
|
||||||
|
since_ms: i64,
|
||||||
|
) -> Result<Vec<StoredKvEntry>, StoreError> {
|
||||||
|
let mut stmt = self.conn.prepare(
|
||||||
|
r#"SELECT kv_id, key, value, updated_at_ms
|
||||||
|
FROM kv_entries
|
||||||
|
WHERE kv_id = ?1 AND updated_at_ms >= ?2
|
||||||
|
ORDER BY updated_at_ms, key"#,
|
||||||
|
)?;
|
||||||
|
let rows = stmt.query_map(params![kv_id, since_ms], |row| {
|
||||||
|
Ok(StoredKvEntry {
|
||||||
|
kv_id: row.get(0)?,
|
||||||
|
key: row.get(1)?,
|
||||||
|
value: row.get(2)?,
|
||||||
|
updated_at_ms: row.get(3)?,
|
||||||
|
})
|
||||||
|
})?;
|
||||||
|
rows.collect::<Result<Vec<_>, _>>()
|
||||||
|
.map_err(StoreError::from)
|
||||||
|
}
|
||||||
|
|
||||||
pub fn insert_document_resource(
|
pub fn insert_document_resource(
|
||||||
&self,
|
&self,
|
||||||
document: &StoredDocumentResource,
|
document: &StoredDocumentResource,
|
||||||
|
|
@ -1518,6 +1556,7 @@ mod tests {
|
||||||
store.get_kv_store_by_name("prefs").expect("get kv"),
|
store.get_kv_store_by_name("prefs").expect("get kv"),
|
||||||
Some(kv.clone())
|
Some(kv.clone())
|
||||||
);
|
);
|
||||||
|
assert_eq!(store.list_kv_stores().expect("list kv"), vec![kv.clone()]);
|
||||||
|
|
||||||
let entry = StoredKvEntry {
|
let entry = StoredKvEntry {
|
||||||
kv_id: kv.kv_id.clone(),
|
kv_id: kv.kv_id.clone(),
|
||||||
|
|
@ -1530,7 +1569,13 @@ mod tests {
|
||||||
store
|
store
|
||||||
.get_kv_entry("kv:prefs", "apps/foo/theme")
|
.get_kv_entry("kv:prefs", "apps/foo/theme")
|
||||||
.expect("get entry"),
|
.expect("get entry"),
|
||||||
Some(entry)
|
Some(entry.clone())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
store
|
||||||
|
.list_kv_entries_since("kv:prefs", 1)
|
||||||
|
.expect("list entries since"),
|
||||||
|
vec![entry]
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -141,8 +141,14 @@ are future work.
|
||||||
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
|
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
|
||||||
ops for `kv.write_key:<key>` so prefix grants can be tested before networked
|
ops for `kv.write_key:<key>` so prefix grants can be tested before networked
|
||||||
callers exist. The local node/agent retains owner access for administration.
|
callers exist. The local node/agent retains owner access for administration.
|
||||||
Iroh Documents namespaces, remote caller identity, and replication are future
|
Iroh Documents namespaces remain the target backend, but the bootstrap can sync
|
||||||
work.
|
named KV stores over the protected Iroh control ALPN. `geth kv sync <node-id>
|
||||||
|
<name>` requires `kv.read` on the remote `resource:kv:<name>`, transfers entries
|
||||||
|
at or beyond a per-peer/per-KV high-water cursor, and imports only values that
|
||||||
|
are at least as new as the local entry timestamp. The daemon background
|
||||||
|
live-sync loop runs the same KV sync for local KV stores and known peers.
|
||||||
|
Private value encryption should use resource secret epochs before payloads are
|
||||||
|
exposed to remote peers.
|
||||||
|
|
||||||
`geth-document` currently registers local document resources and stores
|
`geth-document` currently registers local document resources and stores
|
||||||
validated JSON state in the local SQLite metadata store through
|
validated JSON state in the local SQLite metadata store through
|
||||||
|
|
|
||||||
|
|
@ -264,6 +264,11 @@ authorization and durable-state boundaries clear.
|
||||||
- `[x]` Tests cover allowed and denied prefix-scoped KV write explanations.
|
- `[x]` Tests cover allowed and denied prefix-scoped KV write explanations.
|
||||||
- `[x]` `geth kv set --subject <principal>` enforces local capability
|
- `[x]` `geth kv set --subject <principal>` enforces local capability
|
||||||
decisions for non-local test callers.
|
decisions for non-local test callers.
|
||||||
|
- `[x]` `geth kv sync <node-id> <name>` pulls authorized updates from an
|
||||||
|
imported peer over Iroh.
|
||||||
|
- `[x]` Remote KV sync requires `kv.read` on `resource:kv:<name>`.
|
||||||
|
- `[x]` Background live-sync refreshes local KV stores from known peers using
|
||||||
|
per-peer/per-KV high-water cursors.
|
||||||
- `[ ]` KV metadata is replicated through Iroh Documents.
|
- `[ ]` KV metadata is replicated through Iroh Documents.
|
||||||
|
|
||||||
- `[~]` Iroh-gossip pubsub integration.
|
- `[~]` Iroh-gossip pubsub integration.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue