eric/geth
Watch
1
0
Fork
You've already forked geth
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add authorized file-root sync

Browse source
This commit is contained in:
Eric Wendland 2026-05-20 13:22:40 +02:00
commit 1caaedda90
7 changed files with 489 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

9
AGENTS.md
View file

@ -138,9 +138,12 @@ Roadmap items should be actionable and checkable:
- The CAS crate can build deterministic tree objects for local file trees and
store those manifests as CAS blobs. The daemon can register and scan local
file roots, reporting create/update/delete/rename changes without writing back
to the working tree. Durable local file-conflict records can be listed and
resolved manually; automatic cross-node conflict detection and file sync are
still roadmap work.
to the working tree. `geth cas root sync <node-id> <name>` can pull authorized
remote tree metadata and CAS tree bytes, recording a peer-qualified remote
root with path `remote:<node>:<name>` without applying files or overwriting
same-named local roots. Durable local file-conflict records can be listed and
resolved manually; automatic cross-node conflict detection and file application
are still roadmap work.
- DB resources can be registered locally and report local-only status plus a
read-only SQLite schema summary/hash and `crsql_changes` metadata when
present. The DB crate and daemon can extract typed read-only `crsql_changes`

4
README.md
View file

@ -108,7 +108,9 @@ The bootstrap implementation provides:
`unpin`, `cleanup`, `providers`, `list`; remote fetch accepts
`--bearer-secret <secret>`
- local CAS tree objects describe file trees and are stored as CAS blobs
- local file-root commands: `geth cas root add/list/scan`
- local file-root commands: `geth cas root add/list/scan/sync`; root sync pulls
authorized remote tree metadata and CAS tree bytes into a peer-qualified
remote root without writing files
- local file conflict metadata commands:
`geth cas conflict record/list/resolve`
- local DB resource registration: `geth db add <name> <path>` and

55
crates/geth-cli/src/lib.rs
View file

@ -286,9 +286,20 @@ pub enum CasCommand {
#[derive(Debug, Subcommand)]
pub enum CasRootCommand {
Add { name: String, path: PathBuf },
Add {
name: String,
path: PathBuf,
},
List,
Scan { name: String },
Scan {
name: String,
},
Sync {
node: String,
name: String,
#[arg(long)]
bearer_secret: Option<String>,
},
}
#[derive(Debug, Subcommand)]
@ -731,6 +742,15 @@ fn request_for_command(command: Command) -> Result<ControlRequest> {
CasRootCommand::Add { name, path } => ControlRequest::CasRootAdd { name, path },
CasRootCommand::List => ControlRequest::CasRootList,
CasRootCommand::Scan { name } => ControlRequest::CasRootScan { name },
CasRootCommand::Sync {
node,
name,
bearer_secret,
} => ControlRequest::CasRootSync {
node,
name,
bearer_secret,
},
},
CasCommand::Conflict { command } => match command {
CasConflictCommand::Record {
@ -1244,6 +1264,37 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> {
}
println!("note: {}", scan.note);
}
ControlResponse::CasRootSynced {
peer_node_id,
peer_agent_id,
endpoint_id,
name,
root,
tree_bytes_imported,
allowed,
reason,
note,
} => {
if let Some(root) = root {
println!("synced file root: {name}");
println!("peer: {peer_node_id}");
println!("path: {}", root.path);
println!(
"tree: {}",
root.latest_tree
.map(|hash| hash.to_string())
.unwrap_or_else(|| "unscanned".to_owned())
);
println!("tree_bytes_imported: {tree_bytes_imported}");
} else {
println!("file root sync denied by {peer_node_id}");
}
println!("agent: {peer_agent_id}");
println!("endpoint: {endpoint_id}");
println!("allowed: {allowed}");
println!("reason: {reason}");
println!("note: {note}");
}
ControlResponse::CasConflictRecorded { conflict } => {
println!("recorded conflict: {}", conflict.id);
print_file_conflict(&conflict);

82
crates/geth-control/src/lib.rs
View file

@ -81,6 +81,11 @@ pub enum ControlRequest {
CasRootScan {
name: String,
},
CasRootSync {
node: String,
name: String,
bearer_secret: Option<String>,
},
CasConflictRecord {
root: String,
path: String,
@ -387,6 +392,17 @@ pub enum ControlResponse {
CasRootScanned {
scan: FileRootScan,
},
CasRootSynced {
peer_node_id: String,
peer_agent_id: String,
endpoint_id: String,
name: String,
root: Option<FileRoot>,
tree_bytes_imported: bool,
allowed: bool,
reason: String,
note: String,
},
CasConflictRecorded {
conflict: FileConflict,
},
@ -689,6 +705,12 @@ pub enum PeerControlRequest {
nonce: String,
bearer_proof: Option<BearerProof>,
},
CasRootSync {
peer_card: PeerCard,
name: String,
nonce: String,
bearer_proof: Option<BearerProof>,
},
SshCertSync {
peer_card: PeerCard,
since_ms: i64,
@ -803,6 +825,20 @@ pub enum PeerControlResponse {
nonce: String,
note: String,
},
CasRootSynced {
node_id: String,
agent_id: String,
endpoint_id: String,
remote_endpoint_id: String,
name: String,
root: Option<FileRoot>,
tree_content_base64: Option<String>,
allowed: bool,
reason: String,
evaluated_ops: usize,
nonce: String,
note: String,
},
SshCertSynced {
node_id: String,
agent_id: String,
@ -1382,6 +1418,32 @@ mod tests {
request
);
let request = ControlRequest::CasRootSync {
node: "node:peer".to_owned(),
name: "notes".to_owned(),
bearer_secret: None,
};
assert_eq!(
decode_request(&encode_request(&request).expect("encode")).expect("decode"),
request
);
let response = ControlResponse::CasRootSynced {
peer_node_id: "node:peer".to_owned(),
peer_agent_id: "agent:peer".to_owned(),
endpoint_id: "endpoint:peer".to_owned(),
name: "notes".to_owned(),
root: None,
tree_bytes_imported: false,
allowed: false,
reason: "no grant".to_owned(),
note: "file-root sync".to_owned(),
};
assert_eq!(
decode_response(&encode_response(&response).expect("encode")).expect("decode"),
response
);
let request = ControlRequest::CasConflictResolve {
conflict_id: "file-conflict:notes:1".to_owned(),
resolution: "keep-local".to_owned(),
@ -1488,6 +1550,26 @@ mod tests {
response
);
let response = PeerControlResponse::CasRootSynced {
node_id: "node:peer".to_owned(),
agent_id: "agent:peer".to_owned(),
endpoint_id: "endpoint:peer".to_owned(),
remote_endpoint_id: "endpoint:caller".to_owned(),
name: "notes".to_owned(),
root: None,
tree_content_base64: None,
allowed: false,
reason: "no grant".to_owned(),
evaluated_ops: 0,
nonce: "nonce".to_owned(),
note: "file-root sync".to_owned(),
};
assert_eq!(
decode_peer_response(&encode_peer_response(&response).expect("encode"))
.expect("decode"),
response
);
let request = PeerControlRequest::DbSync {
peer_card: PeerCard {
node_id: "node:caller".into(),

328
crates/geth-node/src/lib.rs
View file

@ -257,6 +257,11 @@ pub async fn handle_request_async(
hash,
bearer_secret,
} => cas_fetch_from_peer(node, &peer_node, hash, bearer_secret).await,
ControlRequest::CasRootSync {
node: peer_node,
name,
bearer_secret,
} => cas_root_sync_from_peer(node, &peer_node, &name, bearer_secret).await,
ControlRequest::SshCertSync {
node: peer_node,
bearer_secret,
@ -581,6 +586,7 @@ async fn peer_ping(node: &LocalNode, peer_node: &str) -> Result<ControlResponse,
)),
PeerControlResponse::SyncStatus { .. }
| PeerControlResponse::CasFetched { .. }
| PeerControlResponse::CasRootSynced { .. }
| PeerControlResponse::SshCertSynced { .. }
| PeerControlResponse::SshRevocationSynced { .. }
| PeerControlResponse::KvSynced { .. }
@ -697,6 +703,7 @@ async fn peer_auth_check(
)),
PeerControlResponse::SyncStatus { .. }
| PeerControlResponse::CasFetched { .. }
| PeerControlResponse::CasRootSynced { .. }
| PeerControlResponse::SshCertSynced { .. }
| PeerControlResponse::SshRevocationSynced { .. }
| PeerControlResponse::KvSynced { .. }
@ -845,6 +852,7 @@ async fn cas_fetch_from_peer(
PeerControlResponse::Pong { .. }
| PeerControlResponse::AuthChecked { .. }
| PeerControlResponse::SyncStatus { .. }
| PeerControlResponse::CasRootSynced { .. }
| PeerControlResponse::SshCertSynced { .. }
| PeerControlResponse::SshRevocationSynced { .. }
| PeerControlResponse::KvSynced { .. }
@ -860,6 +868,137 @@ async fn cas_fetch_from_peer(
}
}
async fn cas_root_sync_from_peer(
node: &LocalNode,
peer_node: &str,
name: &str,
bearer_secret: Option<String>,
) -> Result<ControlResponse, NodeError> {
geth_cas::validate_file_root_name(name)?;
let response = request_peer_control(node, peer_node, "cas-root-sync", |peer_card, nonce| {
PeerControlRequest::CasRootSync {
peer_card,
name: name.to_owned(),
nonce: nonce.clone(),
bearer_proof: bearer_proof(
bearer_secret,
&format!("resource:cas-tree:{name}"),
"cas.fetch",
&nonce,
),
}
})
.await?;
match response {
PeerControlResponse::CasRootSynced {
node_id,
agent_id,
endpoint_id,
name: response_name,
root,
tree_content_base64,
allowed,
reason,
note,
..
} if response_name == name => {
if !allowed {
return Ok(ControlResponse::CasRootSynced {
peer_node_id: node_id,
peer_agent_id: agent_id,
endpoint_id,
name: response_name,
root: None,
tree_bytes_imported: false,
allowed,
reason,
note,
});
}
let store = Store::open(&node.paths.metadata_db())?;
let mut tree_bytes_imported = false;
let root = root
.map(|remote_root| {
let stored_name = remote_file_root_name(&node_id, &remote_root.name);
let stored_path = format!("remote:{node_id}:{}", remote_root.name);
if let Some(existing) = store.get_file_root_by_name(&stored_name)? {
if existing.path != stored_path {
return Err(NodeError::IrohPeer(format!(
"refusing to overwrite existing file root {} at {}",
existing.name, existing.path
)));
}
}
if let (Some(hash), Some(content)) = (
remote_root.latest_tree.clone(),
tree_content_base64.as_ref(),
) {
let decoded = base64::engine::general_purpose::STANDARD
.decode(content)
.map_err(|error| NodeError::IrohPeer(error.to_string()))?;
let info = LocalCas::new(node.paths.cas_dir()).add_bytes(&decoded)?;
if info.hash != hash {
return Err(NodeError::IrohPeer(format!(
"peer returned file-root tree bytes hash {} for announced {}",
info.hash, hash
)));
}
store.record_cas_object(
info.hash.as_str(),
info.size_bytes,
&info.path.to_string_lossy(),
)?;
tree_bytes_imported = true;
}
let stored = StoredFileRoot {
root_id: format!("file-root:remote:{node_id}:{}", remote_root.name),
resource_id: remote_root.resource.clone(),
name: stored_name,
path: stored_path,
latest_tree_hash: remote_root.latest_tree.as_ref().map(ToString::to_string),
latest_tree_json: None,
updated_at_ms: remote_root.updated_at_ms,
};
store.upsert_file_root(&stored)?;
Ok(file_root_from_stored(&stored))
})
.transpose()?;
Ok(ControlResponse::CasRootSynced {
peer_node_id: node_id,
peer_agent_id: agent_id,
endpoint_id,
name: response_name,
root,
tree_bytes_imported,
allowed,
reason,
note,
})
}
PeerControlResponse::CasRootSynced { .. } => Err(NodeError::IrohPeer(
"peer file-root sync response did not match request".to_owned(),
)),
PeerControlResponse::Error { message } => Err(NodeError::IrohPeer(message)),
_ => Err(NodeError::IrohPeer(
"peer returned wrong response type to file-root sync".to_owned(),
)),
}
}
fn remote_file_root_name(peer_node_id: &str, remote_name: &str) -> String {
let peer = peer_node_id
.chars()
.map(|ch| {
if ch.is_ascii_alphanumeric() || matches!(ch, '.' | '_' | '-') {
ch
} else {
'-'
}
})
.collect::<String>();
format!("remote-{peer}-{remote_name}")
}
async fn ssh_cert_sync_from_peer(
node: &LocalNode,
peer_node: &str,
@ -1932,6 +2071,10 @@ async fn request_peer_control(
nonce: response_nonce,
..
}
| PeerControlResponse::CasRootSynced {
nonce: response_nonce,
..
}
| PeerControlResponse::SshProxyConnected {
nonce: response_nonce,
..
@ -2253,6 +2396,73 @@ async fn handle_iroh_control_connection(
}
}
}
PeerControlRequest::CasRootSync {
peer_card,
name,
nonce,
bearer_proof,
} => {
geth_cas::validate_file_root_name(&name)?;
peer_card.validate_candidate()?;
ensure_peer_card_matches_endpoint(&peer_card, &remote_endpoint_id)?;
let discovered = DiscoveredPeer::candidate(
peer_card.clone(),
UnixMillis(geth_store::now_ms()),
DiscoverySource::PeerExchange,
)?;
let store = Store::open(&node.paths.metadata_db())?;
store.upsert_peer_card(&StoredPeerCard {
peer_id: peer_card.node_id.to_string(),
card_json: serde_json::to_string(&peer_card)?,
updated_at_ms: discovered.discovered_at.0,
})?;
let resource = format!("resource:cas-tree:{name}");
let capability = "cas.fetch".to_owned();
let explanation = explain_peer_or_bearer(
&store,
peer_card.node_id.as_str(),
&resource,
&capability,
&nonce,
bearer_proof.as_ref(),
)?;
let (root, tree_content_base64) = if explanation.allowed {
if let Some(root) = store.get_file_root_by_name(&name)? {
let root_view = file_root_from_stored(&root);
let tree_content_base64 = root
.latest_tree_hash
.as_ref()
.map(|hash| -> Result<Option<String>, NodeError> {
let hash = BlobHash::new(hash.clone());
let content = LocalCas::new(node.paths.cas_dir()).read_bytes(&hash)?;
Ok(Some(
base64::engine::general_purpose::STANDARD.encode(content),
))
})
.transpose()?
.flatten();
(Some(root_view), tree_content_base64)
} else {
(None, None)
}
} else {
(None, None)
};
PeerControlResponse::CasRootSynced {
node_id: node.node_id.clone(),
agent_id: node.agent_id.clone(),
endpoint_id: node.iroh_status.endpoint_id.clone().unwrap_or_default(),
remote_endpoint_id,
name,
root,
tree_content_base64,
allowed: explanation.allowed,
reason: explanation.reason,
evaluated_ops: explanation.evaluated_ops,
nonce,
note: "file-root sync authenticated endpoint/card binding and required cas.fetch on the remote cas-tree resource; it imports tree metadata and CAS tree bytes but never writes files".to_owned(),
}
}
PeerControlRequest::SshCertSync {
peer_card,
since_ms,
@ -2952,6 +3162,7 @@ pub fn handle_request(
ControlRequest::PeerPing { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::PeerAuthCheck { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::CasFetch { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::CasRootSync { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::SshCertSync { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::SshRevocationSync { .. } => Err(NodeError::IrohEndpointUnavailable),
ControlRequest::KvSync { .. } => Err(NodeError::IrohEndpointUnavailable),
@ -5246,6 +5457,39 @@ mod tests {
let right_blob = LocalCas::new(right.paths.cas_dir())
.add_bytes(b"remote cas bytes")
.expect("right cas add");
let left_file_root = left_home.path().join("files");
std::fs::create_dir_all(&left_file_root).expect("create left file root");
std::fs::write(left_file_root.join("note.txt"), b"local tree").expect("write left file");
handle_request(
&left,
ControlRequest::CasRootAdd {
name: "shared".to_owned(),
path: left_file_root.clone(),
},
)
.expect("left file root add");
let right_file_root = right_home.path().join("files");
std::fs::create_dir_all(&right_file_root).expect("create right file root");
std::fs::write(right_file_root.join("note.txt"), b"synced tree").expect("write right file");
handle_request(
&right,
ControlRequest::CasRootAdd {
name: "shared".to_owned(),
path: right_file_root,
},
)
.expect("right file root add");
let right_tree_hash = match handle_request(
&right,
ControlRequest::CasRootScan {
name: "shared".to_owned(),
},
)
.expect("right file root scan")
{
ControlResponse::CasRootScanned { scan } => scan.tree.hash,
other => panic!("unexpected right file root scan response: {other:?}"),
};
let right_pubkey = right_home.path().join("request.pub");
std::fs::write(
&right_pubkey,
@ -5423,6 +5667,30 @@ mod tests {
other => panic!("unexpected denied CAS fetch response: {other:?}"),
}
let denied_root_sync = handle_request_async(
&left,
ControlRequest::CasRootSync {
node: right_card.node_id.to_string(),
name: "shared".to_owned(),
bearer_secret: None,
},
)
.await
.expect("denied file root sync");
match denied_root_sync {
ControlResponse::CasRootSynced {
allowed,
root,
reason,
..
} => {
assert!(!allowed);
assert!(root.is_none());
assert!(reason.contains("no active direct or group grant"));
}
other => panic!("unexpected denied file root sync response: {other:?}"),
}
let denied_kv_sync = handle_request_async(
&left,
ControlRequest::KvSync {
@ -5672,6 +5940,16 @@ mod tests {
},
)
.expect("grant left peer");
handle_request(
&right,
ControlRequest::AuthGrant {
subject: left.node_id.clone(),
resource: "resource:cas-tree:shared".to_owned(),
capability: "cas.fetch".to_owned(),
grant_id: Some("grant:left-cas-tree-fetch".to_owned()),
},
)
.expect("grant left file root fetch");
handle_request(
&right,
ControlRequest::AuthGrant {
@ -5817,6 +6095,56 @@ mod tests {
assert_eq!(providers.len(), 1);
assert_eq!(providers[0].peer_node_id, right.node_id);
let root_sync = handle_request_async(
&left,
ControlRequest::CasRootSync {
node: right_card.node_id.to_string(),
name: "shared".to_owned(),
bearer_secret: None,
},
)
.await
.expect("allowed file root sync");
match root_sync {
ControlResponse::CasRootSynced {
allowed,
root,
tree_bytes_imported,
reason,
note,
..
} => {
assert!(allowed);
assert!(tree_bytes_imported);
let root = root.expect("synced root");
assert_ne!(root.name, "shared");
assert!(root.name.starts_with("remote-"));
assert!(root.name.ends_with("-shared"));
assert_eq!(root.latest_tree, Some(right_tree_hash.clone()));
assert!(root.path.starts_with("remote:"));
assert!(reason.contains("direct grant"));
assert!(note.contains("never writes files"));
}
other => panic!("unexpected allowed file root sync response: {other:?}"),
}
assert!(
LocalCas::new(left.paths.cas_dir())
.has(&right_tree_hash)
.expect("left has synced tree object")
);
let local_shared = match handle_request(
&left,
ControlRequest::CasRootScan {
name: "shared".to_owned(),
},
)
.expect("left local shared root remains scannable")
{
ControlResponse::CasRootScanned { scan } => scan.root,
other => panic!("unexpected left local file root scan response: {other:?}"),
};
assert_eq!(local_shared.path, left_file_root.to_string_lossy());
let kv_synced = handle_request_async(
&left,
ControlRequest::KvSync {

12
docs/architecture.md
View file

@ -127,10 +127,14 @@ blobs. The CAS crate can build deterministic tree objects that describe
directories, files, executable bits, and file blob hashes; those tree objects
are stored as CAS blobs. The daemon can register local file roots and scan them
into CAS tree objects while reporting create/update/delete/rename changes. These
scans are local metadata only and never overwrite the working tree. The daemon
also has durable local file-conflict records with explicit resolution choices;
future cross-node file sync will create those records automatically instead of
silently applying ambiguous remote changes.
scans are local metadata only and never overwrite the working tree. A peer can
pull authorized remote file-root tree metadata with `geth cas root sync <node>
<name>` when it has `cas.fetch` on `resource:cas-tree:<name>`; sync imports the
remote CAS tree bytes and records a peer-qualified remote root whose path is
`remote:<node>:<name>` without applying files. The daemon also has durable local
file-conflict records with
explicit resolution choices; future cross-node file sync will create those
records automatically instead of silently applying ambiguous remote changes.
As a bootstrap network path, `geth cas fetch <node-id> <hash>` dials an
imported signed peer card over the daemon-owned Iroh control ALPN. The serving

10
docs/roadmap.md
View file

@ -510,7 +510,15 @@ and future group key evolution.
- `[x]` Scan detects create/update/delete/rename changes.
- `[x]` Scan output documents that geth never overwrites file roots without a
recorded future sync decision.
- `[ ]` File roots can be synced across nodes.
- `[x]` `geth cas root sync <node-id> <name>` pulls authorized remote
file-root tree metadata over the protected Iroh control path.
- `[x]` Remote file-root sync requires `cas.fetch` on
`resource:cas-tree:<name>`.
- `[x]` Sync imports CAS tree bytes and records a peer-qualified remote root
with path `remote:<node>:<name>` without writing files into the working tree
or overwriting same-named local roots.
- `[ ]` Future completion applies file-root sync safely with conflict
detection and explicit resolution.
- `[~]` Conflict handling.
Acceptance criteria: