diff --git a/AGENTS.md b/AGENTS.md index 03186df..6afd6bf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -142,9 +142,10 @@ Roadmap items should be actionable and checkable: remote tree metadata and CAS tree bytes, recording a peer-qualified remote root with path `remote::` without applying files or overwriting same-named local roots. Background live-sync refreshes authorized remote file - roots from sync-status `cas-tree:` watermarks. Durable local - file-conflict records can be listed and resolved manually; automatic - cross-node conflict detection and file application are still roadmap work. + roots from sync-status `cas-tree:` watermarks. `geth cas root apply` + materializes missing files/directories from a CAS tree without deleting extras + or overwriting local edits, and records conflicts for manual resolution. + Richer three-way file application remains roadmap work. - DB resources can be registered locally and report local-only status plus a read-only SQLite schema summary/hash and `crsql_changes` metadata when present. The DB crate and daemon can extract typed read-only `crsql_changes` diff --git a/README.md b/README.md index 0bfd448..e4ac872 100644 --- a/README.md +++ b/README.md @@ -108,9 +108,10 @@ The bootstrap implementation provides: `unpin`, `cleanup`, `providers`, `list`; remote fetch accepts `--bearer-secret ` - local CAS tree objects describe file trees and are stored as CAS blobs -- local file-root commands: `geth cas root add/list/scan/sync`; root sync pulls - authorized remote tree metadata and CAS tree bytes into a peer-qualified - remote root without writing files +- local file-root commands: `geth cas root add/list/scan/sync/apply`; root sync + pulls authorized remote tree metadata and CAS tree bytes into a peer-qualified + remote root, and apply materializes a tree without deleting files or + overwriting local edits - local file conflict metadata commands: `geth cas conflict record/list/resolve` - local DB resource registration: `geth db add ` and @@ -180,6 +181,9 @@ local daemon skip unchanged or unauthorized streams. File roots advertise `cas-tree:` watermarks when the caller has `cas.fetch`; background live-sync imports updated tree metadata and CAS tree bytes into peer-qualified remote roots without writing files. +`geth cas root apply --to ` can then materialize that tree locally: +it creates missing directories/files, never deletes extra files, never +overwrites differing local files, and records conflicts for manual resolution. Named KV stores participate in the same live-sync loop once they exist locally: manual `geth kv sync ` and background ticks require `kv.read` on the remote `resource:kv:` and import only remote entries that are not diff --git a/crates/geth-cli/src/lib.rs b/crates/geth-cli/src/lib.rs index 195a151..47cad12 100644 --- a/crates/geth-cli/src/lib.rs +++ b/crates/geth-cli/src/lib.rs @@ -300,6 +300,13 @@ pub enum CasRootCommand { #[arg(long)] bearer_secret: Option, }, + Apply { + source: String, + #[arg(long)] + to: PathBuf, + #[arg(long)] + dry_run: bool, + }, } #[derive(Debug, Subcommand)] @@ -751,6 +758,15 @@ fn request_for_command(command: Command) -> Result { name, bearer_secret, }, + CasRootCommand::Apply { + source, + to, + dry_run, + } => ControlRequest::CasRootApply { + source, + target: to, + dry_run, + }, }, CasCommand::Conflict { command } => match command { CasConflictCommand::Record { @@ -1295,6 +1311,32 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> { println!("reason: {reason}"); println!("note: {note}"); } + ControlResponse::CasRootApplied { + source, + target, + files_written, + dirs_created, + conflicts, + dry_run, + note, + } => { + println!("applied file root: {source}"); + println!("target: {}", target.display()); + println!("dry_run: {dry_run}"); + println!("files_written: {files_written}"); + println!("dirs_created: {dirs_created}"); + println!("conflicts: {}", conflicts.len()); + for conflict in conflicts { + println!( + "{}\t{}\t{}\t{}", + conflict.id, + conflict.path, + conflict.kind.as_str(), + conflict.status.as_str() + ); + } + println!("note: {note}"); + } ControlResponse::CasConflictRecorded { conflict } => { println!("recorded conflict: {}", conflict.id); print_file_conflict(&conflict); diff --git a/crates/geth-control/src/lib.rs b/crates/geth-control/src/lib.rs index 1120e50..a765712 100644 --- a/crates/geth-control/src/lib.rs +++ b/crates/geth-control/src/lib.rs @@ -86,6 +86,11 @@ pub enum ControlRequest { name: String, bearer_secret: Option, }, + CasRootApply { + source: String, + target: PathBuf, + dry_run: bool, + }, CasConflictRecord { root: String, path: String, @@ -403,6 +408,15 @@ pub enum ControlResponse { reason: String, note: String, }, + CasRootApplied { + source: String, + target: PathBuf, + files_written: usize, + dirs_created: usize, + conflicts: Vec, + dry_run: bool, + note: String, + }, CasConflictRecorded { conflict: FileConflict, }, @@ -1444,6 +1458,30 @@ mod tests { response ); + let request = ControlRequest::CasRootApply { + source: "remote-node-peer-notes".to_owned(), + target: PathBuf::from("/tmp/notes"), + dry_run: true, + }; + assert_eq!( + decode_request(&encode_request(&request).expect("encode")).expect("decode"), + request + ); + + let response = ControlResponse::CasRootApplied { + source: "remote-node-peer-notes".to_owned(), + target: PathBuf::from("/tmp/notes"), + files_written: 1, + dirs_created: 1, + conflicts: Vec::new(), + dry_run: true, + note: "safe apply".to_owned(), + }; + assert_eq!( + decode_response(&encode_response(&response).expect("encode")).expect("decode"), + response + ); + let request = ControlRequest::CasConflictResolve { conflict_id: "file-conflict:notes:1".to_owned(), resolution: "keep-local".to_owned(), diff --git a/crates/geth-node/src/lib.rs b/crates/geth-node/src/lib.rs index 0e9ae6d..fbc7a17 100644 --- a/crates/geth-node/src/lib.rs +++ b/crates/geth-node/src/lib.rs @@ -3,8 +3,8 @@ pub mod service; use base64::Engine; use geth_auth::{AuthExplanation, AuthOp, AuthOpKind}; use geth_cas::{ - BlobInfoSummary, FileConflict, FileConflictKind, FileConflictResolution, FileConflictStatus, - FileRoot, FileRootScan, LocalCas, hash_path, + BlobInfoSummary, CasTreeEntryKind, CasTreeObject, FileConflict, FileConflictKind, + FileConflictResolution, FileConflictStatus, FileRoot, FileRootScan, LocalCas, hash_path, }; use geth_config::{GethConfig, GethPaths, RelayMode}; use geth_control::{ @@ -43,7 +43,7 @@ use geth_types::{ ResourceName, SshCertId, SshCertRequestId, UnixMillis, }; use std::collections::{BTreeMap, VecDeque}; -use std::path::Path; +use std::path::{Component, Path, PathBuf}; use std::sync::{Arc, Mutex}; use std::time::Duration; use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; @@ -3397,6 +3397,11 @@ pub fn handle_request( }, }) } + ControlRequest::CasRootApply { + source, + target, + dry_run, + } => apply_file_root_tree(&store, node, &source, target, dry_run), ControlRequest::CasConflictRecord { root, path, @@ -4628,6 +4633,203 @@ fn file_conflict_from_stored(stored: StoredFileConflict) -> Result Result { + geth_cas::validate_file_root_name(source)?; + let source_root = store + .get_file_root_by_name(source)? + .ok_or_else(|| NodeError::ResourceNotFound(format!("file-root:{source}")))?; + let tree_hash = source_root + .latest_tree_hash + .as_ref() + .ok_or_else(|| NodeError::ResourceNotFound(format!("file-root-tree:{source}")))?; + let tree_hash = BlobHash::new(tree_hash.clone()); + let cas = LocalCas::new(node.paths.cas_dir()); + let tree_bytes = cas.read_bytes(&tree_hash)?; + let tree: CasTreeObject = geth_codec::decode_canonical(&tree_bytes)?; + + if target.exists() && !target.is_dir() { + return Err(NodeError::Cas(geth_cas::CasError::TreeRootNotDirectory( + target.display().to_string(), + ))); + } + if !dry_run { + std::fs::create_dir_all(&target)?; + } + + let mut files_written = 0; + let mut dirs_created = 0; + let mut conflicts = Vec::new(); + for entry in &tree.entries { + let out = safe_tree_output_path(&target, &entry.path)?; + match entry.kind { + CasTreeEntryKind::Directory => { + if out.exists() { + if !out.is_dir() { + conflicts.push(record_apply_conflict( + store, + &source_root, + &entry.path, + &tree_hash, + "remote directory conflicts with a local non-directory path", + dry_run, + )?); + } + continue; + } + dirs_created += 1; + if !dry_run { + std::fs::create_dir_all(&out)?; + } + } + CasTreeEntryKind::File => { + let Some(blob) = &entry.blob else { + conflicts.push(record_apply_conflict( + store, + &source_root, + &entry.path, + &tree_hash, + "remote file entry is missing a CAS blob hash", + dry_run, + )?); + continue; + }; + if out.exists() { + if !out.is_file() { + conflicts.push(record_apply_conflict( + store, + &source_root, + &entry.path, + &tree_hash, + "remote file conflicts with a local non-file path", + dry_run, + )?); + continue; + } + if hash_path(&out)? != *blob { + conflicts.push(record_apply_conflict( + store, + &source_root, + &entry.path, + &tree_hash, + "local file differs from remote CAS tree; refusing to overwrite", + dry_run, + )?); + } + continue; + } + files_written += 1; + if !dry_run { + let bytes = cas.read_bytes(blob)?; + write_file_atomic(&out, &bytes)?; + set_executable_bit(&out, entry.executable)?; + } + } + } + } + + Ok(ControlResponse::CasRootApplied { + source: source.to_owned(), + target, + files_written, + dirs_created, + conflicts, + dry_run, + note: "safe file-root apply creates missing paths but never deletes extra files or overwrites differing local files; conflicts are recorded for manual resolution".to_owned(), + }) +} + +fn safe_tree_output_path(root: &Path, relative: &str) -> Result { + let relative_path = Path::new(relative); + if relative_path.is_absolute() { + return Err(NodeError::Cas(geth_cas::CasError::NonRelativeTreePath( + relative.to_owned(), + ))); + } + for component in relative_path.components() { + if !matches!(component, Component::Normal(_)) { + return Err(NodeError::Cas(geth_cas::CasError::NonRelativeTreePath( + relative.to_owned(), + ))); + } + } + Ok(root.join(relative_path)) +} + +fn record_apply_conflict( + store: &Store, + root: &StoredFileRoot, + path: &str, + remote_tree: &BlobHash, + detail: &str, + dry_run: bool, +) -> Result { + let created_at = geth_store::now_ms(); + let stored = StoredFileConflict { + conflict_id: generated_file_conflict_id( + &root.name, + path, + FileConflictKind::ConcurrentEdit.as_str(), + created_at, + ), + root_name: root.name.clone(), + resource_id: root.resource_id.clone(), + path: path.to_owned(), + kind: FileConflictKind::ConcurrentEdit.as_str().to_owned(), + status: FileConflictStatus::Open.as_str().to_owned(), + base_tree_hash: None, + local_tree_hash: None, + remote_tree_hash: Some(remote_tree.to_string()), + detail: detail.to_owned(), + resolution: None, + resolution_note: None, + created_at_ms: created_at, + resolved_at_ms: None, + }; + if !dry_run { + store.upsert_file_conflict(&stored)?; + } + file_conflict_from_stored(stored) +} + +fn write_file_atomic(path: &Path, bytes: &[u8]) -> Result<(), NodeError> { + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + let tmp = path.with_extension(format!( + "geth-tmp-{}", + geth_crypto::blake3_hex(path.display().to_string().as_bytes()) + )); + std::fs::write(&tmp, bytes)?; + std::fs::rename(tmp, path)?; + Ok(()) +} + +#[cfg(unix)] +fn set_executable_bit(path: &Path, executable: bool) -> Result<(), NodeError> { + use std::os::unix::fs::PermissionsExt; + let mut permissions = std::fs::metadata(path)?.permissions(); + let mut mode = permissions.mode(); + if executable { + mode |= 0o111; + } else { + mode &= !0o111; + } + permissions.set_mode(mode); + std::fs::set_permissions(path, permissions)?; + Ok(()) +} + +#[cfg(not(unix))] +fn set_executable_bit(_path: &Path, _executable: bool) -> Result<(), NodeError> { + Ok(()) +} + fn ensure_resource_exists(store: &Store, resource_id: &str) -> Result<(), NodeError> { if store .list_resources()? diff --git a/crates/geth/tests/bootstrap.rs b/crates/geth/tests/bootstrap.rs index 4a5b6e1..2be221a 100644 --- a/crates/geth/tests/bootstrap.rs +++ b/crates/geth/tests/bootstrap.rs @@ -394,6 +394,91 @@ fn cas_file_root_add_list_scan_detects_changes() { } } +#[test] +fn cas_file_root_apply_writes_missing_files_and_records_conflicts() { + let home = tempfile::tempdir().expect("tempdir"); + let paths = geth_config::GethPaths::from_home(home.path()); + let node = geth_node::init_node(&paths).expect("init node"); + let source_path = home.path().join("source-root"); + std::fs::create_dir_all(source_path.join("dir")).expect("create source root"); + std::fs::write(source_path.join("a.txt"), b"remote").expect("write remote a"); + std::fs::write(source_path.join("dir").join("b.txt"), b"remote b").expect("write remote b"); + + geth_node::handle_request( + &node, + geth_control::ControlRequest::CasRootAdd { + name: "remote-node-peer-shared".to_owned(), + path: source_path, + }, + ) + .expect("add source root"); + let tree = match geth_node::handle_request( + &node, + geth_control::ControlRequest::CasRootScan { + name: "remote-node-peer-shared".to_owned(), + }, + ) + .expect("scan source root") + { + geth_control::ControlResponse::CasRootScanned { scan } => scan.tree.hash, + other => panic!("unexpected response: {other:?}"), + }; + + let target_path = home.path().join("target-root"); + std::fs::create_dir_all(&target_path).expect("create target root"); + std::fs::write(target_path.join("a.txt"), b"local edit").expect("write local a"); + + let response = geth_node::handle_request( + &node, + geth_control::ControlRequest::CasRootApply { + source: "remote-node-peer-shared".to_owned(), + target: target_path.clone(), + dry_run: false, + }, + ) + .expect("apply source root"); + match response { + geth_control::ControlResponse::CasRootApplied { + files_written, + dirs_created, + conflicts, + note, + .. + } => { + assert_eq!(files_written, 1); + assert_eq!(dirs_created, 1); + assert_eq!(conflicts.len(), 1); + assert_eq!(conflicts[0].path, "a.txt"); + assert_eq!(conflicts[0].remote_tree, Some(tree)); + assert!(note.contains("never deletes")); + } + other => panic!("unexpected response: {other:?}"), + } + assert_eq!( + std::fs::read(target_path.join("dir").join("b.txt")).expect("read applied b"), + b"remote b" + ); + assert_eq!( + std::fs::read(target_path.join("a.txt")).expect("read local a"), + b"local edit" + ); + + let response = geth_node::handle_request( + &node, + geth_control::ControlRequest::CasConflictList { + root: Some("remote-node-peer-shared".to_owned()), + }, + ) + .expect("list apply conflicts"); + match response { + geth_control::ControlResponse::CasConflictList { conflicts } => { + assert_eq!(conflicts.len(), 1); + assert_eq!(conflicts[0].path, "a.txt"); + } + other => panic!("unexpected response: {other:?}"), + } +} + #[test] fn cas_file_conflict_record_list_and_resolve() { let home = tempfile::tempdir().expect("tempdir"); diff --git a/docs/architecture.md b/docs/architecture.md index 3fcb32e..73a3435 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -133,9 +133,13 @@ pull authorized remote file-root tree metadata with `geth cas root sync remote CAS tree bytes and records a peer-qualified remote root whose path is `remote::` without applying files. File roots also participate in the daemon background live-sync loop through authorized `cas-tree:` -watermarks. The daemon also has durable local file-conflict records with -explicit resolution choices; future cross-node file sync will create those -records automatically instead of silently applying ambiguous remote changes. +watermarks. `geth cas root apply --to ` is the first conservative +materialization path: it creates missing files and directories from the CAS tree, +does not delete extra local files, does not overwrite differing local files, and +records conflicts for manual resolution. The daemon also has durable local +file-conflict records with explicit resolution choices; future cross-node file +sync will extend those records instead of silently applying ambiguous remote +changes. As a bootstrap network path, `geth cas fetch ` dials an imported signed peer card over the daemon-owned Iroh control ALPN. The serving diff --git a/docs/roadmap.md b/docs/roadmap.md index eabd515..5b9fe4e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -519,8 +519,12 @@ and future group key evolution. or overwriting same-named local roots. - `[x]` Background live-sync imports updated authorized file-root trees from sync-status `cas-tree:` watermarks and stores per-peer cursors. - - `[ ]` Future completion applies file-root sync safely with conflict - detection and explicit resolution. + - `[x]` `geth cas root apply --to ` materializes missing files + and directories from a CAS tree without deleting extras or overwriting local + edits. + - `[x]` Apply records durable conflicts for differing local paths. + - `[ ]` Future completion adds a richer three-way apply using base/local/remote + trees for automatic safe updates and deletes. - `[~]` Conflict handling. Acceptance criteria: