Add safe file-root apply

This commit is contained in:
Eric Wendland 2026-05-20 13:30:57 +02:00
commit 5c91635ea2
8 changed files with 394 additions and 14 deletions

View file

@ -142,9 +142,10 @@ Roadmap items should be actionable and checkable:
remote tree metadata and CAS tree bytes, recording a peer-qualified remote remote tree metadata and CAS tree bytes, recording a peer-qualified remote
root with path `remote:<node>:<name>` without applying files or overwriting root with path `remote:<node>:<name>` without applying files or overwriting
same-named local roots. Background live-sync refreshes authorized remote file same-named local roots. Background live-sync refreshes authorized remote file
roots from sync-status `cas-tree:<name>` watermarks. Durable local roots from sync-status `cas-tree:<name>` watermarks. `geth cas root apply`
file-conflict records can be listed and resolved manually; automatic materializes missing files/directories from a CAS tree without deleting extras
cross-node conflict detection and file application are still roadmap work. or overwriting local edits, and records conflicts for manual resolution.
Richer three-way file application remains roadmap work.
- DB resources can be registered locally and report local-only status plus a - DB resources can be registered locally and report local-only status plus a
read-only SQLite schema summary/hash and `crsql_changes` metadata when read-only SQLite schema summary/hash and `crsql_changes` metadata when
present. The DB crate and daemon can extract typed read-only `crsql_changes` present. The DB crate and daemon can extract typed read-only `crsql_changes`

View file

@ -108,9 +108,10 @@ The bootstrap implementation provides:
`unpin`, `cleanup`, `providers`, `list`; remote fetch accepts `unpin`, `cleanup`, `providers`, `list`; remote fetch accepts
`--bearer-secret <secret>` `--bearer-secret <secret>`
- local CAS tree objects describe file trees and are stored as CAS blobs - local CAS tree objects describe file trees and are stored as CAS blobs
- local file-root commands: `geth cas root add/list/scan/sync`; root sync pulls - local file-root commands: `geth cas root add/list/scan/sync/apply`; root sync
authorized remote tree metadata and CAS tree bytes into a peer-qualified pulls authorized remote tree metadata and CAS tree bytes into a peer-qualified
remote root without writing files remote root, and apply materializes a tree without deleting files or
overwriting local edits
- local file conflict metadata commands: - local file conflict metadata commands:
`geth cas conflict record/list/resolve` `geth cas conflict record/list/resolve`
- local DB resource registration: `geth db add <name> <path>` and - local DB resource registration: `geth db add <name> <path>` and
@ -180,6 +181,9 @@ local daemon skip unchanged or unauthorized streams.
File roots advertise `cas-tree:<name>` watermarks when the caller has File roots advertise `cas-tree:<name>` watermarks when the caller has
`cas.fetch`; background live-sync imports updated tree metadata and CAS tree `cas.fetch`; background live-sync imports updated tree metadata and CAS tree
bytes into peer-qualified remote roots without writing files. bytes into peer-qualified remote roots without writing files.
`geth cas root apply <root> --to <path>` can then materialize that tree locally:
it creates missing directories/files, never deletes extra files, never
overwrites differing local files, and records conflicts for manual resolution.
Named KV stores participate in the same live-sync loop once they exist locally: Named KV stores participate in the same live-sync loop once they exist locally:
manual `geth kv sync <node-id> <name>` and background ticks require `kv.read` manual `geth kv sync <node-id> <name>` and background ticks require `kv.read`
on the remote `resource:kv:<name>` and import only remote entries that are not on the remote `resource:kv:<name>` and import only remote entries that are not

View file

@ -300,6 +300,13 @@ pub enum CasRootCommand {
#[arg(long)] #[arg(long)]
bearer_secret: Option<String>, bearer_secret: Option<String>,
}, },
Apply {
source: String,
#[arg(long)]
to: PathBuf,
#[arg(long)]
dry_run: bool,
},
} }
#[derive(Debug, Subcommand)] #[derive(Debug, Subcommand)]
@ -751,6 +758,15 @@ fn request_for_command(command: Command) -> Result<ControlRequest> {
name, name,
bearer_secret, bearer_secret,
}, },
CasRootCommand::Apply {
source,
to,
dry_run,
} => ControlRequest::CasRootApply {
source,
target: to,
dry_run,
},
}, },
CasCommand::Conflict { command } => match command { CasCommand::Conflict { command } => match command {
CasConflictCommand::Record { CasConflictCommand::Record {
@ -1295,6 +1311,32 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> {
println!("reason: {reason}"); println!("reason: {reason}");
println!("note: {note}"); println!("note: {note}");
} }
ControlResponse::CasRootApplied {
source,
target,
files_written,
dirs_created,
conflicts,
dry_run,
note,
} => {
println!("applied file root: {source}");
println!("target: {}", target.display());
println!("dry_run: {dry_run}");
println!("files_written: {files_written}");
println!("dirs_created: {dirs_created}");
println!("conflicts: {}", conflicts.len());
for conflict in conflicts {
println!(
"{}\t{}\t{}\t{}",
conflict.id,
conflict.path,
conflict.kind.as_str(),
conflict.status.as_str()
);
}
println!("note: {note}");
}
ControlResponse::CasConflictRecorded { conflict } => { ControlResponse::CasConflictRecorded { conflict } => {
println!("recorded conflict: {}", conflict.id); println!("recorded conflict: {}", conflict.id);
print_file_conflict(&conflict); print_file_conflict(&conflict);

View file

@ -86,6 +86,11 @@ pub enum ControlRequest {
name: String, name: String,
bearer_secret: Option<String>, bearer_secret: Option<String>,
}, },
CasRootApply {
source: String,
target: PathBuf,
dry_run: bool,
},
CasConflictRecord { CasConflictRecord {
root: String, root: String,
path: String, path: String,
@ -403,6 +408,15 @@ pub enum ControlResponse {
reason: String, reason: String,
note: String, note: String,
}, },
CasRootApplied {
source: String,
target: PathBuf,
files_written: usize,
dirs_created: usize,
conflicts: Vec<FileConflict>,
dry_run: bool,
note: String,
},
CasConflictRecorded { CasConflictRecorded {
conflict: FileConflict, conflict: FileConflict,
}, },
@ -1444,6 +1458,30 @@ mod tests {
response response
); );
let request = ControlRequest::CasRootApply {
source: "remote-node-peer-notes".to_owned(),
target: PathBuf::from("/tmp/notes"),
dry_run: true,
};
assert_eq!(
decode_request(&encode_request(&request).expect("encode")).expect("decode"),
request
);
let response = ControlResponse::CasRootApplied {
source: "remote-node-peer-notes".to_owned(),
target: PathBuf::from("/tmp/notes"),
files_written: 1,
dirs_created: 1,
conflicts: Vec::new(),
dry_run: true,
note: "safe apply".to_owned(),
};
assert_eq!(
decode_response(&encode_response(&response).expect("encode")).expect("decode"),
response
);
let request = ControlRequest::CasConflictResolve { let request = ControlRequest::CasConflictResolve {
conflict_id: "file-conflict:notes:1".to_owned(), conflict_id: "file-conflict:notes:1".to_owned(),
resolution: "keep-local".to_owned(), resolution: "keep-local".to_owned(),

View file

@ -3,8 +3,8 @@ pub mod service;
use base64::Engine; use base64::Engine;
use geth_auth::{AuthExplanation, AuthOp, AuthOpKind}; use geth_auth::{AuthExplanation, AuthOp, AuthOpKind};
use geth_cas::{ use geth_cas::{
BlobInfoSummary, FileConflict, FileConflictKind, FileConflictResolution, FileConflictStatus, BlobInfoSummary, CasTreeEntryKind, CasTreeObject, FileConflict, FileConflictKind,
FileRoot, FileRootScan, LocalCas, hash_path, FileConflictResolution, FileConflictStatus, FileRoot, FileRootScan, LocalCas, hash_path,
}; };
use geth_config::{GethConfig, GethPaths, RelayMode}; use geth_config::{GethConfig, GethPaths, RelayMode};
use geth_control::{ use geth_control::{
@ -43,7 +43,7 @@ use geth_types::{
ResourceName, SshCertId, SshCertRequestId, UnixMillis, ResourceName, SshCertId, SshCertRequestId, UnixMillis,
}; };
use std::collections::{BTreeMap, VecDeque}; use std::collections::{BTreeMap, VecDeque};
use std::path::Path; use std::path::{Component, Path, PathBuf};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::Duration; use std::time::Duration;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
@ -3397,6 +3397,11 @@ pub fn handle_request(
}, },
}) })
} }
ControlRequest::CasRootApply {
source,
target,
dry_run,
} => apply_file_root_tree(&store, node, &source, target, dry_run),
ControlRequest::CasConflictRecord { ControlRequest::CasConflictRecord {
root, root,
path, path,
@ -4628,6 +4633,203 @@ fn file_conflict_from_stored(stored: StoredFileConflict) -> Result<FileConflict,
}) })
} }
fn apply_file_root_tree(
store: &Store,
node: &LocalNode,
source: &str,
target: PathBuf,
dry_run: bool,
) -> Result<ControlResponse, NodeError> {
geth_cas::validate_file_root_name(source)?;
let source_root = store
.get_file_root_by_name(source)?
.ok_or_else(|| NodeError::ResourceNotFound(format!("file-root:{source}")))?;
let tree_hash = source_root
.latest_tree_hash
.as_ref()
.ok_or_else(|| NodeError::ResourceNotFound(format!("file-root-tree:{source}")))?;
let tree_hash = BlobHash::new(tree_hash.clone());
let cas = LocalCas::new(node.paths.cas_dir());
let tree_bytes = cas.read_bytes(&tree_hash)?;
let tree: CasTreeObject = geth_codec::decode_canonical(&tree_bytes)?;
if target.exists() && !target.is_dir() {
return Err(NodeError::Cas(geth_cas::CasError::TreeRootNotDirectory(
target.display().to_string(),
)));
}
if !dry_run {
std::fs::create_dir_all(&target)?;
}
let mut files_written = 0;
let mut dirs_created = 0;
let mut conflicts = Vec::new();
for entry in &tree.entries {
let out = safe_tree_output_path(&target, &entry.path)?;
match entry.kind {
CasTreeEntryKind::Directory => {
if out.exists() {
if !out.is_dir() {
conflicts.push(record_apply_conflict(
store,
&source_root,
&entry.path,
&tree_hash,
"remote directory conflicts with a local non-directory path",
dry_run,
)?);
}
continue;
}
dirs_created += 1;
if !dry_run {
std::fs::create_dir_all(&out)?;
}
}
CasTreeEntryKind::File => {
let Some(blob) = &entry.blob else {
conflicts.push(record_apply_conflict(
store,
&source_root,
&entry.path,
&tree_hash,
"remote file entry is missing a CAS blob hash",
dry_run,
)?);
continue;
};
if out.exists() {
if !out.is_file() {
conflicts.push(record_apply_conflict(
store,
&source_root,
&entry.path,
&tree_hash,
"remote file conflicts with a local non-file path",
dry_run,
)?);
continue;
}
if hash_path(&out)? != *blob {
conflicts.push(record_apply_conflict(
store,
&source_root,
&entry.path,
&tree_hash,
"local file differs from remote CAS tree; refusing to overwrite",
dry_run,
)?);
}
continue;
}
files_written += 1;
if !dry_run {
let bytes = cas.read_bytes(blob)?;
write_file_atomic(&out, &bytes)?;
set_executable_bit(&out, entry.executable)?;
}
}
}
}
Ok(ControlResponse::CasRootApplied {
source: source.to_owned(),
target,
files_written,
dirs_created,
conflicts,
dry_run,
note: "safe file-root apply creates missing paths but never deletes extra files or overwrites differing local files; conflicts are recorded for manual resolution".to_owned(),
})
}
fn safe_tree_output_path(root: &Path, relative: &str) -> Result<PathBuf, NodeError> {
let relative_path = Path::new(relative);
if relative_path.is_absolute() {
return Err(NodeError::Cas(geth_cas::CasError::NonRelativeTreePath(
relative.to_owned(),
)));
}
for component in relative_path.components() {
if !matches!(component, Component::Normal(_)) {
return Err(NodeError::Cas(geth_cas::CasError::NonRelativeTreePath(
relative.to_owned(),
)));
}
}
Ok(root.join(relative_path))
}
fn record_apply_conflict(
store: &Store,
root: &StoredFileRoot,
path: &str,
remote_tree: &BlobHash,
detail: &str,
dry_run: bool,
) -> Result<FileConflict, NodeError> {
let created_at = geth_store::now_ms();
let stored = StoredFileConflict {
conflict_id: generated_file_conflict_id(
&root.name,
path,
FileConflictKind::ConcurrentEdit.as_str(),
created_at,
),
root_name: root.name.clone(),
resource_id: root.resource_id.clone(),
path: path.to_owned(),
kind: FileConflictKind::ConcurrentEdit.as_str().to_owned(),
status: FileConflictStatus::Open.as_str().to_owned(),
base_tree_hash: None,
local_tree_hash: None,
remote_tree_hash: Some(remote_tree.to_string()),
detail: detail.to_owned(),
resolution: None,
resolution_note: None,
created_at_ms: created_at,
resolved_at_ms: None,
};
if !dry_run {
store.upsert_file_conflict(&stored)?;
}
file_conflict_from_stored(stored)
}
fn write_file_atomic(path: &Path, bytes: &[u8]) -> Result<(), NodeError> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let tmp = path.with_extension(format!(
"geth-tmp-{}",
geth_crypto::blake3_hex(path.display().to_string().as_bytes())
));
std::fs::write(&tmp, bytes)?;
std::fs::rename(tmp, path)?;
Ok(())
}
#[cfg(unix)]
fn set_executable_bit(path: &Path, executable: bool) -> Result<(), NodeError> {
use std::os::unix::fs::PermissionsExt;
let mut permissions = std::fs::metadata(path)?.permissions();
let mut mode = permissions.mode();
if executable {
mode |= 0o111;
} else {
mode &= !0o111;
}
permissions.set_mode(mode);
std::fs::set_permissions(path, permissions)?;
Ok(())
}
#[cfg(not(unix))]
fn set_executable_bit(_path: &Path, _executable: bool) -> Result<(), NodeError> {
Ok(())
}
fn ensure_resource_exists(store: &Store, resource_id: &str) -> Result<(), NodeError> { fn ensure_resource_exists(store: &Store, resource_id: &str) -> Result<(), NodeError> {
if store if store
.list_resources()? .list_resources()?

View file

@ -394,6 +394,91 @@ fn cas_file_root_add_list_scan_detects_changes() {
} }
} }
#[test]
fn cas_file_root_apply_writes_missing_files_and_records_conflicts() {
let home = tempfile::tempdir().expect("tempdir");
let paths = geth_config::GethPaths::from_home(home.path());
let node = geth_node::init_node(&paths).expect("init node");
let source_path = home.path().join("source-root");
std::fs::create_dir_all(source_path.join("dir")).expect("create source root");
std::fs::write(source_path.join("a.txt"), b"remote").expect("write remote a");
std::fs::write(source_path.join("dir").join("b.txt"), b"remote b").expect("write remote b");
geth_node::handle_request(
&node,
geth_control::ControlRequest::CasRootAdd {
name: "remote-node-peer-shared".to_owned(),
path: source_path,
},
)
.expect("add source root");
let tree = match geth_node::handle_request(
&node,
geth_control::ControlRequest::CasRootScan {
name: "remote-node-peer-shared".to_owned(),
},
)
.expect("scan source root")
{
geth_control::ControlResponse::CasRootScanned { scan } => scan.tree.hash,
other => panic!("unexpected response: {other:?}"),
};
let target_path = home.path().join("target-root");
std::fs::create_dir_all(&target_path).expect("create target root");
std::fs::write(target_path.join("a.txt"), b"local edit").expect("write local a");
let response = geth_node::handle_request(
&node,
geth_control::ControlRequest::CasRootApply {
source: "remote-node-peer-shared".to_owned(),
target: target_path.clone(),
dry_run: false,
},
)
.expect("apply source root");
match response {
geth_control::ControlResponse::CasRootApplied {
files_written,
dirs_created,
conflicts,
note,
..
} => {
assert_eq!(files_written, 1);
assert_eq!(dirs_created, 1);
assert_eq!(conflicts.len(), 1);
assert_eq!(conflicts[0].path, "a.txt");
assert_eq!(conflicts[0].remote_tree, Some(tree));
assert!(note.contains("never deletes"));
}
other => panic!("unexpected response: {other:?}"),
}
assert_eq!(
std::fs::read(target_path.join("dir").join("b.txt")).expect("read applied b"),
b"remote b"
);
assert_eq!(
std::fs::read(target_path.join("a.txt")).expect("read local a"),
b"local edit"
);
let response = geth_node::handle_request(
&node,
geth_control::ControlRequest::CasConflictList {
root: Some("remote-node-peer-shared".to_owned()),
},
)
.expect("list apply conflicts");
match response {
geth_control::ControlResponse::CasConflictList { conflicts } => {
assert_eq!(conflicts.len(), 1);
assert_eq!(conflicts[0].path, "a.txt");
}
other => panic!("unexpected response: {other:?}"),
}
}
#[test] #[test]
fn cas_file_conflict_record_list_and_resolve() { fn cas_file_conflict_record_list_and_resolve() {
let home = tempfile::tempdir().expect("tempdir"); let home = tempfile::tempdir().expect("tempdir");

View file

@ -133,9 +133,13 @@ pull authorized remote file-root tree metadata with `geth cas root sync <node>
remote CAS tree bytes and records a peer-qualified remote root whose path is remote CAS tree bytes and records a peer-qualified remote root whose path is
`remote:<node>:<name>` without applying files. File roots also participate in `remote:<node>:<name>` without applying files. File roots also participate in
the daemon background live-sync loop through authorized `cas-tree:<name>` the daemon background live-sync loop through authorized `cas-tree:<name>`
watermarks. The daemon also has durable local file-conflict records with watermarks. `geth cas root apply <root> --to <path>` is the first conservative
explicit resolution choices; future cross-node file sync will create those materialization path: it creates missing files and directories from the CAS tree,
records automatically instead of silently applying ambiguous remote changes. does not delete extra local files, does not overwrite differing local files, and
records conflicts for manual resolution. The daemon also has durable local
file-conflict records with explicit resolution choices; future cross-node file
sync will extend those records instead of silently applying ambiguous remote
changes.
As a bootstrap network path, `geth cas fetch <node-id> <hash>` dials an As a bootstrap network path, `geth cas fetch <node-id> <hash>` dials an
imported signed peer card over the daemon-owned Iroh control ALPN. The serving imported signed peer card over the daemon-owned Iroh control ALPN. The serving

View file

@ -519,8 +519,12 @@ and future group key evolution.
or overwriting same-named local roots. or overwriting same-named local roots.
- `[x]` Background live-sync imports updated authorized file-root trees from - `[x]` Background live-sync imports updated authorized file-root trees from
sync-status `cas-tree:<name>` watermarks and stores per-peer cursors. sync-status `cas-tree:<name>` watermarks and stores per-peer cursors.
- `[ ]` Future completion applies file-root sync safely with conflict - `[x]` `geth cas root apply <root> --to <path>` materializes missing files
detection and explicit resolution. and directories from a CAS tree without deleting extras or overwriting local
edits.
- `[x]` Apply records durable conflicts for differing local paths.
- `[ ]` Future completion adds a richer three-way apply using base/local/remote
trees for automatic safe updates and deletes.
- `[~]` Conflict handling. - `[~]` Conflict handling.
Acceptance criteria: Acceptance criteria: