Track CAS providers

This commit is contained in:
Eric Wendland 2026-05-19 15:37:02 +02:00
commit c9803ea7f2
8 changed files with 161 additions and 11 deletions

View file

@ -127,8 +127,9 @@ Roadmap items should be actionable and checkable:
- Local CAS supports pin/unpin metadata, surfaced through `cas list`, and - Local CAS supports pin/unpin metadata, surfaced through `cas list`, and
`cas cleanup` evicts unpinned blobs while retaining pinned blobs. The daemon `cas cleanup` evicts unpinned blobs while retaining pinned blobs. The daemon
can fetch CAS blobs from an imported signed peer card over Iroh when the peer can fetch CAS blobs from an imported signed peer card over Iroh when the peer
grants `cas.fetch` on `resource:cas:local`; full `iroh-blobs` provider grants `cas.fetch` on `resource:cas:local`; successful fetches record local
integration is still roadmap work. provider metadata visible through `geth cas providers <hash>`. Full
`iroh-blobs` provider integration is still roadmap work.
- The CAS crate can build deterministic tree objects for local file trees and - The CAS crate can build deterministic tree objects for local file trees and
store those manifests as CAS blobs. The daemon can register and scan local store those manifests as CAS blobs. The daemon can register and scan local
file roots, reporting create/update/delete/rename changes without writing back file roots, reporting create/update/delete/rename changes without writing back

View file

@ -93,7 +93,7 @@ The bootstrap implementation provides:
- `geth auth grant <subject> <resource> <capability> [--grant-id <id>]` - `geth auth grant <subject> <resource> <capability> [--grant-id <id>]`
- `geth auth revoke <resource> <grant-id>` - `geth auth revoke <resource> <grant-id>`
- local filesystem CAS commands: `add`, `get`, `fetch`, `hash`, `has`, `pin`, - local filesystem CAS commands: `add`, `get`, `fetch`, `hash`, `has`, `pin`,
`unpin`, `cleanup`, `list` `unpin`, `cleanup`, `providers`, `list`
- local CAS tree objects describe file trees and are stored as CAS blobs - local CAS tree objects describe file trees and are stored as CAS blobs
- local file-root commands: `geth cas root add/list/scan` - local file-root commands: `geth cas root add/list/scan`
- local file conflict metadata commands: - local file conflict metadata commands:
@ -136,9 +136,10 @@ card binds the actual Iroh EndpointID before reducing resource-local auth ops.
`geth cas fetch <node-id> <hash>` uses the same protected Iroh control path to `geth cas fetch <node-id> <hash>` uses the same protected Iroh control path to
request a blob from a peer. The remote daemon only returns bytes when the caller request a blob from a peer. The remote daemon only returns bytes when the caller
has `cas.fetch` on `resource:cas:local`, and the caller verifies that the bytes has `cas.fetch` on `resource:cas:local`, and the caller verifies that the bytes
hash to the requested BLAKE3 CAS hash before storing them locally. This is the hash to the requested BLAKE3 CAS hash before storing them locally. Successful
bootstrap transfer path; future work will move provider/fetch behavior to fetches record the serving peer as a local provider, visible with
`iroh-blobs`. `geth cas providers <hash>`. This is the bootstrap transfer path; future work
will move provider/fetch behavior to `iroh-blobs`.
`geth ssh cert sync <node-id>` requires `ssh_cert.sync` on `resource:ssh:certs` `geth ssh cert sync <node-id>` requires `ssh_cert.sync` on `resource:ssh:certs`
at the peer. `geth ssh revocation sync <node-id>` requires at the peer. `geth ssh revocation sync <node-id>` requires
`ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge `ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge

View file

@ -243,6 +243,9 @@ pub enum CasCommand {
#[arg(long)] #[arg(long)]
dry_run: bool, dry_run: bool,
}, },
Providers {
hash: String,
},
List, List,
Root { Root {
#[command(subcommand)] #[command(subcommand)]
@ -593,6 +596,7 @@ fn request_for_command(command: Command) -> Result<ControlRequest> {
CasCommand::Pin { hash } => ControlRequest::CasPin { hash: hash.into() }, CasCommand::Pin { hash } => ControlRequest::CasPin { hash: hash.into() },
CasCommand::Unpin { hash } => ControlRequest::CasUnpin { hash: hash.into() }, CasCommand::Unpin { hash } => ControlRequest::CasUnpin { hash: hash.into() },
CasCommand::Cleanup { dry_run } => ControlRequest::CasCleanup { dry_run }, CasCommand::Cleanup { dry_run } => ControlRequest::CasCleanup { dry_run },
CasCommand::Providers { hash } => ControlRequest::CasProviders { hash: hash.into() },
CasCommand::List => ControlRequest::CasList, CasCommand::List => ControlRequest::CasList,
CasCommand::Root { command } => match command { CasCommand::Root { command } => match command {
CasRootCommand::Add { name, path } => ControlRequest::CasRootAdd { name, path }, CasRootCommand::Add { name, path } => ControlRequest::CasRootAdd { name, path },
@ -979,6 +983,16 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> {
println!("{}\t{} bytes\t{}", blob.hash, blob.size_bytes, pin); println!("{}\t{} bytes\t{}", blob.hash, blob.size_bytes, pin);
} }
} }
ControlResponse::CasProviders { hash, providers } => {
println!("hash: {hash}");
println!("providers: {}", providers.len());
for provider in providers {
println!(
"{}\t{}\t{}",
provider.peer_node_id, provider.endpoint_id, provider.last_seen_ms
);
}
}
ControlResponse::CasRootAdded { root } => { ControlResponse::CasRootAdded { root } => {
println!("added file root: {}", root.name); println!("added file root: {}", root.name);
println!("id: {}", root.id); println!("id: {}", root.id);

View file

@ -67,6 +67,9 @@ pub enum ControlRequest {
CasCleanup { CasCleanup {
dry_run: bool, dry_run: bool,
}, },
CasProviders {
hash: BlobHash,
},
CasList, CasList,
CasRootAdd { CasRootAdd {
name: String, name: String,
@ -324,6 +327,10 @@ pub enum ControlResponse {
CasList { CasList {
blobs: Vec<CasBlob>, blobs: Vec<CasBlob>,
}, },
CasProviders {
hash: BlobHash,
providers: Vec<CasProvider>,
},
CasRootAdded { CasRootAdded {
root: FileRoot, root: FileRoot,
}, },
@ -569,6 +576,13 @@ pub struct CasBlob {
pub pinned: bool, pub pinned: bool,
} }
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct CasProvider {
pub peer_node_id: String,
pub endpoint_id: String,
pub last_seen_ms: i64,
}
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "kebab-case")] #[serde(tag = "type", rename_all = "kebab-case")]
pub enum PeerControlRequest { pub enum PeerControlRequest {
@ -881,6 +895,27 @@ mod tests {
request request
); );
let request = ControlRequest::CasProviders {
hash: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef".into(),
};
assert_eq!(
decode_request(&encode_request(&request).expect("encode")).expect("decode"),
request
);
let response = ControlResponse::CasProviders {
hash: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef".into(),
providers: vec![CasProvider {
peer_node_id: "node:peer".to_owned(),
endpoint_id: "endpoint:peer".to_owned(),
last_seen_ms: 42,
}],
};
assert_eq!(
decode_response(&encode_response(&response).expect("encode")).expect("decode"),
response
);
let request = ControlRequest::SshRevocationExport { let request = ControlRequest::SshRevocationExport {
out: PathBuf::from("revocations.krl-spec"), out: PathBuf::from("revocations.krl-spec"),
format: "openssh-krl-spec".to_owned(), format: "openssh-krl-spec".to_owned(),

View file

@ -8,7 +8,7 @@ use geth_cas::{
}; };
use geth_config::{GethConfig, GethPaths, RelayMode}; use geth_config::{GethConfig, GethPaths, RelayMode};
use geth_control::{ use geth_control::{
CasBlob, ControlRequest, ControlResponse, KeychainStatusResponse, NodeIdResponse, CasBlob, CasProvider, ControlRequest, ControlResponse, KeychainStatusResponse, NodeIdResponse,
PeerControlRequest, PeerControlResponse, StatusResponse, SyncWatermark, PeerControlRequest, PeerControlResponse, StatusResponse, SyncWatermark,
}; };
use geth_crypto::AgentKey; use geth_crypto::AgentKey;
@ -799,6 +799,7 @@ async fn cas_fetch_from_peer(
info.size_bytes, info.size_bytes,
&info.path.to_string_lossy(), &info.path.to_string_lossy(),
)?; )?;
store.record_cas_provider(info.hash.as_str(), &node_id, &endpoint_id)?;
Ok(ControlResponse::CasFetched { Ok(ControlResponse::CasFetched {
peer_node_id: node_id, peer_node_id: node_id,
peer_agent_id: agent_id, peer_agent_id: agent_id,
@ -2592,6 +2593,21 @@ pub fn handle_request(
dry_run, dry_run,
}) })
} }
ControlRequest::CasProviders { hash } => {
geth_cas::validate_hash(&hash)?;
Ok(ControlResponse::CasProviders {
providers: store
.list_cas_providers(hash.as_str())?
.into_iter()
.map(|provider| CasProvider {
peer_node_id: provider.peer_node_id,
endpoint_id: provider.endpoint_id,
last_seen_ms: provider.last_seen_ms,
})
.collect(),
hash,
})
}
ControlRequest::CasList => { ControlRequest::CasList => {
let cas = LocalCas::new(node.paths.cas_dir()); let cas = LocalCas::new(node.paths.cas_dir());
let blobs = cas let blobs = cas
@ -4651,6 +4667,12 @@ mod tests {
} }
other => panic!("unexpected allowed CAS fetch response: {other:?}"), other => panic!("unexpected allowed CAS fetch response: {other:?}"),
} }
let providers = Store::open(&left_paths.metadata_db())
.expect("open left after cas fetch")
.list_cas_providers(right_blob.hash.as_str())
.expect("list cas providers");
assert_eq!(providers.len(), 1);
assert_eq!(providers[0].peer_node_id, right.node_id);
let kv_synced = handle_request_async( let kv_synced = handle_request_async(
&left, &left,

View file

@ -100,6 +100,13 @@ impl Store {
hash TEXT PRIMARY KEY, hash TEXT PRIMARY KEY,
pinned_at_ms INTEGER NOT NULL pinned_at_ms INTEGER NOT NULL
); );
CREATE TABLE IF NOT EXISTS cas_providers (
hash TEXT NOT NULL,
peer_node_id TEXT NOT NULL,
endpoint_id TEXT NOT NULL,
last_seen_ms INTEGER NOT NULL,
PRIMARY KEY(hash, peer_node_id)
);
CREATE TABLE IF NOT EXISTS kv_stores ( CREATE TABLE IF NOT EXISTS kv_stores (
kv_id TEXT PRIMARY KEY, kv_id TEXT PRIMARY KEY,
resource_id TEXT NOT NULL, resource_id TEXT NOT NULL,
@ -746,6 +753,39 @@ impl Store {
.map_err(StoreError::from) .map_err(StoreError::from)
} }
pub fn record_cas_provider(
&self,
hash: &str,
peer_node_id: &str,
endpoint_id: &str,
) -> Result<(), StoreError> {
self.conn.execute(
r#"INSERT OR REPLACE INTO cas_providers(hash, peer_node_id, endpoint_id, last_seen_ms)
VALUES (?1, ?2, ?3, ?4)"#,
params![hash, peer_node_id, endpoint_id, now_ms()],
)?;
Ok(())
}
pub fn list_cas_providers(&self, hash: &str) -> Result<Vec<CasProvider>, StoreError> {
let mut stmt = self.conn.prepare(
r#"SELECT hash, peer_node_id, endpoint_id, last_seen_ms
FROM cas_providers
WHERE hash = ?1
ORDER BY last_seen_ms DESC, peer_node_id"#,
)?;
let rows = stmt.query_map(params![hash], |row| {
Ok(CasProvider {
hash: row.get(0)?,
peer_node_id: row.get(1)?,
endpoint_id: row.get(2)?,
last_seen_ms: row.get(3)?,
})
})?;
rows.collect::<Result<Vec<_>, _>>()
.map_err(StoreError::from)
}
pub fn upsert_peer_card(&self, peer_card: &StoredPeerCard) -> Result<(), StoreError> { pub fn upsert_peer_card(&self, peer_card: &StoredPeerCard) -> Result<(), StoreError> {
self.conn.execute( self.conn.execute(
"INSERT OR REPLACE INTO peer_cards(peer_id, card_json, updated_at_ms) VALUES (?1, ?2, ?3)", "INSERT OR REPLACE INTO peer_cards(peer_id, card_json, updated_at_ms) VALUES (?1, ?2, ?3)",
@ -1132,6 +1172,14 @@ pub struct StoredResource {
pub status: String, pub status: String,
} }
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct CasProvider {
pub hash: String,
pub peer_node_id: String,
pub endpoint_id: String,
pub last_seen_ms: i64,
}
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct StoredDbResource { pub struct StoredDbResource {
pub db_id: String, pub db_id: String,
@ -1452,6 +1500,30 @@ mod tests {
assert!(store.list_cas_pins().expect("pins").is_empty()); assert!(store.list_cas_pins().expect("pins").is_empty());
} }
#[test]
fn cas_providers_roundtrip_by_hash() {
let store = Store::open_memory().expect("open");
let hash = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
store
.record_cas_provider(hash, "node:laptop", "endpoint:laptop")
.expect("record provider");
let providers = store.list_cas_providers(hash).expect("list providers");
assert_eq!(providers.len(), 1);
assert_eq!(providers[0].hash, hash);
assert_eq!(providers[0].peer_node_id, "node:laptop");
assert_eq!(providers[0].endpoint_id, "endpoint:laptop");
assert!(
store
.list_cas_providers(
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
)
.expect("list missing")
.is_empty()
);
}
#[test] #[test]
fn deleting_cas_object_removes_pin_metadata() { fn deleting_cas_object_removes_pin_metadata() {
let store = Store::open_memory().expect("open"); let store = Store::open_memory().expect("open");

View file

@ -130,9 +130,11 @@ imported signed peer card over the daemon-owned Iroh control ALPN. The serving
daemon validates the caller's peer-card signature and observed Iroh EndpointID, daemon validates the caller's peer-card signature and observed Iroh EndpointID,
then reduces local auth ops and requires `cas.fetch` on `resource:cas:local` then reduces local auth ops and requires `cas.fetch` on `resource:cas:local`
before returning blob bytes. The requester verifies that the returned bytes hash before returning blob bytes. The requester verifies that the returned bytes hash
to the requested BLAKE3 CAS hash before storing them. Iroh-blobs, provider to the requested BLAKE3 CAS hash before storing them. Successful fetches update
tracking, encrypted blobs, richer cache policies, cross-node file roots, and durable local provider metadata keyed by CAS hash and peer node, which can be
automatic conflict detection are future work. inspected through `geth cas providers <hash>`. Iroh-blobs, encrypted blobs,
richer cache policies, cross-node file roots, and automatic conflict detection
are future work.
`geth-db` currently registers local SQLite paths as DB resources and reports `geth-db` currently registers local SQLite paths as DB resources and reports
local-only sync status plus a read-only SQLite schema summary/hash. It also local-only sync status plus a read-only SQLite schema summary/hash. It also

View file

@ -248,9 +248,12 @@ authorization and durable-state boundaries clear.
- `[x]` The requester verifies returned bytes against the requested BLAKE3 - `[x]` The requester verifies returned bytes against the requested BLAKE3
CAS hash before storing them locally. CAS hash before storing them locally.
- `[x]` Local add/get/hash/has/list behavior remains backward compatible. - `[x]` Local add/get/hash/has/list behavior remains backward compatible.
- `[x]` Successful fetches record the serving peer as a local provider for
the CAS hash.
- `[x]` `geth cas providers <hash>` lists locally known providers.
- `[x]` Tests cover local provider metadata storage.
- `[ ]` Replace the bootstrap control-ALPN transfer with `iroh-blobs` - `[ ]` Replace the bootstrap control-ALPN transfer with `iroh-blobs`
provider/fetch behavior. provider/fetch behavior.
- `[ ]` Provider tracking is recorded locally.
- `[x]` CAS pin and cache policy. - `[x]` CAS pin and cache policy.
Acceptance criteria: Acceptance criteria: