Track CAS providers
This commit is contained in:
parent
df70b81a1d
commit
c9803ea7f2
8 changed files with 161 additions and 11 deletions
|
|
@ -127,8 +127,9 @@ Roadmap items should be actionable and checkable:
|
||||||
- Local CAS supports pin/unpin metadata, surfaced through `cas list`, and
|
- Local CAS supports pin/unpin metadata, surfaced through `cas list`, and
|
||||||
`cas cleanup` evicts unpinned blobs while retaining pinned blobs. The daemon
|
`cas cleanup` evicts unpinned blobs while retaining pinned blobs. The daemon
|
||||||
can fetch CAS blobs from an imported signed peer card over Iroh when the peer
|
can fetch CAS blobs from an imported signed peer card over Iroh when the peer
|
||||||
grants `cas.fetch` on `resource:cas:local`; full `iroh-blobs` provider
|
grants `cas.fetch` on `resource:cas:local`; successful fetches record local
|
||||||
integration is still roadmap work.
|
provider metadata visible through `geth cas providers <hash>`. Full
|
||||||
|
`iroh-blobs` provider integration is still roadmap work.
|
||||||
- The CAS crate can build deterministic tree objects for local file trees and
|
- The CAS crate can build deterministic tree objects for local file trees and
|
||||||
store those manifests as CAS blobs. The daemon can register and scan local
|
store those manifests as CAS blobs. The daemon can register and scan local
|
||||||
file roots, reporting create/update/delete/rename changes without writing back
|
file roots, reporting create/update/delete/rename changes without writing back
|
||||||
|
|
|
||||||
|
|
@ -93,7 +93,7 @@ The bootstrap implementation provides:
|
||||||
- `geth auth grant <subject> <resource> <capability> [--grant-id <id>]`
|
- `geth auth grant <subject> <resource> <capability> [--grant-id <id>]`
|
||||||
- `geth auth revoke <resource> <grant-id>`
|
- `geth auth revoke <resource> <grant-id>`
|
||||||
- local filesystem CAS commands: `add`, `get`, `fetch`, `hash`, `has`, `pin`,
|
- local filesystem CAS commands: `add`, `get`, `fetch`, `hash`, `has`, `pin`,
|
||||||
`unpin`, `cleanup`, `list`
|
`unpin`, `cleanup`, `providers`, `list`
|
||||||
- local CAS tree objects describe file trees and are stored as CAS blobs
|
- local CAS tree objects describe file trees and are stored as CAS blobs
|
||||||
- local file-root commands: `geth cas root add/list/scan`
|
- local file-root commands: `geth cas root add/list/scan`
|
||||||
- local file conflict metadata commands:
|
- local file conflict metadata commands:
|
||||||
|
|
@ -136,9 +136,10 @@ card binds the actual Iroh EndpointID before reducing resource-local auth ops.
|
||||||
`geth cas fetch <node-id> <hash>` uses the same protected Iroh control path to
|
`geth cas fetch <node-id> <hash>` uses the same protected Iroh control path to
|
||||||
request a blob from a peer. The remote daemon only returns bytes when the caller
|
request a blob from a peer. The remote daemon only returns bytes when the caller
|
||||||
has `cas.fetch` on `resource:cas:local`, and the caller verifies that the bytes
|
has `cas.fetch` on `resource:cas:local`, and the caller verifies that the bytes
|
||||||
hash to the requested BLAKE3 CAS hash before storing them locally. This is the
|
hash to the requested BLAKE3 CAS hash before storing them locally. Successful
|
||||||
bootstrap transfer path; future work will move provider/fetch behavior to
|
fetches record the serving peer as a local provider, visible with
|
||||||
`iroh-blobs`.
|
`geth cas providers <hash>`. This is the bootstrap transfer path; future work
|
||||||
|
will move provider/fetch behavior to `iroh-blobs`.
|
||||||
`geth ssh cert sync <node-id>` requires `ssh_cert.sync` on `resource:ssh:certs`
|
`geth ssh cert sync <node-id>` requires `ssh_cert.sync` on `resource:ssh:certs`
|
||||||
at the peer. `geth ssh revocation sync <node-id>` requires
|
at the peer. `geth ssh revocation sync <node-id>` requires
|
||||||
`ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge
|
`ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge
|
||||||
|
|
|
||||||
|
|
@ -243,6 +243,9 @@ pub enum CasCommand {
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
dry_run: bool,
|
dry_run: bool,
|
||||||
},
|
},
|
||||||
|
Providers {
|
||||||
|
hash: String,
|
||||||
|
},
|
||||||
List,
|
List,
|
||||||
Root {
|
Root {
|
||||||
#[command(subcommand)]
|
#[command(subcommand)]
|
||||||
|
|
@ -593,6 +596,7 @@ fn request_for_command(command: Command) -> Result<ControlRequest> {
|
||||||
CasCommand::Pin { hash } => ControlRequest::CasPin { hash: hash.into() },
|
CasCommand::Pin { hash } => ControlRequest::CasPin { hash: hash.into() },
|
||||||
CasCommand::Unpin { hash } => ControlRequest::CasUnpin { hash: hash.into() },
|
CasCommand::Unpin { hash } => ControlRequest::CasUnpin { hash: hash.into() },
|
||||||
CasCommand::Cleanup { dry_run } => ControlRequest::CasCleanup { dry_run },
|
CasCommand::Cleanup { dry_run } => ControlRequest::CasCleanup { dry_run },
|
||||||
|
CasCommand::Providers { hash } => ControlRequest::CasProviders { hash: hash.into() },
|
||||||
CasCommand::List => ControlRequest::CasList,
|
CasCommand::List => ControlRequest::CasList,
|
||||||
CasCommand::Root { command } => match command {
|
CasCommand::Root { command } => match command {
|
||||||
CasRootCommand::Add { name, path } => ControlRequest::CasRootAdd { name, path },
|
CasRootCommand::Add { name, path } => ControlRequest::CasRootAdd { name, path },
|
||||||
|
|
@ -979,6 +983,16 @@ fn print_response(response: ControlResponse, json: bool) -> Result<()> {
|
||||||
println!("{}\t{} bytes\t{}", blob.hash, blob.size_bytes, pin);
|
println!("{}\t{} bytes\t{}", blob.hash, blob.size_bytes, pin);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
ControlResponse::CasProviders { hash, providers } => {
|
||||||
|
println!("hash: {hash}");
|
||||||
|
println!("providers: {}", providers.len());
|
||||||
|
for provider in providers {
|
||||||
|
println!(
|
||||||
|
"{}\t{}\t{}",
|
||||||
|
provider.peer_node_id, provider.endpoint_id, provider.last_seen_ms
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
ControlResponse::CasRootAdded { root } => {
|
ControlResponse::CasRootAdded { root } => {
|
||||||
println!("added file root: {}", root.name);
|
println!("added file root: {}", root.name);
|
||||||
println!("id: {}", root.id);
|
println!("id: {}", root.id);
|
||||||
|
|
|
||||||
|
|
@ -67,6 +67,9 @@ pub enum ControlRequest {
|
||||||
CasCleanup {
|
CasCleanup {
|
||||||
dry_run: bool,
|
dry_run: bool,
|
||||||
},
|
},
|
||||||
|
CasProviders {
|
||||||
|
hash: BlobHash,
|
||||||
|
},
|
||||||
CasList,
|
CasList,
|
||||||
CasRootAdd {
|
CasRootAdd {
|
||||||
name: String,
|
name: String,
|
||||||
|
|
@ -324,6 +327,10 @@ pub enum ControlResponse {
|
||||||
CasList {
|
CasList {
|
||||||
blobs: Vec<CasBlob>,
|
blobs: Vec<CasBlob>,
|
||||||
},
|
},
|
||||||
|
CasProviders {
|
||||||
|
hash: BlobHash,
|
||||||
|
providers: Vec<CasProvider>,
|
||||||
|
},
|
||||||
CasRootAdded {
|
CasRootAdded {
|
||||||
root: FileRoot,
|
root: FileRoot,
|
||||||
},
|
},
|
||||||
|
|
@ -569,6 +576,13 @@ pub struct CasBlob {
|
||||||
pub pinned: bool,
|
pub pinned: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct CasProvider {
|
||||||
|
pub peer_node_id: String,
|
||||||
|
pub endpoint_id: String,
|
||||||
|
pub last_seen_ms: i64,
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||||
#[serde(tag = "type", rename_all = "kebab-case")]
|
#[serde(tag = "type", rename_all = "kebab-case")]
|
||||||
pub enum PeerControlRequest {
|
pub enum PeerControlRequest {
|
||||||
|
|
@ -881,6 +895,27 @@ mod tests {
|
||||||
request
|
request
|
||||||
);
|
);
|
||||||
|
|
||||||
|
let request = ControlRequest::CasProviders {
|
||||||
|
hash: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef".into(),
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
decode_request(&encode_request(&request).expect("encode")).expect("decode"),
|
||||||
|
request
|
||||||
|
);
|
||||||
|
|
||||||
|
let response = ControlResponse::CasProviders {
|
||||||
|
hash: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef".into(),
|
||||||
|
providers: vec![CasProvider {
|
||||||
|
peer_node_id: "node:peer".to_owned(),
|
||||||
|
endpoint_id: "endpoint:peer".to_owned(),
|
||||||
|
last_seen_ms: 42,
|
||||||
|
}],
|
||||||
|
};
|
||||||
|
assert_eq!(
|
||||||
|
decode_response(&encode_response(&response).expect("encode")).expect("decode"),
|
||||||
|
response
|
||||||
|
);
|
||||||
|
|
||||||
let request = ControlRequest::SshRevocationExport {
|
let request = ControlRequest::SshRevocationExport {
|
||||||
out: PathBuf::from("revocations.krl-spec"),
|
out: PathBuf::from("revocations.krl-spec"),
|
||||||
format: "openssh-krl-spec".to_owned(),
|
format: "openssh-krl-spec".to_owned(),
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,7 @@ use geth_cas::{
|
||||||
};
|
};
|
||||||
use geth_config::{GethConfig, GethPaths, RelayMode};
|
use geth_config::{GethConfig, GethPaths, RelayMode};
|
||||||
use geth_control::{
|
use geth_control::{
|
||||||
CasBlob, ControlRequest, ControlResponse, KeychainStatusResponse, NodeIdResponse,
|
CasBlob, CasProvider, ControlRequest, ControlResponse, KeychainStatusResponse, NodeIdResponse,
|
||||||
PeerControlRequest, PeerControlResponse, StatusResponse, SyncWatermark,
|
PeerControlRequest, PeerControlResponse, StatusResponse, SyncWatermark,
|
||||||
};
|
};
|
||||||
use geth_crypto::AgentKey;
|
use geth_crypto::AgentKey;
|
||||||
|
|
@ -799,6 +799,7 @@ async fn cas_fetch_from_peer(
|
||||||
info.size_bytes,
|
info.size_bytes,
|
||||||
&info.path.to_string_lossy(),
|
&info.path.to_string_lossy(),
|
||||||
)?;
|
)?;
|
||||||
|
store.record_cas_provider(info.hash.as_str(), &node_id, &endpoint_id)?;
|
||||||
Ok(ControlResponse::CasFetched {
|
Ok(ControlResponse::CasFetched {
|
||||||
peer_node_id: node_id,
|
peer_node_id: node_id,
|
||||||
peer_agent_id: agent_id,
|
peer_agent_id: agent_id,
|
||||||
|
|
@ -2592,6 +2593,21 @@ pub fn handle_request(
|
||||||
dry_run,
|
dry_run,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
ControlRequest::CasProviders { hash } => {
|
||||||
|
geth_cas::validate_hash(&hash)?;
|
||||||
|
Ok(ControlResponse::CasProviders {
|
||||||
|
providers: store
|
||||||
|
.list_cas_providers(hash.as_str())?
|
||||||
|
.into_iter()
|
||||||
|
.map(|provider| CasProvider {
|
||||||
|
peer_node_id: provider.peer_node_id,
|
||||||
|
endpoint_id: provider.endpoint_id,
|
||||||
|
last_seen_ms: provider.last_seen_ms,
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
hash,
|
||||||
|
})
|
||||||
|
}
|
||||||
ControlRequest::CasList => {
|
ControlRequest::CasList => {
|
||||||
let cas = LocalCas::new(node.paths.cas_dir());
|
let cas = LocalCas::new(node.paths.cas_dir());
|
||||||
let blobs = cas
|
let blobs = cas
|
||||||
|
|
@ -4651,6 +4667,12 @@ mod tests {
|
||||||
}
|
}
|
||||||
other => panic!("unexpected allowed CAS fetch response: {other:?}"),
|
other => panic!("unexpected allowed CAS fetch response: {other:?}"),
|
||||||
}
|
}
|
||||||
|
let providers = Store::open(&left_paths.metadata_db())
|
||||||
|
.expect("open left after cas fetch")
|
||||||
|
.list_cas_providers(right_blob.hash.as_str())
|
||||||
|
.expect("list cas providers");
|
||||||
|
assert_eq!(providers.len(), 1);
|
||||||
|
assert_eq!(providers[0].peer_node_id, right.node_id);
|
||||||
|
|
||||||
let kv_synced = handle_request_async(
|
let kv_synced = handle_request_async(
|
||||||
&left,
|
&left,
|
||||||
|
|
|
||||||
|
|
@ -100,6 +100,13 @@ impl Store {
|
||||||
hash TEXT PRIMARY KEY,
|
hash TEXT PRIMARY KEY,
|
||||||
pinned_at_ms INTEGER NOT NULL
|
pinned_at_ms INTEGER NOT NULL
|
||||||
);
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS cas_providers (
|
||||||
|
hash TEXT NOT NULL,
|
||||||
|
peer_node_id TEXT NOT NULL,
|
||||||
|
endpoint_id TEXT NOT NULL,
|
||||||
|
last_seen_ms INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY(hash, peer_node_id)
|
||||||
|
);
|
||||||
CREATE TABLE IF NOT EXISTS kv_stores (
|
CREATE TABLE IF NOT EXISTS kv_stores (
|
||||||
kv_id TEXT PRIMARY KEY,
|
kv_id TEXT PRIMARY KEY,
|
||||||
resource_id TEXT NOT NULL,
|
resource_id TEXT NOT NULL,
|
||||||
|
|
@ -746,6 +753,39 @@ impl Store {
|
||||||
.map_err(StoreError::from)
|
.map_err(StoreError::from)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn record_cas_provider(
|
||||||
|
&self,
|
||||||
|
hash: &str,
|
||||||
|
peer_node_id: &str,
|
||||||
|
endpoint_id: &str,
|
||||||
|
) -> Result<(), StoreError> {
|
||||||
|
self.conn.execute(
|
||||||
|
r#"INSERT OR REPLACE INTO cas_providers(hash, peer_node_id, endpoint_id, last_seen_ms)
|
||||||
|
VALUES (?1, ?2, ?3, ?4)"#,
|
||||||
|
params![hash, peer_node_id, endpoint_id, now_ms()],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn list_cas_providers(&self, hash: &str) -> Result<Vec<CasProvider>, StoreError> {
|
||||||
|
let mut stmt = self.conn.prepare(
|
||||||
|
r#"SELECT hash, peer_node_id, endpoint_id, last_seen_ms
|
||||||
|
FROM cas_providers
|
||||||
|
WHERE hash = ?1
|
||||||
|
ORDER BY last_seen_ms DESC, peer_node_id"#,
|
||||||
|
)?;
|
||||||
|
let rows = stmt.query_map(params![hash], |row| {
|
||||||
|
Ok(CasProvider {
|
||||||
|
hash: row.get(0)?,
|
||||||
|
peer_node_id: row.get(1)?,
|
||||||
|
endpoint_id: row.get(2)?,
|
||||||
|
last_seen_ms: row.get(3)?,
|
||||||
|
})
|
||||||
|
})?;
|
||||||
|
rows.collect::<Result<Vec<_>, _>>()
|
||||||
|
.map_err(StoreError::from)
|
||||||
|
}
|
||||||
|
|
||||||
pub fn upsert_peer_card(&self, peer_card: &StoredPeerCard) -> Result<(), StoreError> {
|
pub fn upsert_peer_card(&self, peer_card: &StoredPeerCard) -> Result<(), StoreError> {
|
||||||
self.conn.execute(
|
self.conn.execute(
|
||||||
"INSERT OR REPLACE INTO peer_cards(peer_id, card_json, updated_at_ms) VALUES (?1, ?2, ?3)",
|
"INSERT OR REPLACE INTO peer_cards(peer_id, card_json, updated_at_ms) VALUES (?1, ?2, ?3)",
|
||||||
|
|
@ -1132,6 +1172,14 @@ pub struct StoredResource {
|
||||||
pub status: String,
|
pub status: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||||
|
pub struct CasProvider {
|
||||||
|
pub hash: String,
|
||||||
|
pub peer_node_id: String,
|
||||||
|
pub endpoint_id: String,
|
||||||
|
pub last_seen_ms: i64,
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||||
pub struct StoredDbResource {
|
pub struct StoredDbResource {
|
||||||
pub db_id: String,
|
pub db_id: String,
|
||||||
|
|
@ -1452,6 +1500,30 @@ mod tests {
|
||||||
assert!(store.list_cas_pins().expect("pins").is_empty());
|
assert!(store.list_cas_pins().expect("pins").is_empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cas_providers_roundtrip_by_hash() {
|
||||||
|
let store = Store::open_memory().expect("open");
|
||||||
|
let hash = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
|
||||||
|
store
|
||||||
|
.record_cas_provider(hash, "node:laptop", "endpoint:laptop")
|
||||||
|
.expect("record provider");
|
||||||
|
|
||||||
|
let providers = store.list_cas_providers(hash).expect("list providers");
|
||||||
|
|
||||||
|
assert_eq!(providers.len(), 1);
|
||||||
|
assert_eq!(providers[0].hash, hash);
|
||||||
|
assert_eq!(providers[0].peer_node_id, "node:laptop");
|
||||||
|
assert_eq!(providers[0].endpoint_id, "endpoint:laptop");
|
||||||
|
assert!(
|
||||||
|
store
|
||||||
|
.list_cas_providers(
|
||||||
|
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
|
||||||
|
)
|
||||||
|
.expect("list missing")
|
||||||
|
.is_empty()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn deleting_cas_object_removes_pin_metadata() {
|
fn deleting_cas_object_removes_pin_metadata() {
|
||||||
let store = Store::open_memory().expect("open");
|
let store = Store::open_memory().expect("open");
|
||||||
|
|
|
||||||
|
|
@ -130,9 +130,11 @@ imported signed peer card over the daemon-owned Iroh control ALPN. The serving
|
||||||
daemon validates the caller's peer-card signature and observed Iroh EndpointID,
|
daemon validates the caller's peer-card signature and observed Iroh EndpointID,
|
||||||
then reduces local auth ops and requires `cas.fetch` on `resource:cas:local`
|
then reduces local auth ops and requires `cas.fetch` on `resource:cas:local`
|
||||||
before returning blob bytes. The requester verifies that the returned bytes hash
|
before returning blob bytes. The requester verifies that the returned bytes hash
|
||||||
to the requested BLAKE3 CAS hash before storing them. Iroh-blobs, provider
|
to the requested BLAKE3 CAS hash before storing them. Successful fetches update
|
||||||
tracking, encrypted blobs, richer cache policies, cross-node file roots, and
|
durable local provider metadata keyed by CAS hash and peer node, which can be
|
||||||
automatic conflict detection are future work.
|
inspected through `geth cas providers <hash>`. Iroh-blobs, encrypted blobs,
|
||||||
|
richer cache policies, cross-node file roots, and automatic conflict detection
|
||||||
|
are future work.
|
||||||
|
|
||||||
`geth-db` currently registers local SQLite paths as DB resources and reports
|
`geth-db` currently registers local SQLite paths as DB resources and reports
|
||||||
local-only sync status plus a read-only SQLite schema summary/hash. It also
|
local-only sync status plus a read-only SQLite schema summary/hash. It also
|
||||||
|
|
|
||||||
|
|
@ -248,9 +248,12 @@ authorization and durable-state boundaries clear.
|
||||||
- `[x]` The requester verifies returned bytes against the requested BLAKE3
|
- `[x]` The requester verifies returned bytes against the requested BLAKE3
|
||||||
CAS hash before storing them locally.
|
CAS hash before storing them locally.
|
||||||
- `[x]` Local add/get/hash/has/list behavior remains backward compatible.
|
- `[x]` Local add/get/hash/has/list behavior remains backward compatible.
|
||||||
|
- `[x]` Successful fetches record the serving peer as a local provider for
|
||||||
|
the CAS hash.
|
||||||
|
- `[x]` `geth cas providers <hash>` lists locally known providers.
|
||||||
|
- `[x]` Tests cover local provider metadata storage.
|
||||||
- `[ ]` Replace the bootstrap control-ALPN transfer with `iroh-blobs`
|
- `[ ]` Replace the bootstrap control-ALPN transfer with `iroh-blobs`
|
||||||
provider/fetch behavior.
|
provider/fetch behavior.
|
||||||
- `[ ]` Provider tracking is recorded locally.
|
|
||||||
|
|
||||||
- `[x]` CAS pin and cache policy.
|
- `[x]` CAS pin and cache policy.
|
||||||
Acceptance criteria:
|
Acceptance criteria:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue