eric/geth
Watch
1
0
Fork
You've already forked geth
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Live sync SSH metadata in background

Browse source
This commit is contained in:
Eric Wendland 2026-05-18 18:29:45 +02:00
commit 68153be5d5
9 changed files with 318 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

6
AGENTS.md
View file

@ -108,7 +108,8 @@ Roadmap items should be actionable and checkable:
types, manual signed peer-card export/import/list commands, `geth peer ping`
and `geth peer auth-check` over Iroh, signed peer-card LAN discovery payloads,
authorized `geth cas fetch`, `geth ssh cert sync`, and
`geth ssh revocation sync` over the Iroh control ALPN, untrusted
`geth ssh revocation sync` over the Iroh control ALPN, a background SSH
metadata live-sync loop with per-peer cursors in `module_state`, untrusted
discovery-backend trait, custom relay-map config, and Iroh local-network
discovery toggle exist.
- Canonical signed-operation envelopes exist for keychain/auth signature
@ -160,7 +161,8 @@ Roadmap items should be actionable and checkable:
public-key and certificate binary KRL revocations when `ssh-keygen` is
available. Authorized peers can pull SSH certificate-flow metadata with
`ssh_cert.sync` on `resource:ssh:certs` and revocation metadata with
`ssh_revocation.sync` on `resource:ssh:revocations`; this is pull-only
`ssh_revocation.sync` on `resource:ssh:revocations`. The daemon live-syncs
known peers every 30 seconds using per-peer cursors; this is pull-only
metadata sync, not yet a CRDT/resource-log replication model.
- cr-sqlite, iroh-docs, iroh-blobs provider/fetch, Automerge sync, broader auth
enforcement, and Keyhive/BeeKEM-style authorization are future roadmap items

1
Cargo.lock generated
View file

@ -1249,6 +1249,7 @@ dependencies = [
"geth-store",
"geth-types",
"iroh",
"serde",
"serde_json",
"swarm-discovery",
"tempfile",

5
README.md
View file

@ -138,7 +138,10 @@ bootstrap transfer path; future work will move provider/fetch behavior to
at the peer. `geth ssh revocation sync <node-id>` requires
`ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge
authorized peer metadata into the local store for offline listing and later
approval/signing workflows.
approval/signing workflows. While the daemon is running, it also performs a
background live-sync tick for known peers every 30 seconds. Live-sync stores
per-peer high-water cursors in local metadata so repeated ticks request only
newer SSH certificate-flow and revocation records.
Importing or pinging a peer card never grants capabilities by itself.
When `[iroh].local_discovery = true`, the daemon also advertises and discovers
signed peer cards on LAN using a geth-specific mDNS TXT payload. That payload is

6
crates/geth-control/src/lib.rs
View file

@ -513,10 +513,12 @@ pub enum PeerControlRequest {
},
SshCertSync {
peer_card: PeerCard,
since_ms: i64,
nonce: String,
},
SshRevocationSync {
peer_card: PeerCard,
since_ms: i64,
nonce: String,
},
}
@ -567,6 +569,7 @@ pub enum PeerControlResponse {
remote_endpoint_id: String,
requests: Vec<SshCertRequest>,
certificates: Vec<SshCertificateRecord>,
high_water_ms: i64,
allowed: bool,
reason: String,
evaluated_ops: usize,
@ -579,6 +582,7 @@ pub enum PeerControlResponse {
endpoint_id: String,
remote_endpoint_id: String,
revocations: Vec<SshRevocationEntry>,
high_water_ms: i64,
allowed: bool,
reason: String,
evaluated_ops: usize,
@ -873,6 +877,7 @@ mod tests {
remote_endpoint_id: "endpoint:caller".to_owned(),
requests: Vec::new(),
certificates: Vec::new(),
high_water_ms: 42,
allowed: false,
reason: "no grant".to_owned(),
evaluated_ops: 0,
@ -891,6 +896,7 @@ mod tests {
endpoint_id: "endpoint:peer".to_owned(),
remote_endpoint_id: "endpoint:caller".to_owned(),
revocations: Vec::new(),
high_water_ms: 42,
allowed: false,
reason: "no grant".to_owned(),
evaluated_ops: 0,

1
crates/geth-node/Cargo.toml
View file

@ -7,6 +7,7 @@ license.workspace = true
[dependencies]
base64.workspace = true
serde.workspace = true
serde_json.workspace = true
thiserror.workspace = true
tokio.workspace = true

180
crates/geth-node/src/lib.rs
View file

@ -33,8 +33,9 @@ use geth_ssh_identity::{
};
use geth_store::{
Store, StoredAuthOp, StoredDbResource, StoredDocumentResource, StoredFileConflict,
StoredFileRoot, StoredKeychainOp, StoredKvEntry, StoredKvStore, StoredPeerCard, StoredResource,
StoredResourceSecret, StoredSshCertRequest, StoredSshCertificate, StoredSshRevocation,
StoredFileRoot, StoredKeychainOp, StoredKvEntry, StoredKvStore, StoredModuleState,
StoredPeerCard, StoredResource, StoredResourceSecret, StoredSshCertRequest,
StoredSshCertificate, StoredSshRevocation,
};
use geth_types::{
AuthOpId, BlobHash, Capability, KeyId, NodeId, PrincipalId, ResourceId, ResourceKind,
@ -43,6 +44,7 @@ use geth_types::{
use std::collections::{BTreeMap, VecDeque};
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::{UnixListener, UnixStream};
@ -148,8 +150,14 @@ struct PipeRuntime {
}
const PUBSUB_RING_LIMIT: usize = 256;
const LIVE_SYNC_INTERVAL: Duration = Duration::from_secs(30);
const PIPE_CONNECTION_RING_LIMIT: usize = 256;
#[derive(Debug, serde::Deserialize, serde::Serialize)]
struct LiveSyncCursor {
cursor_ms: i64,
}
pub fn init_node(paths: &GethPaths) -> Result<LocalNode, NodeError> {
paths.ensure_base_dirs()?;
if !paths.config_file().exists() {
@ -194,6 +202,7 @@ pub async fn run_daemon(paths: GethPaths) -> Result<(), NodeError> {
.clone()
{
spawn_iroh_control_accept_loop(node.clone(), endpoint);
spawn_background_live_sync(node.clone());
}
let _peer_card_lan_discovery = start_peer_card_lan_discovery(&node).await;
if Path::new(&paths.socket_path()).exists() {
@ -778,8 +787,17 @@ async fn ssh_cert_sync_from_peer(
node: &LocalNode,
peer_node: &str,
) -> Result<ControlResponse, NodeError> {
let since_ms = load_live_sync_cursor(
&Store::open(&node.paths.metadata_db())?,
peer_node,
"ssh-certs",
)?;
let response = request_peer_control(node, peer_node, "ssh-cert-sync", |peer_card, nonce| {
PeerControlRequest::SshCertSync { peer_card, nonce }
PeerControlRequest::SshCertSync {
peer_card,
since_ms,
nonce,
}
})
.await?;
match response {
@ -789,6 +807,7 @@ async fn ssh_cert_sync_from_peer(
endpoint_id,
requests,
certificates,
high_water_ms,
allowed,
reason,
note,
@ -815,6 +834,7 @@ async fn ssh_cert_sync_from_peer(
for certificate in &certificates {
store.insert_ssh_certificate(&stored_from_ssh_certificate(certificate))?;
}
store_live_sync_cursor(&store, peer_node, "ssh-certs", high_water_ms)?;
Ok(ControlResponse::SshCertSynced {
peer_node_id: node_id,
peer_agent_id: agent_id,
@ -837,11 +857,20 @@ async fn ssh_revocation_sync_from_peer(
node: &LocalNode,
peer_node: &str,
) -> Result<ControlResponse, NodeError> {
let since_ms = load_live_sync_cursor(
&Store::open(&node.paths.metadata_db())?,
peer_node,
"ssh-revocations",
)?;
let response = request_peer_control(
node,
peer_node,
"ssh-revocation-sync",
|peer_card, nonce| PeerControlRequest::SshRevocationSync { peer_card, nonce },
|peer_card, nonce| PeerControlRequest::SshRevocationSync {
peer_card,
since_ms,
nonce,
},
)
.await?;
match response {
@ -850,6 +879,7 @@ async fn ssh_revocation_sync_from_peer(
agent_id,
endpoint_id,
revocations,
high_water_ms,
allowed,
reason,
note,
@ -871,6 +901,7 @@ async fn ssh_revocation_sync_from_peer(
for revocation in &revocations {
store.insert_ssh_revocation(&stored_from_ssh_revocation(revocation))?;
}
store_live_sync_cursor(&store, peer_node, "ssh-revocations", high_water_ms)?;
Ok(ControlResponse::SshRevocationSynced {
peer_node_id: node_id,
peer_agent_id: agent_id,
@ -888,6 +919,33 @@ async fn ssh_revocation_sync_from_peer(
}
}
fn live_sync_cursor_key(peer_node: &str, stream: &str) -> String {
format!("live-sync:{peer_node}:{stream}")
}
fn load_live_sync_cursor(store: &Store, peer_node: &str, stream: &str) -> Result<i64, NodeError> {
let key = live_sync_cursor_key(peer_node, stream);
let Some(state) = store.get_module_state(&key)? else {
return Ok(0);
};
let cursor: LiveSyncCursor = serde_json::from_str(&state.state_json)?;
Ok(cursor.cursor_ms)
}
fn store_live_sync_cursor(
store: &Store,
peer_node: &str,
stream: &str,
cursor_ms: i64,
) -> Result<(), NodeError> {
store.put_module_state(&StoredModuleState {
module: live_sync_cursor_key(peer_node, stream),
state_json: serde_json::to_string(&LiveSyncCursor { cursor_ms })?,
updated_at_ms: geth_store::now_ms(),
})?;
Ok(())
}
async fn request_peer_control(
node: &LocalNode,
peer_node: &str,
@ -975,6 +1033,42 @@ fn spawn_iroh_control_accept_loop(node: LocalNode, endpoint: GethIrohEndpoint) {
});
}
fn spawn_background_live_sync(node: LocalNode) {
tokio::spawn(async move {
if let Err(error) = run_live_sync_once(&node).await {
tracing::debug!(%error, "initial live sync tick failed");
}
let mut interval = tokio::time::interval(LIVE_SYNC_INTERVAL);
loop {
interval.tick().await;
if let Err(error) = run_live_sync_once(&node).await {
tracing::debug!(%error, "live sync tick failed");
}
}
});
}
async fn run_live_sync_once(node: &LocalNode) -> Result<(), NodeError> {
if node
.iroh_endpoint
.lock()
.map_err(|_| NodeError::RuntimeLockPoisoned)?
.is_none()
{
return Ok(());
}
let peers = Store::open(&node.paths.metadata_db())?.list_peer_cards()?;
for peer in peers {
if let Err(error) = ssh_cert_sync_from_peer(node, &peer.peer_id).await {
tracing::debug!(peer = %peer.peer_id, %error, "SSH cert live sync failed");
}
if let Err(error) = ssh_revocation_sync_from_peer(node, &peer.peer_id).await {
tracing::debug!(peer = %peer.peer_id, %error, "SSH revocation live sync failed");
}
}
Ok(())
}
async fn handle_iroh_control_connection(
node: LocalNode,
incoming: iroh::endpoint::Incoming,
@ -1132,7 +1226,11 @@ async fn handle_iroh_control_connection(
}
}
}
PeerControlRequest::SshCertSync { peer_card, nonce } => {
PeerControlRequest::SshCertSync {
peer_card,
since_ms,
nonce,
} => {
peer_card.validate_candidate()?;
ensure_peer_card_matches_endpoint(&peer_card, &remote_endpoint_id)?;
let discovered = DiscoveredPeer::candidate(
@ -1148,6 +1246,7 @@ async fn handle_iroh_control_connection(
})?;
let resource = "resource:ssh:certs".to_owned();
let capability = "ssh_cert.sync".to_owned();
let high_water_ms = geth_store::now_ms();
let explanation = geth_auth::explain_auth_ops(
&load_auth_ops_for_resource(&store, &resource)?,
PrincipalId::new(peer_card.node_id.to_string()),
@ -1157,12 +1256,12 @@ async fn handle_iroh_control_connection(
let (requests, certificates) = if explanation.allowed {
(
store
.list_ssh_cert_requests()?
.list_ssh_cert_requests_since(since_ms)?
.into_iter()
.map(ssh_cert_request_from_stored)
.collect::<Result<Vec<_>, _>>()?,
store
.list_ssh_certificates()?
.list_ssh_certificates_since(since_ms)?
.into_iter()
.map(ssh_certificate_from_stored)
.collect(),
@ -1177,6 +1276,7 @@ async fn handle_iroh_control_connection(
remote_endpoint_id,
requests,
certificates,
high_water_ms,
allowed: explanation.allowed,
reason: explanation.reason,
evaluated_ops: explanation.evaluated_ops,
@ -1184,7 +1284,11 @@ async fn handle_iroh_control_connection(
note: "SSH certificate metadata sync authenticated endpoint/card binding and required ssh_cert.sync on resource:ssh:certs".to_owned(),
}
}
PeerControlRequest::SshRevocationSync { peer_card, nonce } => {
PeerControlRequest::SshRevocationSync {
peer_card,
since_ms,
nonce,
} => {
peer_card.validate_candidate()?;
ensure_peer_card_matches_endpoint(&peer_card, &remote_endpoint_id)?;
let discovered = DiscoveredPeer::candidate(
@ -1200,6 +1304,7 @@ async fn handle_iroh_control_connection(
})?;
let resource = "resource:ssh:revocations".to_owned();
let capability = "ssh_revocation.sync".to_owned();
let high_water_ms = geth_store::now_ms();
let explanation = geth_auth::explain_auth_ops(
&load_auth_ops_for_resource(&store, &resource)?,
PrincipalId::new(peer_card.node_id.to_string()),
@ -1208,7 +1313,7 @@ async fn handle_iroh_control_connection(
);
let revocations = if explanation.allowed {
store
.list_ssh_revocations()?
.list_ssh_revocations_since(since_ms)?
.into_iter()
.map(ssh_revocation_from_stored)
.collect::<Result<Vec<_>, _>>()?
@ -1221,6 +1326,7 @@ async fn handle_iroh_control_connection(
endpoint_id: node.iroh_status.endpoint_id.clone().unwrap_or_default(),
remote_endpoint_id,
revocations,
high_water_ms,
allowed: explanation.allowed,
reason: explanation.reason,
evaluated_ops: explanation.evaluated_ops,
@ -3138,6 +3244,62 @@ mod tests {
.any(|revocation| revocation.revocation_id == right_revocation_id.as_str())
);
tokio::time::sleep(Duration::from_millis(2)).await;
let second_pubkey = right_home.path().join("request-2.pub");
std::fs::write(
&second_pubkey,
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGV0aDI= node2\n",
)
.expect("write second public key");
let second_requested = handle_request(
&right,
ControlRequest::SshCertRequest {
public_key_path: second_pubkey,
cert_kind: "user".to_owned(),
principals: vec!["admin".to_owned()],
requested_validity: Some("+4w".to_owned()),
renewal_of: None,
reason: Some("background sync test".to_owned()),
},
)
.expect("right second ssh cert request");
let second_request_id = match second_requested {
ControlResponse::SshCertRequested { request } => request.id,
other => panic!("unexpected second SSH cert request response: {other:?}"),
};
let second_revocation = handle_request(
&right,
ControlRequest::SshRevocationAdd {
kind: "key-id".to_owned(),
target: "newly-revoked-key".to_owned(),
reason: Some("background sync test".to_owned()),
},
)
.expect("right second ssh revocation");
let second_revocation_id = match second_revocation {
ControlResponse::SshRevocationAdded { revocation } => revocation.id,
other => panic!("unexpected second SSH revocation response: {other:?}"),
};
run_live_sync_once(&left)
.await
.expect("background live sync tick");
let left_store = Store::open(&left_paths.metadata_db()).expect("open left after live sync");
assert!(
left_store
.list_ssh_cert_requests()
.expect("list live-synced requests")
.iter()
.any(|request| request.request_id == second_request_id.as_str())
);
assert!(
left_store
.list_ssh_revocations()
.expect("list live-synced revocations")
.iter()
.any(|revocation| revocation.revocation_id == second_revocation_id.as_str())
);
left_endpoint.shutdown().await;
right_endpoint.shutdown().await;
}

118
crates/geth-store/src/lib.rs
View file

@ -709,6 +709,31 @@ impl Store {
.map_err(StoreError::from)
}
pub fn put_module_state(&self, state: &StoredModuleState) -> Result<(), StoreError> {
self.conn.execute(
r#"INSERT OR REPLACE INTO module_state(module, state_json, updated_at_ms)
VALUES (?1, ?2, ?3)"#,
params![state.module, state.state_json, state.updated_at_ms],
)?;
Ok(())
}
pub fn get_module_state(&self, module: &str) -> Result<Option<StoredModuleState>, StoreError> {
let mut stmt = self.conn.prepare(
"SELECT module, state_json, updated_at_ms FROM module_state WHERE module = ?1",
)?;
let mut rows = stmt.query(params![module])?;
if let Some(row) = rows.next()? {
Ok(Some(StoredModuleState {
module: row.get(0)?,
state_json: row.get(1)?,
updated_at_ms: row.get(2)?,
}))
} else {
Ok(None)
}
}
pub fn insert_auth_op(&self, op: &StoredAuthOp) -> Result<(), StoreError> {
self.conn.execute(
r#"INSERT OR REPLACE INTO auth_ops(op_id, resource_id, op_json, created_at_ms)
@ -846,6 +871,20 @@ impl Store {
.map_err(StoreError::from)
}
pub fn list_ssh_cert_requests_since(
&self,
since_ms: i64,
) -> Result<Vec<StoredSshCertRequest>, StoreError> {
let mut stmt = self.conn.prepare(
r#"SELECT request_id, requester_node, public_key, public_key_fingerprint, cert_kind,
principals_json, requested_validity, renewal_of, reason, status, created_at_ms
FROM ssh_cert_requests WHERE created_at_ms >= ?1 ORDER BY created_at_ms, request_id"#,
)?;
let rows = stmt.query_map(params![since_ms], stored_ssh_cert_request_from_row)?;
rows.collect::<Result<Vec<_>, _>>()
.map_err(StoreError::from)
}
pub fn insert_ssh_certificate(
&self,
certificate: &StoredSshCertificate,
@ -883,6 +922,27 @@ impl Store {
.map_err(StoreError::from)
}
pub fn list_ssh_certificates_since(
&self,
since_ms: i64,
) -> Result<Vec<StoredSshCertificate>, StoreError> {
let mut stmt = self.conn.prepare(
r#"SELECT cert_id, request_id, certificate, certificate_fingerprint, imported_at_ms
FROM ssh_certificates WHERE imported_at_ms >= ?1 ORDER BY imported_at_ms, cert_id"#,
)?;
let rows = stmt.query_map(params![since_ms], |row| {
Ok(StoredSshCertificate {
cert_id: row.get(0)?,
request_id: row.get(1)?,
certificate: row.get(2)?,
certificate_fingerprint: row.get(3)?,
imported_at_ms: row.get(4)?,
})
})?;
rows.collect::<Result<Vec<_>, _>>()
.map_err(StoreError::from)
}
pub fn insert_ssh_revocation(
&self,
revocation: &StoredSshRevocation,
@ -921,6 +981,28 @@ impl Store {
rows.collect::<Result<Vec<_>, _>>()
.map_err(StoreError::from)
}
pub fn list_ssh_revocations_since(
&self,
since_ms: i64,
) -> Result<Vec<StoredSshRevocation>, StoreError> {
let mut stmt = self.conn.prepare(
r#"SELECT revocation_id, kind, target, reason, created_at_ms, published
FROM ssh_revocations WHERE created_at_ms >= ?1 ORDER BY created_at_ms, revocation_id"#,
)?;
let rows = stmt.query_map(params![since_ms], |row| {
Ok(StoredSshRevocation {
revocation_id: row.get(0)?,
kind: row.get(1)?,
target: row.get(2)?,
reason: row.get(3)?,
created_at_ms: row.get(4)?,
published: row.get::<_, i64>(5)? != 0,
})
})?;
rows.collect::<Result<Vec<_>, _>>()
.map_err(StoreError::from)
}
}
fn stored_ssh_cert_request_from_row(
@ -1064,6 +1146,13 @@ pub struct StoredPeerCard {
pub updated_at_ms: i64,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct StoredModuleState {
pub module: String,
pub state_json: String,
pub updated_at_ms: i64,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct StoredAuthOp {
pub op_id: String,
@ -1173,6 +1262,18 @@ mod tests {
.expect("insert revocation");
assert_eq!(
store.list_ssh_revocations().expect("list revocations"),
vec![revocation.clone()]
);
assert_eq!(
store
.list_ssh_cert_requests_since(0)
.expect("list requests since"),
vec![request]
);
assert_eq!(
store
.list_ssh_revocations_since(1)
.expect("list revocations since"),
vec![revocation]
);
}
@ -1193,6 +1294,23 @@ mod tests {
assert_eq!(store.list_peer_cards().expect("list"), vec![peer_card]);
}
#[test]
fn module_state_roundtrip() {
let store = Store::open_memory().expect("open");
let state = StoredModuleState {
module: "live-sync:node:laptop:ssh-certs".to_owned(),
state_json: r#"{"cursor_ms":42}"#.to_owned(),
updated_at_ms: 43,
};
store.put_module_state(&state).expect("put state");
assert_eq!(
store
.get_module_state("live-sync:node:laptop:ssh-certs")
.expect("get state"),
Some(state)
);
}
#[test]
fn auth_ops_roundtrip_by_resource() {
let store = Store::open_memory().expect("open");

8
docs/architecture.md
View file

@ -84,7 +84,11 @@ revocations are stored as signed-list-ready records. The bootstrap can pull
certificate-flow metadata over Iroh with `geth ssh cert sync <node-id>` when the
peer grants `ssh_cert.sync` on `resource:ssh:certs`, and revocation metadata with
`geth ssh revocation sync <node-id>` when the peer grants `ssh_revocation.sync`
on `resource:ssh:revocations`.
on `resource:ssh:revocations`. The daemon also runs a 30-second background
live-sync tick for known peers and records per-peer high-water cursors in
`module_state`, so repeated ticks request only records at or beyond the last
remote cursor. Boundary duplicates are harmless because records are keyed by
stable IDs and inserted with replace semantics.
## Resource Model
@ -169,6 +173,8 @@ not enumerable through OpenSSH tooling, so geth treats binary import as
unsupported and asks for JSONL or the spec source. Revocation lists are not yet
full CRDT-replicated resources, but the daemon can already pull cert-flow and
revocation metadata from authorized peers over the protected Iroh control ALPN.
Manual sync commands and the background live-sync loop share the same capability
checks and cursor state.
## Keychain, Auth, And Secrets

6
docs/roadmap.md
View file

@ -214,6 +214,10 @@ resource-scoped capability decisions.
`geth ssh cert sync <node-id>`.
- `[x]` Authorized peers can pull SSH revocation metadata with
`geth ssh revocation sync <node-id>`.
- `[x]` The daemon background live-sync loop refreshes known peers without a
manual command.
- `[x]` SSH metadata live-sync stores per-peer high-water cursors in
`module_state` and requests only records at or beyond the cursor.
- `[ ]` Future completion requires auth checks for local request, approve,
import, publish, and read capabilities.
@ -318,6 +322,8 @@ Goal: add authorized stream-oriented management workflows over Iroh.
`resource:ssh:revocations`.
- `[x]` Consumers can list current certs/revocations from local state while
offline after sync.
- `[x]` Background live-sync uses the same protected Iroh path and cursor
state as manual sync.
- `[ ]` Replace pull-only metadata sync with a resource log or CRDT model.
- `[ ]` Conflicting or unsigned records are rejected or quarantined.