Make live sync configurable

This commit is contained in:
Eric Wendland 2026-05-20 13:34:16 +02:00
commit 3eb6b623e7
6 changed files with 106 additions and 14 deletions

View file

@ -204,9 +204,10 @@ Roadmap items should be actionable and checkable:
`resource:ssh:revocations` for non-owner subjects. Authorized peers can pull `resource:ssh:revocations` for non-owner subjects. Authorized peers can pull
SSH certificate-flow metadata with `ssh_cert.sync` on `resource:ssh:certs` and SSH certificate-flow metadata with `ssh_cert.sync` on `resource:ssh:certs` and
revocation metadata with `ssh_revocation.sync` on revocation metadata with `ssh_revocation.sync` on
`resource:ssh:revocations`. The daemon live-syncs known peers every 30 `resource:ssh:revocations`. The daemon live-syncs known peers using per-peer
seconds using per-peer cursors. Conflicting records with already-known ids are cursors; `[sync] live_sync_enabled` and `live_sync_interval_ms` in
rejected during sync import; this is pull-only metadata sync, not yet a `config.toml` control that loop. Conflicting records with already-known ids
are rejected during sync import; this is pull-only metadata sync, not yet a
signed CRDT/resource-log replication model. signed CRDT/resource-log replication model.
- cr-sqlite apply, iroh-docs, iroh-blobs provider/fetch, Automerge sync, - cr-sqlite apply, iroh-docs, iroh-blobs provider/fetch, Automerge sync,
broader auth enforcement, and Keyhive/BeeKEM-style authorization are future broader auth enforcement, and Keyhive/BeeKEM-style authorization are future

View file

@ -170,10 +170,12 @@ at the peer. `geth ssh revocation sync <node-id>` requires
`ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge `ssh_revocation.sync` on `resource:ssh:revocations`. Both commands merge
authorized peer metadata into the local store for offline listing and later authorized peer metadata into the local store for offline listing and later
approval/signing workflows. While the daemon is running, it also performs a approval/signing workflows. While the daemon is running, it also performs a
background live-sync tick for known peers every 30 seconds. Live-sync stores background live-sync tick for known peers. The default interval is 30 seconds
per-peer high-water cursors in local metadata so repeated ticks request only and can be changed in `config.toml` with `[sync] live_sync_enabled` and
newer SSH certificate-flow and revocation records. Sync import preserves local `live_sync_interval_ms`. Live-sync stores per-peer high-water cursors in local
metadata by rejecting conflicting records with ids that already exist locally. metadata so repeated ticks request only newer SSH certificate-flow and
revocation records. Sync import preserves local metadata by rejecting
conflicting records with ids that already exist locally.
Before probing individual modules, the daemon asks the peer for an authorized Before probing individual modules, the daemon asks the peer for an authorized
sync-status summary over Iroh. The peer only returns stream watermarks for sync-status summary over Iroh. The peer only returns stream watermarks for
resources where the caller already has the matching capability, letting the resources where the caller already has the matching capability, letting the

View file

@ -13,6 +13,10 @@ local_discovery = true
# #
# [iroh.relay_maps.home] # [iroh.relay_maps.home]
# urls = ["https://relay.example.com"] # urls = ["https://relay.example.com"]
#
[sync]
live_sync_enabled = true
live_sync_interval_ms = 30000
"#; "#;
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
@ -116,11 +120,16 @@ pub enum ConfigError {
EmptyRelayMap(String), EmptyRelayMap(String),
#[error("invalid iroh relay URL in map `{map}`: {url}")] #[error("invalid iroh relay URL in map `{map}`: {url}")]
InvalidRelayUrl { map: String, url: String }, InvalidRelayUrl { map: String, url: String },
#[error("invalid sync live_sync_enabled value: {0}")]
InvalidLiveSyncEnabled(String),
#[error("invalid sync live_sync_interval_ms value: {0}")]
InvalidLiveSyncInterval(String),
} }
#[derive(Clone, Debug, Default, PartialEq, Eq)] #[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct GethConfig { pub struct GethConfig {
pub iroh: IrohConfig, pub iroh: IrohConfig,
pub sync: SyncConfig,
} }
impl GethConfig { impl GethConfig {
@ -140,6 +149,7 @@ impl GethConfig {
let document = text.parse::<toml_edit::DocumentMut>()?; let document = text.parse::<toml_edit::DocumentMut>()?;
let relay_maps = parse_relay_maps(&document)?; let relay_maps = parse_relay_maps(&document)?;
let local_discovery = parse_local_discovery(&document)?; let local_discovery = parse_local_discovery(&document)?;
let sync = parse_sync_config(&document)?;
let relay_mode = match document.get("iroh").and_then(|iroh| iroh.get("relay_mode")) { let relay_mode = match document.get("iroh").and_then(|iroh| iroh.get("relay_mode")) {
Some(item) => { Some(item) => {
let value = item let value = item
@ -156,6 +166,7 @@ impl GethConfig {
relay_maps, relay_maps,
local_discovery, local_discovery,
}, },
sync,
}) })
} }
} }
@ -177,6 +188,21 @@ impl Default for IrohConfig {
} }
} }
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct SyncConfig {
pub live_sync_enabled: bool,
pub live_sync_interval_ms: u64,
}
impl Default for SyncConfig {
fn default() -> Self {
Self {
live_sync_enabled: true,
live_sync_interval_ms: 30_000,
}
}
}
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct RelayMapConfig { pub struct RelayMapConfig {
pub urls: Vec<String>, pub urls: Vec<String>,
@ -297,6 +323,37 @@ fn parse_local_discovery(document: &toml_edit::DocumentMut) -> Result<bool, Conf
} }
} }
fn parse_sync_config(document: &toml_edit::DocumentMut) -> Result<SyncConfig, ConfigError> {
let live_sync_enabled = match document
.get("sync")
.and_then(|sync| sync.get("live_sync_enabled"))
{
Some(item) => item
.as_bool()
.ok_or_else(|| ConfigError::InvalidLiveSyncEnabled(item.to_string()))?,
None => true,
};
let live_sync_interval_ms = match document
.get("sync")
.and_then(|sync| sync.get("live_sync_interval_ms"))
{
Some(item) => {
let interval = item
.as_integer()
.ok_or_else(|| ConfigError::InvalidLiveSyncInterval(item.to_string()))?;
if interval < 100 {
return Err(ConfigError::InvalidLiveSyncInterval(interval.to_string()));
}
interval as u64
}
None => 30_000,
};
Ok(SyncConfig {
live_sync_enabled,
live_sync_interval_ms,
})
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@ -306,6 +363,8 @@ mod tests {
let config = GethConfig::parse(GethConfig::default_toml()).expect("parse config"); let config = GethConfig::parse(GethConfig::default_toml()).expect("parse config");
assert_eq!(config.iroh.relay_mode, RelayMode::Default); assert_eq!(config.iroh.relay_mode, RelayMode::Default);
assert!(config.iroh.local_discovery); assert!(config.iroh.local_discovery);
assert!(config.sync.live_sync_enabled);
assert_eq!(config.sync.live_sync_interval_ms, 30_000);
} }
#[test] #[test]
@ -381,6 +440,28 @@ mod tests {
assert!(!config.iroh.local_discovery); assert!(!config.iroh.local_discovery);
} }
#[test]
fn config_parses_live_sync_settings() {
let config =
GethConfig::parse("[sync]\nlive_sync_enabled = false\nlive_sync_interval_ms = 250\n")
.expect("parse");
assert!(!config.sync.live_sync_enabled);
assert_eq!(config.sync.live_sync_interval_ms, 250);
}
#[test]
fn config_rejects_invalid_live_sync_settings() {
let error = GethConfig::parse("[sync]\nlive_sync_enabled = \"yes\"\n").expect_err("error");
assert!(matches!(error, ConfigError::InvalidLiveSyncEnabled(_)));
let error = GethConfig::parse("[sync]\nlive_sync_interval_ms = 99\n").expect_err("error");
assert!(matches!(error, ConfigError::InvalidLiveSyncInterval(_)));
let error =
GethConfig::parse("[sync]\nlive_sync_interval_ms = \"fast\"\n").expect_err("error");
assert!(matches!(error, ConfigError::InvalidLiveSyncInterval(_)));
}
#[test] #[test]
fn config_rejects_non_bool_local_discovery() { fn config_rejects_non_bool_local_discovery() {
let error = GethConfig::parse("[iroh]\nlocal_discovery = \"yes\"\n").expect_err("error"); let error = GethConfig::parse("[iroh]\nlocal_discovery = \"yes\"\n").expect_err("error");

View file

@ -153,7 +153,6 @@ struct PipeRuntime {
} }
const PUBSUB_RING_LIMIT: usize = 256; const PUBSUB_RING_LIMIT: usize = 256;
const LIVE_SYNC_INTERVAL: Duration = Duration::from_secs(30);
const PIPE_CONNECTION_RING_LIMIT: usize = 256; const PIPE_CONNECTION_RING_LIMIT: usize = 256;
#[derive(Debug, serde::Deserialize, serde::Serialize)] #[derive(Debug, serde::Deserialize, serde::Serialize)]
@ -205,7 +204,13 @@ pub async fn run_daemon(paths: GethPaths) -> Result<(), NodeError> {
.clone() .clone()
{ {
spawn_iroh_control_accept_loop(node.clone(), endpoint); spawn_iroh_control_accept_loop(node.clone(), endpoint);
spawn_background_live_sync(node.clone()); let config = GethConfig::load(&node.paths.config_file())?;
if config.sync.live_sync_enabled {
spawn_background_live_sync(
node.clone(),
Duration::from_millis(config.sync.live_sync_interval_ms),
);
}
} }
let _peer_card_lan_discovery = start_peer_card_lan_discovery(&node).await; let _peer_card_lan_discovery = start_peer_card_lan_discovery(&node).await;
if Path::new(&paths.socket_path()).exists() { if Path::new(&paths.socket_path()).exists() {
@ -2134,12 +2139,12 @@ fn spawn_iroh_control_accept_loop(node: LocalNode, endpoint: GethIrohEndpoint) {
}); });
} }
fn spawn_background_live_sync(node: LocalNode) { fn spawn_background_live_sync(node: LocalNode, interval_duration: Duration) {
tokio::spawn(async move { tokio::spawn(async move {
if let Err(error) = run_live_sync_once(&node).await { if let Err(error) = run_live_sync_once(&node).await {
tracing::debug!(%error, "initial live sync tick failed"); tracing::debug!(%error, "initial live sync tick failed");
} }
let mut interval = tokio::time::interval(LIVE_SYNC_INTERVAL); let mut interval = tokio::time::interval(interval_duration);
loop { loop {
interval.tick().await; interval.tick().await;
if let Err(error) = run_live_sync_once(&node).await { if let Err(error) = run_live_sync_once(&node).await {

View file

@ -91,11 +91,12 @@ as signed-list-ready records. The bootstrap can pull
certificate-flow metadata over Iroh with `geth ssh cert sync <node-id>` when the certificate-flow metadata over Iroh with `geth ssh cert sync <node-id>` when the
peer grants `ssh_cert.sync` on `resource:ssh:certs`, and revocation metadata with peer grants `ssh_cert.sync` on `resource:ssh:certs`, and revocation metadata with
`geth ssh revocation sync <node-id>` when the peer grants `ssh_revocation.sync` `geth ssh revocation sync <node-id>` when the peer grants `ssh_revocation.sync`
on `resource:ssh:revocations`. The daemon also runs a 30-second background on `resource:ssh:revocations`. The daemon also runs a configurable background
live-sync tick for known peers and records per-peer high-water cursors in live-sync tick for known peers and records per-peer high-water cursors in
`module_state`, so repeated ticks request only records at or beyond the last `module_state`, so repeated ticks request only records at or beyond the last
remote cursor. Boundary duplicates are harmless because records are keyed by remote cursor. The default interval is 30 seconds and can be changed under
stable IDs and inserted with replace semantics. `[sync]` in `config.toml`. Boundary duplicates are harmless because records are
keyed by stable IDs and inserted with replace semantics.
Before issuing per-module pulls, the daemon can request an authorized sync Before issuing per-module pulls, the daemon can request an authorized sync
status summary over the same protected Iroh control ALPN. The serving peer status summary over the same protected Iroh control ALPN. The serving peer
validates endpoint/card binding and returns only watermarks for streams where validates endpoint/card binding and returns only watermarks for streams where

View file

@ -254,6 +254,8 @@ resource-scoped capability decisions.
`geth ssh revocation sync <node-id>`. `geth ssh revocation sync <node-id>`.
- `[x]` The daemon background live-sync loop refreshes known peers without a - `[x]` The daemon background live-sync loop refreshes known peers without a
manual command. manual command.
- `[x]` Background live-sync can be disabled or retuned with `[sync]`
`live_sync_enabled` and `live_sync_interval_ms`.
- `[x]` SSH metadata live-sync stores per-peer high-water cursors in - `[x]` SSH metadata live-sync stores per-peer high-water cursors in
`module_state` and requests only records at or beyond the cursor. `module_state` and requests only records at or beyond the cursor.
- `[x]` Local SSH cert request/read/approve/import commands can enforce - `[x]` Local SSH cert request/read/approve/import commands can enforce