geth/docs/architecture.md

# Architecture

`geth` is a single-binary local-first mesh runtime. One executable provides both
daemon mode and control mode. The daemon owns local identity, metadata storage,
the shared Iroh endpoint, resource registry, module routing, and local control
socket. Control commands connect to the Unix socket and send typed JSONL
requests.

Service management is also exposed through the single binary. `geth daemon
service ...` installs and controls a user-level service definition for the local
daemon. The initial backends are systemd user units on Linux, launchd user agents
on macOS, and per-user scheduled tasks on Windows. Geth does not install itself
as a privileged system service.

## Iroh-Only Remote Communication

Remote geth node-to-node communication is Iroh-only. The daemon will own one
shared Iroh endpoint and register module protocols on ALPNs such as
`/geth/cas/1`, `/geth/kv/1`, `/geth/pipe/1`, and `/geth/ssh-proxy/1`.

The first pinned Iroh integration uses `iroh = 0.90.0`, because newer
Rust-1.85-compatible candidates in the 0.93-0.95 range failed to compile through
a transitive `ed25519-dalek` prerelease dependency. `geth-iroh` wraps
`iroh::Endpoint::builder()`, configures geth ALPNs with `Builder::alpns`, uses
`Builder::relay_mode`, persists an `iroh::SecretKey` as hex-encoded 32-byte key
material, and shuts down through `Endpoint::close().await`. The default config
uses Iroh's default relay policy; local-only/offline development can set
`[iroh].relay_mode = "disabled"`. Named custom relay maps are configured under
`[iroh.relay_maps.<name>]`, selected with `relay_mode = "custom"` plus
`relay_map = "<name>"`, validated at config load, and reported in status as
`custom:<name>` without exposing relay URLs.

Module ALPNs are registered through `geth-iroh`'s protocol router scaffold. The
router owns the default protocol descriptors, rejects duplicate ALPN
registrations, and returns explicit unknown-ALPN errors. It does not yet accept
or dispatch remote streams; peer authentication and module handlers are later
Phase 1 work.

The target product should use Iroh relay support for practical internet
connectivity and mDNS/LAN discovery for local networks. These are connectivity
and candidate-discovery mechanisms only. They do not grant trust, mutate
authorization state, or make EndpointID knowledge sufficient for access.
The current daemon can enable Iroh's local-network discovery service through
`[iroh].local_discovery = true`, which is the default. This publishes and
discovers Iroh node addressing. `geth peer export/import/list` supports manual
exchange of signed peer cards as untrusted candidates. Peer cards include the
Iroh EndpointID plus relay/direct address candidates when the daemon can observe
them. `geth peer ping <node-id>` dials an imported peer card over Iroh and
exchanges signed peer-card metadata. `geth peer auth-check <node-id>
<resource> <capability>` sends a protected Iroh control request that validates
the caller's signed peer card against the actual Iroh EndpointID before
evaluating resource-local capabilities. When local discovery is enabled, the
daemon also advertises and discovers signed peer cards through a geth-specific
mDNS service. The LAN payload is TXT-encoded signed metadata only; remote geth
traffic still uses Iroh.

Peer cards are the discovery payload. A peer card carries node ID, agent ID,
endpoint candidates, timestamp, signing public key, and an Ed25519 signature
over a canonical payload. Imported and ping-discovered peer cards are stored as
untrusted metadata in `peer_cards`; trust reduction is future work. `auth
explain` reports when a subject is only a discovered peer candidate and denies
access. The peer ping path authenticates the Iroh endpoint and peer-card
signature, but it does not authorize any resource module. Protected peer
control requests must also prove that the signed peer card binds the observed
Iroh EndpointID, then reduce resource auth ops; an EndpointID alone is not
accepted as a resource principal.

The daemon starts this endpoint during `geth daemon run` and keeps it alive for
the daemon lifetime. When endpoint startup succeeds, the Iroh EndpointID is
recorded as a transport binding for the stable geth node identity. If local UDP
binding is unavailable, the daemon keeps local control running and reports the
Iroh startup error through status output.

SSH keys are not transport keys. They are admin trust anchors and signing
identities for keychain and authorization operations. The bootstrap `geth ssh
proxy <node-id>` command performs an authorized Iroh control-plane handshake:
the remote daemon validates the caller's signed peer card against the observed
Iroh EndpointID and requires `ssh_proxy.connect` on
`resource:ssh-proxy:local`. It returns connection metadata only. Carrying SSH
bytes over an Iroh stream is implemented on the dedicated `/geth/ssh-proxy/1`
ALPN and only connects to remote `127.0.0.1:22` after authorization.
`geth ssh admin-shell <node-id> <help|status|node-id>` is a separate restricted
geth admin workflow over the protected Iroh control path. It requires
`ssh_proxy.admin_shell` and executes only built-in geth commands, never host
shell commands. Neither path makes SSH a geth transport backend.

SSH certificate flows use the same split. Nodes can request new OpenSSH
certificates or renewals through geth metadata. A machine with the CA key or
YubiKey can approve the request and run an explicit `ssh-keygen -s ...` command,
or pass `--sign` to execute that command immediately and import the resulting
certificate into local metadata for distribution. This relies on the local
OpenSSH ecosystem, so hardware-backed keys remain mediated by `ssh-keygen` and
the host's agent/security-key flow. Certificate and key revocations are stored
as signed-list-ready records. The bootstrap can pull
certificate-flow metadata over Iroh with `geth ssh cert sync <node-id>` when the
peer grants `ssh_cert.sync` on `resource:ssh:certs`, and revocation metadata with
`geth ssh revocation sync <node-id>` when the peer grants `ssh_revocation.sync`
on `resource:ssh:revocations`. The daemon also runs a configurable background
live-sync tick for known peers and records per-peer high-water cursors in
`module_state`, so repeated ticks request only records at or beyond the last
remote cursor. The default interval is 30 seconds and can be changed under
`[sync]` in `config.toml`. Boundary duplicates are harmless because records are
keyed by stable IDs and inserted with replace semantics.
Before issuing per-module pulls, the daemon can request an authorized sync
status summary over the same protected Iroh control ALPN. The serving peer
validates endpoint/card binding and returns only watermarks for streams where
the caller already has the required resource capability, which reduces blind
polling without letting discovery reveal private resource names.
Keychain and auth operation logs are also advertised through this watermark
path. Pulls are delta-style by per-peer cursor, but every received operation is
still verified against trusted-admin OpenSSH signatures before import.
Operators can run `geth sync now [node]` to trigger the same best-effort pass
immediately and `geth sync status` to inspect locally recorded last-attempt,
last-success, cursor, import/rejection counts, and errors for each peer stream.

## Resource Model

Everything meaningful is modeled as a resource. Resources have a kind, name,
authority reference, local role, replication policy, retention policy, and
status. Authorization is resource-scoped and capability-based.

Resource kinds:

- `db`: cr-sqlite-backed SQLite synchronization
- `kv`: Iroh Documents backed key-value data
- `pipe`: authorized byte streams and forwarding
- `document`: Automerge CRDT documents
- `pubsub`: lossy notifications and presence
- `cas`: content-addressed blobs
- `ssh-proxy`: SSH/admin proxying over Iroh

## Module Overview

`geth-cas` is implemented locally first using BLAKE3 hashes and filesystem blob
storage. Local pin/unpin metadata is tracked in SQLite and surfaced in
`cas list`. `cas cleanup` removes unpinned local blobs while retaining pinned
blobs. The CAS crate can build deterministic tree objects that describe
directories, files, executable bits, and file blob hashes; those tree objects
are stored as CAS blobs. The daemon can register local file roots and scan them
into CAS tree objects while reporting create/update/delete/rename changes. These
scans are local metadata only and never overwrite the working tree. A peer can
pull authorized remote file-root tree metadata with `geth cas root sync <node>
<name>` when it has `cas.fetch` on `resource:cas-tree:<name>`; sync imports the
remote CAS tree bytes and records a peer-qualified remote root whose path is
`remote:<node>:<name>` without applying files. File roots also participate in
the daemon background live-sync loop through authorized `cas-tree:<name>`
watermarks. Repeated remote file-root syncs retain the previous imported remote
tree as the base and compare base/local/remote tree state. When both local and
remote roots changed, geth records durable concurrent edit, delete/edit, or
divergent rename conflicts instead of applying remote content. `geth cas root
apply <root> --to <path>` is the first conservative
materialization path: it creates missing files and directories from the CAS tree,
does not delete extra local files, does not overwrite differing local files, and
records conflicts for manual resolution. The daemon also has durable local
file-conflict records with explicit resolution choices; richer three-way apply
will extend those records instead of silently applying ambiguous remote changes.

As a bootstrap network path, `geth cas fetch <node-id> <hash>` dials an
imported signed peer card over the daemon-owned Iroh control ALPN. The serving
daemon validates the caller's peer-card signature and observed Iroh EndpointID,
then reduces local auth ops and requires `cas.fetch` on `resource:cas:local`
before returning blob bytes. The requester verifies that the returned bytes hash
to the requested BLAKE3 CAS hash before storing them. Successful fetches update
durable local provider metadata keyed by CAS hash and peer node, which can be
inspected through `geth cas providers <hash>`. Iroh-blobs provider/fetch
integration and richer three-way file application are future work.

`geth-db` currently registers local SQLite paths as DB resources and reports
local-only sync status plus a read-only SQLite schema summary/hash. It also
inspects `crsql_changes` metadata when that table or view exists, reporting
change count, columns, and max `db_version`. The crate and local daemon can
extract read-only typed change batches from `crsql_changes` with schema metadata
through `db changes`. As a staged network path, `geth db sync <node-id> <name>`
uses the protected Iroh control ALPN to request remote typed change batches when
the caller has `db.sync` on the remote `resource:db:<name>`. The requester
checks remote schema metadata against its local DB before applying changes and
advancing its per-peer/per-DB cursor. Compatible remote batches are inserted
into the local `crsql_changes` table or view before the cursor advances.
Loading/configuring the cr-sqlite extension for real application databases
remains the database owner's responsibility; the bootstrap tests use
deterministic fixture tables.

`geth-kv` currently provides a SQLite-backed local fallback for named KV stores
through `kv create/set/get`. `kv set --subject <principal>` evaluates local auth
ops for `kv.write_key:<key>` so prefix grants can be tested before networked
callers exist. The local node/agent retains owner access for administration.
Iroh Documents namespaces remain the target backend, but the bootstrap can sync
named KV stores over the protected Iroh control ALPN. `geth kv sync <node-id>
<name>` requires `kv.read` on the remote `resource:kv:<name>`, transfers entries
at or beyond a per-peer/per-KV high-water cursor, and imports only values that
are at least as new as the local entry timestamp. The daemon background
live-sync loop runs the same KV sync for local KV stores and known peers.
Private value encryption should use resource secret epochs before payloads are
exposed to remote peers.

`geth-document` currently registers local document resources and stores
validated JSON state in the local SQLite metadata store through
`document create/status/set/get`. This is a bootstrap editing surface, not yet
Automerge CRDT state. `geth document sync <node-id> <name>` can pull remote JSON
state over the protected Iroh control ALPN when the peer grants `document.read`
on `resource:document:<name>`. The daemon background live-sync loop runs the same
sync for local documents and known peers using per-peer/per-document cursors.
The import rule is last-writer-wins by document timestamp. Automerge state
encoding and sync are future work.

`geth-pubsub` currently supports local publish/subscribe snapshots through the
daemon control protocol. Messages live in a bounded in-memory ring buffer and
are lost when the daemon stops. This is deliberate: pubsub is a lossy wakeup and
presence channel, not authoritative storage. Durable facts must be written to
CAS, KV, document, or DB resources before pubsub is used as a wakeup. `geth
pubsub pub <topic> <message> --node <node-id>` can publish to an imported peer
over the protected Iroh control ALPN. The remote daemon validates endpoint/card
binding and requires
`pubsub.publish` on `resource:pubsub:<topic>` before recording the message in
its local ring buffer. `geth pubsub sub <topic> --node <node-id>` can read an
authorized peer's current snapshot for that topic over the same protected path
when the caller has `pubsub.subscribe` on `resource:pubsub:<topic>`. Iroh-gossip
replication and private topics are future work.

`geth-pipe` currently supports `pipe listen/connect/send/recv` against a
daemon-lifetime runtime. `geth pipe connect <name> --node <node-id>` sends an
authorized remote connect request over the protected Iroh control ALPN. The
remote daemon validates endpoint/card binding and requires `pipe.connect` on
`resource:pipe:<name>` before recording the connection attempt and reporting
whether a listener exists. `geth pipe send <name> [message|--in <path>|--in -] --node <node-id>`
uses the dedicated `/geth/pipe/1` ALPN to write a byte message to a peer
listener after the same endpoint/card and capability checks. `geth pipe recv
<name>` drains local daemon-lifetime messages. `geth pipe listen <name> --node
<node-id>` can also ask a peer to register a daemon-lifetime listener after
checking `pipe.listen` on the same resource. `geth pipe forward-tcp --listen
127.0.0.1:<local-port> --node <node-id> --target 127.0.0.1:<remote-port>` runs a
local loopback listener and opens one authorized `/geth/pipe/1` byte stream per
accepted connection. The remote daemon validates endpoint/card binding and
requires `pipe.forward` on `resource:pipe-tcp:<target>` before connecting to the
remote loopback TCP target. TCP forwarding is loopback-only in the prototype;
`geth pipe forward-unix --listen <local-socket> --node <node-id> --target
<remote-socket>` uses the same authorized Iroh pipe stream and requires
`pipe.forward` on `resource:pipe-unix:<target>` before connecting to an absolute
remote Unix socket path.

`geth-ssh-proxy` defines proxy target and connection metadata. `geth ssh proxy
<node>` is a streaming command intended for OpenSSH `ProxyCommand`: the CLI
streams stdin/stdout through the local daemon, the local daemon dials the remote
daemon with `/geth/ssh-proxy/1`, the remote daemon validates the signed peer card
against the observed Iroh EndpointID, reduces `ssh_proxy.connect` on
`resource:ssh-proxy:local`, and only then connects the Iroh stream to
`127.0.0.1:22`. OpenSSH still performs its normal login authentication over the
resulting byte stream. SSH is not used as a geth transport backend.

`geth-ssh-identity` defines SSH trust namespaces plus certificate request,
approval, certificate import, and revocation-list data models. The bootstrap
persists these flows locally and exports revocations as JSONL or OpenSSH KRL
specification text. Certificate approval normally emits the exact
`ssh-keygen -s ...` command, and `approve --sign` can run that command, import
the resulting OpenSSH certificate, and mark the request signed. It can also
invoke `ssh-keygen -k` to produce a binary OpenSSH KRL; serial and key-ID KRL
entries require a CA public key via `--ca-public`, matching OpenSSH behavior.
It can import geth JSONL revocation
exports and OpenSSH KRL specification source files. Binary OpenSSH KRL files are
not enumerable through OpenSSH tooling, so geth treats binary import as
unsupported and asks for JSONL or the spec source. Revocation lists are not yet
full CRDT-replicated resources, but the daemon can already pull cert-flow and
revocation metadata from authorized peers over the protected Iroh control ALPN.
Manual sync commands and the background live-sync loop share the same capability
checks and cursor state. Sync import rejects conflicting records with ids that
already exist locally instead of replacing local metadata. Local SSH certificate
and revocation metadata commands also accept an optional subject principal for
authorization testing: non-owner subjects must hold `ssh_cert.*` capabilities on
`resource:ssh:certs` or `ssh_revocation.*` capabilities on
`resource:ssh:revocations` before requests, approval/import/read operations, or
revocation publish/read/import operations
are accepted. The live-sync loop first asks for authorized stream watermarks and
skips module pulls whose remote high-water value has not advanced. File-root
live-sync uses only sync-status-advertised `cas-tree:<name>` streams, because
the local node otherwise does not know which roots the peer is willing to
expose.

## Keychain, Auth, And Secrets

The identity plane is `geth-keychain`: admin keys, users, devices, nodes, agents,
and endpoint bindings. Endpoint rotation must not destroy higher-level node
identity. Keychain operations reduce into an active view containing current
admin keys, users, devices, node records, agent bindings, and endpoint-to-node
bindings. Revoked identity subtrees are excluded from that active view. `geth
init --admin-key <pub> --signing-key <key> --node-name <name>` records an
owner/admin key, user, device, node, and agent binding as keychain operations
and signs them with OpenSSH under `geth.keychain.v1@geth.local`. Both keys are
required when owner setup options are used, so the node does not create unsigned
owner statements by accident. `geth node list` shows the active reduced node
view. `geth node rename` and `geth node revoke` record signed keychain
operations and require `--signing-key`. Endpoint rotation is explicit:
`geth node endpoint-add` and `geth node endpoint-revoke` record signed
`NodeEndpointAdd` and `NodeEndpointRevoke` keychain operations. `geth keychain
sync <node>` pulls keychain operations and signatures from an imported peer over
Iroh and imports only operations with a valid OpenSSH signature from a currently
trusted admin key over the canonical payload. This is currently a pull-based
signed operation log, not a CRDT or Keyhive-style convergent authority.

New devices can use the node enrollment flow instead of hand-editing keychain
state. `geth node enroll request` creates a canonical, agent-key-signed request
containing the requesting node ID, agent ID, requested node name, optional Iroh
endpoint, and requested resource capabilities. The request can be submitted over
Iroh to an imported owner peer or moved as a JSON file to the owner machine.
`geth node enroll approve` runs on the owner/YubiKey machine and records signed
keychain operations for the device/node/agent/endpoint binding plus signed auth
operations for approved capabilities. `geth node enroll sync <owner-node>` pulls
both signed logs so the new node can see its approved identity and permissions.

The authorization plane is `geth-auth`: resource-local signed operation logs,
grants, revocations, groups, and `auth explain`. Auth operations reduce into a
current permission view for resources, grants, groups, and bearer access. The
library can explain direct and group grants. The daemon persists local auth
grant/revoke operations and `geth auth explain` evaluates that local operation
log. `geth node grant`, `geth node revoke-grant`, `geth auth grant`, and `geth
auth revoke` require `--signing-key` in the CLI and store OpenSSH-signed auth
operations. Enrollment approval uses the same signed auth operation path. Auth
sync imports only auth operations signed by currently trusted admin keys.
Broader delegated authority and module enforcement are still future work.

Capability evaluation supports exact matches plus explicit scoped forms. For KV,
`kv.write_prefix:<prefix>` grants writes requested as `kv.write_key:<key>` only
when the key is under that prefix; `kv.write` remains the broad write
capability. Command-level KV enforcement is still future work.

Both keychain and auth operations use `geth-codec` canonical envelopes for
signature payloads. The envelope includes a version, an explicit signature
namespace, and the operation payload encoded with postcard. JSON remains useful
for CLI/control output, but it is not the signed representation.

The payload access plane is `geth-secrets`: resource master secrets, epochs,
key envelopes, bearer secrets, and rotation. Revocation for private data is
modeled initially as secret epoch rotation. The daemon persists resource secret
epoch metadata through `secret create/rotate/status`. Bearer access is recorded
as resource-scoped auth operations and rejects trust-mutation capabilities such
as `auth.delegate`, `auth.revoke`, and `node.enroll`. Bearer creation returns a
private bearer token once and stores a separate public bearer id plus token
verifier in the auth log. Bearer challenge/proof commands derive deterministic
BLAKE3 keyed responses from the private token, resource, nonce, and requested
capabilities, then verify them against active resource-scoped bearer grants.
Remote resource operations can carry optional bearer proofs over the protected
Iroh control path; a valid proof authorizes only the requested resource
capability and does not create node trust. The daemon does not yet store payload
key material, encrypt resource data, or distribute key envelopes.

## Multi-User Direction

The project is structured for future multi-user local-first authorization:

- authorization is replicated data, not one mutable ACL blob
- resources can carry or delegate to their own auth state
- users, devices, nodes, agents, and endpoints are separate principals
- capabilities are the underlying permission unit
- bearer access is resource-scoped and does not mutate the trust graph
- offline revocation is eventual
- encryption key distribution is part of authorization

## Keyhive/BeeKEM Roadmap

Resource secret epochs are the v0/v1 approximation for private payload access.
Later designs can add Keyhive-like convergent capabilities and BeeKEM/CGKA-style
group key evolution. The bootstrap does not implement BeeKEM and does not claim
strong forward secrecy or post-compromise security.

## Security Invariants

- All remote node-to-node communication is over Iroh.
- SSH is not a geth transport.
- SSH keys are admin trust anchors and signing identities.
- Agent/node keys handle routine local identity.
- Discovery is untrusted.
- Knowing an EndpointID does not grant access.
- Bearer secrets are resource-scoped capabilities.
- Bearer access does not imply trust graph mutation rights.
- Authorization is capability-based and resource-scoped.
- SSH certificate issuance must be explicitly approved by an authorized
  principal before signing.
- SSH certificate and key revocations are durable metadata that should be
  distributed over Iroh, not fetched through unauthenticated discovery.
- Network and control decoders treat input as untrusted.
- Service installation targets user service managers, not system service
  managers.