Draft 0.5: identity/registry spec; Phase A signed MA registry, mailbox challenge auth, key rotation, skew rejection

AGENTS.md

@@ -1,9 +1,9 @@
 # AGENTS.md
 
 ## Repo shape
-- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.4) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
+- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.5) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
 - `frxd` is the member node (init/add/index/serve/relay/query/status); `frx` is the thin client (search/query/status). Relay and node roles are separate subcommands.
-- Commands: `cargo build`, `cargo test` (68 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
+- Commands: `cargo build`, `cargo test` (80 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; registry `tests/registry.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
 - E2E pattern: relay + nodes in-process on ephemeral ports with tempdir corpora; use `tests/common/mod.rs` helpers (`spawn_relay*`, `query_envelope`, `poll_messages`, `register`) for new coverage. Raw relay polls return envelopes (payload under `body`), not response bodies.
 
 ## Editing the spec
@@ -14,25 +14,28 @@
 - §10 Open Issues are known gaps, not oversights (e.g., signature canonicalization blocks Phase-1 interop). Check it before "fixing" something.
 - Use the spec's vocabulary — member/querier/responder, aggregates, source/enrichment members — not client/server or search-engine terms.
 - A new `MUST` is only legitimate if it is observable at the boundary, deterministically verifiable by a peer, beneficial to the counterparty, and not derivable from local policy. Ranking/ordering/presentation fails this test and stays local (§5); scores never travel (I6).
+- I2 is not blanket anti-centralization: shared coordination (identity, admission, contract) is centralized in the MA because common state is cheaper held once; decisions that consume local information (matching, relevance, sharing, retention) stay local. Off-wire conduct (link handling, retention, gating) is contract, not conformance.
 
 ## Implementation notes
-- Envelope signing is provisional (`src/crypto.rs`): `FRX/0.4` + fields + sorted-key canonical JSON body. §10's signature canonicalization open issue is unsolved — never present this scheme as interoperable.
+- Draft 0.5 specifies JCS (RFC 8785) envelope signing over `{type, from, key, ts, nonce, body}` with `from` = MA-hosted identifier and `key` = pubkey. The code still signs the provisional `FRX/0.4` scheme (`src/crypto.rs`) until Phase B lands — never present current code as interoperable.
 - Built: envelope/query/response, Tantivy index, aggregates, member directory. Not built: dashboard UI, directory watching, TLS, lineage/delegation. Responses travel relay-mediated unicast; transport is HTTP long-poll, not SSE.
 - Economics is out of protocol scope (I4: aggregates advise, contracts govern): no receipt, citation, pricing, or settlement fields or message types exist or may be added.
-- Relay verifies signatures, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops.
+- Relay verifies signatures and ±300s timestamp skew, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops. Mailbox polls require proof of key possession: `GET /v1/challenge` then a signed single-use nonce, so knowing a pubkey is not enough to drain its queue.
 - Responder searches only collections marked shared (I9), stays silent when nothing matches, and emits results with honest `truncated`/`more_available` and no scores (I6). BM25 order is a local implementation detail, not protocol surface.
 - Index layout: Tantivy at `<data_dir>/index`, collections manifest at `<data_dir>/collections.toml`; `exposure` (metadata|full) gates whether `content` is returned.
 - Egress checks live in the responder path (`src/node.rs` `respond`), not the relay — keep private collections unreachable there.
 - Appendix B is executable policy: `tests/purges.rs` has one absence test per rejected mechanism (12 rows). Add a negative test there before ever re-proposing one, and only if the rationale is addressed.
 - `add`/`reindex` reset a collection (delete by manifest `name`) before re-adding, so deleted files don't linger; collection identity is its name, and same-named collections replace each other.
 - Relay backpressure is global: any member's full queue 429s every publisher until drained (visible per §3, but one lagging member can stall the firehose — revisit before scale).
-- Member directory lives at `<data_dir>/members.toml` (name, pubkey, class source|enrichment), mtime-reloaded so MA updates need no restart; empty directory = open bootstrap mode (admission is MA policy, §10). Receivers drop content-bearing responses from members listed as enrichment (metadata-only, §6).
+- Member authority (Draft 0.5 §6): the MA-signed registry snapshot is authoritative when configured (`[node] registry` = file path or URL, `ma_key` pinned; monotonic version — rollback and forgery close the node; file path is mtime-reloaded, URL is fetched at start + every 60s and cached to `<data_dir>/registry-cache.json`, so outage fails static). Keys carry optional validity windows (`not_before`/`not_after`); rotation = `registry add-key` then `revoke-key`.
+- `<data_dir>/members.toml` (name, pubkey, class, `previous` keys, mtime-reloaded) is a dev/local fallback used only when no registry is configured; empty directory without a registry is open bootstrap only when `dev_bootstrap = true` (RFC §6: explicit dev flag). Receivers drop content-bearing responses from enrichment-class senders (metadata-only, §6).
+- MA tooling: `frxd registry init|add|add-key|revoke-key|remove|list|show|serve` (signed `registry.json` + `ma-key.hex` in `--dir`); `frxd init --id/--registry/--ma-key`; `frxd key show|rotate`; `member add --previous <old>` for the fallback path.
 - Aggregate semantics are our implementation choices from a terse spec: requests are `aggregate` envelopes carrying only `period`; replies carry `sent` (broadcasts that month) / `passed` (responses consumed from that member); granularity floor is enforced as YYYY or YYYY-MM only (finer rejected), yearly rolls up months. Revisit with §10 sufficiency review.
 
 ## Known gaps (Phase 2/3, intentional — don't fake them)
 - No dashboard UI, no directory watching (new files need `reindex`), no TLS, no user-supplied URL ingestion, no node-side (bilateral) rate limiting.
 - Receipts/settlement are out of protocol scope, not unimplemented (I4; Appendix B row 2). Lineage and delegation remain §10 open issues — unspecified without a supply stream, so not buildable as written; don't invent them silently.
-- Node query dedup is by `qid` only; there is no envelope replay/nonce window (RFC doesn't require one).
+- Node query dedup is by `qid` only; replay inside the ±300s skew window remains possible (no nonce cache at nodes), relays have no directory/admission, and there is no end-to-end encryption — relays see everything in clear.
 
 ## Technical plans (deliberately not in the RFC)
 - Record plans here — not as spec edits — when they are implementation/demo choices rather than protocol surface.
@@ -42,3 +45,4 @@
 - frxd modes (one binary, config toggles, no code required of publishers): querier (broadcast/local-first search), responder (match incoming queries against shared collections, sign), local index (watch dirs, extract text, explicit shared marking per I9). Use RFC terms querier/responder, not "subscriber/publisher".
 - Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role.
 - Matching accuracy is a project-health concern: start lexical (Tantivy), plan a hybrid cheap lexical gate + optional local embedding rerank (two-stage ingestion, Appendix A); embedding model stays local and replaceable (I2/I5).
+- Identity/registry (RFC Draft 0.5 §4/§6): MA-hosted FQDN identifiers first (`<label>.frx.<ma-domain>`, no DNS needed by users), signed versioned registry snapshot with the MA key pinned; envelope `from` = identifier, `key` = pubkey; registry outage fails static. Member-hosted identities, MA anchor rollover, and unicast confidentiality are §10 open. Implementation phases: A = signed registry snapshot with the current wire format (built and tested, no wire change); B = identifier + `key` + JCS on the wire. Prioritize frictionless onboarding (users may be department-level and cannot create DNS).