Draft 0.5: identity/registry spec; Phase A signed MA registry, mailbox challenge auth, key rotation, skew rejection
This commit is contained in:
@@ -1,9 +1,9 @@
|
|||||||
# AGENTS.md
|
# AGENTS.md
|
||||||
|
|
||||||
## Repo shape
|
## Repo shape
|
||||||
- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.4) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
|
- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.5) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
|
||||||
- `frxd` is the member node (init/add/index/serve/relay/query/status); `frx` is the thin client (search/query/status). Relay and node roles are separate subcommands.
|
- `frxd` is the member node (init/add/index/serve/relay/query/status); `frx` is the thin client (search/query/status). Relay and node roles are separate subcommands.
|
||||||
- Commands: `cargo build`, `cargo test` (68 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
|
- Commands: `cargo build`, `cargo test` (80 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; registry `tests/registry.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
|
||||||
- E2E pattern: relay + nodes in-process on ephemeral ports with tempdir corpora; use `tests/common/mod.rs` helpers (`spawn_relay*`, `query_envelope`, `poll_messages`, `register`) for new coverage. Raw relay polls return envelopes (payload under `body`), not response bodies.
|
- E2E pattern: relay + nodes in-process on ephemeral ports with tempdir corpora; use `tests/common/mod.rs` helpers (`spawn_relay*`, `query_envelope`, `poll_messages`, `register`) for new coverage. Raw relay polls return envelopes (payload under `body`), not response bodies.
|
||||||
|
|
||||||
## Editing the spec
|
## Editing the spec
|
||||||
@@ -14,25 +14,28 @@
|
|||||||
- §10 Open Issues are known gaps, not oversights (e.g., signature canonicalization blocks Phase-1 interop). Check it before "fixing" something.
|
- §10 Open Issues are known gaps, not oversights (e.g., signature canonicalization blocks Phase-1 interop). Check it before "fixing" something.
|
||||||
- Use the spec's vocabulary — member/querier/responder, aggregates, source/enrichment members — not client/server or search-engine terms.
|
- Use the spec's vocabulary — member/querier/responder, aggregates, source/enrichment members — not client/server or search-engine terms.
|
||||||
- A new `MUST` is only legitimate if it is observable at the boundary, deterministically verifiable by a peer, beneficial to the counterparty, and not derivable from local policy. Ranking/ordering/presentation fails this test and stays local (§5); scores never travel (I6).
|
- A new `MUST` is only legitimate if it is observable at the boundary, deterministically verifiable by a peer, beneficial to the counterparty, and not derivable from local policy. Ranking/ordering/presentation fails this test and stays local (§5); scores never travel (I6).
|
||||||
|
- I2 is not blanket anti-centralization: shared coordination (identity, admission, contract) is centralized in the MA because common state is cheaper held once; decisions that consume local information (matching, relevance, sharing, retention) stay local. Off-wire conduct (link handling, retention, gating) is contract, not conformance.
|
||||||
|
|
||||||
## Implementation notes
|
## Implementation notes
|
||||||
- Envelope signing is provisional (`src/crypto.rs`): `FRX/0.4` + fields + sorted-key canonical JSON body. §10's signature canonicalization open issue is unsolved — never present this scheme as interoperable.
|
- Draft 0.5 specifies JCS (RFC 8785) envelope signing over `{type, from, key, ts, nonce, body}` with `from` = MA-hosted identifier and `key` = pubkey. The code still signs the provisional `FRX/0.4` scheme (`src/crypto.rs`) until Phase B lands — never present current code as interoperable.
|
||||||
- Built: envelope/query/response, Tantivy index, aggregates, member directory. Not built: dashboard UI, directory watching, TLS, lineage/delegation. Responses travel relay-mediated unicast; transport is HTTP long-poll, not SSE.
|
- Built: envelope/query/response, Tantivy index, aggregates, member directory. Not built: dashboard UI, directory watching, TLS, lineage/delegation. Responses travel relay-mediated unicast; transport is HTTP long-poll, not SSE.
|
||||||
- Economics is out of protocol scope (I4: aggregates advise, contracts govern): no receipt, citation, pricing, or settlement fields or message types exist or may be added.
|
- Economics is out of protocol scope (I4: aggregates advise, contracts govern): no receipt, citation, pricing, or settlement fields or message types exist or may be added.
|
||||||
- Relay verifies signatures, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops.
|
- Relay verifies signatures and ±300s timestamp skew, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops. Mailbox polls require proof of key possession: `GET /v1/challenge` then a signed single-use nonce, so knowing a pubkey is not enough to drain its queue.
|
||||||
- Responder searches only collections marked shared (I9), stays silent when nothing matches, and emits results with honest `truncated`/`more_available` and no scores (I6). BM25 order is a local implementation detail, not protocol surface.
|
- Responder searches only collections marked shared (I9), stays silent when nothing matches, and emits results with honest `truncated`/`more_available` and no scores (I6). BM25 order is a local implementation detail, not protocol surface.
|
||||||
- Index layout: Tantivy at `<data_dir>/index`, collections manifest at `<data_dir>/collections.toml`; `exposure` (metadata|full) gates whether `content` is returned.
|
- Index layout: Tantivy at `<data_dir>/index`, collections manifest at `<data_dir>/collections.toml`; `exposure` (metadata|full) gates whether `content` is returned.
|
||||||
- Egress checks live in the responder path (`src/node.rs` `respond`), not the relay — keep private collections unreachable there.
|
- Egress checks live in the responder path (`src/node.rs` `respond`), not the relay — keep private collections unreachable there.
|
||||||
- Appendix B is executable policy: `tests/purges.rs` has one absence test per rejected mechanism (12 rows). Add a negative test there before ever re-proposing one, and only if the rationale is addressed.
|
- Appendix B is executable policy: `tests/purges.rs` has one absence test per rejected mechanism (12 rows). Add a negative test there before ever re-proposing one, and only if the rationale is addressed.
|
||||||
- `add`/`reindex` reset a collection (delete by manifest `name`) before re-adding, so deleted files don't linger; collection identity is its name, and same-named collections replace each other.
|
- `add`/`reindex` reset a collection (delete by manifest `name`) before re-adding, so deleted files don't linger; collection identity is its name, and same-named collections replace each other.
|
||||||
- Relay backpressure is global: any member's full queue 429s every publisher until drained (visible per §3, but one lagging member can stall the firehose — revisit before scale).
|
- Relay backpressure is global: any member's full queue 429s every publisher until drained (visible per §3, but one lagging member can stall the firehose — revisit before scale).
|
||||||
- Member directory lives at `<data_dir>/members.toml` (name, pubkey, class source|enrichment), mtime-reloaded so MA updates need no restart; empty directory = open bootstrap mode (admission is MA policy, §10). Receivers drop content-bearing responses from members listed as enrichment (metadata-only, §6).
|
- Member authority (Draft 0.5 §6): the MA-signed registry snapshot is authoritative when configured (`[node] registry` = file path or URL, `ma_key` pinned; monotonic version — rollback and forgery close the node; file path is mtime-reloaded, URL is fetched at start + every 60s and cached to `<data_dir>/registry-cache.json`, so outage fails static). Keys carry optional validity windows (`not_before`/`not_after`); rotation = `registry add-key` then `revoke-key`.
|
||||||
|
- `<data_dir>/members.toml` (name, pubkey, class, `previous` keys, mtime-reloaded) is a dev/local fallback used only when no registry is configured; empty directory without a registry is open bootstrap only when `dev_bootstrap = true` (RFC §6: explicit dev flag). Receivers drop content-bearing responses from enrichment-class senders (metadata-only, §6).
|
||||||
|
- MA tooling: `frxd registry init|add|add-key|revoke-key|remove|list|show|serve` (signed `registry.json` + `ma-key.hex` in `--dir`); `frxd init --id/--registry/--ma-key`; `frxd key show|rotate`; `member add --previous <old>` for the fallback path.
|
||||||
- Aggregate semantics are our implementation choices from a terse spec: requests are `aggregate` envelopes carrying only `period`; replies carry `sent` (broadcasts that month) / `passed` (responses consumed from that member); granularity floor is enforced as YYYY or YYYY-MM only (finer rejected), yearly rolls up months. Revisit with §10 sufficiency review.
|
- Aggregate semantics are our implementation choices from a terse spec: requests are `aggregate` envelopes carrying only `period`; replies carry `sent` (broadcasts that month) / `passed` (responses consumed from that member); granularity floor is enforced as YYYY or YYYY-MM only (finer rejected), yearly rolls up months. Revisit with §10 sufficiency review.
|
||||||
|
|
||||||
## Known gaps (Phase 2/3, intentional — don't fake them)
|
## Known gaps (Phase 2/3, intentional — don't fake them)
|
||||||
- No dashboard UI, no directory watching (new files need `reindex`), no TLS, no user-supplied URL ingestion, no node-side (bilateral) rate limiting.
|
- No dashboard UI, no directory watching (new files need `reindex`), no TLS, no user-supplied URL ingestion, no node-side (bilateral) rate limiting.
|
||||||
- Receipts/settlement are out of protocol scope, not unimplemented (I4; Appendix B row 2). Lineage and delegation remain §10 open issues — unspecified without a supply stream, so not buildable as written; don't invent them silently.
|
- Receipts/settlement are out of protocol scope, not unimplemented (I4; Appendix B row 2). Lineage and delegation remain §10 open issues — unspecified without a supply stream, so not buildable as written; don't invent them silently.
|
||||||
- Node query dedup is by `qid` only; there is no envelope replay/nonce window (RFC doesn't require one).
|
- Node query dedup is by `qid` only; replay inside the ±300s skew window remains possible (no nonce cache at nodes), relays have no directory/admission, and there is no end-to-end encryption — relays see everything in clear.
|
||||||
|
|
||||||
## Technical plans (deliberately not in the RFC)
|
## Technical plans (deliberately not in the RFC)
|
||||||
- Record plans here — not as spec edits — when they are implementation/demo choices rather than protocol surface.
|
- Record plans here — not as spec edits — when they are implementation/demo choices rather than protocol surface.
|
||||||
@@ -42,3 +45,4 @@
|
|||||||
- frxd modes (one binary, config toggles, no code required of publishers): querier (broadcast/local-first search), responder (match incoming queries against shared collections, sign), local index (watch dirs, extract text, explicit shared marking per I9). Use RFC terms querier/responder, not "subscriber/publisher".
|
- frxd modes (one binary, config toggles, no code required of publishers): querier (broadcast/local-first search), responder (match incoming queries against shared collections, sign), local index (watch dirs, extract text, explicit shared marking per I9). Use RFC terms querier/responder, not "subscriber/publisher".
|
||||||
- Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role.
|
- Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role.
|
||||||
- Matching accuracy is a project-health concern: start lexical (Tantivy), plan a hybrid cheap lexical gate + optional local embedding rerank (two-stage ingestion, Appendix A); embedding model stays local and replaceable (I2/I5).
|
- Matching accuracy is a project-health concern: start lexical (Tantivy), plan a hybrid cheap lexical gate + optional local embedding rerank (two-stage ingestion, Appendix A); embedding model stays local and replaceable (I2/I5).
|
||||||
|
- Identity/registry (RFC Draft 0.5 §4/§6): MA-hosted FQDN identifiers first (`<label>.frx.<ma-domain>`, no DNS needed by users), signed versioned registry snapshot with the MA key pinned; envelope `from` = identifier, `key` = pubkey; registry outage fails static. Member-hosted identities, MA anchor rollover, and unicast confidentiality are §10 open. Implementation phases: A = signed registry snapshot with the current wire format (built and tested, no wire change); B = identifier + `key` + JCS on the wire. Prioritize frictionless onboarding (users may be department-level and cannot create DNS).
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
FRX — Federated Retrieval Exchange
|
FRX — Federated Retrieval Exchange
|
||||||
|
|
||||||
Status: Draft 0.4. Experimental. Reference implementation: frxd (Rust).
|
Status: Draft 0.5. Experimental. Reference implementation: frxd (Rust).
|
||||||
|
|
||||||
1. Summary
|
1. Summary
|
||||||
|
|
||||||
@@ -9,7 +9,7 @@ FRX is a membership federation for retrieval. Content owners answer broadcast qu
|
|||||||
2. Invariants
|
2. Invariants
|
||||||
|
|
||||||
I1 Ingress consent — content enters a member's index only via the publisher adding its own content or a user-supplied URL.
|
I1 Ingress consent — content enters a member's index only via the publisher adding its own content or a user-supplied URL.
|
||||||
I2 Local sovereignty — nothing centralizes a decision a member could make locally.
|
I2 Local judgment — decisions that consume local information (matching, relevance, sharing, retention) are made locally. Shared coordination (identity, admission, contract) is centralized in the MA, because common state is cheaper held once.
|
||||||
I3 Broadcast privacy — broadcast payloads MUST NOT contain third-party private content. Queries derived from scoring others' posts MUST be canonicalized (claims/entities). First-party user-initiated search text MAY be sent as typed.
|
I3 Broadcast privacy — broadcast payloads MUST NOT contain third-party private content. Queries derived from scoring others' posts MUST be canonicalized (claims/entities). First-party user-initiated search text MAY be sent as typed.
|
||||||
I4 Channel separation — statements advise (aggregates), contracts govern (membership); never cross-wired. The protocol carries no pricing, metering, or settlement.
|
I4 Channel separation — statements advise (aggregates), contracts govern (membership); never cross-wired. The protocol carries no pricing, metering, or settlement.
|
||||||
I5 Role symmetry — no privileged roles. Any member may originate queries or responses; no peer may require remote work per incoming query (I7). Neither role is privileged.
|
I5 Role symmetry — no privileged roles. Any member may originate queries or responses; no peer may require remote work per incoming query (I7). Neither role is privileged.
|
||||||
@@ -33,7 +33,7 @@ Silence is conformant and informative: an unanswered query means no member's ava
|
|||||||
|
|
||||||
4. Messages
|
4. Messages
|
||||||
|
|
||||||
Envelope (all messages): {type, from, ts, nonce, body, sig} — Ed25519, key listed in the member directory.
|
Envelope (all messages): {type, from, key, ts, nonce, body, sig}. `from` is the sender's member identifier; `key` is the Ed25519 public key used to sign; `sig` covers the canonical form of the other fields (RFC 8785 JCS under a versioned prefix; bodies carry no floating-point numbers). A receiver verifies the signature under `key`, then verifies that `key` is authorized for `from` by the member registry (§6).
|
||||||
|
|
||||||
query
|
query
|
||||||
|
|
||||||
@@ -65,21 +65,21 @@ There is no publish/announce message. Document metadata is carried in responses
|
|||||||
|
|
||||||
5. Local Policy Domains
|
5. Local Policy Domains
|
||||||
|
|
||||||
Protocol-silent by design (I2): ranking, ordering, presentation, relevance gating, reputation counters and throttles, verification/spot-checks, caching and invalidation, claim minting, mode (eager/lazy), external fallback, sharing policy above the I9 floor. Advisory reputation bureaus MAY exist; no member is bound.
|
Protocol-silent by design (I2): ranking, ordering, presentation, relevance gating, reputation counters and throttles, verification/spot-checks, caching and invalidation, claim minting, external fallback, sharing policy above the I9 floor. Advisory reputation bureaus MAY exist; no member is bound.
|
||||||
|
|
||||||
6. Membership
|
6. Membership
|
||||||
|
|
||||||
The MA governs identity, contract, expulsion — who, never quality. Admission cost is the Sybil defense. Expulsion grounds: fabrication, admission fraud, sustained abuse — never low quality. Escalation: local throttle → advisory aggregates → MA warning → delisting → expulsion. Aggregates are inadmissible as sanction evidence (I4). Membership classes: source members (own content) and enrichment members (derived corpora, e.g. GDELT/CC-NEWS bots — metadata-only exposure, transformation logic open and auditable).
|
The MA governs identity, contract, expulsion — who, never quality. Identifiers are MA-hosted FQDNs (`<label>.frx.<ma-domain>`); no member-controlled DNS is required. Member-hosted identifiers — keys published in the member's own domain and allowlisted by the MA — are planned, not yet normative. The MA maintains a signed, versioned registry snapshot listing identifiers, class, and authorized keys with validity windows. Nodes pin the MA key; the snapshot is the sole authority for the key→identifier binding. Rotation publishes a successor key before retiring its predecessor; revocation removes a key or shortens its validity. Registry outage is fail-static: the last validated snapshot stays in force, and open bootstrap requires an explicit development flag. Admission cost is the Sybil defense. Expulsion grounds: fabrication, admission fraud, sustained abuse — never low quality. Escalation: local throttle → advisory aggregates → MA warning → delisting → expulsion. Aggregates are inadmissible as sanction evidence (I4). Conduct not observable on the wire — link handling, retention, gating — is governed by contract; the protocol neither observes nor adjudicates it. Membership classes: source members (own content) and enrichment members (derived corpora, e.g. GDELT/CC-NEWS bots — metadata-only exposure, transformation logic open and auditable).
|
||||||
|
|
||||||
7. Reference Implementation — frxd
|
7. Reference Implementation — frxd
|
||||||
|
|
||||||
A single static Rust binary. Install, point at a directory, done. It is simultaneously: (a) a personal search engine over local files, (b) a conformant FRX member.
|
A single static Rust binary. Install, point at a directory, done. It is simultaneously: (a) a personal search engine over local files, (b) a conformant FRX member.
|
||||||
|
|
||||||
First run: generate keypair, write config.toml, open localhost web UI (plus frx search CLI). Index target directories with Tantivy; watch for changes; extract text from txt/md/html (PDF optional). Nothing is shared until a collection is explicitly marked shared (I9).
|
First run: generate keypair, join the MA registry (no domain or DNS required), write config.toml, open localhost web UI (plus frx search CLI). Index target directories with Tantivy; watch for changes; extract text from txt/md/html (PDF optional). Nothing is shared until a collection is explicitly marked shared (I9).
|
||||||
|
|
||||||
As responder: subscribe to the live query stream via configured relays; match incoming queries against shared collections only (receiver-local lexical/embedding match); respond within budget with honest truncation; sign.
|
As responder: subscribe to the live query stream via configured relays; match incoming queries against shared collections only (receiver-local lexical/embedding match); respond within budget with honest truncation; sign.
|
||||||
|
|
||||||
As consumer: search is local-first; a network toggle broadcasts the query and merges responses, provenance-marked ("your files" / "member X"). Fetched content is retained (fetch-on-miss-and-retain): the kept-set converges to the demand-weighted corpus.
|
As consumer: search is local-first; a network toggle broadcasts the query and merges responses, provenance-marked ("your files" / "member X").
|
||||||
|
|
||||||
Dashboard: sent / passed per period (self-derived from local counters; aggregates requested from queriers on demand).
|
Dashboard: sent / passed per period (self-derived from local counters; aggregates requested from queriers on demand).
|
||||||
|
|
||||||
@@ -93,20 +93,22 @@ Query visibility is total among members; abstraction level and membership are th
|
|||||||
|
|
||||||
9. Conformance
|
9. Conformance
|
||||||
|
|
||||||
A conforming implementation: signs all messages with a listed member key; publishes queries to the firehose only; respects I3 and I9; respects max_results; truncates honestly; sends results without scores; enforces visible transport backpressure, never silent transport drops; serves aggregates on request at or above the granularity floor; ingests content only via publisher-added content or user-supplied URL; implements no dispute messages.
|
A conforming implementation: signs all messages with a key authorized for its identifier in the member registry; publishes queries to the firehose only; respects I3 and I9; respects max_results; truncates honestly; sends results without scores; enforces visible transport backpressure, never silent transport drops; serves aggregates on request at or above the granularity floor; ingests content only via publisher-added content or user-supplied URL; implements no dispute messages.
|
||||||
|
|
||||||
10. Open Issues
|
10. Open Issues
|
||||||
|
|
||||||
Consumer admission tier — automated/invite admission for distributed binaries without weakening the Sybil defense (MA policy, gates §7 adoption).
|
Consumer admission tier — automated/invite admission for distributed binaries without weakening the Sybil defense (MA policy, gates §7 adoption).
|
||||||
Relay discovery and default-relay governance (shipped defaults are soft centralization; mitigate with multiple defaults + one-command self-host).
|
Relay discovery and default-relay governance (shipped defaults are soft centralization; mitigate with multiple defaults + one-command self-host).
|
||||||
Delegation grant mechanism.
|
Delegation grant mechanism.
|
||||||
Signature canonicalization scheme (blocks Phase-1 interop; required before two implementations can exchange a valid envelope).
|
Member-hosted identifiers — keys published in the member's own DNS instead of the MA registry; MA-hosted is normative until specified.
|
||||||
|
MA anchor rollover — successor commitment and overlap for the registry signing key.
|
||||||
|
Unicast confidentiality — response and aggregate payloads are visible to relays; no end-to-end scheme is specified.
|
||||||
Document lineage (revision/supersedes) and delegation without a supply stream — previously carried by publish; now unspecified.
|
Document lineage (revision/supersedes) and delegation without a supply stream — previously carried by publish; now unspecified.
|
||||||
Claim/entity minting conventions — recommended, non-normative (I8).
|
Claim/entity minting conventions — recommended, non-normative (I8).
|
||||||
|
|
||||||
Appendix A. Recommended Local Practices (Non-Normative)
|
Appendix A. Recommended Local Practices (Non-Normative)
|
||||||
|
|
||||||
Two-stage ingestion (cheap gate before any LLM attention); per-peer sent/passed counters, throttling on ratios; verdict memoization and claim normalization; demand-driven prefetch (fetch and retain on attention); hot-set replication; backfill seeding (Wikipedia/Wikidata, GDELT, CC-NEWS); measure query-to-claim collapse and domain concentration before sizing anything.
|
Two-stage ingestion (cheap gate before any LLM attention); per-peer sent/passed counters, throttling on ratios; verdict memoization and claim normalization; hot-set replication; backfill seeding (Wikipedia/Wikidata, GDELT, CC-NEWS); measure query-to-claim collapse and domain concentration before sizing anything.
|
||||||
|
|
||||||
Appendix B. Purge Log (Normative)
|
Appendix B. Purge Log (Normative)
|
||||||
|
|
||||||
|
|||||||
+246
-5
@@ -1,12 +1,20 @@
|
|||||||
use std::path::Path;
|
use std::path::{Path, PathBuf};
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use anyhow::{Context, Result, anyhow};
|
use anyhow::{Context, Result, anyhow};
|
||||||
|
use axum::extract::State;
|
||||||
|
use axum::http::StatusCode;
|
||||||
|
use axum::response::{IntoResponse, Response};
|
||||||
|
use axum::routing::get;
|
||||||
|
use axum::{Json, Router};
|
||||||
|
use tokio::net::TcpListener;
|
||||||
|
|
||||||
use crate::config::{CLASS_ENRICHMENT, CLASS_SOURCE, Config, Member, load_members, save_members};
|
use crate::config::{CLASS_ENRICHMENT, CLASS_SOURCE, Config, Member, load_members, save_members};
|
||||||
|
use crate::crypto::{Keypair, now_ts};
|
||||||
use crate::index::{Collection, LocalIndex, load_collections, save_collections};
|
use crate::index::{Collection, LocalIndex, load_collections, save_collections};
|
||||||
use crate::message::{EXPOSURE_FULL, EXPOSURE_METADATA};
|
use crate::message::{EXPOSURE_FULL, EXPOSURE_METADATA};
|
||||||
use crate::node;
|
use crate::node;
|
||||||
|
use crate::registry::{self, KeyEntry, RegistryDoc, RegistryMember, SignedRegistry};
|
||||||
use crate::render;
|
use crate::render;
|
||||||
|
|
||||||
pub fn add(
|
pub fn add(
|
||||||
@@ -92,12 +100,27 @@ pub async fn query(
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) -> Result<()> {
|
fn normalize_key(value: &str) -> Result<String> {
|
||||||
let config = Config::load(config_path)?;
|
let bytes = hex::decode(value).context("pubkey must be hex")?;
|
||||||
let bytes = hex::decode(pubkey).context("pubkey must be hex")?;
|
|
||||||
if bytes.len() != 32 {
|
if bytes.len() != 32 {
|
||||||
return Err(anyhow!("pubkey must be 32 bytes (64 hex characters)"));
|
return Err(anyhow!("pubkey must be 32 bytes (64 hex characters)"));
|
||||||
}
|
}
|
||||||
|
Ok(value.to_ascii_lowercase())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn member_add(
|
||||||
|
config_path: &Path,
|
||||||
|
name: &str,
|
||||||
|
pubkey: &str,
|
||||||
|
class: &str,
|
||||||
|
previous: &[String],
|
||||||
|
) -> Result<()> {
|
||||||
|
let config = Config::load(config_path)?;
|
||||||
|
let pubkey = normalize_key(pubkey)?;
|
||||||
|
let previous = previous
|
||||||
|
.iter()
|
||||||
|
.map(|key| normalize_key(key))
|
||||||
|
.collect::<Result<Vec<_>>>()?;
|
||||||
let class = if class == CLASS_ENRICHMENT {
|
let class = if class == CLASS_ENRICHMENT {
|
||||||
CLASS_ENRICHMENT
|
CLASS_ENRICHMENT
|
||||||
} else {
|
} else {
|
||||||
@@ -107,8 +130,9 @@ pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) ->
|
|||||||
members.retain(|member| member.name != name && member.pubkey != pubkey);
|
members.retain(|member| member.name != name && member.pubkey != pubkey);
|
||||||
members.push(Member {
|
members.push(Member {
|
||||||
name: name.to_string(),
|
name: name.to_string(),
|
||||||
pubkey: pubkey.to_ascii_lowercase(),
|
pubkey,
|
||||||
class: class.to_string(),
|
class: class.to_string(),
|
||||||
|
previous,
|
||||||
});
|
});
|
||||||
save_members(&config.members_path(), &members)?;
|
save_members(&config.members_path(), &members)?;
|
||||||
println!(
|
println!(
|
||||||
@@ -118,6 +142,30 @@ pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) ->
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn key_show(config_path: &Path) -> Result<()> {
|
||||||
|
let config = Config::load(config_path)?;
|
||||||
|
println!("{}", config.load_key()?.public_hex());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn key_rotate(config_path: &Path) -> Result<()> {
|
||||||
|
let config = Config::load(config_path)?;
|
||||||
|
let old = config.load_key()?;
|
||||||
|
let backup = config.key_path().with_extension("hex.bak");
|
||||||
|
std::fs::copy(config.key_path(), &backup)?;
|
||||||
|
let new_key = Keypair::generate();
|
||||||
|
config.save_key(&new_key)?;
|
||||||
|
println!("old pubkey {}", old.public_hex());
|
||||||
|
println!("new pubkey {}", new_key.public_hex());
|
||||||
|
println!("old key saved to {}", backup.display());
|
||||||
|
println!(
|
||||||
|
"peers accept the new key via: frxd member add <name> {} --previous {}",
|
||||||
|
new_key.public_hex(),
|
||||||
|
old.public_hex()
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
pub fn member_remove(config_path: &Path, name: &str) -> Result<()> {
|
pub fn member_remove(config_path: &Path, name: &str) -> Result<()> {
|
||||||
let config = Config::load(config_path)?;
|
let config = Config::load(config_path)?;
|
||||||
let mut members = load_members(&config.members_path())?;
|
let mut members = load_members(&config.members_path())?;
|
||||||
@@ -184,6 +232,199 @@ pub async fn aggregates(
|
|||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn registry_key_path(dir: &Path) -> PathBuf {
|
||||||
|
dir.join("ma-key.hex")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn registry_doc_path(dir: &Path) -> PathBuf {
|
||||||
|
dir.join("registry.json")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn open_registry(dir: &Path) -> Result<(Keypair, SignedRegistry)> {
|
||||||
|
let raw = std::fs::read_to_string(registry_key_path(dir))
|
||||||
|
.with_context(|| format!("reading MA key from {}", registry_key_path(dir).display()))?;
|
||||||
|
let key = Keypair::from_hex(&raw)?;
|
||||||
|
let signed = registry::load_registry(®istry_doc_path(dir))
|
||||||
|
.with_context(|| format!("reading registry from {}", registry_doc_path(dir).display()))?;
|
||||||
|
Ok((key, signed))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn mutate_registry(dir: &Path, mutate: impl FnOnce(&mut RegistryDoc) -> Result<()>) -> Result<u64> {
|
||||||
|
let (ma, signed) = open_registry(dir)?;
|
||||||
|
let mut doc = signed.doc;
|
||||||
|
mutate(&mut doc)?;
|
||||||
|
doc.version += 1;
|
||||||
|
doc.issued_at = now_ts();
|
||||||
|
let signed = registry::sign_registry(doc, &ma);
|
||||||
|
registry::save_registry(®istry_doc_path(dir), &signed)?;
|
||||||
|
Ok(signed.doc.version)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_init(dir: &Path) -> Result<()> {
|
||||||
|
let registry_path = registry_doc_path(dir);
|
||||||
|
if registry_path.exists() {
|
||||||
|
return Err(anyhow!(
|
||||||
|
"registry {} already exists",
|
||||||
|
registry_path.display()
|
||||||
|
));
|
||||||
|
}
|
||||||
|
std::fs::create_dir_all(dir)?;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
std::fs::write(registry_key_path(dir), ma.to_hex())?;
|
||||||
|
crate::config::set_private_permissions(®istry_key_path(dir))?;
|
||||||
|
let doc = RegistryDoc {
|
||||||
|
version: 1,
|
||||||
|
issued_at: now_ts(),
|
||||||
|
ma_key: String::new(),
|
||||||
|
members: Vec::new(),
|
||||||
|
};
|
||||||
|
let signed = registry::sign_registry(doc, &ma);
|
||||||
|
registry::save_registry(®istry_path, &signed)?;
|
||||||
|
println!("MA key {}", ma.public_hex());
|
||||||
|
println!("registry {}", registry_path.display());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_add(
|
||||||
|
dir: &Path,
|
||||||
|
id: &str,
|
||||||
|
pubkey: &str,
|
||||||
|
class: &str,
|
||||||
|
not_before: Option<u64>,
|
||||||
|
not_after: Option<u64>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let pubkey = normalize_key(pubkey)?;
|
||||||
|
let class = if class == CLASS_ENRICHMENT {
|
||||||
|
CLASS_ENRICHMENT
|
||||||
|
} else {
|
||||||
|
CLASS_SOURCE
|
||||||
|
};
|
||||||
|
mutate_registry(dir, |doc| {
|
||||||
|
if doc.members.iter().any(|member| member.id == id) {
|
||||||
|
return Err(anyhow!("member {id} already listed"));
|
||||||
|
}
|
||||||
|
doc.members.push(RegistryMember {
|
||||||
|
id: id.to_string(),
|
||||||
|
class: class.to_string(),
|
||||||
|
keys: vec![KeyEntry {
|
||||||
|
key: pubkey.clone(),
|
||||||
|
not_before: not_before.unwrap_or_else(now_ts),
|
||||||
|
not_after,
|
||||||
|
}],
|
||||||
|
});
|
||||||
|
Ok(())
|
||||||
|
})?;
|
||||||
|
println!("added {id} ({class})");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_add_key(
|
||||||
|
dir: &Path,
|
||||||
|
id: &str,
|
||||||
|
pubkey: &str,
|
||||||
|
not_before: Option<u64>,
|
||||||
|
not_after: Option<u64>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let pubkey = normalize_key(pubkey)?;
|
||||||
|
mutate_registry(dir, |doc| {
|
||||||
|
let Some(member) = doc.members.iter_mut().find(|member| member.id == id) else {
|
||||||
|
return Err(anyhow!("no member named {id}"));
|
||||||
|
};
|
||||||
|
if member.keys.iter().any(|entry| entry.key == pubkey) {
|
||||||
|
return Err(anyhow!("key already authorized for {id}"));
|
||||||
|
}
|
||||||
|
member.keys.push(KeyEntry {
|
||||||
|
key: pubkey.clone(),
|
||||||
|
not_before: not_before.unwrap_or_else(now_ts),
|
||||||
|
not_after,
|
||||||
|
});
|
||||||
|
Ok(())
|
||||||
|
})?;
|
||||||
|
println!("added key for {id}");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_revoke_key(dir: &Path, id: &str, pubkey: &str) -> Result<()> {
|
||||||
|
let pubkey = normalize_key(pubkey)?;
|
||||||
|
mutate_registry(dir, |doc| {
|
||||||
|
let Some(member) = doc.members.iter_mut().find(|member| member.id == id) else {
|
||||||
|
return Err(anyhow!("no member named {id}"));
|
||||||
|
};
|
||||||
|
let before = member.keys.len();
|
||||||
|
member.keys.retain(|entry| entry.key != pubkey);
|
||||||
|
if member.keys.len() == before {
|
||||||
|
return Err(anyhow!("key not authorized for {id}"));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
})?;
|
||||||
|
println!("revoked key for {id}");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_remove(dir: &Path, id: &str) -> Result<()> {
|
||||||
|
mutate_registry(dir, |doc| {
|
||||||
|
let before = doc.members.len();
|
||||||
|
doc.members.retain(|member| member.id != id);
|
||||||
|
if doc.members.len() == before {
|
||||||
|
return Err(anyhow!("no member named {id}"));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
})?;
|
||||||
|
println!("removed {id}");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_list(dir: &Path) -> Result<()> {
|
||||||
|
let (_, signed) = open_registry(dir)?;
|
||||||
|
for member in &signed.doc.members {
|
||||||
|
let keys = member.keys.len();
|
||||||
|
println!("{} [{}] ({} key(s))", member.id, member.class, keys);
|
||||||
|
for entry in &member.keys {
|
||||||
|
let window = match entry.not_after {
|
||||||
|
Some(end) => format!("valid {}..{}", entry.not_before, end),
|
||||||
|
None => format!("valid from {}", entry.not_before),
|
||||||
|
};
|
||||||
|
println!(" {} ({window})", entry.key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_show(dir: &Path) -> Result<()> {
|
||||||
|
let (_, signed) = open_registry(dir)?;
|
||||||
|
println!("ma_key {}", signed.doc.ma_key);
|
||||||
|
println!("version {}", signed.doc.version);
|
||||||
|
println!("issued_at {}", signed.doc.issued_at);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn registry_serve(dir: &Path, listen: &str) -> Result<()> {
|
||||||
|
let state = dir.to_path_buf();
|
||||||
|
let app = Router::new()
|
||||||
|
.route("/health", get(registry_health))
|
||||||
|
.route("/registry.json", get(registry_snapshot))
|
||||||
|
.with_state(state);
|
||||||
|
let listener = TcpListener::bind(listen).await?;
|
||||||
|
println!("registry serving on http://{}", listener.local_addr()?);
|
||||||
|
axum::serve(listener, app).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn registry_health() -> &'static str {
|
||||||
|
"ok"
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn registry_snapshot(State(dir): State<PathBuf>) -> Response {
|
||||||
|
match registry::load_registry(®istry_doc_path(&dir)) {
|
||||||
|
Ok(signed) => Json(signed).into_response(),
|
||||||
|
Err(_) => (
|
||||||
|
StatusCode::NOT_FOUND,
|
||||||
|
Json(serde_json::json!({ "error": "registry not found" })),
|
||||||
|
)
|
||||||
|
.into_response(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn status(config_path: &Path) -> Result<()> {
|
pub async fn status(config_path: &Path) -> Result<()> {
|
||||||
let config = Config::load(config_path)?;
|
let config = Config::load(config_path)?;
|
||||||
let base = format!("http://{}", config.node.listen);
|
let base = format!("http://{}", config.node.listen);
|
||||||
|
|||||||
+19
-1
@@ -18,8 +18,16 @@ pub struct Config {
|
|||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
pub struct NodeSection {
|
pub struct NodeSection {
|
||||||
pub name: String,
|
pub name: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub id: Option<String>,
|
||||||
pub listen: String,
|
pub listen: String,
|
||||||
pub relays: Vec<String>,
|
pub relays: Vec<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub registry: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub ma_key: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub dev_bootstrap: bool,
|
||||||
#[serde(default = "default_true")]
|
#[serde(default = "default_true")]
|
||||||
pub responder: bool,
|
pub responder: bool,
|
||||||
}
|
}
|
||||||
@@ -33,6 +41,8 @@ pub struct Member {
|
|||||||
pub pubkey: String,
|
pub pubkey: String,
|
||||||
#[serde(default = "default_class")]
|
#[serde(default = "default_class")]
|
||||||
pub class: String,
|
pub class: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub previous: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Default, Serialize, Deserialize)]
|
#[derive(Debug, Default, Serialize, Deserialize)]
|
||||||
@@ -117,8 +127,12 @@ impl Config {
|
|||||||
Self {
|
Self {
|
||||||
node: NodeSection {
|
node: NodeSection {
|
||||||
name: name.to_string(),
|
name: name.to_string(),
|
||||||
|
id: None,
|
||||||
listen: listen.to_string(),
|
listen: listen.to_string(),
|
||||||
relays: vec![relay.to_string()],
|
relays: vec![relay.to_string()],
|
||||||
|
registry: None,
|
||||||
|
ma_key: None,
|
||||||
|
dev_bootstrap: false,
|
||||||
responder: true,
|
responder: true,
|
||||||
},
|
},
|
||||||
query: QuerySection::default(),
|
query: QuerySection::default(),
|
||||||
@@ -164,6 +178,10 @@ impl Config {
|
|||||||
self.data_dir().join("members.toml")
|
self.data_dir().join("members.toml")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn registry_cache_path(&self) -> PathBuf {
|
||||||
|
self.data_dir().join("registry-cache.json")
|
||||||
|
}
|
||||||
|
|
||||||
pub fn load_key(&self) -> Result<Keypair> {
|
pub fn load_key(&self) -> Result<Keypair> {
|
||||||
let raw = fs::read_to_string(self.key_path())
|
let raw = fs::read_to_string(self.key_path())
|
||||||
.with_context(|| format!("reading key {}", self.key_path().display()))?;
|
.with_context(|| format!("reading key {}", self.key_path().display()))?;
|
||||||
@@ -179,7 +197,7 @@ impl Config {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn set_private_permissions(path: &Path) -> Result<()> {
|
pub fn set_private_permissions(path: &Path) -> Result<()> {
|
||||||
#[cfg(unix)]
|
#[cfg(unix)]
|
||||||
{
|
{
|
||||||
use std::os::unix::fs::PermissionsExt;
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
|||||||
+24
-16
@@ -97,6 +97,29 @@ fn write_canonical(value: &Value, out: &mut String) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn poll_signing_bytes(member: &str, nonce: &str) -> Vec<u8> {
|
||||||
|
format!("{}\npoll\n{}\n{}", PROTOCOL, member, nonce).into_bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn verify_signature(public_hex: &str, message: &[u8], sig_hex: &str) -> Result<()> {
|
||||||
|
let key_bytes = hex::decode(public_hex).context("public key is not hex")?;
|
||||||
|
let key_bytes: [u8; 32] = key_bytes
|
||||||
|
.as_slice()
|
||||||
|
.try_into()
|
||||||
|
.map_err(|_| anyhow!("public key must be 32 bytes"))?;
|
||||||
|
let verifying_key =
|
||||||
|
VerifyingKey::from_bytes(&key_bytes).map_err(|e| anyhow!("bad public key: {e}"))?;
|
||||||
|
let sig_bytes = hex::decode(sig_hex).context("signature is not hex")?;
|
||||||
|
let sig_bytes: [u8; 64] = sig_bytes
|
||||||
|
.as_slice()
|
||||||
|
.try_into()
|
||||||
|
.map_err(|_| anyhow!("signature must be 64 bytes"))?;
|
||||||
|
let signature = Signature::from_bytes(&sig_bytes);
|
||||||
|
verifying_key
|
||||||
|
.verify_strict(message, &signature)
|
||||||
|
.map_err(|_| anyhow!("signature verification failed"))
|
||||||
|
}
|
||||||
|
|
||||||
pub fn signing_bytes(envelope: &Envelope) -> Vec<u8> {
|
pub fn signing_bytes(envelope: &Envelope) -> Vec<u8> {
|
||||||
format!(
|
format!(
|
||||||
"{}\n{}\n{}\n{}\n{}\n{}",
|
"{}\n{}\n{}\n{}\n{}\n{}",
|
||||||
@@ -111,22 +134,7 @@ pub fn signing_bytes(envelope: &Envelope) -> Vec<u8> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
pub fn verify_envelope(envelope: &Envelope) -> Result<()> {
|
pub fn verify_envelope(envelope: &Envelope) -> Result<()> {
|
||||||
let key_bytes = hex::decode(&envelope.from).context("from is not hex")?;
|
verify_signature(&envelope.from, &signing_bytes(envelope), &envelope.sig)
|
||||||
let key_bytes: [u8; 32] = key_bytes
|
|
||||||
.as_slice()
|
|
||||||
.try_into()
|
|
||||||
.map_err(|_| anyhow!("from must be a 32-byte ed25519 public key"))?;
|
|
||||||
let verifying_key =
|
|
||||||
VerifyingKey::from_bytes(&key_bytes).map_err(|e| anyhow!("bad public key: {e}"))?;
|
|
||||||
let sig_bytes = hex::decode(&envelope.sig).context("sig is not hex")?;
|
|
||||||
let sig_bytes: [u8; 64] = sig_bytes
|
|
||||||
.as_slice()
|
|
||||||
.try_into()
|
|
||||||
.map_err(|_| anyhow!("sig must be 64 bytes"))?;
|
|
||||||
let signature = Signature::from_bytes(&sig_bytes);
|
|
||||||
verifying_key
|
|
||||||
.verify_strict(&signing_bytes(envelope), &signature)
|
|
||||||
.map_err(|_| anyhow!("signature verification failed"))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn now_ts() -> u64 {
|
pub fn now_ts() -> u64 {
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ pub mod extract;
|
|||||||
pub mod index;
|
pub mod index;
|
||||||
pub mod message;
|
pub mod message;
|
||||||
pub mod node;
|
pub mod node;
|
||||||
|
pub mod registry;
|
||||||
pub mod relay;
|
pub mod relay;
|
||||||
pub mod render;
|
pub mod render;
|
||||||
|
|
||||||
|
|||||||
+105
-2
@@ -33,6 +33,12 @@ enum Command {
|
|||||||
data_dir: String,
|
data_dir: String,
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
force: bool,
|
force: bool,
|
||||||
|
#[arg(long)]
|
||||||
|
id: Option<String>,
|
||||||
|
#[arg(long)]
|
||||||
|
registry: Option<String>,
|
||||||
|
#[arg(long)]
|
||||||
|
ma_key: Option<String>,
|
||||||
},
|
},
|
||||||
Add {
|
Add {
|
||||||
path: PathBuf,
|
path: PathBuf,
|
||||||
@@ -70,6 +76,16 @@ enum Command {
|
|||||||
#[command(subcommand)]
|
#[command(subcommand)]
|
||||||
command: MemberCommand,
|
command: MemberCommand,
|
||||||
},
|
},
|
||||||
|
Key {
|
||||||
|
#[command(subcommand)]
|
||||||
|
command: KeyCommand,
|
||||||
|
},
|
||||||
|
Registry {
|
||||||
|
#[arg(long, default_value = "./frx-registry")]
|
||||||
|
dir: PathBuf,
|
||||||
|
#[command(subcommand)]
|
||||||
|
command: RegistryCommand,
|
||||||
|
},
|
||||||
Aggregates {
|
Aggregates {
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
from: Option<String>,
|
from: Option<String>,
|
||||||
@@ -87,6 +103,8 @@ enum MemberCommand {
|
|||||||
pubkey: String,
|
pubkey: String,
|
||||||
#[arg(long, default_value = "source")]
|
#[arg(long, default_value = "source")]
|
||||||
class: String,
|
class: String,
|
||||||
|
#[arg(long = "previous")]
|
||||||
|
previous: Vec<String>,
|
||||||
},
|
},
|
||||||
Remove {
|
Remove {
|
||||||
name: String,
|
name: String,
|
||||||
@@ -94,6 +112,48 @@ enum MemberCommand {
|
|||||||
List,
|
List,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Subcommand)]
|
||||||
|
enum KeyCommand {
|
||||||
|
Show,
|
||||||
|
Rotate,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Subcommand)]
|
||||||
|
enum RegistryCommand {
|
||||||
|
Init,
|
||||||
|
Add {
|
||||||
|
id: String,
|
||||||
|
pubkey: String,
|
||||||
|
#[arg(long, default_value = "source")]
|
||||||
|
class: String,
|
||||||
|
#[arg(long)]
|
||||||
|
not_before: Option<u64>,
|
||||||
|
#[arg(long)]
|
||||||
|
not_after: Option<u64>,
|
||||||
|
},
|
||||||
|
AddKey {
|
||||||
|
id: String,
|
||||||
|
pubkey: String,
|
||||||
|
#[arg(long)]
|
||||||
|
not_before: Option<u64>,
|
||||||
|
#[arg(long)]
|
||||||
|
not_after: Option<u64>,
|
||||||
|
},
|
||||||
|
RevokeKey {
|
||||||
|
id: String,
|
||||||
|
pubkey: String,
|
||||||
|
},
|
||||||
|
Remove {
|
||||||
|
id: String,
|
||||||
|
},
|
||||||
|
List,
|
||||||
|
Show,
|
||||||
|
Serve {
|
||||||
|
#[arg(long, default_value = "127.0.0.1:7800")]
|
||||||
|
listen: String,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Clone, Copy, ValueEnum)]
|
#[derive(Clone, Copy, ValueEnum)]
|
||||||
enum Exposure {
|
enum Exposure {
|
||||||
Metadata,
|
Metadata,
|
||||||
@@ -110,6 +170,9 @@ async fn main() -> Result<()> {
|
|||||||
relay,
|
relay,
|
||||||
data_dir,
|
data_dir,
|
||||||
force,
|
force,
|
||||||
|
id,
|
||||||
|
registry,
|
||||||
|
ma_key,
|
||||||
} => {
|
} => {
|
||||||
if cli.config.exists() && !force {
|
if cli.config.exists() && !force {
|
||||||
bail!(
|
bail!(
|
||||||
@@ -117,7 +180,19 @@ async fn main() -> Result<()> {
|
|||||||
cli.config.display()
|
cli.config.display()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
let config = Config::new(&name, &listen, &relay, &data_dir);
|
if registry.is_some() && ma_key.is_none() {
|
||||||
|
bail!("--ma-key is required with --registry");
|
||||||
|
}
|
||||||
|
let mut config = Config::new(&name, &listen, &relay, &data_dir);
|
||||||
|
config.node.id = id;
|
||||||
|
config.node.registry = registry;
|
||||||
|
config.node.ma_key = ma_key;
|
||||||
|
if config.node.registry.is_none() {
|
||||||
|
config.node.dev_bootstrap = true;
|
||||||
|
println!(
|
||||||
|
"warning: no registry configured; development open bootstrap (any valid key is accepted; do not deploy)"
|
||||||
|
);
|
||||||
|
}
|
||||||
let key = Keypair::generate();
|
let key = Keypair::generate();
|
||||||
config.save_key(&key)?;
|
config.save_key(&key)?;
|
||||||
config.save(&cli.config)?;
|
config.save(&cli.config)?;
|
||||||
@@ -167,10 +242,38 @@ async fn main() -> Result<()> {
|
|||||||
name,
|
name,
|
||||||
pubkey,
|
pubkey,
|
||||||
class,
|
class,
|
||||||
} => commands::member_add(&cli.config, &name, &pubkey, &class)?,
|
previous,
|
||||||
|
} => commands::member_add(&cli.config, &name, &pubkey, &class, &previous)?,
|
||||||
MemberCommand::Remove { name } => commands::member_remove(&cli.config, &name)?,
|
MemberCommand::Remove { name } => commands::member_remove(&cli.config, &name)?,
|
||||||
MemberCommand::List => commands::member_list(&cli.config)?,
|
MemberCommand::List => commands::member_list(&cli.config)?,
|
||||||
},
|
},
|
||||||
|
Command::Key { command } => match command {
|
||||||
|
KeyCommand::Show => commands::key_show(&cli.config)?,
|
||||||
|
KeyCommand::Rotate => commands::key_rotate(&cli.config)?,
|
||||||
|
},
|
||||||
|
Command::Registry { dir, command } => match command {
|
||||||
|
RegistryCommand::Init => commands::registry_init(&dir)?,
|
||||||
|
RegistryCommand::Add {
|
||||||
|
id,
|
||||||
|
pubkey,
|
||||||
|
class,
|
||||||
|
not_before,
|
||||||
|
not_after,
|
||||||
|
} => commands::registry_add(&dir, &id, &pubkey, &class, not_before, not_after)?,
|
||||||
|
RegistryCommand::AddKey {
|
||||||
|
id,
|
||||||
|
pubkey,
|
||||||
|
not_before,
|
||||||
|
not_after,
|
||||||
|
} => commands::registry_add_key(&dir, &id, &pubkey, not_before, not_after)?,
|
||||||
|
RegistryCommand::RevokeKey { id, pubkey } => {
|
||||||
|
commands::registry_revoke_key(&dir, &id, &pubkey)?
|
||||||
|
}
|
||||||
|
RegistryCommand::Remove { id } => commands::registry_remove(&dir, &id)?,
|
||||||
|
RegistryCommand::List => commands::registry_list(&dir)?,
|
||||||
|
RegistryCommand::Show => commands::registry_show(&dir)?,
|
||||||
|
RegistryCommand::Serve { listen } => commands::registry_serve(&dir, &listen).await?,
|
||||||
|
},
|
||||||
Command::Aggregates {
|
Command::Aggregates {
|
||||||
from,
|
from,
|
||||||
period,
|
period,
|
||||||
|
|||||||
@@ -11,6 +11,12 @@ pub const TYPE_AGGREGATE: &str = "aggregate";
|
|||||||
pub const EXPOSURE_METADATA: &str = "metadata";
|
pub const EXPOSURE_METADATA: &str = "metadata";
|
||||||
pub const EXPOSURE_FULL: &str = "full";
|
pub const EXPOSURE_FULL: &str = "full";
|
||||||
|
|
||||||
|
pub const MAX_CLOCK_SKEW_SECS: u64 = 300;
|
||||||
|
|
||||||
|
pub fn timestamp_is_fresh(ts: u64, now: u64) -> bool {
|
||||||
|
now.abs_diff(ts) <= MAX_CLOCK_SKEW_SECS
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
pub struct Envelope {
|
pub struct Envelope {
|
||||||
#[serde(rename = "type")]
|
#[serde(rename = "type")]
|
||||||
|
|||||||
+166
-10
@@ -17,12 +17,13 @@ use tokio::net::TcpListener;
|
|||||||
use tokio::task::JoinHandle;
|
use tokio::task::JoinHandle;
|
||||||
|
|
||||||
use crate::config::{CLASS_ENRICHMENT, Config, Member, load_members};
|
use crate::config::{CLASS_ENRICHMENT, Config, Member, load_members};
|
||||||
use crate::crypto::Keypair;
|
use crate::crypto::{Keypair, now_ts, poll_signing_bytes};
|
||||||
use crate::index::{LocalIndex, SearchHit, response_items};
|
use crate::index::{LocalIndex, SearchHit, response_items};
|
||||||
use crate::message::{
|
use crate::message::{
|
||||||
AggregateBody, Envelope, QueryBody, ResponseBody, ResponseItem, TYPE_AGGREGATE, TYPE_QUERY,
|
AggregateBody, Envelope, QueryBody, ResponseBody, ResponseItem, TYPE_AGGREGATE, TYPE_QUERY,
|
||||||
TYPE_RESPONSE, build_response,
|
TYPE_RESPONSE, build_response, timestamp_is_fresh,
|
||||||
};
|
};
|
||||||
|
use crate::registry::{SignedRegistry, authorized_keys, load_registry, verify_registry};
|
||||||
|
|
||||||
#[derive(Default)]
|
#[derive(Default)]
|
||||||
struct Aggregates {
|
struct Aggregates {
|
||||||
@@ -36,6 +37,8 @@ pub struct Node {
|
|||||||
index: LocalIndex,
|
index: LocalIndex,
|
||||||
members: RwLock<Vec<Member>>,
|
members: RwLock<Vec<Member>>,
|
||||||
members_mtime: Mutex<Option<SystemTime>>,
|
members_mtime: Mutex<Option<SystemTime>>,
|
||||||
|
registry: Mutex<Option<SignedRegistry>>,
|
||||||
|
registry_mtime: Mutex<Option<SystemTime>>,
|
||||||
aggregates: Mutex<Aggregates>,
|
aggregates: Mutex<Aggregates>,
|
||||||
seen: Mutex<HashSet<String>>,
|
seen: Mutex<HashSet<String>>,
|
||||||
pending: Mutex<HashMap<String, Vec<(String, ResponseBody)>>>,
|
pending: Mutex<HashMap<String, Vec<(String, ResponseBody)>>>,
|
||||||
@@ -119,12 +122,14 @@ impl Node {
|
|||||||
.timeout(Duration::from_secs(15))
|
.timeout(Duration::from_secs(15))
|
||||||
.build()
|
.build()
|
||||||
.context("building http client")?;
|
.context("building http client")?;
|
||||||
Ok(Arc::new(Self {
|
let node = Arc::new(Self {
|
||||||
config,
|
config,
|
||||||
key,
|
key,
|
||||||
index,
|
index,
|
||||||
members: RwLock::new(members),
|
members: RwLock::new(members),
|
||||||
members_mtime: Mutex::new(members_mtime),
|
members_mtime: Mutex::new(members_mtime),
|
||||||
|
registry: Mutex::new(None),
|
||||||
|
registry_mtime: Mutex::new(None),
|
||||||
aggregates: Mutex::new(Aggregates::default()),
|
aggregates: Mutex::new(Aggregates::default()),
|
||||||
seen: Mutex::new(HashSet::new()),
|
seen: Mutex::new(HashSet::new()),
|
||||||
pending: Mutex::new(HashMap::new()),
|
pending: Mutex::new(HashMap::new()),
|
||||||
@@ -132,7 +137,90 @@ impl Node {
|
|||||||
client,
|
client,
|
||||||
sent: AtomicU64::new(0),
|
sent: AtomicU64::new(0),
|
||||||
received: AtomicU64::new(0),
|
received: AtomicU64::new(0),
|
||||||
}))
|
});
|
||||||
|
node.load_initial_registry();
|
||||||
|
Ok(node)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn load_initial_registry(&self) {
|
||||||
|
let Some(source) = &self.config.node.registry else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let result = if source.starts_with("http://") || source.starts_with("https://") {
|
||||||
|
load_registry(&self.config.registry_cache_path())
|
||||||
|
} else {
|
||||||
|
let path = std::path::PathBuf::from(source);
|
||||||
|
let loaded = load_registry(&path);
|
||||||
|
if loaded.is_ok() {
|
||||||
|
let mtime = fs::metadata(&path)
|
||||||
|
.and_then(|metadata| metadata.modified())
|
||||||
|
.ok();
|
||||||
|
*self.registry_mtime.lock().expect("registry mtime lock") = mtime;
|
||||||
|
}
|
||||||
|
loaded
|
||||||
|
};
|
||||||
|
if let Ok(signed) = result {
|
||||||
|
if let Err(error) = self.verify_and_apply_registry(signed) {
|
||||||
|
eprintln!("cached registry rejected: {error}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn verify_and_apply_registry(&self, signed: SignedRegistry) -> Result<()> {
|
||||||
|
let ma_key = self
|
||||||
|
.config
|
||||||
|
.node
|
||||||
|
.ma_key
|
||||||
|
.as_deref()
|
||||||
|
.ok_or_else(|| anyhow!("registry configured without ma_key"))?;
|
||||||
|
verify_registry(&signed, ma_key)?;
|
||||||
|
{
|
||||||
|
let current = self.registry.lock().expect("registry lock");
|
||||||
|
if let Some(existing) = current.as_ref() {
|
||||||
|
if signed.doc.version <= existing.doc.version {
|
||||||
|
return Err(anyhow!("registry version rollback rejected"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*self.registry.lock().expect("registry lock") = Some(signed);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn refresh_registry(&self) {
|
||||||
|
let Some(source) = &self.config.node.registry else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if source.starts_with("http://") || source.starts_with("https://") {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let path = std::path::PathBuf::from(source);
|
||||||
|
let mtime = fs::metadata(&path)
|
||||||
|
.and_then(|metadata| metadata.modified())
|
||||||
|
.ok();
|
||||||
|
{
|
||||||
|
let last = self.registry_mtime.lock().expect("registry mtime lock");
|
||||||
|
if *last == mtime {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let Ok(signed) = load_registry(&path) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
*self.registry_mtime.lock().expect("registry mtime lock") = mtime;
|
||||||
|
if let Err(error) = self.verify_and_apply_registry(signed) {
|
||||||
|
eprintln!("registry update rejected: {error}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn fetch_registry(&self) -> Result<()> {
|
||||||
|
let Some(url) = &self.config.node.registry else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
let response = self.client.get(url).send().await?;
|
||||||
|
let signed: SignedRegistry = response.json().await?;
|
||||||
|
self.verify_and_apply_registry(signed.clone())?;
|
||||||
|
crate::registry::save_registry(&self.config.registry_cache_path(), &signed)?;
|
||||||
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn refresh_members(&self) {
|
fn refresh_members(&self) {
|
||||||
@@ -156,7 +244,9 @@ impl Node {
|
|||||||
fn member_class(&self, members: &[Member], pubkey: &str) -> Option<String> {
|
fn member_class(&self, members: &[Member], pubkey: &str) -> Option<String> {
|
||||||
members
|
members
|
||||||
.iter()
|
.iter()
|
||||||
.find(|member| member.pubkey == pubkey)
|
.find(|member| {
|
||||||
|
member.pubkey == pubkey || member.previous.iter().any(|key| key == pubkey)
|
||||||
|
})
|
||||||
.map(|member| member.class.clone())
|
.map(|member| member.class.clone())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -213,6 +303,22 @@ impl Node {
|
|||||||
.with_context(|| format!("binding {}", node.config.node.listen))?;
|
.with_context(|| format!("binding {}", node.config.node.listen))?;
|
||||||
let addr = listener.local_addr()?;
|
let addr = listener.local_addr()?;
|
||||||
let mut tasks = Vec::new();
|
let mut tasks = Vec::new();
|
||||||
|
if let Some(source) = &node.config.node.registry {
|
||||||
|
if source.starts_with("http://") || source.starts_with("https://") {
|
||||||
|
if let Err(error) = node.fetch_registry().await {
|
||||||
|
eprintln!("initial registry fetch failed: {error}");
|
||||||
|
}
|
||||||
|
let node_for_registry = node.clone();
|
||||||
|
tasks.push(tokio::spawn(async move {
|
||||||
|
loop {
|
||||||
|
if let Err(error) = node_for_registry.fetch_registry().await {
|
||||||
|
eprintln!("registry fetch failed: {error}");
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_secs(60)).await;
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
}
|
||||||
for relay in node.config.node.relays.clone() {
|
for relay in node.config.node.relays.clone() {
|
||||||
tasks.push(tokio::spawn(poll_relay(node.clone(), relay)));
|
tasks.push(tokio::spawn(poll_relay(node.clone(), relay)));
|
||||||
}
|
}
|
||||||
@@ -333,11 +439,31 @@ impl Node {
|
|||||||
if envelope.verify().is_err() {
|
if envelope.verify().is_err() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if !timestamp_is_fresh(envelope.ts, crate::crypto::now_ts()) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
self.refresh_members();
|
self.refresh_members();
|
||||||
let (class, listed) = {
|
self.refresh_registry();
|
||||||
|
let (class, listed) = if self.config.node.registry.is_some() {
|
||||||
|
let registry = self.registry.lock().expect("registry lock");
|
||||||
|
let authorized = registry
|
||||||
|
.as_ref()
|
||||||
|
.map(|signed| authorized_keys(signed, now_ts()));
|
||||||
|
match authorized {
|
||||||
|
Some(map) => {
|
||||||
|
let entry = map.get(&envelope.from);
|
||||||
|
(entry.map(|(_, class)| class.clone()), entry.is_some())
|
||||||
|
}
|
||||||
|
None => (None, false),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
let members = self.members.read().expect("members lock");
|
let members = self.members.read().expect("members lock");
|
||||||
let class = self.member_class(&members, &envelope.from);
|
let class = self.member_class(&members, &envelope.from);
|
||||||
let listed = members.is_empty() || class.is_some();
|
let listed = if members.is_empty() {
|
||||||
|
self.config.node.dev_bootstrap
|
||||||
|
} else {
|
||||||
|
class.is_some()
|
||||||
|
};
|
||||||
(class, listed)
|
(class, listed)
|
||||||
};
|
};
|
||||||
if !listed {
|
if !listed {
|
||||||
@@ -504,13 +630,35 @@ impl Node {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn fetch_challenge(node: &Node, base: &str, member: &str) -> Option<String> {
|
||||||
|
let response = node
|
||||||
|
.client
|
||||||
|
.get(format!("{base}/v1/challenge?member={member}"))
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.ok()?;
|
||||||
|
if !response.status().is_success() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let payload: Value = response.json().await.ok()?;
|
||||||
|
payload
|
||||||
|
.get("nonce")
|
||||||
|
.and_then(Value::as_str)
|
||||||
|
.map(str::to_string)
|
||||||
|
}
|
||||||
|
|
||||||
async fn poll_relay(node: Arc<Node>, relay: String) {
|
async fn poll_relay(node: Arc<Node>, relay: String) {
|
||||||
let base = relay.trim_end_matches('/').to_string();
|
let base = relay.trim_end_matches('/').to_string();
|
||||||
|
let member = node.key.public_hex();
|
||||||
loop {
|
loop {
|
||||||
|
let Some(nonce) = fetch_challenge(&node, &base, &member).await else {
|
||||||
|
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let signature = node.key.sign(&poll_signing_bytes(&member, &nonce));
|
||||||
let url = format!(
|
let url = format!(
|
||||||
"{}/v1/poll?member={}&timeout_ms=20000",
|
"{}/v1/poll?member={}&nonce={}&sig={}&timeout_ms=20000",
|
||||||
base,
|
base, member, nonce, signature
|
||||||
node.key.public_hex()
|
|
||||||
);
|
);
|
||||||
match node.client.get(&url).send().await {
|
match node.client.get(&url).send().await {
|
||||||
Ok(response) if response.status() == reqwest::StatusCode::NO_CONTENT => continue,
|
Ok(response) if response.status() == reqwest::StatusCode::NO_CONTENT => continue,
|
||||||
@@ -582,8 +730,16 @@ async fn local_query(
|
|||||||
|
|
||||||
async fn local_status(State(node): State<Arc<Node>>) -> Response {
|
async fn local_status(State(node): State<Arc<Node>>) -> Response {
|
||||||
let aggregates = node.aggregate_for(¤t_period(), None);
|
let aggregates = node.aggregate_for(¤t_period(), None);
|
||||||
|
let registry_version = node
|
||||||
|
.registry
|
||||||
|
.lock()
|
||||||
|
.expect("registry lock")
|
||||||
|
.as_ref()
|
||||||
|
.map(|signed| signed.doc.version);
|
||||||
Json(json!({
|
Json(json!({
|
||||||
"name": node.config.node.name,
|
"name": node.config.node.name,
|
||||||
|
"id": node.config.node.id,
|
||||||
|
"registry_version": registry_version,
|
||||||
"pubkey": node.key.public_hex(),
|
"pubkey": node.key.public_hex(),
|
||||||
"listen": node.config.node.listen,
|
"listen": node.config.node.listen,
|
||||||
"relays": node.config.node.relays,
|
"relays": node.config.node.relays,
|
||||||
|
|||||||
+176
@@ -0,0 +1,176 @@
|
|||||||
|
use std::collections::HashMap;
|
||||||
|
use std::fs;
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
|
use anyhow::{Context, Result, anyhow};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
use crate::PROTOCOL;
|
||||||
|
#[cfg(test)]
|
||||||
|
use crate::crypto::now_ts;
|
||||||
|
use crate::crypto::{Keypair, canonical_json, verify_signature};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct KeyEntry {
|
||||||
|
pub key: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub not_before: u64,
|
||||||
|
#[serde(default)]
|
||||||
|
pub not_after: Option<u64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct RegistryMember {
|
||||||
|
pub id: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub class: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub keys: Vec<KeyEntry>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct RegistryDoc {
|
||||||
|
pub version: u64,
|
||||||
|
pub issued_at: u64,
|
||||||
|
pub ma_key: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub members: Vec<RegistryMember>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct SignedRegistry {
|
||||||
|
#[serde(flatten)]
|
||||||
|
pub doc: RegistryDoc,
|
||||||
|
pub sig: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn registry_signing_bytes(doc: &RegistryDoc) -> Vec<u8> {
|
||||||
|
let value = serde_json::to_value(doc).expect("registry doc serializes");
|
||||||
|
format!("{}\nregistry\n{}", PROTOCOL, canonical_json(&value)).into_bytes()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn sign_registry(mut doc: RegistryDoc, key: &Keypair) -> SignedRegistry {
|
||||||
|
doc.ma_key = key.public_hex();
|
||||||
|
let sig = key.sign(®istry_signing_bytes(&doc));
|
||||||
|
SignedRegistry { doc, sig }
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn verify_registry(signed: &SignedRegistry, ma_key: &str) -> Result<()> {
|
||||||
|
if signed.doc.ma_key != ma_key {
|
||||||
|
return Err(anyhow!("registry signed by unexpected MA key"));
|
||||||
|
}
|
||||||
|
verify_signature(ma_key, ®istry_signing_bytes(&signed.doc), &signed.sig)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn load_registry(path: &Path) -> Result<SignedRegistry> {
|
||||||
|
let raw = fs::read_to_string(path).context("reading registry")?;
|
||||||
|
serde_json::from_str(&raw).context("parsing registry")
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn save_registry(path: &Path, signed: &SignedRegistry) -> Result<()> {
|
||||||
|
if let Some(parent) = path.parent() {
|
||||||
|
fs::create_dir_all(parent)?;
|
||||||
|
}
|
||||||
|
fs::write(path, serde_json::to_string_pretty(signed)?)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn authorized_keys(signed: &SignedRegistry, now: u64) -> HashMap<String, (String, String)> {
|
||||||
|
let mut authorized = HashMap::new();
|
||||||
|
for member in &signed.doc.members {
|
||||||
|
for key in &member.keys {
|
||||||
|
let valid = now >= key.not_before && key.not_after.map(|end| now < end).unwrap_or(true);
|
||||||
|
if valid {
|
||||||
|
authorized
|
||||||
|
.entry(key.key.clone())
|
||||||
|
.or_insert((member.id.clone(), member.class.clone()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
authorized
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::crypto::Keypair;
|
||||||
|
|
||||||
|
fn make_registry(
|
||||||
|
key: &Keypair,
|
||||||
|
entries: Vec<(&str, &str, u64, Option<u64>)>,
|
||||||
|
) -> SignedRegistry {
|
||||||
|
let members = entries
|
||||||
|
.into_iter()
|
||||||
|
.map(|(id, key_hex, not_before, not_after)| RegistryMember {
|
||||||
|
id: id.to_string(),
|
||||||
|
class: "source".to_string(),
|
||||||
|
keys: vec![KeyEntry {
|
||||||
|
key: key_hex.to_string(),
|
||||||
|
not_before,
|
||||||
|
not_after,
|
||||||
|
}],
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
sign_registry(
|
||||||
|
RegistryDoc {
|
||||||
|
version: 1,
|
||||||
|
issued_at: now_ts(),
|
||||||
|
ma_key: String::new(),
|
||||||
|
members,
|
||||||
|
},
|
||||||
|
key,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn signed_snapshot_verifies_and_forgery_fails() {
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let member = Keypair::generate();
|
||||||
|
let registry = make_registry(&ma, vec![("alice", &member.public_hex(), 0, None)]);
|
||||||
|
verify_registry(®istry, &ma.public_hex()).unwrap();
|
||||||
|
|
||||||
|
let mut forged = registry.clone();
|
||||||
|
forged.doc.members[0].id = "mallory".to_string();
|
||||||
|
forged.sig = ma.sign(®istry_signing_bytes(&forged.doc));
|
||||||
|
assert!(verify_registry(&forged, &ma.public_hex()).is_ok());
|
||||||
|
assert!(
|
||||||
|
verify_registry(&forged, &Keypair::generate().public_hex()).is_err(),
|
||||||
|
"wrong MA key must be rejected"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tampered_snapshot_fails_verification() {
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let member = Keypair::generate();
|
||||||
|
let mut registry = make_registry(&ma, vec![("alice", &member.public_hex(), 0, None)]);
|
||||||
|
registry.doc.version = 99;
|
||||||
|
assert!(verify_registry(®istry, &ma.public_hex()).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn validity_windows_gate_authorization() {
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let member = Keypair::generate();
|
||||||
|
let now = now_ts();
|
||||||
|
let mut registry = make_registry(
|
||||||
|
&ma,
|
||||||
|
vec![("future", &member.public_hex(), now + 3600, None)],
|
||||||
|
);
|
||||||
|
let mut expired = registry.doc.members.clone();
|
||||||
|
expired.push(RegistryMember {
|
||||||
|
id: "expired".to_string(),
|
||||||
|
class: "source".to_string(),
|
||||||
|
keys: vec![KeyEntry {
|
||||||
|
key: Keypair::generate().public_hex(),
|
||||||
|
not_before: 0,
|
||||||
|
not_after: Some(now.saturating_sub(1)),
|
||||||
|
}],
|
||||||
|
});
|
||||||
|
registry.doc.members = expired;
|
||||||
|
let registry = sign_registry(registry.doc, &ma);
|
||||||
|
|
||||||
|
let map = authorized_keys(®istry, now);
|
||||||
|
assert!(map.is_empty(), "future or expired keys must not authorize");
|
||||||
|
}
|
||||||
|
}
|
||||||
+73
-2
@@ -13,17 +13,25 @@ use serde::Deserialize;
|
|||||||
use serde_json::json;
|
use serde_json::json;
|
||||||
use tokio::net::TcpListener;
|
use tokio::net::TcpListener;
|
||||||
|
|
||||||
use crate::message::{Envelope, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE};
|
use crate::crypto::{now_ts, poll_signing_bytes, random_nonce, verify_signature};
|
||||||
|
use crate::message::{Envelope, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE, timestamp_is_fresh};
|
||||||
|
|
||||||
pub const DEFAULT_CAPACITY: usize = 256;
|
pub const DEFAULT_CAPACITY: usize = 256;
|
||||||
|
const CHALLENGE_TTL: Duration = Duration::from_secs(120);
|
||||||
|
|
||||||
#[derive(Default)]
|
#[derive(Default)]
|
||||||
struct MemberQueue {
|
struct MemberQueue {
|
||||||
items: VecDeque<(u64, Envelope)>,
|
items: VecDeque<(u64, Envelope)>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
struct Challenge {
|
||||||
|
member: String,
|
||||||
|
created: Instant,
|
||||||
|
}
|
||||||
|
|
||||||
struct Inner {
|
struct Inner {
|
||||||
members: HashMap<String, MemberQueue>,
|
members: HashMap<String, MemberQueue>,
|
||||||
|
challenges: HashMap<String, Challenge>,
|
||||||
seq: u64,
|
seq: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -37,6 +45,7 @@ impl Relay {
|
|||||||
Arc::new(Self {
|
Arc::new(Self {
|
||||||
inner: Mutex::new(Inner {
|
inner: Mutex::new(Inner {
|
||||||
members: HashMap::new(),
|
members: HashMap::new(),
|
||||||
|
challenges: HashMap::new(),
|
||||||
seq: 0,
|
seq: 0,
|
||||||
}),
|
}),
|
||||||
capacity,
|
capacity,
|
||||||
@@ -79,6 +88,7 @@ impl Relay {
|
|||||||
pub fn router(relay: Arc<Relay>) -> Router {
|
pub fn router(relay: Arc<Relay>) -> Router {
|
||||||
Router::new()
|
Router::new()
|
||||||
.route("/health", get(health))
|
.route("/health", get(health))
|
||||||
|
.route("/v1/challenge", get(challenge))
|
||||||
.route("/v1/publish", post(publish))
|
.route("/v1/publish", post(publish))
|
||||||
.route("/v1/unicast", post(unicast))
|
.route("/v1/unicast", post(unicast))
|
||||||
.route("/v1/poll", get(poll))
|
.route("/v1/poll", get(poll))
|
||||||
@@ -97,14 +107,48 @@ pub async fn bind(addr: &str) -> Result<(TcpListener, SocketAddr)> {
|
|||||||
Ok((listener, local))
|
Ok((listener, local))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn valid_key(member: &str) -> bool {
|
||||||
|
matches!(hex::decode(member), Ok(bytes) if bytes.len() == 32)
|
||||||
|
}
|
||||||
|
|
||||||
async fn health() -> &'static str {
|
async fn health() -> &'static str {
|
||||||
"ok"
|
"ok"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct ChallengeParams {
|
||||||
|
member: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn challenge(
|
||||||
|
State(relay): State<Arc<Relay>>,
|
||||||
|
Query(params): Query<ChallengeParams>,
|
||||||
|
) -> Response {
|
||||||
|
if !valid_key(¶ms.member) {
|
||||||
|
return bad_request("valid member key required");
|
||||||
|
}
|
||||||
|
let nonce = random_nonce();
|
||||||
|
let mut inner = relay.inner.lock().expect("relay lock");
|
||||||
|
inner
|
||||||
|
.challenges
|
||||||
|
.retain(|_, challenge| challenge.created.elapsed() < CHALLENGE_TTL);
|
||||||
|
inner.challenges.insert(
|
||||||
|
nonce.clone(),
|
||||||
|
Challenge {
|
||||||
|
member: params.member,
|
||||||
|
created: Instant::now(),
|
||||||
|
},
|
||||||
|
);
|
||||||
|
(StatusCode::OK, Json(json!({ "nonce": nonce }))).into_response()
|
||||||
|
}
|
||||||
|
|
||||||
async fn publish(State(relay): State<Arc<Relay>>, Json(envelope): Json<Envelope>) -> Response {
|
async fn publish(State(relay): State<Arc<Relay>>, Json(envelope): Json<Envelope>) -> Response {
|
||||||
if envelope.verify().is_err() {
|
if envelope.verify().is_err() {
|
||||||
return bad_request("invalid signature");
|
return bad_request("invalid signature");
|
||||||
}
|
}
|
||||||
|
if !timestamp_is_fresh(envelope.ts, now_ts()) {
|
||||||
|
return bad_request("stale timestamp");
|
||||||
|
}
|
||||||
if envelope.msg_type != TYPE_QUERY {
|
if envelope.msg_type != TYPE_QUERY {
|
||||||
return bad_request("relay carries broadcast queries only");
|
return bad_request("relay carries broadcast queries only");
|
||||||
}
|
}
|
||||||
@@ -131,6 +175,9 @@ async fn unicast(
|
|||||||
if envelope.verify().is_err() {
|
if envelope.verify().is_err() {
|
||||||
return bad_request("invalid signature");
|
return bad_request("invalid signature");
|
||||||
}
|
}
|
||||||
|
if !timestamp_is_fresh(envelope.ts, now_ts()) {
|
||||||
|
return bad_request("stale timestamp");
|
||||||
|
}
|
||||||
if envelope.msg_type != TYPE_RESPONSE && envelope.msg_type != TYPE_AGGREGATE {
|
if envelope.msg_type != TYPE_RESPONSE && envelope.msg_type != TYPE_AGGREGATE {
|
||||||
return bad_request("unicast carries responses and aggregates only");
|
return bad_request("unicast carries responses and aggregates only");
|
||||||
}
|
}
|
||||||
@@ -148,13 +195,33 @@ async fn unicast(
|
|||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
struct PollParams {
|
struct PollParams {
|
||||||
member: String,
|
member: String,
|
||||||
|
nonce: String,
|
||||||
|
sig: String,
|
||||||
timeout_ms: Option<u64>,
|
timeout_ms: Option<u64>,
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn poll(State(relay): State<Arc<Relay>>, Query(params): Query<PollParams>) -> Response {
|
async fn poll(State(relay): State<Arc<Relay>>, Query(params): Query<PollParams>) -> Response {
|
||||||
if params.member.is_empty() || hex::decode(¶ms.member).is_err() {
|
if !valid_key(¶ms.member) {
|
||||||
return bad_request("valid member key required");
|
return bad_request("valid member key required");
|
||||||
}
|
}
|
||||||
|
if verify_signature(
|
||||||
|
¶ms.member,
|
||||||
|
&poll_signing_bytes(¶ms.member, ¶ms.nonce),
|
||||||
|
¶ms.sig,
|
||||||
|
)
|
||||||
|
.is_err()
|
||||||
|
{
|
||||||
|
return unauthorized("invalid poll signature");
|
||||||
|
}
|
||||||
|
{
|
||||||
|
let mut inner = relay.inner.lock().expect("relay lock");
|
||||||
|
match inner.challenges.remove(¶ms.nonce) {
|
||||||
|
Some(challenge)
|
||||||
|
if challenge.member == params.member
|
||||||
|
&& challenge.created.elapsed() < CHALLENGE_TTL => {}
|
||||||
|
_ => return unauthorized("unknown, expired, or reused challenge"),
|
||||||
|
}
|
||||||
|
}
|
||||||
let timeout = Duration::from_millis(params.timeout_ms.unwrap_or(25_000).min(60_000));
|
let timeout = Duration::from_millis(params.timeout_ms.unwrap_or(25_000).min(60_000));
|
||||||
let deadline = Instant::now() + timeout;
|
let deadline = Instant::now() + timeout;
|
||||||
loop {
|
loop {
|
||||||
@@ -181,6 +248,10 @@ fn bad_request(message: &str) -> Response {
|
|||||||
(StatusCode::BAD_REQUEST, Json(json!({ "error": message }))).into_response()
|
(StatusCode::BAD_REQUEST, Json(json!({ "error": message }))).into_response()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn unauthorized(message: &str) -> Response {
|
||||||
|
(StatusCode::UNAUTHORIZED, Json(json!({ "error": message }))).into_response()
|
||||||
|
}
|
||||||
|
|
||||||
fn backpressure(status: StatusCode) -> Response {
|
fn backpressure(status: StatusCode) -> Response {
|
||||||
(
|
(
|
||||||
status,
|
status,
|
||||||
|
|||||||
@@ -198,11 +198,13 @@ async fn enrichment_members_cannot_send_content() {
|
|||||||
name: "bob".to_string(),
|
name: "bob".to_string(),
|
||||||
pubkey: bob_full.pubkey.clone(),
|
pubkey: bob_full.pubkey.clone(),
|
||||||
class: CLASS_ENRICHMENT.to_string(),
|
class: CLASS_ENRICHMENT.to_string(),
|
||||||
|
previous: Vec::new(),
|
||||||
},
|
},
|
||||||
Member {
|
Member {
|
||||||
name: "carol".to_string(),
|
name: "carol".to_string(),
|
||||||
pubkey: carol_meta.pubkey.clone(),
|
pubkey: carol_meta.pubkey.clone(),
|
||||||
class: CLASS_ENRICHMENT.to_string(),
|
class: CLASS_ENRICHMENT.to_string(),
|
||||||
|
previous: Vec::new(),
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
@@ -256,6 +258,7 @@ async fn source_members_may_send_content() {
|
|||||||
name: "bob".to_string(),
|
name: "bob".to_string(),
|
||||||
pubkey: bob.pubkey.clone(),
|
pubkey: bob.pubkey.clone(),
|
||||||
class: CLASS_SOURCE.to_string(),
|
class: CLASS_SOURCE.to_string(),
|
||||||
|
previous: Vec::new(),
|
||||||
}],
|
}],
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|||||||
+141
@@ -183,6 +183,147 @@ fn cli_init_refuses_overwrite_without_force() {
|
|||||||
run_ok(&forced);
|
run_ok(&forced);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn http_get(port: u16, path: &str) -> String {
|
||||||
|
use std::io::{Read, Write};
|
||||||
|
let mut stream = std::net::TcpStream::connect(("127.0.0.1", port)).unwrap();
|
||||||
|
stream
|
||||||
|
.write_all(format!("GET {path} HTTP/1.0\r\nHost: 127.0.0.1\r\n\r\n").as_bytes())
|
||||||
|
.unwrap();
|
||||||
|
let mut out = String::new();
|
||||||
|
stream.read_to_string(&mut out).unwrap();
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cli_registry_lifecycle() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let dir = root.path().join("reg");
|
||||||
|
let dir_arg = dir.display().to_string();
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"init".to_string(),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("MA key"), "{stdout}");
|
||||||
|
assert!(dir.join("registry.json").exists());
|
||||||
|
#[cfg(unix)]
|
||||||
|
{
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
let mode = fs::metadata(dir.join("ma-key.hex"))
|
||||||
|
.unwrap()
|
||||||
|
.permissions()
|
||||||
|
.mode()
|
||||||
|
& 0o777;
|
||||||
|
assert_eq!(mode, 0o600);
|
||||||
|
}
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"show".to_string(),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("version 1"));
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"add".to_string(),
|
||||||
|
"alice.frx.example".to_string(),
|
||||||
|
"ab".repeat(32),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("added alice.frx.example"));
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"add-key".to_string(),
|
||||||
|
"alice.frx.example".to_string(),
|
||||||
|
"cd".repeat(32),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("added key"));
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"list".to_string(),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("alice.frx.example [source] (2 key(s))"));
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"revoke-key".to_string(),
|
||||||
|
"alice.frx.example".to_string(),
|
||||||
|
"ab".repeat(32),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("revoked key"));
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg.clone(),
|
||||||
|
"list".to_string(),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("(1 key(s))"));
|
||||||
|
|
||||||
|
let port = common::free_port();
|
||||||
|
let _service = spawn_service(
|
||||||
|
&[
|
||||||
|
"registry".to_string(),
|
||||||
|
"--dir".to_string(),
|
||||||
|
dir_arg,
|
||||||
|
"serve".to_string(),
|
||||||
|
"--listen".to_string(),
|
||||||
|
format!("127.0.0.1:{port}"),
|
||||||
|
],
|
||||||
|
"registry serving",
|
||||||
|
);
|
||||||
|
let body = http_get(port, "/registry.json");
|
||||||
|
assert!(body.contains("\"members\""), "{body}");
|
||||||
|
assert!(body.contains("alice.frx.example"), "{body}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cli_key_rotation() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let config = root.path().join("frxd.toml");
|
||||||
|
let data = root.path().join("data");
|
||||||
|
let config_arg = config.display().to_string();
|
||||||
|
run_ok(&init_args(&config, "alice", 0, 1, &data));
|
||||||
|
|
||||||
|
let first = run_ok(&[
|
||||||
|
"--config".to_string(),
|
||||||
|
config_arg.clone(),
|
||||||
|
"key".to_string(),
|
||||||
|
"show".to_string(),
|
||||||
|
]);
|
||||||
|
let first = first.trim().to_string();
|
||||||
|
assert_eq!(first.len(), 64);
|
||||||
|
|
||||||
|
let stdout = run_ok(&[
|
||||||
|
"--config".to_string(),
|
||||||
|
config_arg.clone(),
|
||||||
|
"key".to_string(),
|
||||||
|
"rotate".to_string(),
|
||||||
|
]);
|
||||||
|
assert!(stdout.contains("old pubkey"), "{stdout}");
|
||||||
|
assert!(stdout.contains("new pubkey"), "{stdout}");
|
||||||
|
|
||||||
|
let second = run_ok(&[
|
||||||
|
"--config".to_string(),
|
||||||
|
config_arg,
|
||||||
|
"key".to_string(),
|
||||||
|
"show".to_string(),
|
||||||
|
]);
|
||||||
|
assert_ne!(first, second.trim(), "key did not change");
|
||||||
|
assert!(data.join("key.hex.bak").exists());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn cli_full_network_pipeline() {
|
fn cli_full_network_pipeline() {
|
||||||
let root = tempfile::tempdir().unwrap();
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
|||||||
+31
-6
@@ -13,8 +13,12 @@ pub fn config_for(dir: &Path, name: &str, relay_url: &str) -> Config {
|
|||||||
let config = Config {
|
let config = Config {
|
||||||
node: NodeSection {
|
node: NodeSection {
|
||||||
name: name.to_string(),
|
name: name.to_string(),
|
||||||
|
id: None,
|
||||||
listen: "127.0.0.1:0".to_string(),
|
listen: "127.0.0.1:0".to_string(),
|
||||||
relays: vec![relay_url.to_string()],
|
relays: vec![relay_url.to_string()],
|
||||||
|
registry: None,
|
||||||
|
ma_key: None,
|
||||||
|
dev_bootstrap: true,
|
||||||
responder: true,
|
responder: true,
|
||||||
},
|
},
|
||||||
query: QuerySection {
|
query: QuerySection {
|
||||||
@@ -60,17 +64,33 @@ pub fn query_envelope(key: &Keypair, text: &str, max_results: usize) -> Envelope
|
|||||||
Envelope::new(key, TYPE_QUERY, serde_json::to_value(&body).unwrap())
|
Envelope::new(key, TYPE_QUERY, serde_json::to_value(&body).unwrap())
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn register(client: &reqwest::Client, relay_url: &str, member: &str) {
|
pub async fn challenge(client: &reqwest::Client, relay_url: &str, member: &str) -> Option<String> {
|
||||||
poll(client, relay_url, member, 30).await;
|
let response = client
|
||||||
|
.get(format!("{relay_url}/v1/challenge?member={member}"))
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
if !response.status().is_success() {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let payload: Value = response.json().await.unwrap();
|
||||||
|
payload
|
||||||
|
.get("nonce")
|
||||||
|
.and_then(Value::as_str)
|
||||||
|
.map(str::to_string)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn register(client: &reqwest::Client, relay_url: &str, key: &Keypair) {
|
||||||
|
poll(client, relay_url, key, 30).await;
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn poll_messages(
|
pub async fn poll_messages(
|
||||||
client: &reqwest::Client,
|
client: &reqwest::Client,
|
||||||
relay_url: &str,
|
relay_url: &str,
|
||||||
member: &str,
|
key: &Keypair,
|
||||||
timeout_ms: u64,
|
timeout_ms: u64,
|
||||||
) -> Vec<Value> {
|
) -> Vec<Value> {
|
||||||
let response = poll(client, relay_url, member, timeout_ms).await;
|
let response = poll(client, relay_url, key, timeout_ms).await;
|
||||||
if response.status() == reqwest::StatusCode::NO_CONTENT {
|
if response.status() == reqwest::StatusCode::NO_CONTENT {
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
}
|
}
|
||||||
@@ -149,12 +169,17 @@ pub async fn unicast(
|
|||||||
pub async fn poll(
|
pub async fn poll(
|
||||||
client: &reqwest::Client,
|
client: &reqwest::Client,
|
||||||
relay_url: &str,
|
relay_url: &str,
|
||||||
member: &str,
|
key: &Keypair,
|
||||||
timeout_ms: u64,
|
timeout_ms: u64,
|
||||||
) -> reqwest::Response {
|
) -> reqwest::Response {
|
||||||
|
let member = key.public_hex();
|
||||||
|
let nonce = challenge(client, relay_url, &member)
|
||||||
|
.await
|
||||||
|
.expect("relay challenge");
|
||||||
|
let sig = key.sign(&frxd::crypto::poll_signing_bytes(&member, &nonce));
|
||||||
client
|
client
|
||||||
.get(format!(
|
.get(format!(
|
||||||
"{relay_url}/v1/poll?member={member}&timeout_ms={timeout_ms}"
|
"{relay_url}/v1/poll?member={member}&nonce={nonce}&sig={sig}&timeout_ms={timeout_ms}"
|
||||||
))
|
))
|
||||||
.send()
|
.send()
|
||||||
.await
|
.await
|
||||||
|
|||||||
@@ -101,8 +101,8 @@ async fn multi_relay_deduplicates_and_responds_once() {
|
|||||||
|
|
||||||
let http = client();
|
let http = client();
|
||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
register(&http, &relay_one, &alice.public_hex()).await;
|
register(&http, &relay_one, &alice).await;
|
||||||
register(&http, &relay_two, &alice.public_hex()).await;
|
register(&http, &relay_two, &alice).await;
|
||||||
let envelope = query_envelope(&alice, "rust", 5);
|
let envelope = query_envelope(&alice, "rust", 5);
|
||||||
assert!(
|
assert!(
|
||||||
publish(&http, &relay_one, &envelope)
|
publish(&http, &relay_one, &envelope)
|
||||||
@@ -121,7 +121,7 @@ async fn multi_relay_deduplicates_and_responds_once() {
|
|||||||
let mut responses = Vec::new();
|
let mut responses = Vec::new();
|
||||||
while responses.is_empty() && Instant::now() < deadline {
|
while responses.is_empty() && Instant::now() < deadline {
|
||||||
for relay in [&relay_one, &relay_two] {
|
for relay in [&relay_one, &relay_two] {
|
||||||
let messages = poll_messages(&http, relay, &alice.public_hex(), 200).await;
|
let messages = poll_messages(&http, relay, &alice, 200).await;
|
||||||
responses.extend(
|
responses.extend(
|
||||||
messages
|
messages
|
||||||
.iter()
|
.iter()
|
||||||
@@ -190,7 +190,7 @@ async fn poll_returns_queued_batch_in_one_call() {
|
|||||||
let relay_url = spawn_relay().await;
|
let relay_url = spawn_relay().await;
|
||||||
let http = client();
|
let http = client();
|
||||||
let member = Keypair::generate();
|
let member = Keypair::generate();
|
||||||
register(&http, &relay_url, &member.public_hex()).await;
|
register(&http, &relay_url, &member).await;
|
||||||
for index in 0..3 {
|
for index in 0..3 {
|
||||||
let envelope = query_envelope(&member, &format!("query {index}"), 5);
|
let envelope = query_envelope(&member, &format!("query {index}"), 5);
|
||||||
assert!(
|
assert!(
|
||||||
@@ -200,10 +200,10 @@ async fn poll_returns_queued_batch_in_one_call() {
|
|||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
let messages = poll_messages(&http, &relay_url, &member.public_hex(), 300).await;
|
let messages = poll_messages(&http, &relay_url, &member, 300).await;
|
||||||
assert_eq!(messages.len(), 3);
|
assert_eq!(messages.len(), 3);
|
||||||
assert!(
|
assert!(
|
||||||
poll_messages(&http, &relay_url, &member.public_hex(), 50)
|
poll_messages(&http, &relay_url, &member, 50)
|
||||||
.await
|
.await
|
||||||
.is_empty()
|
.is_empty()
|
||||||
);
|
);
|
||||||
|
|||||||
+154
-26
@@ -6,7 +6,7 @@ use std::path::{Path, PathBuf};
|
|||||||
use std::time::{Duration, Instant};
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
use common::{
|
use common::{
|
||||||
ask, client, collection, config_for, messages_of_type, poll, poll_messages, publish,
|
ask, challenge, client, collection, config_for, messages_of_type, poll, poll_messages, publish,
|
||||||
query_envelope, register, spawn_relay, spawn_relay_with_capacity, unicast,
|
query_envelope, register, spawn_relay, spawn_relay_with_capacity, unicast,
|
||||||
};
|
};
|
||||||
use frxd::crypto::Keypair;
|
use frxd::crypto::Keypair;
|
||||||
@@ -63,14 +63,14 @@ async fn start_node_with_corpus(
|
|||||||
async fn collect_responses(
|
async fn collect_responses(
|
||||||
http: &reqwest::Client,
|
http: &reqwest::Client,
|
||||||
relay_url: &str,
|
relay_url: &str,
|
||||||
member: &str,
|
key: &Keypair,
|
||||||
timeout_ms: u64,
|
timeout_ms: u64,
|
||||||
) -> Vec<Value> {
|
) -> Vec<Value> {
|
||||||
let start = Instant::now();
|
let start = Instant::now();
|
||||||
let deadline = Duration::from_millis(timeout_ms);
|
let deadline = Duration::from_millis(timeout_ms);
|
||||||
let mut responses = Vec::new();
|
let mut responses = Vec::new();
|
||||||
while start.elapsed() < deadline {
|
while start.elapsed() < deadline {
|
||||||
let messages = poll_messages(http, relay_url, member, 200).await;
|
let messages = poll_messages(http, relay_url, key, 200).await;
|
||||||
responses.extend(messages_of_type(&messages, TYPE_RESPONSE));
|
responses.extend(messages_of_type(&messages, TYPE_RESPONSE));
|
||||||
if !responses.is_empty() {
|
if !responses.is_empty() {
|
||||||
break;
|
break;
|
||||||
@@ -86,7 +86,7 @@ async fn raw_query(
|
|||||||
text: &str,
|
text: &str,
|
||||||
timeout_ms: u64,
|
timeout_ms: u64,
|
||||||
) -> Vec<Value> {
|
) -> Vec<Value> {
|
||||||
register(http, relay_url, &asker.public_hex()).await;
|
register(http, relay_url, asker).await;
|
||||||
let envelope = query_envelope(asker, text, 5);
|
let envelope = query_envelope(asker, text, 5);
|
||||||
assert!(
|
assert!(
|
||||||
publish(http, relay_url, &envelope)
|
publish(http, relay_url, &envelope)
|
||||||
@@ -94,7 +94,7 @@ async fn raw_query(
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
collect_responses(http, relay_url, &asker.public_hex(), timeout_ms).await
|
collect_responses(http, relay_url, asker, timeout_ms).await
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -330,7 +330,7 @@ async fn entities_are_optional_hints() {
|
|||||||
.await;
|
.await;
|
||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
let http = client();
|
let http = client();
|
||||||
register(&http, &relay_url, &alice.public_hex()).await;
|
register(&http, &relay_url, &alice).await;
|
||||||
let body = json!({
|
let body = json!({
|
||||||
"qid": "entities-test",
|
"qid": "entities-test",
|
||||||
"text": "rust",
|
"text": "rust",
|
||||||
@@ -344,7 +344,7 @@ async fn entities_are_optional_hints() {
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||||
assert_eq!(responses.len(), 1);
|
assert_eq!(responses.len(), 1);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
responses[0].pointer("/body/qid").and_then(Value::as_str),
|
responses[0].pointer("/body/qid").and_then(Value::as_str),
|
||||||
@@ -367,6 +367,7 @@ async fn member_directory_filters_senders() {
|
|||||||
name: "alice".to_string(),
|
name: "alice".to_string(),
|
||||||
pubkey: alice.public_hex(),
|
pubkey: alice.public_hex(),
|
||||||
class: frxd::config::CLASS_SOURCE.to_string(),
|
class: frxd::config::CLASS_SOURCE.to_string(),
|
||||||
|
previous: Vec::new(),
|
||||||
}],
|
}],
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
@@ -382,7 +383,7 @@ async fn member_directory_filters_senders() {
|
|||||||
let trusted = raw_query(&http, &relay_url, &alice, "rust", 700).await;
|
let trusted = raw_query(&http, &relay_url, &alice, "rust", 700).await;
|
||||||
assert_eq!(trusted.len(), 1, "trusted member got no response");
|
assert_eq!(trusted.len(), 1, "trusted member got no response");
|
||||||
|
|
||||||
register(&http, &relay_url, &untrusted.public_hex()).await;
|
register(&http, &relay_url, &untrusted).await;
|
||||||
let envelope = query_envelope(&untrusted, "rust", 5);
|
let envelope = query_envelope(&untrusted, "rust", 5);
|
||||||
assert!(
|
assert!(
|
||||||
publish(&http, &relay_url, &envelope)
|
publish(&http, &relay_url, &envelope)
|
||||||
@@ -390,17 +391,98 @@ async fn member_directory_filters_senders() {
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
let denied = collect_responses(&http, &relay_url, &untrusted.public_hex(), 400).await;
|
let denied = collect_responses(&http, &relay_url, &untrusted, 400).await;
|
||||||
assert!(denied.is_empty(), "untrusted member received a response");
|
assert!(denied.is_empty(), "untrusted member received a response");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn rotated_keys_are_accepted_through_previous_listing() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let old_key = Keypair::generate();
|
||||||
|
let new_key = Keypair::generate();
|
||||||
|
|
||||||
|
let docs = corpus_dir(root.path(), "bob", &[("doc.txt", "rotation rust document")]);
|
||||||
|
let config = config_for(&root.path().join("bob"), "bob", &relay_url);
|
||||||
|
let listing = |previous: Vec<String>| {
|
||||||
|
vec![frxd::config::Member {
|
||||||
|
name: "carol".to_string(),
|
||||||
|
pubkey: new_key.public_hex(),
|
||||||
|
class: frxd::config::CLASS_SOURCE.to_string(),
|
||||||
|
previous,
|
||||||
|
}]
|
||||||
|
};
|
||||||
|
frxd::config::save_members(&config.members_path(), &listing(vec![old_key.public_hex()]))
|
||||||
|
.unwrap();
|
||||||
|
{
|
||||||
|
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||||
|
index
|
||||||
|
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let _bob = Node::start(config.clone()).await.unwrap();
|
||||||
|
|
||||||
|
let http = client();
|
||||||
|
register(&http, &relay_url, &old_key).await;
|
||||||
|
let envelope = query_envelope(&old_key, "rust", 5);
|
||||||
|
assert!(
|
||||||
|
publish(&http, &relay_url, &envelope)
|
||||||
|
.await
|
||||||
|
.status()
|
||||||
|
.is_success()
|
||||||
|
);
|
||||||
|
let accepted = collect_responses(&http, &relay_url, &old_key, 700).await;
|
||||||
|
assert_eq!(accepted.len(), 1, "listed previous key was rejected");
|
||||||
|
|
||||||
|
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||||
|
frxd::config::save_members(&config.members_path(), &listing(Vec::new())).unwrap();
|
||||||
|
let envelope = query_envelope(&old_key, "rust", 5);
|
||||||
|
assert!(
|
||||||
|
publish(&http, &relay_url, &envelope)
|
||||||
|
.await
|
||||||
|
.status()
|
||||||
|
.is_success()
|
||||||
|
);
|
||||||
|
let revoked = collect_responses(&http, &relay_url, &old_key, 500).await;
|
||||||
|
assert!(revoked.is_empty(), "revoked key was still accepted");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn stale_envelopes_are_rejected() {
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let http = client();
|
||||||
|
let key = Keypair::generate();
|
||||||
|
|
||||||
|
let mut envelope = query_envelope(&key, "stale", 5);
|
||||||
|
envelope.ts = frxd::crypto::now_ts().saturating_sub(3600);
|
||||||
|
envelope.sig = key.sign(&frxd::crypto::signing_bytes(&envelope));
|
||||||
|
assert_eq!(
|
||||||
|
publish(&http, &relay_url, &envelope).await.status(),
|
||||||
|
reqwest::StatusCode::BAD_REQUEST
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut response = Envelope::new(
|
||||||
|
&key,
|
||||||
|
TYPE_RESPONSE,
|
||||||
|
json!({"qid": "q1", "results": [], "truncated": false, "more_available": 0, "cursor": null}),
|
||||||
|
);
|
||||||
|
response.ts = frxd::crypto::now_ts().saturating_sub(3600);
|
||||||
|
response.sig = key.sign(&frxd::crypto::signing_bytes(&response));
|
||||||
|
assert_eq!(
|
||||||
|
unicast(&http, &relay_url, &key.public_hex(), &response)
|
||||||
|
.await
|
||||||
|
.status(),
|
||||||
|
reqwest::StatusCode::BAD_REQUEST
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
async fn backpressure_is_visible_and_recoverable() {
|
async fn backpressure_is_visible_and_recoverable() {
|
||||||
let relay_url = spawn_relay_with_capacity(1).await;
|
let relay_url = spawn_relay_with_capacity(1).await;
|
||||||
let http = client();
|
let http = client();
|
||||||
let member = Keypair::generate();
|
let member = Keypair::generate();
|
||||||
let publisher = Keypair::generate();
|
let publisher = Keypair::generate();
|
||||||
register(&http, &relay_url, &member.public_hex()).await;
|
register(&http, &relay_url, &member).await;
|
||||||
|
|
||||||
let first = query_envelope(&publisher, "one", 5);
|
let first = query_envelope(&publisher, "one", 5);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
@@ -419,13 +501,11 @@ async fn backpressure_is_visible_and_recoverable() {
|
|||||||
);
|
);
|
||||||
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll_messages(&http, &relay_url, &member.public_hex(), 300)
|
poll_messages(&http, &relay_url, &member, 300).await.len(),
|
||||||
.await
|
|
||||||
.len(),
|
|
||||||
1
|
1
|
||||||
);
|
);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll_messages(&http, &relay_url, &publisher.public_hex(), 300)
|
poll_messages(&http, &relay_url, &publisher, 300)
|
||||||
.await
|
.await
|
||||||
.len(),
|
.len(),
|
||||||
1
|
1
|
||||||
@@ -444,8 +524,8 @@ async fn unicast_is_need_to_know() {
|
|||||||
let http = client();
|
let http = client();
|
||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
let bob = Keypair::generate();
|
let bob = Keypair::generate();
|
||||||
register(&http, &relay_url, &alice.public_hex()).await;
|
register(&http, &relay_url, &alice).await;
|
||||||
register(&http, &relay_url, &bob.public_hex()).await;
|
register(&http, &relay_url, &bob).await;
|
||||||
|
|
||||||
let query = query_envelope(&alice, "rust", 5);
|
let query = query_envelope(&alice, "rust", 5);
|
||||||
assert!(
|
assert!(
|
||||||
@@ -467,9 +547,9 @@ async fn unicast_is_need_to_know() {
|
|||||||
reqwest::StatusCode::OK
|
reqwest::StatusCode::OK
|
||||||
);
|
);
|
||||||
|
|
||||||
let alice_messages = poll_messages(&http, &relay_url, &alice.public_hex(), 300).await;
|
let alice_messages = poll_messages(&http, &relay_url, &alice, 300).await;
|
||||||
assert_eq!(messages_of_type(&alice_messages, TYPE_RESPONSE).len(), 1);
|
assert_eq!(messages_of_type(&alice_messages, TYPE_RESPONSE).len(), 1);
|
||||||
let bob_messages = poll_messages(&http, &relay_url, &bob.public_hex(), 300).await;
|
let bob_messages = poll_messages(&http, &relay_url, &bob, 300).await;
|
||||||
assert!(
|
assert!(
|
||||||
messages_of_type(&bob_messages, TYPE_RESPONSE).is_empty(),
|
messages_of_type(&bob_messages, TYPE_RESPONSE).is_empty(),
|
||||||
"unicast response leaked to another member"
|
"unicast response leaked to another member"
|
||||||
@@ -516,10 +596,58 @@ async fn tampered_envelopes_are_rejected() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
async fn invalid_poll_member_is_rejected() {
|
async fn poll_requires_proof_of_key() {
|
||||||
let relay_url = spawn_relay().await;
|
let relay_url = spawn_relay().await;
|
||||||
let response = poll(&client(), &relay_url, "not-hex-at-all", 30).await;
|
let http = client();
|
||||||
assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST);
|
let member = Keypair::generate();
|
||||||
|
let imposter = Keypair::generate();
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
challenge(&http, &relay_url, "not-hex-at-all")
|
||||||
|
.await
|
||||||
|
.is_none()
|
||||||
|
);
|
||||||
|
|
||||||
|
let nonce = challenge(&http, &relay_url, &member.public_hex())
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let forged = imposter.sign(&frxd::crypto::poll_signing_bytes(
|
||||||
|
&member.public_hex(),
|
||||||
|
&nonce,
|
||||||
|
));
|
||||||
|
let response = http
|
||||||
|
.get(format!(
|
||||||
|
"{relay_url}/v1/poll?member={}&nonce={nonce}&sig={forged}&timeout_ms=30",
|
||||||
|
member.public_hex()
|
||||||
|
))
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(response.status(), reqwest::StatusCode::UNAUTHORIZED);
|
||||||
|
|
||||||
|
let response = poll(&http, &relay_url, &member, 30).await;
|
||||||
|
assert_eq!(response.status(), reqwest::StatusCode::NO_CONTENT);
|
||||||
|
|
||||||
|
let nonce = challenge(&http, &relay_url, &member.public_hex())
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
let sig = member.sign(&frxd::crypto::poll_signing_bytes(
|
||||||
|
&member.public_hex(),
|
||||||
|
&nonce,
|
||||||
|
));
|
||||||
|
let url = format!(
|
||||||
|
"{relay_url}/v1/poll?member={}&nonce={nonce}&sig={sig}&timeout_ms=30",
|
||||||
|
member.public_hex()
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
http.get(&url).send().await.unwrap().status(),
|
||||||
|
reqwest::StatusCode::NO_CONTENT
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
http.get(&url).send().await.unwrap().status(),
|
||||||
|
reqwest::StatusCode::UNAUTHORIZED,
|
||||||
|
"challenge nonce must be single-use"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
@@ -724,7 +852,7 @@ async fn aggregates_are_unicast_only_and_ignored_by_nodes() {
|
|||||||
.await;
|
.await;
|
||||||
let http = client();
|
let http = client();
|
||||||
let sender = Keypair::generate();
|
let sender = Keypair::generate();
|
||||||
register(&http, &relay_url, &bob.pubkey).await;
|
register(&http, &relay_url, &bob.node.key).await;
|
||||||
|
|
||||||
let broadcast = Envelope::new(
|
let broadcast = Envelope::new(
|
||||||
&sender,
|
&sender,
|
||||||
@@ -798,7 +926,7 @@ async fn duplicate_delivery_is_answered_once() {
|
|||||||
.await;
|
.await;
|
||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
let http = client();
|
let http = client();
|
||||||
register(&http, &relay_url, &alice.public_hex()).await;
|
register(&http, &relay_url, &alice).await;
|
||||||
let envelope = query_envelope(&alice, "rust", 5);
|
let envelope = query_envelope(&alice, "rust", 5);
|
||||||
assert!(
|
assert!(
|
||||||
publish(&http, &relay_url, &envelope)
|
publish(&http, &relay_url, &envelope)
|
||||||
@@ -812,7 +940,7 @@ async fn duplicate_delivery_is_answered_once() {
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
responses.len(),
|
responses.len(),
|
||||||
1,
|
1,
|
||||||
@@ -904,7 +1032,7 @@ async fn query_body_without_entities_parses() {
|
|||||||
.await;
|
.await;
|
||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
let http = client();
|
let http = client();
|
||||||
register(&http, &relay_url, &alice.public_hex()).await;
|
register(&http, &relay_url, &alice).await;
|
||||||
let envelope = Envelope::new(
|
let envelope = Envelope::new(
|
||||||
&alice,
|
&alice,
|
||||||
TYPE_QUERY,
|
TYPE_QUERY,
|
||||||
@@ -916,7 +1044,7 @@ async fn query_body_without_entities_parses() {
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||||
assert_eq!(responses.len(), 1);
|
assert_eq!(responses.len(), 1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+7
-15
@@ -143,7 +143,7 @@ async fn no_protocol_query_dedup() {
|
|||||||
.status()
|
.status()
|
||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
let response = poll(&http, &relay_url, &key.public_hex(), 300).await;
|
let response = poll(&http, &relay_url, &key, 300).await;
|
||||||
let payload: Value = response.json().await.unwrap();
|
let payload: Value = response.json().await.unwrap();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
payload
|
payload
|
||||||
@@ -242,9 +242,7 @@ async fn no_aggregate_appeals() {
|
|||||||
);
|
);
|
||||||
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll(&http, &relay_url, &member.public_hex(), 30)
|
poll(&http, &relay_url, &member, 30).await.status(),
|
||||||
.await
|
|
||||||
.status(),
|
|
||||||
reqwest::StatusCode::NO_CONTENT
|
reqwest::StatusCode::NO_CONTENT
|
||||||
);
|
);
|
||||||
let aggregate = Envelope::new(
|
let aggregate = Envelope::new(
|
||||||
@@ -299,15 +297,11 @@ async fn no_topic_channels() {
|
|||||||
let alice = Keypair::generate();
|
let alice = Keypair::generate();
|
||||||
let bob = Keypair::generate();
|
let bob = Keypair::generate();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll(&http, &relay_url, &alice.public_hex(), 30)
|
poll(&http, &relay_url, &alice, 30).await.status(),
|
||||||
.await
|
|
||||||
.status(),
|
|
||||||
reqwest::StatusCode::NO_CONTENT
|
reqwest::StatusCode::NO_CONTENT
|
||||||
);
|
);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll(&http, &relay_url, &bob.public_hex(), 30)
|
poll(&http, &relay_url, &bob, 30).await.status(),
|
||||||
.await
|
|
||||||
.status(),
|
|
||||||
reqwest::StatusCode::NO_CONTENT
|
reqwest::StatusCode::NO_CONTENT
|
||||||
);
|
);
|
||||||
let envelope = Envelope::new(
|
let envelope = Envelope::new(
|
||||||
@@ -322,7 +316,7 @@ async fn no_topic_channels() {
|
|||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
for member in [&alice, &bob] {
|
for member in [&alice, &bob] {
|
||||||
let response = poll(&http, &relay_url, &member.public_hex(), 300).await;
|
let response = poll(&http, &relay_url, &member, 300).await;
|
||||||
let payload: Value = response.json().await.unwrap();
|
let payload: Value = response.json().await.unwrap();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
payload
|
payload
|
||||||
@@ -422,7 +416,7 @@ async fn no_durable_replayable_broadcast_stream() {
|
|||||||
.is_success()
|
.is_success()
|
||||||
);
|
);
|
||||||
|
|
||||||
let response = poll(&http, &relay_url, &key.public_hex(), 300).await;
|
let response = poll(&http, &relay_url, &key, 300).await;
|
||||||
let payload: Value = response.json().await.unwrap();
|
let payload: Value = response.json().await.unwrap();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
payload
|
payload
|
||||||
@@ -433,9 +427,7 @@ async fn no_durable_replayable_broadcast_stream() {
|
|||||||
);
|
);
|
||||||
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
poll(&http, &relay_url, &key.public_hex(), 50)
|
poll(&http, &relay_url, &key, 50).await.status(),
|
||||||
.await
|
|
||||||
.status(),
|
|
||||||
reqwest::StatusCode::NO_CONTENT,
|
reqwest::StatusCode::NO_CONTENT,
|
||||||
"relay replayed a drained message"
|
"relay replayed a drained message"
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -0,0 +1,346 @@
|
|||||||
|
mod common;
|
||||||
|
|
||||||
|
use std::fs;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use axum::routing::get;
|
||||||
|
use axum::{Json, Router};
|
||||||
|
use common::{
|
||||||
|
client, collection, config_for, poll_messages, publish, query_envelope, register, spawn_relay,
|
||||||
|
};
|
||||||
|
use frxd::crypto::{Keypair, now_ts};
|
||||||
|
use frxd::index::LocalIndex;
|
||||||
|
use frxd::message::{EXPOSURE_FULL, TYPE_RESPONSE};
|
||||||
|
use frxd::node::{Node, NodeHandle};
|
||||||
|
use frxd::registry::{self, KeyEntry, RegistryDoc, RegistryMember, SignedRegistry};
|
||||||
|
use serde_json::Value;
|
||||||
|
|
||||||
|
fn member_entry(
|
||||||
|
id: &str,
|
||||||
|
key: &Keypair,
|
||||||
|
class: &str,
|
||||||
|
not_before: u64,
|
||||||
|
not_after: Option<u64>,
|
||||||
|
) -> RegistryMember {
|
||||||
|
RegistryMember {
|
||||||
|
id: id.to_string(),
|
||||||
|
class: class.to_string(),
|
||||||
|
keys: vec![KeyEntry {
|
||||||
|
key: key.public_hex(),
|
||||||
|
not_before,
|
||||||
|
not_after,
|
||||||
|
}],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn ma_registry(ma: &Keypair, members: Vec<RegistryMember>, version: u64) -> SignedRegistry {
|
||||||
|
registry::sign_registry(
|
||||||
|
RegistryDoc {
|
||||||
|
version,
|
||||||
|
issued_at: now_ts(),
|
||||||
|
ma_key: String::new(),
|
||||||
|
members,
|
||||||
|
},
|
||||||
|
ma,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn config_with_registry(
|
||||||
|
dir: &Path,
|
||||||
|
name: &str,
|
||||||
|
relay_url: &str,
|
||||||
|
registry: &str,
|
||||||
|
ma_key: &str,
|
||||||
|
) -> frxd::config::Config {
|
||||||
|
let mut config = config_for(dir, name, relay_url);
|
||||||
|
config.node.registry = Some(registry.to_string());
|
||||||
|
config.node.ma_key = Some(ma_key.to_string());
|
||||||
|
config.node.dev_bootstrap = false;
|
||||||
|
config
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn bob_with_docs(root: &Path, relay_url: &str, registry: &str, ma_key: &str) -> NodeHandle {
|
||||||
|
let docs = root.join("bob-docs");
|
||||||
|
fs::create_dir_all(&docs).unwrap();
|
||||||
|
fs::write(docs.join("doc.txt"), "registry rust document").unwrap();
|
||||||
|
let config = config_with_registry(&root.join("bob"), "bob", relay_url, registry, ma_key);
|
||||||
|
{
|
||||||
|
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||||
|
index
|
||||||
|
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
Node::start(config).await.unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn responses_for(
|
||||||
|
http: &reqwest::Client,
|
||||||
|
relay_url: &str,
|
||||||
|
asker: &Keypair,
|
||||||
|
text: &str,
|
||||||
|
timeout_ms: u64,
|
||||||
|
) -> Vec<Value> {
|
||||||
|
register(http, relay_url, asker).await;
|
||||||
|
let envelope = query_envelope(asker, text, 5);
|
||||||
|
assert!(
|
||||||
|
publish(http, relay_url, &envelope)
|
||||||
|
.await
|
||||||
|
.status()
|
||||||
|
.is_success()
|
||||||
|
);
|
||||||
|
let start = Instant::now();
|
||||||
|
let deadline = Duration::from_millis(timeout_ms);
|
||||||
|
let mut responses = Vec::new();
|
||||||
|
while start.elapsed() < deadline {
|
||||||
|
let messages = poll_messages(http, relay_url, asker, 200).await;
|
||||||
|
responses.extend(
|
||||||
|
messages
|
||||||
|
.iter()
|
||||||
|
.filter(|m| m.get("type").and_then(Value::as_str) == Some(TYPE_RESPONSE))
|
||||||
|
.cloned(),
|
||||||
|
);
|
||||||
|
if !responses.is_empty() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
responses
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn registry_gates_membership_and_revocation_propagates() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let alice = Keypair::generate();
|
||||||
|
let registry_path = root.path().join("registry.json");
|
||||||
|
registry::save_registry(
|
||||||
|
®istry_path,
|
||||||
|
&ma_registry(
|
||||||
|
&ma,
|
||||||
|
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let _bob = bob_with_docs(
|
||||||
|
root.path(),
|
||||||
|
&relay_url,
|
||||||
|
®istry_path.display().to_string(),
|
||||||
|
&ma.public_hex(),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let http = client();
|
||||||
|
|
||||||
|
let accepted = responses_for(&http, &relay_url, &alice, "rust", 700).await;
|
||||||
|
assert_eq!(accepted.len(), 1, "listed member got no response");
|
||||||
|
|
||||||
|
let stranger = Keypair::generate();
|
||||||
|
let denied = responses_for(&http, &relay_url, &stranger, "rust", 400).await;
|
||||||
|
assert!(denied.is_empty(), "unlisted key was answered");
|
||||||
|
|
||||||
|
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||||
|
registry::save_registry(®istry_path, &ma_registry(&ma, Vec::new(), 2)).unwrap();
|
||||||
|
let revoked = responses_for(&http, &relay_url, &alice, "rust", 500).await;
|
||||||
|
assert!(revoked.is_empty(), "revoked member still answered");
|
||||||
|
|
||||||
|
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||||
|
registry::save_registry(
|
||||||
|
®istry_path,
|
||||||
|
&ma_registry(
|
||||||
|
&ma,
|
||||||
|
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let rolled_back = responses_for(&http, &relay_url, &alice, "rust", 500).await;
|
||||||
|
assert!(
|
||||||
|
rolled_back.is_empty(),
|
||||||
|
"registry rollback re-admitted a revoked member"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn forged_or_wrong_key_snapshot_closes_the_registry() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let attacker = Keypair::generate();
|
||||||
|
let alice = Keypair::generate();
|
||||||
|
let registry_path = root.path().join("registry.json");
|
||||||
|
registry::save_registry(
|
||||||
|
®istry_path,
|
||||||
|
&ma_registry(
|
||||||
|
&attacker,
|
||||||
|
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let _bob = bob_with_docs(
|
||||||
|
root.path(),
|
||||||
|
&relay_url,
|
||||||
|
®istry_path.display().to_string(),
|
||||||
|
&ma.public_hex(),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let http = client();
|
||||||
|
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||||
|
assert!(
|
||||||
|
denied.is_empty(),
|
||||||
|
"snapshot signed by wrong key was trusted"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn expired_key_is_not_authorized() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let alice = Keypair::generate();
|
||||||
|
let registry_path = root.path().join("registry.json");
|
||||||
|
registry::save_registry(
|
||||||
|
®istry_path,
|
||||||
|
&ma_registry(
|
||||||
|
&ma,
|
||||||
|
vec![member_entry(
|
||||||
|
"alice.frx.example",
|
||||||
|
&alice,
|
||||||
|
"source",
|
||||||
|
0,
|
||||||
|
Some(now_ts().saturating_sub(1)),
|
||||||
|
)],
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let _bob = bob_with_docs(
|
||||||
|
root.path(),
|
||||||
|
&relay_url,
|
||||||
|
®istry_path.display().to_string(),
|
||||||
|
&ma.public_hex(),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let http = client();
|
||||||
|
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||||
|
assert!(denied.is_empty(), "expired key was answered");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn fail_static_uses_last_validated_snapshot() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let alice = Keypair::generate();
|
||||||
|
let snapshot = ma_registry(
|
||||||
|
&ma,
|
||||||
|
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||||
|
1,
|
||||||
|
);
|
||||||
|
|
||||||
|
let cached_dir = root.path().join("bob-cached");
|
||||||
|
let mut config = config_with_registry(
|
||||||
|
&cached_dir,
|
||||||
|
"bob",
|
||||||
|
&relay_url,
|
||||||
|
"http://127.0.0.1:1/registry.json",
|
||||||
|
&ma.public_hex(),
|
||||||
|
);
|
||||||
|
fs::create_dir_all(config.data_dir()).unwrap();
|
||||||
|
registry::save_registry(&config.registry_cache_path(), &snapshot).unwrap();
|
||||||
|
let docs = root.path().join("bob-docs");
|
||||||
|
fs::create_dir_all(&docs).unwrap();
|
||||||
|
fs::write(docs.join("doc.txt"), "registry rust document").unwrap();
|
||||||
|
{
|
||||||
|
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||||
|
index
|
||||||
|
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let _bob = Node::start(config.clone()).await.unwrap();
|
||||||
|
|
||||||
|
let http = client();
|
||||||
|
let accepted = responses_for(&http, &relay_url, &alice, "rust", 700).await;
|
||||||
|
assert_eq!(
|
||||||
|
accepted.len(),
|
||||||
|
1,
|
||||||
|
"cached snapshot should keep last-known-good membership"
|
||||||
|
);
|
||||||
|
drop(_bob);
|
||||||
|
|
||||||
|
let fresh_dir = root.path().join("bob-fresh");
|
||||||
|
let fresh = config_with_registry(
|
||||||
|
&fresh_dir,
|
||||||
|
"bob2",
|
||||||
|
&relay_url,
|
||||||
|
"http://127.0.0.1:1/registry.json",
|
||||||
|
&ma.public_hex(),
|
||||||
|
);
|
||||||
|
{
|
||||||
|
let index = LocalIndex::open(&fresh.index_dir()).unwrap();
|
||||||
|
index
|
||||||
|
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||||
|
.unwrap();
|
||||||
|
}
|
||||||
|
let _bob2 = Node::start(fresh).await.unwrap();
|
||||||
|
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||||
|
assert!(
|
||||||
|
denied.is_empty(),
|
||||||
|
"unreachable registry without cache must close, not open"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||||
|
async fn registry_can_be_served_over_http() {
|
||||||
|
let root = tempfile::tempdir().unwrap();
|
||||||
|
let relay_url = spawn_relay().await;
|
||||||
|
let ma = Keypair::generate();
|
||||||
|
let alice = Keypair::generate();
|
||||||
|
let registry_dir = root.path().join("ma");
|
||||||
|
fs::create_dir_all(®istry_dir).unwrap();
|
||||||
|
registry::save_registry(
|
||||||
|
®istry_dir.join("registry.json"),
|
||||||
|
&ma_registry(
|
||||||
|
&ma,
|
||||||
|
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||||
|
1,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||||
|
let addr = listener.local_addr().unwrap();
|
||||||
|
let state: PathBuf = registry_dir.clone();
|
||||||
|
let handler_state = state.clone();
|
||||||
|
let app = Router::new()
|
||||||
|
.route(
|
||||||
|
"/registry.json",
|
||||||
|
get(move || {
|
||||||
|
let state = handler_state.clone();
|
||||||
|
async move {
|
||||||
|
Json(
|
||||||
|
registry::load_registry(&state.join("registry.json"))
|
||||||
|
.expect("registry file"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.with_state(state);
|
||||||
|
tokio::spawn(async move {
|
||||||
|
let _ = axum::serve(listener, app).await;
|
||||||
|
});
|
||||||
|
let registry_url = format!("http://{addr}/registry.json");
|
||||||
|
|
||||||
|
let _bob = bob_with_docs(root.path(), &relay_url, ®istry_url, &ma.public_hex()).await;
|
||||||
|
let http = client();
|
||||||
|
let accepted = responses_for(&http, &relay_url, &alice, "rust", 900).await;
|
||||||
|
assert_eq!(
|
||||||
|
accepted.len(),
|
||||||
|
1,
|
||||||
|
"http-served registry did not authorize member"
|
||||||
|
);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user