Draft 0.5: identity/registry spec; Phase A signed MA registry, mailbox challenge auth, key rotation, skew rejection
This commit is contained in:
@@ -1,9 +1,9 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Repo shape
|
||||
- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.4) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
|
||||
- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.5) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries).
|
||||
- `frxd` is the member node (init/add/index/serve/relay/query/status); `frx` is the thin client (search/query/status). Relay and node roles are separate subcommands.
|
||||
- Commands: `cargo build`, `cargo test` (68 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
|
||||
- Commands: `cargo build`, `cargo test` (80 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; aggregates + member directory `tests/aggregates.rs`; registry `tests/registry.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config.
|
||||
- E2E pattern: relay + nodes in-process on ephemeral ports with tempdir corpora; use `tests/common/mod.rs` helpers (`spawn_relay*`, `query_envelope`, `poll_messages`, `register`) for new coverage. Raw relay polls return envelopes (payload under `body`), not response bodies.
|
||||
|
||||
## Editing the spec
|
||||
@@ -14,25 +14,28 @@
|
||||
- §10 Open Issues are known gaps, not oversights (e.g., signature canonicalization blocks Phase-1 interop). Check it before "fixing" something.
|
||||
- Use the spec's vocabulary — member/querier/responder, aggregates, source/enrichment members — not client/server or search-engine terms.
|
||||
- A new `MUST` is only legitimate if it is observable at the boundary, deterministically verifiable by a peer, beneficial to the counterparty, and not derivable from local policy. Ranking/ordering/presentation fails this test and stays local (§5); scores never travel (I6).
|
||||
- I2 is not blanket anti-centralization: shared coordination (identity, admission, contract) is centralized in the MA because common state is cheaper held once; decisions that consume local information (matching, relevance, sharing, retention) stay local. Off-wire conduct (link handling, retention, gating) is contract, not conformance.
|
||||
|
||||
## Implementation notes
|
||||
- Envelope signing is provisional (`src/crypto.rs`): `FRX/0.4` + fields + sorted-key canonical JSON body. §10's signature canonicalization open issue is unsolved — never present this scheme as interoperable.
|
||||
- Draft 0.5 specifies JCS (RFC 8785) envelope signing over `{type, from, key, ts, nonce, body}` with `from` = MA-hosted identifier and `key` = pubkey. The code still signs the provisional `FRX/0.4` scheme (`src/crypto.rs`) until Phase B lands — never present current code as interoperable.
|
||||
- Built: envelope/query/response, Tantivy index, aggregates, member directory. Not built: dashboard UI, directory watching, TLS, lineage/delegation. Responses travel relay-mediated unicast; transport is HTTP long-poll, not SSE.
|
||||
- Economics is out of protocol scope (I4: aggregates advise, contracts govern): no receipt, citation, pricing, or settlement fields or message types exist or may be added.
|
||||
- Relay verifies signatures, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops.
|
||||
- Relay verifies signatures and ±300s timestamp skew, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops. Mailbox polls require proof of key possession: `GET /v1/challenge` then a signed single-use nonce, so knowing a pubkey is not enough to drain its queue.
|
||||
- Responder searches only collections marked shared (I9), stays silent when nothing matches, and emits results with honest `truncated`/`more_available` and no scores (I6). BM25 order is a local implementation detail, not protocol surface.
|
||||
- Index layout: Tantivy at `<data_dir>/index`, collections manifest at `<data_dir>/collections.toml`; `exposure` (metadata|full) gates whether `content` is returned.
|
||||
- Egress checks live in the responder path (`src/node.rs` `respond`), not the relay — keep private collections unreachable there.
|
||||
- Appendix B is executable policy: `tests/purges.rs` has one absence test per rejected mechanism (12 rows). Add a negative test there before ever re-proposing one, and only if the rationale is addressed.
|
||||
- `add`/`reindex` reset a collection (delete by manifest `name`) before re-adding, so deleted files don't linger; collection identity is its name, and same-named collections replace each other.
|
||||
- Relay backpressure is global: any member's full queue 429s every publisher until drained (visible per §3, but one lagging member can stall the firehose — revisit before scale).
|
||||
- Member directory lives at `<data_dir>/members.toml` (name, pubkey, class source|enrichment), mtime-reloaded so MA updates need no restart; empty directory = open bootstrap mode (admission is MA policy, §10). Receivers drop content-bearing responses from members listed as enrichment (metadata-only, §6).
|
||||
- Member authority (Draft 0.5 §6): the MA-signed registry snapshot is authoritative when configured (`[node] registry` = file path or URL, `ma_key` pinned; monotonic version — rollback and forgery close the node; file path is mtime-reloaded, URL is fetched at start + every 60s and cached to `<data_dir>/registry-cache.json`, so outage fails static). Keys carry optional validity windows (`not_before`/`not_after`); rotation = `registry add-key` then `revoke-key`.
|
||||
- `<data_dir>/members.toml` (name, pubkey, class, `previous` keys, mtime-reloaded) is a dev/local fallback used only when no registry is configured; empty directory without a registry is open bootstrap only when `dev_bootstrap = true` (RFC §6: explicit dev flag). Receivers drop content-bearing responses from enrichment-class senders (metadata-only, §6).
|
||||
- MA tooling: `frxd registry init|add|add-key|revoke-key|remove|list|show|serve` (signed `registry.json` + `ma-key.hex` in `--dir`); `frxd init --id/--registry/--ma-key`; `frxd key show|rotate`; `member add --previous <old>` for the fallback path.
|
||||
- Aggregate semantics are our implementation choices from a terse spec: requests are `aggregate` envelopes carrying only `period`; replies carry `sent` (broadcasts that month) / `passed` (responses consumed from that member); granularity floor is enforced as YYYY or YYYY-MM only (finer rejected), yearly rolls up months. Revisit with §10 sufficiency review.
|
||||
|
||||
## Known gaps (Phase 2/3, intentional — don't fake them)
|
||||
- No dashboard UI, no directory watching (new files need `reindex`), no TLS, no user-supplied URL ingestion, no node-side (bilateral) rate limiting.
|
||||
- Receipts/settlement are out of protocol scope, not unimplemented (I4; Appendix B row 2). Lineage and delegation remain §10 open issues — unspecified without a supply stream, so not buildable as written; don't invent them silently.
|
||||
- Node query dedup is by `qid` only; there is no envelope replay/nonce window (RFC doesn't require one).
|
||||
- Node query dedup is by `qid` only; replay inside the ±300s skew window remains possible (no nonce cache at nodes), relays have no directory/admission, and there is no end-to-end encryption — relays see everything in clear.
|
||||
|
||||
## Technical plans (deliberately not in the RFC)
|
||||
- Record plans here — not as spec edits — when they are implementation/demo choices rather than protocol surface.
|
||||
@@ -42,3 +45,4 @@
|
||||
- frxd modes (one binary, config toggles, no code required of publishers): querier (broadcast/local-first search), responder (match incoming queries against shared collections, sign), local index (watch dirs, extract text, explicit shared marking per I9). Use RFC terms querier/responder, not "subscriber/publisher".
|
||||
- Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role.
|
||||
- Matching accuracy is a project-health concern: start lexical (Tantivy), plan a hybrid cheap lexical gate + optional local embedding rerank (two-stage ingestion, Appendix A); embedding model stays local and replaceable (I2/I5).
|
||||
- Identity/registry (RFC Draft 0.5 §4/§6): MA-hosted FQDN identifiers first (`<label>.frx.<ma-domain>`, no DNS needed by users), signed versioned registry snapshot with the MA key pinned; envelope `from` = identifier, `key` = pubkey; registry outage fails static. Member-hosted identities, MA anchor rollover, and unicast confidentiality are §10 open. Implementation phases: A = signed registry snapshot with the current wire format (built and tested, no wire change); B = identifier + `key` + JCS on the wire. Prioritize frictionless onboarding (users may be department-level and cannot create DNS).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
FRX — Federated Retrieval Exchange
|
||||
|
||||
Status: Draft 0.4. Experimental. Reference implementation: frxd (Rust).
|
||||
Status: Draft 0.5. Experimental. Reference implementation: frxd (Rust).
|
||||
|
||||
1. Summary
|
||||
|
||||
@@ -9,7 +9,7 @@ FRX is a membership federation for retrieval. Content owners answer broadcast qu
|
||||
2. Invariants
|
||||
|
||||
I1 Ingress consent — content enters a member's index only via the publisher adding its own content or a user-supplied URL.
|
||||
I2 Local sovereignty — nothing centralizes a decision a member could make locally.
|
||||
I2 Local judgment — decisions that consume local information (matching, relevance, sharing, retention) are made locally. Shared coordination (identity, admission, contract) is centralized in the MA, because common state is cheaper held once.
|
||||
I3 Broadcast privacy — broadcast payloads MUST NOT contain third-party private content. Queries derived from scoring others' posts MUST be canonicalized (claims/entities). First-party user-initiated search text MAY be sent as typed.
|
||||
I4 Channel separation — statements advise (aggregates), contracts govern (membership); never cross-wired. The protocol carries no pricing, metering, or settlement.
|
||||
I5 Role symmetry — no privileged roles. Any member may originate queries or responses; no peer may require remote work per incoming query (I7). Neither role is privileged.
|
||||
@@ -33,7 +33,7 @@ Silence is conformant and informative: an unanswered query means no member's ava
|
||||
|
||||
4. Messages
|
||||
|
||||
Envelope (all messages): {type, from, ts, nonce, body, sig} — Ed25519, key listed in the member directory.
|
||||
Envelope (all messages): {type, from, key, ts, nonce, body, sig}. `from` is the sender's member identifier; `key` is the Ed25519 public key used to sign; `sig` covers the canonical form of the other fields (RFC 8785 JCS under a versioned prefix; bodies carry no floating-point numbers). A receiver verifies the signature under `key`, then verifies that `key` is authorized for `from` by the member registry (§6).
|
||||
|
||||
query
|
||||
|
||||
@@ -65,21 +65,21 @@ There is no publish/announce message. Document metadata is carried in responses
|
||||
|
||||
5. Local Policy Domains
|
||||
|
||||
Protocol-silent by design (I2): ranking, ordering, presentation, relevance gating, reputation counters and throttles, verification/spot-checks, caching and invalidation, claim minting, mode (eager/lazy), external fallback, sharing policy above the I9 floor. Advisory reputation bureaus MAY exist; no member is bound.
|
||||
Protocol-silent by design (I2): ranking, ordering, presentation, relevance gating, reputation counters and throttles, verification/spot-checks, caching and invalidation, claim minting, external fallback, sharing policy above the I9 floor. Advisory reputation bureaus MAY exist; no member is bound.
|
||||
|
||||
6. Membership
|
||||
|
||||
The MA governs identity, contract, expulsion — who, never quality. Admission cost is the Sybil defense. Expulsion grounds: fabrication, admission fraud, sustained abuse — never low quality. Escalation: local throttle → advisory aggregates → MA warning → delisting → expulsion. Aggregates are inadmissible as sanction evidence (I4). Membership classes: source members (own content) and enrichment members (derived corpora, e.g. GDELT/CC-NEWS bots — metadata-only exposure, transformation logic open and auditable).
|
||||
The MA governs identity, contract, expulsion — who, never quality. Identifiers are MA-hosted FQDNs (`<label>.frx.<ma-domain>`); no member-controlled DNS is required. Member-hosted identifiers — keys published in the member's own domain and allowlisted by the MA — are planned, not yet normative. The MA maintains a signed, versioned registry snapshot listing identifiers, class, and authorized keys with validity windows. Nodes pin the MA key; the snapshot is the sole authority for the key→identifier binding. Rotation publishes a successor key before retiring its predecessor; revocation removes a key or shortens its validity. Registry outage is fail-static: the last validated snapshot stays in force, and open bootstrap requires an explicit development flag. Admission cost is the Sybil defense. Expulsion grounds: fabrication, admission fraud, sustained abuse — never low quality. Escalation: local throttle → advisory aggregates → MA warning → delisting → expulsion. Aggregates are inadmissible as sanction evidence (I4). Conduct not observable on the wire — link handling, retention, gating — is governed by contract; the protocol neither observes nor adjudicates it. Membership classes: source members (own content) and enrichment members (derived corpora, e.g. GDELT/CC-NEWS bots — metadata-only exposure, transformation logic open and auditable).
|
||||
|
||||
7. Reference Implementation — frxd
|
||||
|
||||
A single static Rust binary. Install, point at a directory, done. It is simultaneously: (a) a personal search engine over local files, (b) a conformant FRX member.
|
||||
|
||||
First run: generate keypair, write config.toml, open localhost web UI (plus frx search CLI). Index target directories with Tantivy; watch for changes; extract text from txt/md/html (PDF optional). Nothing is shared until a collection is explicitly marked shared (I9).
|
||||
First run: generate keypair, join the MA registry (no domain or DNS required), write config.toml, open localhost web UI (plus frx search CLI). Index target directories with Tantivy; watch for changes; extract text from txt/md/html (PDF optional). Nothing is shared until a collection is explicitly marked shared (I9).
|
||||
|
||||
As responder: subscribe to the live query stream via configured relays; match incoming queries against shared collections only (receiver-local lexical/embedding match); respond within budget with honest truncation; sign.
|
||||
|
||||
As consumer: search is local-first; a network toggle broadcasts the query and merges responses, provenance-marked ("your files" / "member X"). Fetched content is retained (fetch-on-miss-and-retain): the kept-set converges to the demand-weighted corpus.
|
||||
As consumer: search is local-first; a network toggle broadcasts the query and merges responses, provenance-marked ("your files" / "member X").
|
||||
|
||||
Dashboard: sent / passed per period (self-derived from local counters; aggregates requested from queriers on demand).
|
||||
|
||||
@@ -93,20 +93,22 @@ Query visibility is total among members; abstraction level and membership are th
|
||||
|
||||
9. Conformance
|
||||
|
||||
A conforming implementation: signs all messages with a listed member key; publishes queries to the firehose only; respects I3 and I9; respects max_results; truncates honestly; sends results without scores; enforces visible transport backpressure, never silent transport drops; serves aggregates on request at or above the granularity floor; ingests content only via publisher-added content or user-supplied URL; implements no dispute messages.
|
||||
A conforming implementation: signs all messages with a key authorized for its identifier in the member registry; publishes queries to the firehose only; respects I3 and I9; respects max_results; truncates honestly; sends results without scores; enforces visible transport backpressure, never silent transport drops; serves aggregates on request at or above the granularity floor; ingests content only via publisher-added content or user-supplied URL; implements no dispute messages.
|
||||
|
||||
10. Open Issues
|
||||
|
||||
Consumer admission tier — automated/invite admission for distributed binaries without weakening the Sybil defense (MA policy, gates §7 adoption).
|
||||
Relay discovery and default-relay governance (shipped defaults are soft centralization; mitigate with multiple defaults + one-command self-host).
|
||||
Delegation grant mechanism.
|
||||
Signature canonicalization scheme (blocks Phase-1 interop; required before two implementations can exchange a valid envelope).
|
||||
Member-hosted identifiers — keys published in the member's own DNS instead of the MA registry; MA-hosted is normative until specified.
|
||||
MA anchor rollover — successor commitment and overlap for the registry signing key.
|
||||
Unicast confidentiality — response and aggregate payloads are visible to relays; no end-to-end scheme is specified.
|
||||
Document lineage (revision/supersedes) and delegation without a supply stream — previously carried by publish; now unspecified.
|
||||
Claim/entity minting conventions — recommended, non-normative (I8).
|
||||
|
||||
Appendix A. Recommended Local Practices (Non-Normative)
|
||||
|
||||
Two-stage ingestion (cheap gate before any LLM attention); per-peer sent/passed counters, throttling on ratios; verdict memoization and claim normalization; demand-driven prefetch (fetch and retain on attention); hot-set replication; backfill seeding (Wikipedia/Wikidata, GDELT, CC-NEWS); measure query-to-claim collapse and domain concentration before sizing anything.
|
||||
Two-stage ingestion (cheap gate before any LLM attention); per-peer sent/passed counters, throttling on ratios; verdict memoization and claim normalization; hot-set replication; backfill seeding (Wikipedia/Wikidata, GDELT, CC-NEWS); measure query-to-claim collapse and domain concentration before sizing anything.
|
||||
|
||||
Appendix B. Purge Log (Normative)
|
||||
|
||||
|
||||
+246
-5
@@ -1,12 +1,20 @@
|
||||
use std::path::Path;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::{Context, Result, anyhow};
|
||||
use axum::extract::State;
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Response};
|
||||
use axum::routing::get;
|
||||
use axum::{Json, Router};
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
use crate::config::{CLASS_ENRICHMENT, CLASS_SOURCE, Config, Member, load_members, save_members};
|
||||
use crate::crypto::{Keypair, now_ts};
|
||||
use crate::index::{Collection, LocalIndex, load_collections, save_collections};
|
||||
use crate::message::{EXPOSURE_FULL, EXPOSURE_METADATA};
|
||||
use crate::node;
|
||||
use crate::registry::{self, KeyEntry, RegistryDoc, RegistryMember, SignedRegistry};
|
||||
use crate::render;
|
||||
|
||||
pub fn add(
|
||||
@@ -92,12 +100,27 @@ pub async fn query(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
let bytes = hex::decode(pubkey).context("pubkey must be hex")?;
|
||||
fn normalize_key(value: &str) -> Result<String> {
|
||||
let bytes = hex::decode(value).context("pubkey must be hex")?;
|
||||
if bytes.len() != 32 {
|
||||
return Err(anyhow!("pubkey must be 32 bytes (64 hex characters)"));
|
||||
}
|
||||
Ok(value.to_ascii_lowercase())
|
||||
}
|
||||
|
||||
pub fn member_add(
|
||||
config_path: &Path,
|
||||
name: &str,
|
||||
pubkey: &str,
|
||||
class: &str,
|
||||
previous: &[String],
|
||||
) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
let pubkey = normalize_key(pubkey)?;
|
||||
let previous = previous
|
||||
.iter()
|
||||
.map(|key| normalize_key(key))
|
||||
.collect::<Result<Vec<_>>>()?;
|
||||
let class = if class == CLASS_ENRICHMENT {
|
||||
CLASS_ENRICHMENT
|
||||
} else {
|
||||
@@ -107,8 +130,9 @@ pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) ->
|
||||
members.retain(|member| member.name != name && member.pubkey != pubkey);
|
||||
members.push(Member {
|
||||
name: name.to_string(),
|
||||
pubkey: pubkey.to_ascii_lowercase(),
|
||||
pubkey,
|
||||
class: class.to_string(),
|
||||
previous,
|
||||
});
|
||||
save_members(&config.members_path(), &members)?;
|
||||
println!(
|
||||
@@ -118,6 +142,30 @@ pub fn member_add(config_path: &Path, name: &str, pubkey: &str, class: &str) ->
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn key_show(config_path: &Path) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
println!("{}", config.load_key()?.public_hex());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn key_rotate(config_path: &Path) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
let old = config.load_key()?;
|
||||
let backup = config.key_path().with_extension("hex.bak");
|
||||
std::fs::copy(config.key_path(), &backup)?;
|
||||
let new_key = Keypair::generate();
|
||||
config.save_key(&new_key)?;
|
||||
println!("old pubkey {}", old.public_hex());
|
||||
println!("new pubkey {}", new_key.public_hex());
|
||||
println!("old key saved to {}", backup.display());
|
||||
println!(
|
||||
"peers accept the new key via: frxd member add <name> {} --previous {}",
|
||||
new_key.public_hex(),
|
||||
old.public_hex()
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn member_remove(config_path: &Path, name: &str) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
let mut members = load_members(&config.members_path())?;
|
||||
@@ -184,6 +232,199 @@ pub async fn aggregates(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn registry_key_path(dir: &Path) -> PathBuf {
|
||||
dir.join("ma-key.hex")
|
||||
}
|
||||
|
||||
fn registry_doc_path(dir: &Path) -> PathBuf {
|
||||
dir.join("registry.json")
|
||||
}
|
||||
|
||||
fn open_registry(dir: &Path) -> Result<(Keypair, SignedRegistry)> {
|
||||
let raw = std::fs::read_to_string(registry_key_path(dir))
|
||||
.with_context(|| format!("reading MA key from {}", registry_key_path(dir).display()))?;
|
||||
let key = Keypair::from_hex(&raw)?;
|
||||
let signed = registry::load_registry(®istry_doc_path(dir))
|
||||
.with_context(|| format!("reading registry from {}", registry_doc_path(dir).display()))?;
|
||||
Ok((key, signed))
|
||||
}
|
||||
|
||||
fn mutate_registry(dir: &Path, mutate: impl FnOnce(&mut RegistryDoc) -> Result<()>) -> Result<u64> {
|
||||
let (ma, signed) = open_registry(dir)?;
|
||||
let mut doc = signed.doc;
|
||||
mutate(&mut doc)?;
|
||||
doc.version += 1;
|
||||
doc.issued_at = now_ts();
|
||||
let signed = registry::sign_registry(doc, &ma);
|
||||
registry::save_registry(®istry_doc_path(dir), &signed)?;
|
||||
Ok(signed.doc.version)
|
||||
}
|
||||
|
||||
pub fn registry_init(dir: &Path) -> Result<()> {
|
||||
let registry_path = registry_doc_path(dir);
|
||||
if registry_path.exists() {
|
||||
return Err(anyhow!(
|
||||
"registry {} already exists",
|
||||
registry_path.display()
|
||||
));
|
||||
}
|
||||
std::fs::create_dir_all(dir)?;
|
||||
let ma = Keypair::generate();
|
||||
std::fs::write(registry_key_path(dir), ma.to_hex())?;
|
||||
crate::config::set_private_permissions(®istry_key_path(dir))?;
|
||||
let doc = RegistryDoc {
|
||||
version: 1,
|
||||
issued_at: now_ts(),
|
||||
ma_key: String::new(),
|
||||
members: Vec::new(),
|
||||
};
|
||||
let signed = registry::sign_registry(doc, &ma);
|
||||
registry::save_registry(®istry_path, &signed)?;
|
||||
println!("MA key {}", ma.public_hex());
|
||||
println!("registry {}", registry_path.display());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_add(
|
||||
dir: &Path,
|
||||
id: &str,
|
||||
pubkey: &str,
|
||||
class: &str,
|
||||
not_before: Option<u64>,
|
||||
not_after: Option<u64>,
|
||||
) -> Result<()> {
|
||||
let pubkey = normalize_key(pubkey)?;
|
||||
let class = if class == CLASS_ENRICHMENT {
|
||||
CLASS_ENRICHMENT
|
||||
} else {
|
||||
CLASS_SOURCE
|
||||
};
|
||||
mutate_registry(dir, |doc| {
|
||||
if doc.members.iter().any(|member| member.id == id) {
|
||||
return Err(anyhow!("member {id} already listed"));
|
||||
}
|
||||
doc.members.push(RegistryMember {
|
||||
id: id.to_string(),
|
||||
class: class.to_string(),
|
||||
keys: vec![KeyEntry {
|
||||
key: pubkey.clone(),
|
||||
not_before: not_before.unwrap_or_else(now_ts),
|
||||
not_after,
|
||||
}],
|
||||
});
|
||||
Ok(())
|
||||
})?;
|
||||
println!("added {id} ({class})");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_add_key(
|
||||
dir: &Path,
|
||||
id: &str,
|
||||
pubkey: &str,
|
||||
not_before: Option<u64>,
|
||||
not_after: Option<u64>,
|
||||
) -> Result<()> {
|
||||
let pubkey = normalize_key(pubkey)?;
|
||||
mutate_registry(dir, |doc| {
|
||||
let Some(member) = doc.members.iter_mut().find(|member| member.id == id) else {
|
||||
return Err(anyhow!("no member named {id}"));
|
||||
};
|
||||
if member.keys.iter().any(|entry| entry.key == pubkey) {
|
||||
return Err(anyhow!("key already authorized for {id}"));
|
||||
}
|
||||
member.keys.push(KeyEntry {
|
||||
key: pubkey.clone(),
|
||||
not_before: not_before.unwrap_or_else(now_ts),
|
||||
not_after,
|
||||
});
|
||||
Ok(())
|
||||
})?;
|
||||
println!("added key for {id}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_revoke_key(dir: &Path, id: &str, pubkey: &str) -> Result<()> {
|
||||
let pubkey = normalize_key(pubkey)?;
|
||||
mutate_registry(dir, |doc| {
|
||||
let Some(member) = doc.members.iter_mut().find(|member| member.id == id) else {
|
||||
return Err(anyhow!("no member named {id}"));
|
||||
};
|
||||
let before = member.keys.len();
|
||||
member.keys.retain(|entry| entry.key != pubkey);
|
||||
if member.keys.len() == before {
|
||||
return Err(anyhow!("key not authorized for {id}"));
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
println!("revoked key for {id}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_remove(dir: &Path, id: &str) -> Result<()> {
|
||||
mutate_registry(dir, |doc| {
|
||||
let before = doc.members.len();
|
||||
doc.members.retain(|member| member.id != id);
|
||||
if doc.members.len() == before {
|
||||
return Err(anyhow!("no member named {id}"));
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
println!("removed {id}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_list(dir: &Path) -> Result<()> {
|
||||
let (_, signed) = open_registry(dir)?;
|
||||
for member in &signed.doc.members {
|
||||
let keys = member.keys.len();
|
||||
println!("{} [{}] ({} key(s))", member.id, member.class, keys);
|
||||
for entry in &member.keys {
|
||||
let window = match entry.not_after {
|
||||
Some(end) => format!("valid {}..{}", entry.not_before, end),
|
||||
None => format!("valid from {}", entry.not_before),
|
||||
};
|
||||
println!(" {} ({window})", entry.key);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn registry_show(dir: &Path) -> Result<()> {
|
||||
let (_, signed) = open_registry(dir)?;
|
||||
println!("ma_key {}", signed.doc.ma_key);
|
||||
println!("version {}", signed.doc.version);
|
||||
println!("issued_at {}", signed.doc.issued_at);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn registry_serve(dir: &Path, listen: &str) -> Result<()> {
|
||||
let state = dir.to_path_buf();
|
||||
let app = Router::new()
|
||||
.route("/health", get(registry_health))
|
||||
.route("/registry.json", get(registry_snapshot))
|
||||
.with_state(state);
|
||||
let listener = TcpListener::bind(listen).await?;
|
||||
println!("registry serving on http://{}", listener.local_addr()?);
|
||||
axum::serve(listener, app).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn registry_health() -> &'static str {
|
||||
"ok"
|
||||
}
|
||||
|
||||
async fn registry_snapshot(State(dir): State<PathBuf>) -> Response {
|
||||
match registry::load_registry(®istry_doc_path(&dir)) {
|
||||
Ok(signed) => Json(signed).into_response(),
|
||||
Err(_) => (
|
||||
StatusCode::NOT_FOUND,
|
||||
Json(serde_json::json!({ "error": "registry not found" })),
|
||||
)
|
||||
.into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn status(config_path: &Path) -> Result<()> {
|
||||
let config = Config::load(config_path)?;
|
||||
let base = format!("http://{}", config.node.listen);
|
||||
|
||||
+19
-1
@@ -18,8 +18,16 @@ pub struct Config {
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct NodeSection {
|
||||
pub name: String,
|
||||
#[serde(default)]
|
||||
pub id: Option<String>,
|
||||
pub listen: String,
|
||||
pub relays: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub registry: Option<String>,
|
||||
#[serde(default)]
|
||||
pub ma_key: Option<String>,
|
||||
#[serde(default)]
|
||||
pub dev_bootstrap: bool,
|
||||
#[serde(default = "default_true")]
|
||||
pub responder: bool,
|
||||
}
|
||||
@@ -33,6 +41,8 @@ pub struct Member {
|
||||
pub pubkey: String,
|
||||
#[serde(default = "default_class")]
|
||||
pub class: String,
|
||||
#[serde(default)]
|
||||
pub previous: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Serialize, Deserialize)]
|
||||
@@ -117,8 +127,12 @@ impl Config {
|
||||
Self {
|
||||
node: NodeSection {
|
||||
name: name.to_string(),
|
||||
id: None,
|
||||
listen: listen.to_string(),
|
||||
relays: vec![relay.to_string()],
|
||||
registry: None,
|
||||
ma_key: None,
|
||||
dev_bootstrap: false,
|
||||
responder: true,
|
||||
},
|
||||
query: QuerySection::default(),
|
||||
@@ -164,6 +178,10 @@ impl Config {
|
||||
self.data_dir().join("members.toml")
|
||||
}
|
||||
|
||||
pub fn registry_cache_path(&self) -> PathBuf {
|
||||
self.data_dir().join("registry-cache.json")
|
||||
}
|
||||
|
||||
pub fn load_key(&self) -> Result<Keypair> {
|
||||
let raw = fs::read_to_string(self.key_path())
|
||||
.with_context(|| format!("reading key {}", self.key_path().display()))?;
|
||||
@@ -179,7 +197,7 @@ impl Config {
|
||||
}
|
||||
}
|
||||
|
||||
fn set_private_permissions(path: &Path) -> Result<()> {
|
||||
pub fn set_private_permissions(path: &Path) -> Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
|
||||
+24
-16
@@ -97,6 +97,29 @@ fn write_canonical(value: &Value, out: &mut String) {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn poll_signing_bytes(member: &str, nonce: &str) -> Vec<u8> {
|
||||
format!("{}\npoll\n{}\n{}", PROTOCOL, member, nonce).into_bytes()
|
||||
}
|
||||
|
||||
pub fn verify_signature(public_hex: &str, message: &[u8], sig_hex: &str) -> Result<()> {
|
||||
let key_bytes = hex::decode(public_hex).context("public key is not hex")?;
|
||||
let key_bytes: [u8; 32] = key_bytes
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| anyhow!("public key must be 32 bytes"))?;
|
||||
let verifying_key =
|
||||
VerifyingKey::from_bytes(&key_bytes).map_err(|e| anyhow!("bad public key: {e}"))?;
|
||||
let sig_bytes = hex::decode(sig_hex).context("signature is not hex")?;
|
||||
let sig_bytes: [u8; 64] = sig_bytes
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| anyhow!("signature must be 64 bytes"))?;
|
||||
let signature = Signature::from_bytes(&sig_bytes);
|
||||
verifying_key
|
||||
.verify_strict(message, &signature)
|
||||
.map_err(|_| anyhow!("signature verification failed"))
|
||||
}
|
||||
|
||||
pub fn signing_bytes(envelope: &Envelope) -> Vec<u8> {
|
||||
format!(
|
||||
"{}\n{}\n{}\n{}\n{}\n{}",
|
||||
@@ -111,22 +134,7 @@ pub fn signing_bytes(envelope: &Envelope) -> Vec<u8> {
|
||||
}
|
||||
|
||||
pub fn verify_envelope(envelope: &Envelope) -> Result<()> {
|
||||
let key_bytes = hex::decode(&envelope.from).context("from is not hex")?;
|
||||
let key_bytes: [u8; 32] = key_bytes
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| anyhow!("from must be a 32-byte ed25519 public key"))?;
|
||||
let verifying_key =
|
||||
VerifyingKey::from_bytes(&key_bytes).map_err(|e| anyhow!("bad public key: {e}"))?;
|
||||
let sig_bytes = hex::decode(&envelope.sig).context("sig is not hex")?;
|
||||
let sig_bytes: [u8; 64] = sig_bytes
|
||||
.as_slice()
|
||||
.try_into()
|
||||
.map_err(|_| anyhow!("sig must be 64 bytes"))?;
|
||||
let signature = Signature::from_bytes(&sig_bytes);
|
||||
verifying_key
|
||||
.verify_strict(&signing_bytes(envelope), &signature)
|
||||
.map_err(|_| anyhow!("signature verification failed"))
|
||||
verify_signature(&envelope.from, &signing_bytes(envelope), &envelope.sig)
|
||||
}
|
||||
|
||||
pub fn now_ts() -> u64 {
|
||||
|
||||
@@ -5,6 +5,7 @@ pub mod extract;
|
||||
pub mod index;
|
||||
pub mod message;
|
||||
pub mod node;
|
||||
pub mod registry;
|
||||
pub mod relay;
|
||||
pub mod render;
|
||||
|
||||
|
||||
+105
-2
@@ -33,6 +33,12 @@ enum Command {
|
||||
data_dir: String,
|
||||
#[arg(long)]
|
||||
force: bool,
|
||||
#[arg(long)]
|
||||
id: Option<String>,
|
||||
#[arg(long)]
|
||||
registry: Option<String>,
|
||||
#[arg(long)]
|
||||
ma_key: Option<String>,
|
||||
},
|
||||
Add {
|
||||
path: PathBuf,
|
||||
@@ -70,6 +76,16 @@ enum Command {
|
||||
#[command(subcommand)]
|
||||
command: MemberCommand,
|
||||
},
|
||||
Key {
|
||||
#[command(subcommand)]
|
||||
command: KeyCommand,
|
||||
},
|
||||
Registry {
|
||||
#[arg(long, default_value = "./frx-registry")]
|
||||
dir: PathBuf,
|
||||
#[command(subcommand)]
|
||||
command: RegistryCommand,
|
||||
},
|
||||
Aggregates {
|
||||
#[arg(long)]
|
||||
from: Option<String>,
|
||||
@@ -87,6 +103,8 @@ enum MemberCommand {
|
||||
pubkey: String,
|
||||
#[arg(long, default_value = "source")]
|
||||
class: String,
|
||||
#[arg(long = "previous")]
|
||||
previous: Vec<String>,
|
||||
},
|
||||
Remove {
|
||||
name: String,
|
||||
@@ -94,6 +112,48 @@ enum MemberCommand {
|
||||
List,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum KeyCommand {
|
||||
Show,
|
||||
Rotate,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum RegistryCommand {
|
||||
Init,
|
||||
Add {
|
||||
id: String,
|
||||
pubkey: String,
|
||||
#[arg(long, default_value = "source")]
|
||||
class: String,
|
||||
#[arg(long)]
|
||||
not_before: Option<u64>,
|
||||
#[arg(long)]
|
||||
not_after: Option<u64>,
|
||||
},
|
||||
AddKey {
|
||||
id: String,
|
||||
pubkey: String,
|
||||
#[arg(long)]
|
||||
not_before: Option<u64>,
|
||||
#[arg(long)]
|
||||
not_after: Option<u64>,
|
||||
},
|
||||
RevokeKey {
|
||||
id: String,
|
||||
pubkey: String,
|
||||
},
|
||||
Remove {
|
||||
id: String,
|
||||
},
|
||||
List,
|
||||
Show,
|
||||
Serve {
|
||||
#[arg(long, default_value = "127.0.0.1:7800")]
|
||||
listen: String,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, ValueEnum)]
|
||||
enum Exposure {
|
||||
Metadata,
|
||||
@@ -110,6 +170,9 @@ async fn main() -> Result<()> {
|
||||
relay,
|
||||
data_dir,
|
||||
force,
|
||||
id,
|
||||
registry,
|
||||
ma_key,
|
||||
} => {
|
||||
if cli.config.exists() && !force {
|
||||
bail!(
|
||||
@@ -117,7 +180,19 @@ async fn main() -> Result<()> {
|
||||
cli.config.display()
|
||||
);
|
||||
}
|
||||
let config = Config::new(&name, &listen, &relay, &data_dir);
|
||||
if registry.is_some() && ma_key.is_none() {
|
||||
bail!("--ma-key is required with --registry");
|
||||
}
|
||||
let mut config = Config::new(&name, &listen, &relay, &data_dir);
|
||||
config.node.id = id;
|
||||
config.node.registry = registry;
|
||||
config.node.ma_key = ma_key;
|
||||
if config.node.registry.is_none() {
|
||||
config.node.dev_bootstrap = true;
|
||||
println!(
|
||||
"warning: no registry configured; development open bootstrap (any valid key is accepted; do not deploy)"
|
||||
);
|
||||
}
|
||||
let key = Keypair::generate();
|
||||
config.save_key(&key)?;
|
||||
config.save(&cli.config)?;
|
||||
@@ -167,10 +242,38 @@ async fn main() -> Result<()> {
|
||||
name,
|
||||
pubkey,
|
||||
class,
|
||||
} => commands::member_add(&cli.config, &name, &pubkey, &class)?,
|
||||
previous,
|
||||
} => commands::member_add(&cli.config, &name, &pubkey, &class, &previous)?,
|
||||
MemberCommand::Remove { name } => commands::member_remove(&cli.config, &name)?,
|
||||
MemberCommand::List => commands::member_list(&cli.config)?,
|
||||
},
|
||||
Command::Key { command } => match command {
|
||||
KeyCommand::Show => commands::key_show(&cli.config)?,
|
||||
KeyCommand::Rotate => commands::key_rotate(&cli.config)?,
|
||||
},
|
||||
Command::Registry { dir, command } => match command {
|
||||
RegistryCommand::Init => commands::registry_init(&dir)?,
|
||||
RegistryCommand::Add {
|
||||
id,
|
||||
pubkey,
|
||||
class,
|
||||
not_before,
|
||||
not_after,
|
||||
} => commands::registry_add(&dir, &id, &pubkey, &class, not_before, not_after)?,
|
||||
RegistryCommand::AddKey {
|
||||
id,
|
||||
pubkey,
|
||||
not_before,
|
||||
not_after,
|
||||
} => commands::registry_add_key(&dir, &id, &pubkey, not_before, not_after)?,
|
||||
RegistryCommand::RevokeKey { id, pubkey } => {
|
||||
commands::registry_revoke_key(&dir, &id, &pubkey)?
|
||||
}
|
||||
RegistryCommand::Remove { id } => commands::registry_remove(&dir, &id)?,
|
||||
RegistryCommand::List => commands::registry_list(&dir)?,
|
||||
RegistryCommand::Show => commands::registry_show(&dir)?,
|
||||
RegistryCommand::Serve { listen } => commands::registry_serve(&dir, &listen).await?,
|
||||
},
|
||||
Command::Aggregates {
|
||||
from,
|
||||
period,
|
||||
|
||||
@@ -11,6 +11,12 @@ pub const TYPE_AGGREGATE: &str = "aggregate";
|
||||
pub const EXPOSURE_METADATA: &str = "metadata";
|
||||
pub const EXPOSURE_FULL: &str = "full";
|
||||
|
||||
pub const MAX_CLOCK_SKEW_SECS: u64 = 300;
|
||||
|
||||
pub fn timestamp_is_fresh(ts: u64, now: u64) -> bool {
|
||||
now.abs_diff(ts) <= MAX_CLOCK_SKEW_SECS
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Envelope {
|
||||
#[serde(rename = "type")]
|
||||
|
||||
+166
-10
@@ -17,12 +17,13 @@ use tokio::net::TcpListener;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::config::{CLASS_ENRICHMENT, Config, Member, load_members};
|
||||
use crate::crypto::Keypair;
|
||||
use crate::crypto::{Keypair, now_ts, poll_signing_bytes};
|
||||
use crate::index::{LocalIndex, SearchHit, response_items};
|
||||
use crate::message::{
|
||||
AggregateBody, Envelope, QueryBody, ResponseBody, ResponseItem, TYPE_AGGREGATE, TYPE_QUERY,
|
||||
TYPE_RESPONSE, build_response,
|
||||
TYPE_RESPONSE, build_response, timestamp_is_fresh,
|
||||
};
|
||||
use crate::registry::{SignedRegistry, authorized_keys, load_registry, verify_registry};
|
||||
|
||||
#[derive(Default)]
|
||||
struct Aggregates {
|
||||
@@ -36,6 +37,8 @@ pub struct Node {
|
||||
index: LocalIndex,
|
||||
members: RwLock<Vec<Member>>,
|
||||
members_mtime: Mutex<Option<SystemTime>>,
|
||||
registry: Mutex<Option<SignedRegistry>>,
|
||||
registry_mtime: Mutex<Option<SystemTime>>,
|
||||
aggregates: Mutex<Aggregates>,
|
||||
seen: Mutex<HashSet<String>>,
|
||||
pending: Mutex<HashMap<String, Vec<(String, ResponseBody)>>>,
|
||||
@@ -119,12 +122,14 @@ impl Node {
|
||||
.timeout(Duration::from_secs(15))
|
||||
.build()
|
||||
.context("building http client")?;
|
||||
Ok(Arc::new(Self {
|
||||
let node = Arc::new(Self {
|
||||
config,
|
||||
key,
|
||||
index,
|
||||
members: RwLock::new(members),
|
||||
members_mtime: Mutex::new(members_mtime),
|
||||
registry: Mutex::new(None),
|
||||
registry_mtime: Mutex::new(None),
|
||||
aggregates: Mutex::new(Aggregates::default()),
|
||||
seen: Mutex::new(HashSet::new()),
|
||||
pending: Mutex::new(HashMap::new()),
|
||||
@@ -132,7 +137,90 @@ impl Node {
|
||||
client,
|
||||
sent: AtomicU64::new(0),
|
||||
received: AtomicU64::new(0),
|
||||
}))
|
||||
});
|
||||
node.load_initial_registry();
|
||||
Ok(node)
|
||||
}
|
||||
|
||||
fn load_initial_registry(&self) {
|
||||
let Some(source) = &self.config.node.registry else {
|
||||
return;
|
||||
};
|
||||
let result = if source.starts_with("http://") || source.starts_with("https://") {
|
||||
load_registry(&self.config.registry_cache_path())
|
||||
} else {
|
||||
let path = std::path::PathBuf::from(source);
|
||||
let loaded = load_registry(&path);
|
||||
if loaded.is_ok() {
|
||||
let mtime = fs::metadata(&path)
|
||||
.and_then(|metadata| metadata.modified())
|
||||
.ok();
|
||||
*self.registry_mtime.lock().expect("registry mtime lock") = mtime;
|
||||
}
|
||||
loaded
|
||||
};
|
||||
if let Ok(signed) = result {
|
||||
if let Err(error) = self.verify_and_apply_registry(signed) {
|
||||
eprintln!("cached registry rejected: {error}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn verify_and_apply_registry(&self, signed: SignedRegistry) -> Result<()> {
|
||||
let ma_key = self
|
||||
.config
|
||||
.node
|
||||
.ma_key
|
||||
.as_deref()
|
||||
.ok_or_else(|| anyhow!("registry configured without ma_key"))?;
|
||||
verify_registry(&signed, ma_key)?;
|
||||
{
|
||||
let current = self.registry.lock().expect("registry lock");
|
||||
if let Some(existing) = current.as_ref() {
|
||||
if signed.doc.version <= existing.doc.version {
|
||||
return Err(anyhow!("registry version rollback rejected"));
|
||||
}
|
||||
}
|
||||
}
|
||||
*self.registry.lock().expect("registry lock") = Some(signed);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn refresh_registry(&self) {
|
||||
let Some(source) = &self.config.node.registry else {
|
||||
return;
|
||||
};
|
||||
if source.starts_with("http://") || source.starts_with("https://") {
|
||||
return;
|
||||
}
|
||||
let path = std::path::PathBuf::from(source);
|
||||
let mtime = fs::metadata(&path)
|
||||
.and_then(|metadata| metadata.modified())
|
||||
.ok();
|
||||
{
|
||||
let last = self.registry_mtime.lock().expect("registry mtime lock");
|
||||
if *last == mtime {
|
||||
return;
|
||||
}
|
||||
}
|
||||
let Ok(signed) = load_registry(&path) else {
|
||||
return;
|
||||
};
|
||||
*self.registry_mtime.lock().expect("registry mtime lock") = mtime;
|
||||
if let Err(error) = self.verify_and_apply_registry(signed) {
|
||||
eprintln!("registry update rejected: {error}");
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_registry(&self) -> Result<()> {
|
||||
let Some(url) = &self.config.node.registry else {
|
||||
return Ok(());
|
||||
};
|
||||
let response = self.client.get(url).send().await?;
|
||||
let signed: SignedRegistry = response.json().await?;
|
||||
self.verify_and_apply_registry(signed.clone())?;
|
||||
crate::registry::save_registry(&self.config.registry_cache_path(), &signed)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn refresh_members(&self) {
|
||||
@@ -156,7 +244,9 @@ impl Node {
|
||||
fn member_class(&self, members: &[Member], pubkey: &str) -> Option<String> {
|
||||
members
|
||||
.iter()
|
||||
.find(|member| member.pubkey == pubkey)
|
||||
.find(|member| {
|
||||
member.pubkey == pubkey || member.previous.iter().any(|key| key == pubkey)
|
||||
})
|
||||
.map(|member| member.class.clone())
|
||||
}
|
||||
|
||||
@@ -213,6 +303,22 @@ impl Node {
|
||||
.with_context(|| format!("binding {}", node.config.node.listen))?;
|
||||
let addr = listener.local_addr()?;
|
||||
let mut tasks = Vec::new();
|
||||
if let Some(source) = &node.config.node.registry {
|
||||
if source.starts_with("http://") || source.starts_with("https://") {
|
||||
if let Err(error) = node.fetch_registry().await {
|
||||
eprintln!("initial registry fetch failed: {error}");
|
||||
}
|
||||
let node_for_registry = node.clone();
|
||||
tasks.push(tokio::spawn(async move {
|
||||
loop {
|
||||
if let Err(error) = node_for_registry.fetch_registry().await {
|
||||
eprintln!("registry fetch failed: {error}");
|
||||
}
|
||||
tokio::time::sleep(Duration::from_secs(60)).await;
|
||||
}
|
||||
}));
|
||||
}
|
||||
}
|
||||
for relay in node.config.node.relays.clone() {
|
||||
tasks.push(tokio::spawn(poll_relay(node.clone(), relay)));
|
||||
}
|
||||
@@ -333,11 +439,31 @@ impl Node {
|
||||
if envelope.verify().is_err() {
|
||||
return;
|
||||
}
|
||||
if !timestamp_is_fresh(envelope.ts, crate::crypto::now_ts()) {
|
||||
return;
|
||||
}
|
||||
self.refresh_members();
|
||||
let (class, listed) = {
|
||||
self.refresh_registry();
|
||||
let (class, listed) = if self.config.node.registry.is_some() {
|
||||
let registry = self.registry.lock().expect("registry lock");
|
||||
let authorized = registry
|
||||
.as_ref()
|
||||
.map(|signed| authorized_keys(signed, now_ts()));
|
||||
match authorized {
|
||||
Some(map) => {
|
||||
let entry = map.get(&envelope.from);
|
||||
(entry.map(|(_, class)| class.clone()), entry.is_some())
|
||||
}
|
||||
None => (None, false),
|
||||
}
|
||||
} else {
|
||||
let members = self.members.read().expect("members lock");
|
||||
let class = self.member_class(&members, &envelope.from);
|
||||
let listed = members.is_empty() || class.is_some();
|
||||
let listed = if members.is_empty() {
|
||||
self.config.node.dev_bootstrap
|
||||
} else {
|
||||
class.is_some()
|
||||
};
|
||||
(class, listed)
|
||||
};
|
||||
if !listed {
|
||||
@@ -504,13 +630,35 @@ impl Node {
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_challenge(node: &Node, base: &str, member: &str) -> Option<String> {
|
||||
let response = node
|
||||
.client
|
||||
.get(format!("{base}/v1/challenge?member={member}"))
|
||||
.send()
|
||||
.await
|
||||
.ok()?;
|
||||
if !response.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
let payload: Value = response.json().await.ok()?;
|
||||
payload
|
||||
.get("nonce")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string)
|
||||
}
|
||||
|
||||
async fn poll_relay(node: Arc<Node>, relay: String) {
|
||||
let base = relay.trim_end_matches('/').to_string();
|
||||
let member = node.key.public_hex();
|
||||
loop {
|
||||
let Some(nonce) = fetch_challenge(&node, &base, &member).await else {
|
||||
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||
continue;
|
||||
};
|
||||
let signature = node.key.sign(&poll_signing_bytes(&member, &nonce));
|
||||
let url = format!(
|
||||
"{}/v1/poll?member={}&timeout_ms=20000",
|
||||
base,
|
||||
node.key.public_hex()
|
||||
"{}/v1/poll?member={}&nonce={}&sig={}&timeout_ms=20000",
|
||||
base, member, nonce, signature
|
||||
);
|
||||
match node.client.get(&url).send().await {
|
||||
Ok(response) if response.status() == reqwest::StatusCode::NO_CONTENT => continue,
|
||||
@@ -582,8 +730,16 @@ async fn local_query(
|
||||
|
||||
async fn local_status(State(node): State<Arc<Node>>) -> Response {
|
||||
let aggregates = node.aggregate_for(¤t_period(), None);
|
||||
let registry_version = node
|
||||
.registry
|
||||
.lock()
|
||||
.expect("registry lock")
|
||||
.as_ref()
|
||||
.map(|signed| signed.doc.version);
|
||||
Json(json!({
|
||||
"name": node.config.node.name,
|
||||
"id": node.config.node.id,
|
||||
"registry_version": registry_version,
|
||||
"pubkey": node.key.public_hex(),
|
||||
"listen": node.config.node.listen,
|
||||
"relays": node.config.node.relays,
|
||||
|
||||
+176
@@ -0,0 +1,176 @@
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{Context, Result, anyhow};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::PROTOCOL;
|
||||
#[cfg(test)]
|
||||
use crate::crypto::now_ts;
|
||||
use crate::crypto::{Keypair, canonical_json, verify_signature};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct KeyEntry {
|
||||
pub key: String,
|
||||
#[serde(default)]
|
||||
pub not_before: u64,
|
||||
#[serde(default)]
|
||||
pub not_after: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct RegistryMember {
|
||||
pub id: String,
|
||||
#[serde(default)]
|
||||
pub class: String,
|
||||
#[serde(default)]
|
||||
pub keys: Vec<KeyEntry>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct RegistryDoc {
|
||||
pub version: u64,
|
||||
pub issued_at: u64,
|
||||
pub ma_key: String,
|
||||
#[serde(default)]
|
||||
pub members: Vec<RegistryMember>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SignedRegistry {
|
||||
#[serde(flatten)]
|
||||
pub doc: RegistryDoc,
|
||||
pub sig: String,
|
||||
}
|
||||
|
||||
pub fn registry_signing_bytes(doc: &RegistryDoc) -> Vec<u8> {
|
||||
let value = serde_json::to_value(doc).expect("registry doc serializes");
|
||||
format!("{}\nregistry\n{}", PROTOCOL, canonical_json(&value)).into_bytes()
|
||||
}
|
||||
|
||||
pub fn sign_registry(mut doc: RegistryDoc, key: &Keypair) -> SignedRegistry {
|
||||
doc.ma_key = key.public_hex();
|
||||
let sig = key.sign(®istry_signing_bytes(&doc));
|
||||
SignedRegistry { doc, sig }
|
||||
}
|
||||
|
||||
pub fn verify_registry(signed: &SignedRegistry, ma_key: &str) -> Result<()> {
|
||||
if signed.doc.ma_key != ma_key {
|
||||
return Err(anyhow!("registry signed by unexpected MA key"));
|
||||
}
|
||||
verify_signature(ma_key, ®istry_signing_bytes(&signed.doc), &signed.sig)
|
||||
}
|
||||
|
||||
pub fn load_registry(path: &Path) -> Result<SignedRegistry> {
|
||||
let raw = fs::read_to_string(path).context("reading registry")?;
|
||||
serde_json::from_str(&raw).context("parsing registry")
|
||||
}
|
||||
|
||||
pub fn save_registry(path: &Path, signed: &SignedRegistry) -> Result<()> {
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
fs::write(path, serde_json::to_string_pretty(signed)?)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn authorized_keys(signed: &SignedRegistry, now: u64) -> HashMap<String, (String, String)> {
|
||||
let mut authorized = HashMap::new();
|
||||
for member in &signed.doc.members {
|
||||
for key in &member.keys {
|
||||
let valid = now >= key.not_before && key.not_after.map(|end| now < end).unwrap_or(true);
|
||||
if valid {
|
||||
authorized
|
||||
.entry(key.key.clone())
|
||||
.or_insert((member.id.clone(), member.class.clone()));
|
||||
}
|
||||
}
|
||||
}
|
||||
authorized
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::crypto::Keypair;
|
||||
|
||||
fn make_registry(
|
||||
key: &Keypair,
|
||||
entries: Vec<(&str, &str, u64, Option<u64>)>,
|
||||
) -> SignedRegistry {
|
||||
let members = entries
|
||||
.into_iter()
|
||||
.map(|(id, key_hex, not_before, not_after)| RegistryMember {
|
||||
id: id.to_string(),
|
||||
class: "source".to_string(),
|
||||
keys: vec![KeyEntry {
|
||||
key: key_hex.to_string(),
|
||||
not_before,
|
||||
not_after,
|
||||
}],
|
||||
})
|
||||
.collect();
|
||||
sign_registry(
|
||||
RegistryDoc {
|
||||
version: 1,
|
||||
issued_at: now_ts(),
|
||||
ma_key: String::new(),
|
||||
members,
|
||||
},
|
||||
key,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn signed_snapshot_verifies_and_forgery_fails() {
|
||||
let ma = Keypair::generate();
|
||||
let member = Keypair::generate();
|
||||
let registry = make_registry(&ma, vec![("alice", &member.public_hex(), 0, None)]);
|
||||
verify_registry(®istry, &ma.public_hex()).unwrap();
|
||||
|
||||
let mut forged = registry.clone();
|
||||
forged.doc.members[0].id = "mallory".to_string();
|
||||
forged.sig = ma.sign(®istry_signing_bytes(&forged.doc));
|
||||
assert!(verify_registry(&forged, &ma.public_hex()).is_ok());
|
||||
assert!(
|
||||
verify_registry(&forged, &Keypair::generate().public_hex()).is_err(),
|
||||
"wrong MA key must be rejected"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tampered_snapshot_fails_verification() {
|
||||
let ma = Keypair::generate();
|
||||
let member = Keypair::generate();
|
||||
let mut registry = make_registry(&ma, vec![("alice", &member.public_hex(), 0, None)]);
|
||||
registry.doc.version = 99;
|
||||
assert!(verify_registry(®istry, &ma.public_hex()).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validity_windows_gate_authorization() {
|
||||
let ma = Keypair::generate();
|
||||
let member = Keypair::generate();
|
||||
let now = now_ts();
|
||||
let mut registry = make_registry(
|
||||
&ma,
|
||||
vec![("future", &member.public_hex(), now + 3600, None)],
|
||||
);
|
||||
let mut expired = registry.doc.members.clone();
|
||||
expired.push(RegistryMember {
|
||||
id: "expired".to_string(),
|
||||
class: "source".to_string(),
|
||||
keys: vec![KeyEntry {
|
||||
key: Keypair::generate().public_hex(),
|
||||
not_before: 0,
|
||||
not_after: Some(now.saturating_sub(1)),
|
||||
}],
|
||||
});
|
||||
registry.doc.members = expired;
|
||||
let registry = sign_registry(registry.doc, &ma);
|
||||
|
||||
let map = authorized_keys(®istry, now);
|
||||
assert!(map.is_empty(), "future or expired keys must not authorize");
|
||||
}
|
||||
}
|
||||
+73
-2
@@ -13,17 +13,25 @@ use serde::Deserialize;
|
||||
use serde_json::json;
|
||||
use tokio::net::TcpListener;
|
||||
|
||||
use crate::message::{Envelope, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE};
|
||||
use crate::crypto::{now_ts, poll_signing_bytes, random_nonce, verify_signature};
|
||||
use crate::message::{Envelope, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE, timestamp_is_fresh};
|
||||
|
||||
pub const DEFAULT_CAPACITY: usize = 256;
|
||||
const CHALLENGE_TTL: Duration = Duration::from_secs(120);
|
||||
|
||||
#[derive(Default)]
|
||||
struct MemberQueue {
|
||||
items: VecDeque<(u64, Envelope)>,
|
||||
}
|
||||
|
||||
struct Challenge {
|
||||
member: String,
|
||||
created: Instant,
|
||||
}
|
||||
|
||||
struct Inner {
|
||||
members: HashMap<String, MemberQueue>,
|
||||
challenges: HashMap<String, Challenge>,
|
||||
seq: u64,
|
||||
}
|
||||
|
||||
@@ -37,6 +45,7 @@ impl Relay {
|
||||
Arc::new(Self {
|
||||
inner: Mutex::new(Inner {
|
||||
members: HashMap::new(),
|
||||
challenges: HashMap::new(),
|
||||
seq: 0,
|
||||
}),
|
||||
capacity,
|
||||
@@ -79,6 +88,7 @@ impl Relay {
|
||||
pub fn router(relay: Arc<Relay>) -> Router {
|
||||
Router::new()
|
||||
.route("/health", get(health))
|
||||
.route("/v1/challenge", get(challenge))
|
||||
.route("/v1/publish", post(publish))
|
||||
.route("/v1/unicast", post(unicast))
|
||||
.route("/v1/poll", get(poll))
|
||||
@@ -97,14 +107,48 @@ pub async fn bind(addr: &str) -> Result<(TcpListener, SocketAddr)> {
|
||||
Ok((listener, local))
|
||||
}
|
||||
|
||||
fn valid_key(member: &str) -> bool {
|
||||
matches!(hex::decode(member), Ok(bytes) if bytes.len() == 32)
|
||||
}
|
||||
|
||||
async fn health() -> &'static str {
|
||||
"ok"
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct ChallengeParams {
|
||||
member: String,
|
||||
}
|
||||
|
||||
async fn challenge(
|
||||
State(relay): State<Arc<Relay>>,
|
||||
Query(params): Query<ChallengeParams>,
|
||||
) -> Response {
|
||||
if !valid_key(¶ms.member) {
|
||||
return bad_request("valid member key required");
|
||||
}
|
||||
let nonce = random_nonce();
|
||||
let mut inner = relay.inner.lock().expect("relay lock");
|
||||
inner
|
||||
.challenges
|
||||
.retain(|_, challenge| challenge.created.elapsed() < CHALLENGE_TTL);
|
||||
inner.challenges.insert(
|
||||
nonce.clone(),
|
||||
Challenge {
|
||||
member: params.member,
|
||||
created: Instant::now(),
|
||||
},
|
||||
);
|
||||
(StatusCode::OK, Json(json!({ "nonce": nonce }))).into_response()
|
||||
}
|
||||
|
||||
async fn publish(State(relay): State<Arc<Relay>>, Json(envelope): Json<Envelope>) -> Response {
|
||||
if envelope.verify().is_err() {
|
||||
return bad_request("invalid signature");
|
||||
}
|
||||
if !timestamp_is_fresh(envelope.ts, now_ts()) {
|
||||
return bad_request("stale timestamp");
|
||||
}
|
||||
if envelope.msg_type != TYPE_QUERY {
|
||||
return bad_request("relay carries broadcast queries only");
|
||||
}
|
||||
@@ -131,6 +175,9 @@ async fn unicast(
|
||||
if envelope.verify().is_err() {
|
||||
return bad_request("invalid signature");
|
||||
}
|
||||
if !timestamp_is_fresh(envelope.ts, now_ts()) {
|
||||
return bad_request("stale timestamp");
|
||||
}
|
||||
if envelope.msg_type != TYPE_RESPONSE && envelope.msg_type != TYPE_AGGREGATE {
|
||||
return bad_request("unicast carries responses and aggregates only");
|
||||
}
|
||||
@@ -148,13 +195,33 @@ async fn unicast(
|
||||
#[derive(Deserialize)]
|
||||
struct PollParams {
|
||||
member: String,
|
||||
nonce: String,
|
||||
sig: String,
|
||||
timeout_ms: Option<u64>,
|
||||
}
|
||||
|
||||
async fn poll(State(relay): State<Arc<Relay>>, Query(params): Query<PollParams>) -> Response {
|
||||
if params.member.is_empty() || hex::decode(¶ms.member).is_err() {
|
||||
if !valid_key(¶ms.member) {
|
||||
return bad_request("valid member key required");
|
||||
}
|
||||
if verify_signature(
|
||||
¶ms.member,
|
||||
&poll_signing_bytes(¶ms.member, ¶ms.nonce),
|
||||
¶ms.sig,
|
||||
)
|
||||
.is_err()
|
||||
{
|
||||
return unauthorized("invalid poll signature");
|
||||
}
|
||||
{
|
||||
let mut inner = relay.inner.lock().expect("relay lock");
|
||||
match inner.challenges.remove(¶ms.nonce) {
|
||||
Some(challenge)
|
||||
if challenge.member == params.member
|
||||
&& challenge.created.elapsed() < CHALLENGE_TTL => {}
|
||||
_ => return unauthorized("unknown, expired, or reused challenge"),
|
||||
}
|
||||
}
|
||||
let timeout = Duration::from_millis(params.timeout_ms.unwrap_or(25_000).min(60_000));
|
||||
let deadline = Instant::now() + timeout;
|
||||
loop {
|
||||
@@ -181,6 +248,10 @@ fn bad_request(message: &str) -> Response {
|
||||
(StatusCode::BAD_REQUEST, Json(json!({ "error": message }))).into_response()
|
||||
}
|
||||
|
||||
fn unauthorized(message: &str) -> Response {
|
||||
(StatusCode::UNAUTHORIZED, Json(json!({ "error": message }))).into_response()
|
||||
}
|
||||
|
||||
fn backpressure(status: StatusCode) -> Response {
|
||||
(
|
||||
status,
|
||||
|
||||
@@ -198,11 +198,13 @@ async fn enrichment_members_cannot_send_content() {
|
||||
name: "bob".to_string(),
|
||||
pubkey: bob_full.pubkey.clone(),
|
||||
class: CLASS_ENRICHMENT.to_string(),
|
||||
previous: Vec::new(),
|
||||
},
|
||||
Member {
|
||||
name: "carol".to_string(),
|
||||
pubkey: carol_meta.pubkey.clone(),
|
||||
class: CLASS_ENRICHMENT.to_string(),
|
||||
previous: Vec::new(),
|
||||
},
|
||||
],
|
||||
)
|
||||
@@ -256,6 +258,7 @@ async fn source_members_may_send_content() {
|
||||
name: "bob".to_string(),
|
||||
pubkey: bob.pubkey.clone(),
|
||||
class: CLASS_SOURCE.to_string(),
|
||||
previous: Vec::new(),
|
||||
}],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
+141
@@ -183,6 +183,147 @@ fn cli_init_refuses_overwrite_without_force() {
|
||||
run_ok(&forced);
|
||||
}
|
||||
|
||||
fn http_get(port: u16, path: &str) -> String {
|
||||
use std::io::{Read, Write};
|
||||
let mut stream = std::net::TcpStream::connect(("127.0.0.1", port)).unwrap();
|
||||
stream
|
||||
.write_all(format!("GET {path} HTTP/1.0\r\nHost: 127.0.0.1\r\n\r\n").as_bytes())
|
||||
.unwrap();
|
||||
let mut out = String::new();
|
||||
stream.read_to_string(&mut out).unwrap();
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_registry_lifecycle() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let dir = root.path().join("reg");
|
||||
let dir_arg = dir.display().to_string();
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"init".to_string(),
|
||||
]);
|
||||
assert!(stdout.contains("MA key"), "{stdout}");
|
||||
assert!(dir.join("registry.json").exists());
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let mode = fs::metadata(dir.join("ma-key.hex"))
|
||||
.unwrap()
|
||||
.permissions()
|
||||
.mode()
|
||||
& 0o777;
|
||||
assert_eq!(mode, 0o600);
|
||||
}
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"show".to_string(),
|
||||
]);
|
||||
assert!(stdout.contains("version 1"));
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"add".to_string(),
|
||||
"alice.frx.example".to_string(),
|
||||
"ab".repeat(32),
|
||||
]);
|
||||
assert!(stdout.contains("added alice.frx.example"));
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"add-key".to_string(),
|
||||
"alice.frx.example".to_string(),
|
||||
"cd".repeat(32),
|
||||
]);
|
||||
assert!(stdout.contains("added key"));
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"list".to_string(),
|
||||
]);
|
||||
assert!(stdout.contains("alice.frx.example [source] (2 key(s))"));
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"revoke-key".to_string(),
|
||||
"alice.frx.example".to_string(),
|
||||
"ab".repeat(32),
|
||||
]);
|
||||
assert!(stdout.contains("revoked key"));
|
||||
let stdout = run_ok(&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg.clone(),
|
||||
"list".to_string(),
|
||||
]);
|
||||
assert!(stdout.contains("(1 key(s))"));
|
||||
|
||||
let port = common::free_port();
|
||||
let _service = spawn_service(
|
||||
&[
|
||||
"registry".to_string(),
|
||||
"--dir".to_string(),
|
||||
dir_arg,
|
||||
"serve".to_string(),
|
||||
"--listen".to_string(),
|
||||
format!("127.0.0.1:{port}"),
|
||||
],
|
||||
"registry serving",
|
||||
);
|
||||
let body = http_get(port, "/registry.json");
|
||||
assert!(body.contains("\"members\""), "{body}");
|
||||
assert!(body.contains("alice.frx.example"), "{body}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_key_rotation() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let config = root.path().join("frxd.toml");
|
||||
let data = root.path().join("data");
|
||||
let config_arg = config.display().to_string();
|
||||
run_ok(&init_args(&config, "alice", 0, 1, &data));
|
||||
|
||||
let first = run_ok(&[
|
||||
"--config".to_string(),
|
||||
config_arg.clone(),
|
||||
"key".to_string(),
|
||||
"show".to_string(),
|
||||
]);
|
||||
let first = first.trim().to_string();
|
||||
assert_eq!(first.len(), 64);
|
||||
|
||||
let stdout = run_ok(&[
|
||||
"--config".to_string(),
|
||||
config_arg.clone(),
|
||||
"key".to_string(),
|
||||
"rotate".to_string(),
|
||||
]);
|
||||
assert!(stdout.contains("old pubkey"), "{stdout}");
|
||||
assert!(stdout.contains("new pubkey"), "{stdout}");
|
||||
|
||||
let second = run_ok(&[
|
||||
"--config".to_string(),
|
||||
config_arg,
|
||||
"key".to_string(),
|
||||
"show".to_string(),
|
||||
]);
|
||||
assert_ne!(first, second.trim(), "key did not change");
|
||||
assert!(data.join("key.hex.bak").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_full_network_pipeline() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
|
||||
+31
-6
@@ -13,8 +13,12 @@ pub fn config_for(dir: &Path, name: &str, relay_url: &str) -> Config {
|
||||
let config = Config {
|
||||
node: NodeSection {
|
||||
name: name.to_string(),
|
||||
id: None,
|
||||
listen: "127.0.0.1:0".to_string(),
|
||||
relays: vec![relay_url.to_string()],
|
||||
registry: None,
|
||||
ma_key: None,
|
||||
dev_bootstrap: true,
|
||||
responder: true,
|
||||
},
|
||||
query: QuerySection {
|
||||
@@ -60,17 +64,33 @@ pub fn query_envelope(key: &Keypair, text: &str, max_results: usize) -> Envelope
|
||||
Envelope::new(key, TYPE_QUERY, serde_json::to_value(&body).unwrap())
|
||||
}
|
||||
|
||||
pub async fn register(client: &reqwest::Client, relay_url: &str, member: &str) {
|
||||
poll(client, relay_url, member, 30).await;
|
||||
pub async fn challenge(client: &reqwest::Client, relay_url: &str, member: &str) -> Option<String> {
|
||||
let response = client
|
||||
.get(format!("{relay_url}/v1/challenge?member={member}"))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
if !response.status().is_success() {
|
||||
return None;
|
||||
}
|
||||
let payload: Value = response.json().await.unwrap();
|
||||
payload
|
||||
.get("nonce")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_string)
|
||||
}
|
||||
|
||||
pub async fn register(client: &reqwest::Client, relay_url: &str, key: &Keypair) {
|
||||
poll(client, relay_url, key, 30).await;
|
||||
}
|
||||
|
||||
pub async fn poll_messages(
|
||||
client: &reqwest::Client,
|
||||
relay_url: &str,
|
||||
member: &str,
|
||||
key: &Keypair,
|
||||
timeout_ms: u64,
|
||||
) -> Vec<Value> {
|
||||
let response = poll(client, relay_url, member, timeout_ms).await;
|
||||
let response = poll(client, relay_url, key, timeout_ms).await;
|
||||
if response.status() == reqwest::StatusCode::NO_CONTENT {
|
||||
return Vec::new();
|
||||
}
|
||||
@@ -149,12 +169,17 @@ pub async fn unicast(
|
||||
pub async fn poll(
|
||||
client: &reqwest::Client,
|
||||
relay_url: &str,
|
||||
member: &str,
|
||||
key: &Keypair,
|
||||
timeout_ms: u64,
|
||||
) -> reqwest::Response {
|
||||
let member = key.public_hex();
|
||||
let nonce = challenge(client, relay_url, &member)
|
||||
.await
|
||||
.expect("relay challenge");
|
||||
let sig = key.sign(&frxd::crypto::poll_signing_bytes(&member, &nonce));
|
||||
client
|
||||
.get(format!(
|
||||
"{relay_url}/v1/poll?member={member}&timeout_ms={timeout_ms}"
|
||||
"{relay_url}/v1/poll?member={member}&nonce={nonce}&sig={sig}&timeout_ms={timeout_ms}"
|
||||
))
|
||||
.send()
|
||||
.await
|
||||
|
||||
@@ -101,8 +101,8 @@ async fn multi_relay_deduplicates_and_responds_once() {
|
||||
|
||||
let http = client();
|
||||
let alice = Keypair::generate();
|
||||
register(&http, &relay_one, &alice.public_hex()).await;
|
||||
register(&http, &relay_two, &alice.public_hex()).await;
|
||||
register(&http, &relay_one, &alice).await;
|
||||
register(&http, &relay_two, &alice).await;
|
||||
let envelope = query_envelope(&alice, "rust", 5);
|
||||
assert!(
|
||||
publish(&http, &relay_one, &envelope)
|
||||
@@ -121,7 +121,7 @@ async fn multi_relay_deduplicates_and_responds_once() {
|
||||
let mut responses = Vec::new();
|
||||
while responses.is_empty() && Instant::now() < deadline {
|
||||
for relay in [&relay_one, &relay_two] {
|
||||
let messages = poll_messages(&http, relay, &alice.public_hex(), 200).await;
|
||||
let messages = poll_messages(&http, relay, &alice, 200).await;
|
||||
responses.extend(
|
||||
messages
|
||||
.iter()
|
||||
@@ -190,7 +190,7 @@ async fn poll_returns_queued_batch_in_one_call() {
|
||||
let relay_url = spawn_relay().await;
|
||||
let http = client();
|
||||
let member = Keypair::generate();
|
||||
register(&http, &relay_url, &member.public_hex()).await;
|
||||
register(&http, &relay_url, &member).await;
|
||||
for index in 0..3 {
|
||||
let envelope = query_envelope(&member, &format!("query {index}"), 5);
|
||||
assert!(
|
||||
@@ -200,10 +200,10 @@ async fn poll_returns_queued_batch_in_one_call() {
|
||||
.is_success()
|
||||
);
|
||||
}
|
||||
let messages = poll_messages(&http, &relay_url, &member.public_hex(), 300).await;
|
||||
let messages = poll_messages(&http, &relay_url, &member, 300).await;
|
||||
assert_eq!(messages.len(), 3);
|
||||
assert!(
|
||||
poll_messages(&http, &relay_url, &member.public_hex(), 50)
|
||||
poll_messages(&http, &relay_url, &member, 50)
|
||||
.await
|
||||
.is_empty()
|
||||
);
|
||||
|
||||
+154
-26
@@ -6,7 +6,7 @@ use std::path::{Path, PathBuf};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use common::{
|
||||
ask, client, collection, config_for, messages_of_type, poll, poll_messages, publish,
|
||||
ask, challenge, client, collection, config_for, messages_of_type, poll, poll_messages, publish,
|
||||
query_envelope, register, spawn_relay, spawn_relay_with_capacity, unicast,
|
||||
};
|
||||
use frxd::crypto::Keypair;
|
||||
@@ -63,14 +63,14 @@ async fn start_node_with_corpus(
|
||||
async fn collect_responses(
|
||||
http: &reqwest::Client,
|
||||
relay_url: &str,
|
||||
member: &str,
|
||||
key: &Keypair,
|
||||
timeout_ms: u64,
|
||||
) -> Vec<Value> {
|
||||
let start = Instant::now();
|
||||
let deadline = Duration::from_millis(timeout_ms);
|
||||
let mut responses = Vec::new();
|
||||
while start.elapsed() < deadline {
|
||||
let messages = poll_messages(http, relay_url, member, 200).await;
|
||||
let messages = poll_messages(http, relay_url, key, 200).await;
|
||||
responses.extend(messages_of_type(&messages, TYPE_RESPONSE));
|
||||
if !responses.is_empty() {
|
||||
break;
|
||||
@@ -86,7 +86,7 @@ async fn raw_query(
|
||||
text: &str,
|
||||
timeout_ms: u64,
|
||||
) -> Vec<Value> {
|
||||
register(http, relay_url, &asker.public_hex()).await;
|
||||
register(http, relay_url, asker).await;
|
||||
let envelope = query_envelope(asker, text, 5);
|
||||
assert!(
|
||||
publish(http, relay_url, &envelope)
|
||||
@@ -94,7 +94,7 @@ async fn raw_query(
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
collect_responses(http, relay_url, &asker.public_hex(), timeout_ms).await
|
||||
collect_responses(http, relay_url, asker, timeout_ms).await
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -330,7 +330,7 @@ async fn entities_are_optional_hints() {
|
||||
.await;
|
||||
let alice = Keypair::generate();
|
||||
let http = client();
|
||||
register(&http, &relay_url, &alice.public_hex()).await;
|
||||
register(&http, &relay_url, &alice).await;
|
||||
let body = json!({
|
||||
"qid": "entities-test",
|
||||
"text": "rust",
|
||||
@@ -344,7 +344,7 @@ async fn entities_are_optional_hints() {
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
||||
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||
assert_eq!(responses.len(), 1);
|
||||
assert_eq!(
|
||||
responses[0].pointer("/body/qid").and_then(Value::as_str),
|
||||
@@ -367,6 +367,7 @@ async fn member_directory_filters_senders() {
|
||||
name: "alice".to_string(),
|
||||
pubkey: alice.public_hex(),
|
||||
class: frxd::config::CLASS_SOURCE.to_string(),
|
||||
previous: Vec::new(),
|
||||
}],
|
||||
)
|
||||
.unwrap();
|
||||
@@ -382,7 +383,7 @@ async fn member_directory_filters_senders() {
|
||||
let trusted = raw_query(&http, &relay_url, &alice, "rust", 700).await;
|
||||
assert_eq!(trusted.len(), 1, "trusted member got no response");
|
||||
|
||||
register(&http, &relay_url, &untrusted.public_hex()).await;
|
||||
register(&http, &relay_url, &untrusted).await;
|
||||
let envelope = query_envelope(&untrusted, "rust", 5);
|
||||
assert!(
|
||||
publish(&http, &relay_url, &envelope)
|
||||
@@ -390,17 +391,98 @@ async fn member_directory_filters_senders() {
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let denied = collect_responses(&http, &relay_url, &untrusted.public_hex(), 400).await;
|
||||
let denied = collect_responses(&http, &relay_url, &untrusted, 400).await;
|
||||
assert!(denied.is_empty(), "untrusted member received a response");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn rotated_keys_are_accepted_through_previous_listing() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let old_key = Keypair::generate();
|
||||
let new_key = Keypair::generate();
|
||||
|
||||
let docs = corpus_dir(root.path(), "bob", &[("doc.txt", "rotation rust document")]);
|
||||
let config = config_for(&root.path().join("bob"), "bob", &relay_url);
|
||||
let listing = |previous: Vec<String>| {
|
||||
vec![frxd::config::Member {
|
||||
name: "carol".to_string(),
|
||||
pubkey: new_key.public_hex(),
|
||||
class: frxd::config::CLASS_SOURCE.to_string(),
|
||||
previous,
|
||||
}]
|
||||
};
|
||||
frxd::config::save_members(&config.members_path(), &listing(vec![old_key.public_hex()]))
|
||||
.unwrap();
|
||||
{
|
||||
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||
index
|
||||
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||
.unwrap();
|
||||
}
|
||||
let _bob = Node::start(config.clone()).await.unwrap();
|
||||
|
||||
let http = client();
|
||||
register(&http, &relay_url, &old_key).await;
|
||||
let envelope = query_envelope(&old_key, "rust", 5);
|
||||
assert!(
|
||||
publish(&http, &relay_url, &envelope)
|
||||
.await
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let accepted = collect_responses(&http, &relay_url, &old_key, 700).await;
|
||||
assert_eq!(accepted.len(), 1, "listed previous key was rejected");
|
||||
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
frxd::config::save_members(&config.members_path(), &listing(Vec::new())).unwrap();
|
||||
let envelope = query_envelope(&old_key, "rust", 5);
|
||||
assert!(
|
||||
publish(&http, &relay_url, &envelope)
|
||||
.await
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let revoked = collect_responses(&http, &relay_url, &old_key, 500).await;
|
||||
assert!(revoked.is_empty(), "revoked key was still accepted");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn stale_envelopes_are_rejected() {
|
||||
let relay_url = spawn_relay().await;
|
||||
let http = client();
|
||||
let key = Keypair::generate();
|
||||
|
||||
let mut envelope = query_envelope(&key, "stale", 5);
|
||||
envelope.ts = frxd::crypto::now_ts().saturating_sub(3600);
|
||||
envelope.sig = key.sign(&frxd::crypto::signing_bytes(&envelope));
|
||||
assert_eq!(
|
||||
publish(&http, &relay_url, &envelope).await.status(),
|
||||
reqwest::StatusCode::BAD_REQUEST
|
||||
);
|
||||
|
||||
let mut response = Envelope::new(
|
||||
&key,
|
||||
TYPE_RESPONSE,
|
||||
json!({"qid": "q1", "results": [], "truncated": false, "more_available": 0, "cursor": null}),
|
||||
);
|
||||
response.ts = frxd::crypto::now_ts().saturating_sub(3600);
|
||||
response.sig = key.sign(&frxd::crypto::signing_bytes(&response));
|
||||
assert_eq!(
|
||||
unicast(&http, &relay_url, &key.public_hex(), &response)
|
||||
.await
|
||||
.status(),
|
||||
reqwest::StatusCode::BAD_REQUEST
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn backpressure_is_visible_and_recoverable() {
|
||||
let relay_url = spawn_relay_with_capacity(1).await;
|
||||
let http = client();
|
||||
let member = Keypair::generate();
|
||||
let publisher = Keypair::generate();
|
||||
register(&http, &relay_url, &member.public_hex()).await;
|
||||
register(&http, &relay_url, &member).await;
|
||||
|
||||
let first = query_envelope(&publisher, "one", 5);
|
||||
assert_eq!(
|
||||
@@ -419,13 +501,11 @@ async fn backpressure_is_visible_and_recoverable() {
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
poll_messages(&http, &relay_url, &member.public_hex(), 300)
|
||||
.await
|
||||
.len(),
|
||||
poll_messages(&http, &relay_url, &member, 300).await.len(),
|
||||
1
|
||||
);
|
||||
assert_eq!(
|
||||
poll_messages(&http, &relay_url, &publisher.public_hex(), 300)
|
||||
poll_messages(&http, &relay_url, &publisher, 300)
|
||||
.await
|
||||
.len(),
|
||||
1
|
||||
@@ -444,8 +524,8 @@ async fn unicast_is_need_to_know() {
|
||||
let http = client();
|
||||
let alice = Keypair::generate();
|
||||
let bob = Keypair::generate();
|
||||
register(&http, &relay_url, &alice.public_hex()).await;
|
||||
register(&http, &relay_url, &bob.public_hex()).await;
|
||||
register(&http, &relay_url, &alice).await;
|
||||
register(&http, &relay_url, &bob).await;
|
||||
|
||||
let query = query_envelope(&alice, "rust", 5);
|
||||
assert!(
|
||||
@@ -467,9 +547,9 @@ async fn unicast_is_need_to_know() {
|
||||
reqwest::StatusCode::OK
|
||||
);
|
||||
|
||||
let alice_messages = poll_messages(&http, &relay_url, &alice.public_hex(), 300).await;
|
||||
let alice_messages = poll_messages(&http, &relay_url, &alice, 300).await;
|
||||
assert_eq!(messages_of_type(&alice_messages, TYPE_RESPONSE).len(), 1);
|
||||
let bob_messages = poll_messages(&http, &relay_url, &bob.public_hex(), 300).await;
|
||||
let bob_messages = poll_messages(&http, &relay_url, &bob, 300).await;
|
||||
assert!(
|
||||
messages_of_type(&bob_messages, TYPE_RESPONSE).is_empty(),
|
||||
"unicast response leaked to another member"
|
||||
@@ -516,10 +596,58 @@ async fn tampered_envelopes_are_rejected() {
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn invalid_poll_member_is_rejected() {
|
||||
async fn poll_requires_proof_of_key() {
|
||||
let relay_url = spawn_relay().await;
|
||||
let response = poll(&client(), &relay_url, "not-hex-at-all", 30).await;
|
||||
assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST);
|
||||
let http = client();
|
||||
let member = Keypair::generate();
|
||||
let imposter = Keypair::generate();
|
||||
|
||||
assert!(
|
||||
challenge(&http, &relay_url, "not-hex-at-all")
|
||||
.await
|
||||
.is_none()
|
||||
);
|
||||
|
||||
let nonce = challenge(&http, &relay_url, &member.public_hex())
|
||||
.await
|
||||
.unwrap();
|
||||
let forged = imposter.sign(&frxd::crypto::poll_signing_bytes(
|
||||
&member.public_hex(),
|
||||
&nonce,
|
||||
));
|
||||
let response = http
|
||||
.get(format!(
|
||||
"{relay_url}/v1/poll?member={}&nonce={nonce}&sig={forged}&timeout_ms=30",
|
||||
member.public_hex()
|
||||
))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), reqwest::StatusCode::UNAUTHORIZED);
|
||||
|
||||
let response = poll(&http, &relay_url, &member, 30).await;
|
||||
assert_eq!(response.status(), reqwest::StatusCode::NO_CONTENT);
|
||||
|
||||
let nonce = challenge(&http, &relay_url, &member.public_hex())
|
||||
.await
|
||||
.unwrap();
|
||||
let sig = member.sign(&frxd::crypto::poll_signing_bytes(
|
||||
&member.public_hex(),
|
||||
&nonce,
|
||||
));
|
||||
let url = format!(
|
||||
"{relay_url}/v1/poll?member={}&nonce={nonce}&sig={sig}&timeout_ms=30",
|
||||
member.public_hex()
|
||||
);
|
||||
assert_eq!(
|
||||
http.get(&url).send().await.unwrap().status(),
|
||||
reqwest::StatusCode::NO_CONTENT
|
||||
);
|
||||
assert_eq!(
|
||||
http.get(&url).send().await.unwrap().status(),
|
||||
reqwest::StatusCode::UNAUTHORIZED,
|
||||
"challenge nonce must be single-use"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
@@ -724,7 +852,7 @@ async fn aggregates_are_unicast_only_and_ignored_by_nodes() {
|
||||
.await;
|
||||
let http = client();
|
||||
let sender = Keypair::generate();
|
||||
register(&http, &relay_url, &bob.pubkey).await;
|
||||
register(&http, &relay_url, &bob.node.key).await;
|
||||
|
||||
let broadcast = Envelope::new(
|
||||
&sender,
|
||||
@@ -798,7 +926,7 @@ async fn duplicate_delivery_is_answered_once() {
|
||||
.await;
|
||||
let alice = Keypair::generate();
|
||||
let http = client();
|
||||
register(&http, &relay_url, &alice.public_hex()).await;
|
||||
register(&http, &relay_url, &alice).await;
|
||||
let envelope = query_envelope(&alice, "rust", 5);
|
||||
assert!(
|
||||
publish(&http, &relay_url, &envelope)
|
||||
@@ -812,7 +940,7 @@ async fn duplicate_delivery_is_answered_once() {
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
||||
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||
assert_eq!(
|
||||
responses.len(),
|
||||
1,
|
||||
@@ -904,7 +1032,7 @@ async fn query_body_without_entities_parses() {
|
||||
.await;
|
||||
let alice = Keypair::generate();
|
||||
let http = client();
|
||||
register(&http, &relay_url, &alice.public_hex()).await;
|
||||
register(&http, &relay_url, &alice).await;
|
||||
let envelope = Envelope::new(
|
||||
&alice,
|
||||
TYPE_QUERY,
|
||||
@@ -916,7 +1044,7 @@ async fn query_body_without_entities_parses() {
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await;
|
||||
let responses = collect_responses(&http, &relay_url, &alice, 700).await;
|
||||
assert_eq!(responses.len(), 1);
|
||||
}
|
||||
|
||||
|
||||
+7
-15
@@ -143,7 +143,7 @@ async fn no_protocol_query_dedup() {
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let response = poll(&http, &relay_url, &key.public_hex(), 300).await;
|
||||
let response = poll(&http, &relay_url, &key, 300).await;
|
||||
let payload: Value = response.json().await.unwrap();
|
||||
assert_eq!(
|
||||
payload
|
||||
@@ -242,9 +242,7 @@ async fn no_aggregate_appeals() {
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
poll(&http, &relay_url, &member.public_hex(), 30)
|
||||
.await
|
||||
.status(),
|
||||
poll(&http, &relay_url, &member, 30).await.status(),
|
||||
reqwest::StatusCode::NO_CONTENT
|
||||
);
|
||||
let aggregate = Envelope::new(
|
||||
@@ -299,15 +297,11 @@ async fn no_topic_channels() {
|
||||
let alice = Keypair::generate();
|
||||
let bob = Keypair::generate();
|
||||
assert_eq!(
|
||||
poll(&http, &relay_url, &alice.public_hex(), 30)
|
||||
.await
|
||||
.status(),
|
||||
poll(&http, &relay_url, &alice, 30).await.status(),
|
||||
reqwest::StatusCode::NO_CONTENT
|
||||
);
|
||||
assert_eq!(
|
||||
poll(&http, &relay_url, &bob.public_hex(), 30)
|
||||
.await
|
||||
.status(),
|
||||
poll(&http, &relay_url, &bob, 30).await.status(),
|
||||
reqwest::StatusCode::NO_CONTENT
|
||||
);
|
||||
let envelope = Envelope::new(
|
||||
@@ -322,7 +316,7 @@ async fn no_topic_channels() {
|
||||
.is_success()
|
||||
);
|
||||
for member in [&alice, &bob] {
|
||||
let response = poll(&http, &relay_url, &member.public_hex(), 300).await;
|
||||
let response = poll(&http, &relay_url, &member, 300).await;
|
||||
let payload: Value = response.json().await.unwrap();
|
||||
assert_eq!(
|
||||
payload
|
||||
@@ -422,7 +416,7 @@ async fn no_durable_replayable_broadcast_stream() {
|
||||
.is_success()
|
||||
);
|
||||
|
||||
let response = poll(&http, &relay_url, &key.public_hex(), 300).await;
|
||||
let response = poll(&http, &relay_url, &key, 300).await;
|
||||
let payload: Value = response.json().await.unwrap();
|
||||
assert_eq!(
|
||||
payload
|
||||
@@ -433,9 +427,7 @@ async fn no_durable_replayable_broadcast_stream() {
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
poll(&http, &relay_url, &key.public_hex(), 50)
|
||||
.await
|
||||
.status(),
|
||||
poll(&http, &relay_url, &key, 50).await.status(),
|
||||
reqwest::StatusCode::NO_CONTENT,
|
||||
"relay replayed a drained message"
|
||||
);
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
mod common;
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use axum::routing::get;
|
||||
use axum::{Json, Router};
|
||||
use common::{
|
||||
client, collection, config_for, poll_messages, publish, query_envelope, register, spawn_relay,
|
||||
};
|
||||
use frxd::crypto::{Keypair, now_ts};
|
||||
use frxd::index::LocalIndex;
|
||||
use frxd::message::{EXPOSURE_FULL, TYPE_RESPONSE};
|
||||
use frxd::node::{Node, NodeHandle};
|
||||
use frxd::registry::{self, KeyEntry, RegistryDoc, RegistryMember, SignedRegistry};
|
||||
use serde_json::Value;
|
||||
|
||||
fn member_entry(
|
||||
id: &str,
|
||||
key: &Keypair,
|
||||
class: &str,
|
||||
not_before: u64,
|
||||
not_after: Option<u64>,
|
||||
) -> RegistryMember {
|
||||
RegistryMember {
|
||||
id: id.to_string(),
|
||||
class: class.to_string(),
|
||||
keys: vec![KeyEntry {
|
||||
key: key.public_hex(),
|
||||
not_before,
|
||||
not_after,
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn ma_registry(ma: &Keypair, members: Vec<RegistryMember>, version: u64) -> SignedRegistry {
|
||||
registry::sign_registry(
|
||||
RegistryDoc {
|
||||
version,
|
||||
issued_at: now_ts(),
|
||||
ma_key: String::new(),
|
||||
members,
|
||||
},
|
||||
ma,
|
||||
)
|
||||
}
|
||||
|
||||
fn config_with_registry(
|
||||
dir: &Path,
|
||||
name: &str,
|
||||
relay_url: &str,
|
||||
registry: &str,
|
||||
ma_key: &str,
|
||||
) -> frxd::config::Config {
|
||||
let mut config = config_for(dir, name, relay_url);
|
||||
config.node.registry = Some(registry.to_string());
|
||||
config.node.ma_key = Some(ma_key.to_string());
|
||||
config.node.dev_bootstrap = false;
|
||||
config
|
||||
}
|
||||
|
||||
async fn bob_with_docs(root: &Path, relay_url: &str, registry: &str, ma_key: &str) -> NodeHandle {
|
||||
let docs = root.join("bob-docs");
|
||||
fs::create_dir_all(&docs).unwrap();
|
||||
fs::write(docs.join("doc.txt"), "registry rust document").unwrap();
|
||||
let config = config_with_registry(&root.join("bob"), "bob", relay_url, registry, ma_key);
|
||||
{
|
||||
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||
index
|
||||
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||
.unwrap();
|
||||
}
|
||||
Node::start(config).await.unwrap()
|
||||
}
|
||||
|
||||
async fn responses_for(
|
||||
http: &reqwest::Client,
|
||||
relay_url: &str,
|
||||
asker: &Keypair,
|
||||
text: &str,
|
||||
timeout_ms: u64,
|
||||
) -> Vec<Value> {
|
||||
register(http, relay_url, asker).await;
|
||||
let envelope = query_envelope(asker, text, 5);
|
||||
assert!(
|
||||
publish(http, relay_url, &envelope)
|
||||
.await
|
||||
.status()
|
||||
.is_success()
|
||||
);
|
||||
let start = Instant::now();
|
||||
let deadline = Duration::from_millis(timeout_ms);
|
||||
let mut responses = Vec::new();
|
||||
while start.elapsed() < deadline {
|
||||
let messages = poll_messages(http, relay_url, asker, 200).await;
|
||||
responses.extend(
|
||||
messages
|
||||
.iter()
|
||||
.filter(|m| m.get("type").and_then(Value::as_str) == Some(TYPE_RESPONSE))
|
||||
.cloned(),
|
||||
);
|
||||
if !responses.is_empty() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
responses
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn registry_gates_membership_and_revocation_propagates() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let ma = Keypair::generate();
|
||||
let alice = Keypair::generate();
|
||||
let registry_path = root.path().join("registry.json");
|
||||
registry::save_registry(
|
||||
®istry_path,
|
||||
&ma_registry(
|
||||
&ma,
|
||||
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||
1,
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let _bob = bob_with_docs(
|
||||
root.path(),
|
||||
&relay_url,
|
||||
®istry_path.display().to_string(),
|
||||
&ma.public_hex(),
|
||||
)
|
||||
.await;
|
||||
let http = client();
|
||||
|
||||
let accepted = responses_for(&http, &relay_url, &alice, "rust", 700).await;
|
||||
assert_eq!(accepted.len(), 1, "listed member got no response");
|
||||
|
||||
let stranger = Keypair::generate();
|
||||
let denied = responses_for(&http, &relay_url, &stranger, "rust", 400).await;
|
||||
assert!(denied.is_empty(), "unlisted key was answered");
|
||||
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
registry::save_registry(®istry_path, &ma_registry(&ma, Vec::new(), 2)).unwrap();
|
||||
let revoked = responses_for(&http, &relay_url, &alice, "rust", 500).await;
|
||||
assert!(revoked.is_empty(), "revoked member still answered");
|
||||
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
registry::save_registry(
|
||||
®istry_path,
|
||||
&ma_registry(
|
||||
&ma,
|
||||
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||
1,
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
let rolled_back = responses_for(&http, &relay_url, &alice, "rust", 500).await;
|
||||
assert!(
|
||||
rolled_back.is_empty(),
|
||||
"registry rollback re-admitted a revoked member"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn forged_or_wrong_key_snapshot_closes_the_registry() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let ma = Keypair::generate();
|
||||
let attacker = Keypair::generate();
|
||||
let alice = Keypair::generate();
|
||||
let registry_path = root.path().join("registry.json");
|
||||
registry::save_registry(
|
||||
®istry_path,
|
||||
&ma_registry(
|
||||
&attacker,
|
||||
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||
1,
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let _bob = bob_with_docs(
|
||||
root.path(),
|
||||
&relay_url,
|
||||
®istry_path.display().to_string(),
|
||||
&ma.public_hex(),
|
||||
)
|
||||
.await;
|
||||
let http = client();
|
||||
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||
assert!(
|
||||
denied.is_empty(),
|
||||
"snapshot signed by wrong key was trusted"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn expired_key_is_not_authorized() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let ma = Keypair::generate();
|
||||
let alice = Keypair::generate();
|
||||
let registry_path = root.path().join("registry.json");
|
||||
registry::save_registry(
|
||||
®istry_path,
|
||||
&ma_registry(
|
||||
&ma,
|
||||
vec![member_entry(
|
||||
"alice.frx.example",
|
||||
&alice,
|
||||
"source",
|
||||
0,
|
||||
Some(now_ts().saturating_sub(1)),
|
||||
)],
|
||||
1,
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let _bob = bob_with_docs(
|
||||
root.path(),
|
||||
&relay_url,
|
||||
®istry_path.display().to_string(),
|
||||
&ma.public_hex(),
|
||||
)
|
||||
.await;
|
||||
let http = client();
|
||||
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||
assert!(denied.is_empty(), "expired key was answered");
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn fail_static_uses_last_validated_snapshot() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let ma = Keypair::generate();
|
||||
let alice = Keypair::generate();
|
||||
let snapshot = ma_registry(
|
||||
&ma,
|
||||
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||
1,
|
||||
);
|
||||
|
||||
let cached_dir = root.path().join("bob-cached");
|
||||
let mut config = config_with_registry(
|
||||
&cached_dir,
|
||||
"bob",
|
||||
&relay_url,
|
||||
"http://127.0.0.1:1/registry.json",
|
||||
&ma.public_hex(),
|
||||
);
|
||||
fs::create_dir_all(config.data_dir()).unwrap();
|
||||
registry::save_registry(&config.registry_cache_path(), &snapshot).unwrap();
|
||||
let docs = root.path().join("bob-docs");
|
||||
fs::create_dir_all(&docs).unwrap();
|
||||
fs::write(docs.join("doc.txt"), "registry rust document").unwrap();
|
||||
{
|
||||
let index = LocalIndex::open(&config.index_dir()).unwrap();
|
||||
index
|
||||
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||
.unwrap();
|
||||
}
|
||||
let _bob = Node::start(config.clone()).await.unwrap();
|
||||
|
||||
let http = client();
|
||||
let accepted = responses_for(&http, &relay_url, &alice, "rust", 700).await;
|
||||
assert_eq!(
|
||||
accepted.len(),
|
||||
1,
|
||||
"cached snapshot should keep last-known-good membership"
|
||||
);
|
||||
drop(_bob);
|
||||
|
||||
let fresh_dir = root.path().join("bob-fresh");
|
||||
let fresh = config_with_registry(
|
||||
&fresh_dir,
|
||||
"bob2",
|
||||
&relay_url,
|
||||
"http://127.0.0.1:1/registry.json",
|
||||
&ma.public_hex(),
|
||||
);
|
||||
{
|
||||
let index = LocalIndex::open(&fresh.index_dir()).unwrap();
|
||||
index
|
||||
.add_collection(&collection("docs", &docs, true, EXPOSURE_FULL))
|
||||
.unwrap();
|
||||
}
|
||||
let _bob2 = Node::start(fresh).await.unwrap();
|
||||
let denied = responses_for(&http, &relay_url, &alice, "rust", 400).await;
|
||||
assert!(
|
||||
denied.is_empty(),
|
||||
"unreachable registry without cache must close, not open"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn registry_can_be_served_over_http() {
|
||||
let root = tempfile::tempdir().unwrap();
|
||||
let relay_url = spawn_relay().await;
|
||||
let ma = Keypair::generate();
|
||||
let alice = Keypair::generate();
|
||||
let registry_dir = root.path().join("ma");
|
||||
fs::create_dir_all(®istry_dir).unwrap();
|
||||
registry::save_registry(
|
||||
®istry_dir.join("registry.json"),
|
||||
&ma_registry(
|
||||
&ma,
|
||||
vec![member_entry("alice.frx.example", &alice, "source", 0, None)],
|
||||
1,
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
let state: PathBuf = registry_dir.clone();
|
||||
let handler_state = state.clone();
|
||||
let app = Router::new()
|
||||
.route(
|
||||
"/registry.json",
|
||||
get(move || {
|
||||
let state = handler_state.clone();
|
||||
async move {
|
||||
Json(
|
||||
registry::load_registry(&state.join("registry.json"))
|
||||
.expect("registry file"),
|
||||
)
|
||||
}
|
||||
}),
|
||||
)
|
||||
.with_state(state);
|
||||
tokio::spawn(async move {
|
||||
let _ = axum::serve(listener, app).await;
|
||||
});
|
||||
let registry_url = format!("http://{addr}/registry.json");
|
||||
|
||||
let _bob = bob_with_docs(root.path(), &relay_url, ®istry_url, &ma.public_hex()).await;
|
||||
let http = client();
|
||||
let accepted = responses_for(&http, &relay_url, &alice, "rust", 900).await;
|
||||
assert_eq!(
|
||||
accepted.len(),
|
||||
1,
|
||||
"http-served registry did not authorize member"
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user