Add FRX draft 0.3 spec

rfc.txt

@@ -0,0 +1,132 @@
+FRX — Federated Retrieval Exchange
+
+Status: Draft 0.3. Experimental. Reference implementation: frxd (Rust).
+
+1. Summary
+
+FRX is a membership federation for retrieval. Content owners answer broadcast queries; citation traffic is the payment. The protocol's normative surface is minimal: signed messages, budgets, truncation honesty, aggregate courtesy. Ranking, reputation, caching, filtering, verification, and quality judgment are node-local. There is no supply announcement stream: a responder answers only from content it already holds. A member's mandatory work per incoming query is a local lookup — nothing heavier is required.
+
+2. Invariants
+
+ I1 Ingress consent — content enters a member's index only via the publisher adding its own content or a user-supplied URL.
+ I2 Local sovereignty — nothing centralizes a decision a member could make locally.
+ I3 Broadcast privacy — broadcast payloads MUST NOT contain third-party private content. Queries derived from scoring others' posts MUST be canonicalized (claims/entities). First-party user-initiated search text MAY be sent as typed.
+ I4 Channel separation — the market pays (citations), statements advise (aggregates), contracts punish (membership). Never cross-wired.
+ I5 Mode neutrality — no privileged roles. Eagerness is a local choice: an eager member fetches and retains proactively; a lazy member answers only from content already at hand. Neither holds a privileged role.
+ I6 Ordering travels, scores don't — ranked orderings, never numeric relevance.
+ I7 Local-lookup cost — a member's mandatory work per incoming query is a local lookup. There is no supply stream to maintain, and no heavier mandatory work (fetch, embed, LLM). Cost scales with received query volume at lookup cost.
+ I8 No shared vocabulary — no topic taxonomy; all filtering is receiver-local; the querier never classifies on behalf of receivers.
+ I9 Egress consent — a member serves queries only from collections explicitly marked shared. Default is private.
+
+3. System Model
+
+Members exchange signed messages via dumb relays (multiply-operated transport, zero protocol authority). Queries are broadcast live to all members; relays hold no history and replay nothing. Responses are unicast from responder to querier. Transport DEFAULT: HTTPS + long-poll/SSE. Bilateral rate limits with visible backpressure are normative; application-layer rejection is silent.
+
+STREAM
+TYPE
+VISIBILITY
+ORIGIN
+Demand	query	Broadcast	Any member
+Bid	response	Unicast to querier	Any member holding content
+Courtesy	aggregate	Bilateral, on request	Querier
+Settlement	receipt (app layer)	Public artifact	Citing platform
+
+Silence is conformant and informative: an unanswered query means no member's available content produced a response — not that the content does not exist. A member holding matching content may still choose silence (I2, I5).
+
+4. Messages
+
+Envelope (all messages): {type, from, ts, nonce, body, sig} — Ed25519, key listed in the member directory.
+
+query
+
+jsonc
+
+{ "qid": "...", "text": "<free-form, I3 applies>",
+  "entities": ["Q192630"],            // optional, non-normative hint
+  "budget": { "max_results": 5 } }
+No format mandate. A query no receiver can match earns silence; intelligibility is enforced by retrieval economics, not conformance police (I8).
+
+response
+
+jsonc
+
+{ "qid": "...",
+  "results": [ { "url": "...", "title": "...", "summary": "...",
+                 "published": "...", "exposure": "...", "content": null } ],
+  "truncated": false, "more_available": 0, "cursor": null }
+MUST NOT exceed max_results. MUST set truncated honestly if more ranked results exist within budget (DNS TC-bit pattern). MUST be ordered; MUST NOT carry numeric relevance scores (I6). One result is a conformant, good response. exposure is "metadata" | "full" (paywall compatibility); content is present only when exposure is "full". Republication of another member's response before the querier publishes a citing post is a contract matter; the receipt is the sanctioned republication form, at citation granularity, synchronized to settlement.
+
+aggregate (on request, per member, per period)
+
+jsonc
+
+{ "period": "2026-03", "sent": 12400, "passed": 310, "cited": 44 }
+Counters are the querier's own; no dispute or appeal messages exist, and none may be added (I4). Granularity floor is normative.
+
+There is no publish/announce message. Document metadata is carried in responses (above). Document lineage (revision/supersedes) and delegation to an indexer are unspecified in pull-only mode; see §10.
+
+5. Local Policy Domains
+
+Protocol-silent by design (I2): ranking, relevance gating, reputation counters and throttles, verification/spot-checks, caching and invalidation, claim minting, mode (eager/lazy), external fallback, sharing policy above the I9 floor. Advisory reputation bureaus MAY exist; no member is bound.
+
+6. Membership
+
+The MA governs identity, contract, expulsion — who, never quality. Admission cost is the Sybil defense. Expulsion grounds: fabrication, admission fraud, sustained abuse — never low quality (priced by the citation market: zero citations, zero traffic). Escalation: local throttle → advisory aggregates → MA warning → delisting → expulsion. Aggregates are inadmissible as sanction evidence (I4). Membership classes: source members (own content) and enrichment members (derived corpora, e.g. GDELT/CC-NEWS bots — metadata-only exposure, earn no traffic, sit outside the citation market, transformation logic open and auditable).
+
+7. Reference Implementation — frxd
+
+A single static Rust binary. Install, point at a directory, done. It is simultaneously: (a) a personal search engine over local files, (b) a conformant FRX member.
+
+First run: generate keypair, write config.toml, open localhost web UI (plus frx search CLI). Index target directories with Tantivy; watch for changes; extract text from txt/md/html (PDF optional). Nothing is shared until a collection is explicitly marked shared (I9).
+
+As responder: subscribe to the live query stream via configured relays; match incoming queries against shared collections only (receiver-local lexical/embedding match); respond within budget with honest truncation; sign.
+
+As consumer: search is local-first; a network toggle broadcasts the query and merges responses, provenance-marked ("your files" / "member X"). Click-through preserves attribution — receipts at consumer scale. Fetched content is retained (fetch-on-miss-and-retain): the kept-set converges to the demand-weighted corpus.
+
+Dashboard: sent / passed / cited per period (self-derived from referrer traffic and observable receipts; aggregates requested from queriers on demand). "Your files were cited 14 times this week" is the reward loop.
+
+Stack: tokio, tantivy, axum (localhost UI), reqwest, ed25519-dalek, notify. One binary; frxd relay runs a relay for sovereignty-minded users.
+
+Build order: Phase 1 — envelope, query stream, query, response (demoable between two nodes; consumer packaging is a skin over this). Phase 2 — aggregates, dashboard. Phase 3 — receipts, lineage, delegation (app layer).
+
+8. Security & Privacy Considerations
+
+Query visibility is total among members — the claim stream is the attention product; abstraction level and membership are the boundary (I3). Derived queries SHOULD minimize personal data (I3); each member is responsible for the content of its own messages. Response streams are strategic disclosure (corpus mapping, intake intelligence) — unicast, need-to-know. Amplification is bounded by bilateral transport limits and contract, not routing. Publisher self-promotion is the expected adversarial mode; defense is local (gate, pass-rate throttle, citation-weighted source reputation). Enrichment members' filters are an editorial power — auditable openness is the mitigation.
+
+9. Conformance
+
+A conforming implementation: signs all messages with a listed member key; publishes queries to the firehose only; respects I3 and I9; respects max_results; truncates honestly; sends orderings without scores; enforces visible transport backpressure, never silent transport drops; serves aggregates on request at or above the granularity floor; ingests content only via publisher-added content or user-supplied URL; implements no dispute messages.
+
+10. Open Issues
+
+Consumer admission tier — automated/invite admission for distributed binaries without weakening the Sybil defense (MA policy, gates §7 adoption).
+Relay discovery and default-relay governance (shipped defaults are soft centralization; mitigate with multiple defaults + one-command self-host).
+Delegation grant mechanism.
+Signature canonicalization scheme (blocks Phase-1 interop; required before two implementations can exchange a valid envelope).
+Receipt data model (app layer; specify for cross-member consistency).
+Document lineage (revision/supersedes) and delegation in pull-only mode — previously carried by publish; now unspecified.
+Claim/entity minting conventions — recommended, non-normative (I8).
+
+Appendix A. Recommended Local Practices (Non-Normative)
+
+Two-stage ingestion (cheap gate before any LLM attention); per-peer sent/passed/cited counters, throttling on ratios; verdict memoization and claim normalization; demand-driven prefetch (fetch and retain on attention); hot-set replication; backfill seeding (Wikipedia/Wikidata, GDELT, CC-NEWS); measure query-to-claim collapse and domain concentration before sizing anything.
+
+Appendix B. Purge Log (Normative)
+
+Re-proposals MUST address the rationale.
+
+MECHANISM
+REJECTED BECAUSE
+Per-query bounty, winner-selection, slashing	Pays for query execution when responders own content; race-to-first rewards speed over honesty; anonymous-trust machinery with no anonymous peers
+Protocol query dedup	Undecidable at network layer; duplicates cost receivers ~nothing; dedup is local caching
+k-fetch ingestion attestations	Swarm trust machinery; signatures + local spot-checks suffice
+Result-count etiquette (SERP min/max)	Rendering convention from the one-actor world; quantity is querier-local, selection responder-local; budget + truncation flag suffice
+Global reputation score	Relevance is locally defined; consumption is observable where it happens
+Aggregate appeals	Converts courtesy into litigation; drags local policy into network process
+Topic channels	Shared vocabulary is a governance object; sender-side routing by the least-informed party yields undetectable false negatives; at membership N, broadcast + receiver filtering is cheaper than the coordination
+Broadcast responses ("evidence commons")	Multiplies the heavy stream by N; requires network-layer query equivalence (purged); receipts provide post-settlement publicity
+Normative query canonical form	Receiver-local matching handles arbitrary phrasing; intelligibility is self-enforcing; the protocol constrains only what must not enter a broadcast (I3)
+Supply announce firehose	Pull-only supply: responders answer from content already held; a visible announce stream adds cost and privacy exposure with no discovery benefit, since queries already reach all members
+Durable / replayable broadcast stream	A live, ephemeral broadcast suffices for routing; persistence and history-replay buy nothing and invite retention and erasure problems
+ 
+Two closing notes. First, the open issue that matters most is consumer admission — everything else in §7 is straightforward engineering, but "naive user installs a binary and joins a membership org" has real tension with the admission-cost Sybil defense, and I'd resolve it deliberately rather than by accident (invite codes at launch; a lightweight consumer tier later; or defaulting consumer nodes to delegation through an indexer member until they've earned standing). Second, when the reference implementation exists, the purge log stops being documentation and becomes a test suite: nine unit tests, each asserting the absence of a mechanism. A spec this small can afford to test what it refuses to do.