1 Commits
5 changed files with 43 additions and 38 deletions
+1 -1
View File
@@ -50,6 +50,6 @@
- Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role. - Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role.
- Matching floor: boundary tokenizer (`src/tokenizer.rs` — letter/digit splits so `5555` matches `DLEX5555`, lowercase, ASCII fold, English stopwords+stemmer) → coverage gate (`[match] min_coverage`, default 0.4; 12 term queries require all terms) → title boost 2.0 + phrase boost 3.0 + query-time snippets. Schema changes require a fresh index dir (`open_or_create` errors on mismatch). - Matching floor: boundary tokenizer (`src/tokenizer.rs` — letter/digit splits so `5555` matches `DLEX5555`, lowercase, ASCII fold, English stopwords+stemmer) → coverage gate (`[match] min_coverage`, default 0.4; 12 term queries require all terms) → title boost 2.0 + phrase boost 3.0 + query-time snippets. Schema changes require a fresh index dir (`open_or_create` errors on mismatch).
- Engine seam: `src/engine.rs` `SearchEngine` trait (`search``EngineOutput { hits, total: Option<u64> }`, `doc_count`); `respond()` in `src/node.rs` is the conformance wrapper (budget clamp, truncation from engine total — unknown total forces `truncated = true`). Power users can implement the trait (HTTP adapter or subprocess to an external engine). - Engine seam: `src/engine.rs` `SearchEngine` trait (`search``EngineOutput { hits, total: Option<u64> }`, `doc_count`); `respond()` in `src/node.rs` is the conformance wrapper (budget clamp, truncation from engine total — unknown total forces `truncated = true`). Power users can implement the trait (HTTP adapter or subprocess to an external engine).
- Onboarding: `frxd --onboarding` runs a wizard consuming a credential block (`id=.. token=.. registry=.. ma_key=..`) issued by the MA (`registry serve`; HTML page at `/`, `POST /v1/signup` queues a pending application, `POST /v1/enroll` binds keys and re-signs). Identity registration stays MA-side; the wizard never creates identities, only binds locally generated keys. Applications live in `<registry dir>/applications.json` (mode 600, MA contract data — never in the signed snapshot); `frxd registry approve <id>` promotes one (member stub + credential block whose token is the member's reusable account credential, hashed in `<registry dir>/tokens.json` — authorizes key enrollment for every node the member runs); `frxd registry token <id>` mints another member token, `revoke-token <id>` revokes all of a member's tokens; `frxd registry invite <id>` mints a single-use 24h handoff token (`<registry dir>/invites.json`). Prompts accept empty input as the default; scripted stdin works for tests. - Onboarding: `frxd --onboarding` runs a wizard consuming a credential block (`id=.. token=.. registry=.. ma_key=..`) issued by the MA (`registry serve`; HTML page at `/`, `POST /v1/signup` queues a pending application, `POST /v1/enroll` binds keys and re-signs). Identity registration stays MA-side; the wizard never creates identities, only binds locally generated keys. Applications live in `<registry dir>/applications.json` (mode 600, MA contract data — never in the signed snapshot); `frxd registry approve <id>` promotes one (member stub + credential block whose token is the member's reusable account credential, hashed in `<registry dir>/tokens.json` — authorizes key enrollment for every node the member runs); applicants never self-declare a class — the MA assigns it with `approve --class enrichment` (default source; classes are provenance, not roles — every member may query and respond, I5). `frxd registry token <id>` mints another member token, `revoke-token <id>` revokes all of a member's tokens; `frxd registry invite <id>` mints a single-use 24h handoff token (`<registry dir>/invites.json`). Prompts accept empty input as the default; scripted stdin works for tests.
- Next matching steps: eval harness with a small golden set (precision@k + false-silence rate), then a dense recall leg (model2vec-rs 0.2.1 exists but needs `default-features = false, features = ["fancy-regex", "local-only"]` for musl/airgapped; verify crate + model licenses before bundling), then an optional cross-encoder reranker. Embeddings are for recall; reranking is the precision tier. - Next matching steps: eval harness with a small golden set (precision@k + false-silence rate), then a dense recall leg (model2vec-rs 0.2.1 exists but needs `default-features = false, features = ["fancy-regex", "local-only"]` for musl/airgapped; verify crate + model licenses before bundling), then an optional cross-encoder reranker. Embeddings are for recall; reranking is the precision tier.
- Identity/registry (RFC Draft 0.5 §4/§6): MA-hosted FQDN identifiers first (`<label>.frx.<ma-domain>`, no DNS needed by users), signed versioned registry snapshot with the MA key pinned; envelope `from` = identifier, `key` = pubkey; registry outage fails static. Member-hosted identities, MA anchor rollover, and unicast confidentiality are §10 open. Implementation phases: A (signed registry snapshot) and B (identifier + `key` + JCS on the wire) are built and tested. Prioritize frictionless onboarding (users may be department-level and cannot create DNS). - Identity/registry (RFC Draft 0.5 §4/§6): MA-hosted FQDN identifiers first (`<label>.frx.<ma-domain>`, no DNS needed by users), signed versioned registry snapshot with the MA key pinned; envelope `from` = identifier, `key` = pubkey; registry outage fails static. Member-hosted identities, MA anchor rollover, and unicast confidentiality are §10 open. Implementation phases: A (signed registry snapshot) and B (identifier + `key` + JCS on the wire) are built and tested. Prioritize frictionless onboarding (users may be department-level and cannot create DNS).
+1 -1
View File
@@ -29,7 +29,7 @@ frxd registry --dir ./ma init --zone frx.federatedsearch.org
frxd registry --dir ./ma serve --listen 127.0.0.1:7800 frxd registry --dir ./ma serve --listen 127.0.0.1:7800
``` ```
(put Caddy in front for a real domain). The page at `/` collects the registration form (short name, organization details) and queues it for MA review — `frxd registry --dir <dir> applications` lists applications and `frxd registry --dir <dir> approve <id> --registry-url <url>` creates the member, mints its account credential, and prints the credential block to hand over. The token is reusable: it authorizes key enrollment for every node the member runs (`registry token <id>` mints an additional one; `registry revoke-token <id>` revokes all after a leak). A single-use 24h invite (`registry invite <id>`) remains for constrained handoffs. The block is `id=... token=... registry=... ma_key=...`. Organization details (legal name, representative, contacts, payment) are recorded privately by the MA in `<registry dir>/applications.json` — contract data, never in the public signed snapshot. (put Caddy in front for a real domain). The page at `/` collects the registration form (short name, organization details) and queues it for MA review — `frxd registry --dir <dir> applications` lists applications and `frxd registry --dir <dir> approve <id> --registry-url <url>` creates the member, mints its account credential, and prints the credential block to hand over (`--class enrichment` at approval for derived-corpora members, which are metadata-only; the default `source` fits everyone else — membership itself has no roles or tiers). The token is reusable: it authorizes key enrollment for every node the member runs (`registry token <id>` mints an additional one; `registry revoke-token <id>` revokes all after a leak). A single-use 24h invite (`registry invite <id>`) remains for constrained handoffs. The block is `id=... token=... registry=... ma_key=...`. Organization details (legal name, representative, contacts, payment) are recorded privately by the MA in `<registry dir>/applications.json` — contract data, never in the public signed snapshot.
New member: New member:
+20 -25
View File
@@ -449,15 +449,21 @@ pub fn registry_applications(dir: &Path) -> Result<()> {
Ok(()) Ok(())
} }
/// Approves a pending application: creates the member stub (class from the application), /// Approves a pending application: creates the member stub, mints its account
/// issues a 24h invite, and prints the credential block to hand to the member. /// credential (member token), and prints the credential block to hand over.
pub fn registry_approve(dir: &Path, id: &str, registry_url: Option<&str>) -> Result<()> { /// `--class enrichment` is for derived corpora (metadata-only, §6); default is source.
pub fn registry_approve(
dir: &Path,
id: &str,
registry_url: Option<&str>,
class: Option<&str>,
) -> Result<()> {
let (_, signed) = open_registry(dir)?; let (_, signed) = open_registry(dir)?;
if signed.doc.members.iter().any(|member| member.id == id) { if signed.doc.members.iter().any(|member| member.id == id) {
return Err(anyhow!("member {id} already listed")); return Err(anyhow!("member {id} already listed"));
} }
let application = registry::approve_application(dir, id)?; let _application = registry::approve_application(dir, id)?;
let class = if application.class == CLASS_ENRICHMENT { let class = if class == Some(CLASS_ENRICHMENT) {
CLASS_ENRICHMENT CLASS_ENRICHMENT
} else { } else {
CLASS_SOURCE CLASS_SOURCE
@@ -610,8 +616,6 @@ struct SignupRequest {
#[serde(default)] #[serde(default)]
domain: String, domain: String,
#[serde(default)] #[serde(default)]
class: Option<String>,
#[serde(default)]
payment: String, payment: String,
#[serde(default)] #[serde(default)]
privacy_link: String, privacy_link: String,
@@ -694,11 +698,6 @@ async fn registry_signup(
) )
.into_response(); .into_response();
} }
let class = if request.class.as_deref() == Some(CLASS_ENRICHMENT) {
CLASS_ENRICHMENT
} else {
CLASS_SOURCE
};
let application = registry::Application { let application = registry::Application {
id: id.clone(), id: id.clone(),
org: request.org.clone(), org: request.org.clone(),
@@ -706,7 +705,7 @@ async fn registry_signup(
email: request.email.clone(), email: request.email.clone(),
address: request.address.clone(), address: request.address.clone(),
domain: request.domain.clone(), domain: request.domain.clone(),
class: class.to_string(), class: CLASS_SOURCE.to_string(),
payment: request.payment.clone(), payment: request.payment.clone(),
privacy_link: request.privacy_link.clone(), privacy_link: request.privacy_link.clone(),
status: "pending".to_string(), status: "pending".to_string(),
@@ -858,8 +857,9 @@ scores on the wire, no in-protocol payment.</p>
<h2>1. Register with the membership authority</h2> <h2>1. Register with the membership authority</h2>
<div class="card"> <div class="card">
<p class="muted">This form requests membership from the membership authority (MA). The MA reviews <p class="muted">This form requests membership from the membership authority (MA). Membership has no
your organization details and issues a credential block roles or tiers: every member may broadcast queries and every member may answer them. The MA
reviews your organization details and issues a credential block
(<code>id=... token=... registry=... ma_key=...</code>); the onboarding wizard in step 4 then (<code>id=... token=... registry=... ma_key=...</code>); the onboarding wizard in step 4 then
binds your node's keys to the identifier. The public registry publishes only your identifier, binds your node's keys to the identifier. The public registry publishes only your identifier,
class, keys, and the federation's relays. The organization details below are kept privately by class, keys, and the federation's relays. The organization details below are kept privately by
@@ -878,11 +878,6 @@ the MA for the membership contract — never published, never on the wire.</p>
<input name="address" placeholder="street, city, country"></label> <input name="address" placeholder="street, city, country"></label>
<label>Organization domain (optional)<br> <label>Organization domain (optional)<br>
<input name="domain" placeholder="example.org"></label> <input name="domain" placeholder="example.org"></label>
<label>What you will share<br>
<select name="member_class">
<option value="source">Source member — content I own or host</option>
<option value="enrichment">Enrichment member — derived corpora (metadata-only)</option>
</select></label>
<label>Payment details (billing / payout — e.g. IBAN or payment handle)<br> <label>Payment details (billing / payout — e.g. IBAN or payment handle)<br>
<input name="payment" placeholder="kept private; the protocol itself carries no payment"></label> <input name="payment" placeholder="kept private; the protocol itself carries no payment"></label>
<label>Your privacy statement URL (optional)<br> <label>Your privacy statement URL (optional)<br>
@@ -897,10 +892,10 @@ the MA for the membership contract — never published, never on the wire.</p>
<h2>2. Download</h2> <h2>2. Download</h2>
<div class="card"> <div class="card">
<p>Static Linux x86_64 binaries (musl — no runtime dependencies):</p> <p>Static Linux x86_64 binaries (musl — no runtime dependencies):</p>
<pre class="cmd">curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.4/frxd-linux-amd64 <pre class="cmd">curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.5/frxd-linux-amd64
curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.4/frxd-linux-amd64.sha256 curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.5/frxd-linux-amd64.sha256
curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.4/frx-linux-amd64 curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.5/frx-linux-amd64
curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.4/frx-linux-amd64.sha256</pre> curl -LO https://git.federatedsearch.org/frx/frxd/releases/download/v0.1.5/frx-linux-amd64.sha256</pre>
<p class="muted">All releases: <a href="https://git.federatedsearch.org/frx/frxd/releases">git.federatedsearch.org/frx/frxd/releases</a>. <p class="muted">All releases: <a href="https://git.federatedsearch.org/frx/frxd/releases">git.federatedsearch.org/frx/frxd/releases</a>.
Source and spec (<code>rfc.txt</code>): <a href="https://git.federatedsearch.org/frx/frxd">git.federatedsearch.org/frx/frxd</a>.</p> Source and spec (<code>rfc.txt</code>): <a href="https://git.federatedsearch.org/frx/frxd">git.federatedsearch.org/frx/frxd</a>.</p>
</div> </div>
@@ -965,7 +960,7 @@ f.onsubmit = async (e) => {
body: JSON.stringify({ body: JSON.stringify({
label: f.label.value, label: f.label.value,
org: f.org.value, representative: f.representative.value, email: f.email.value, org: f.org.value, representative: f.representative.value, email: f.email.value,
address: f.address.value, domain: f.domain.value, class: f.member_class.value, address: f.address.value, domain: f.domain.value,
payment: f.payment.value, privacy_link: f.privacy_link.value, payment: f.payment.value, privacy_link: f.privacy_link.value,
attestation: f.attestation.checked, privacy_ack: f.privacy_ack.checked attestation: f.attestation.checked, privacy_ack: f.privacy_ack.checked
}) })
+7 -3
View File
@@ -184,6 +184,8 @@ enum RegistryCommand {
id: String, id: String,
#[arg(long)] #[arg(long)]
registry_url: Option<String>, registry_url: Option<String>,
#[arg(long)]
class: Option<String>,
}, },
Invite { Invite {
id: String, id: String,
@@ -370,9 +372,11 @@ async fn main() -> Result<()> {
RegistryCommand::Remove { id } => commands::registry_remove(&dir, &id)?, RegistryCommand::Remove { id } => commands::registry_remove(&dir, &id)?,
RegistryCommand::List => commands::registry_list(&dir)?, RegistryCommand::List => commands::registry_list(&dir)?,
RegistryCommand::Applications => commands::registry_applications(&dir)?, RegistryCommand::Applications => commands::registry_applications(&dir)?,
RegistryCommand::Approve { id, registry_url } => { RegistryCommand::Approve {
commands::registry_approve(&dir, &id, registry_url.as_deref())? id,
} registry_url,
class,
} => commands::registry_approve(&dir, &id, registry_url.as_deref(), class.as_deref())?,
RegistryCommand::Invite { id, registry_url } => { RegistryCommand::Invite { id, registry_url } => {
commands::registry_invite(&dir, &id, registry_url.as_deref())? commands::registry_invite(&dir, &id, registry_url.as_deref())?
} }
+14 -8
View File
@@ -91,7 +91,7 @@ async fn application_pending_then_approve_then_enroll_binds_key() {
assert_eq!(dup.status(), reqwest::StatusCode::CONFLICT); assert_eq!(dup.status(), reqwest::StatusCode::CONFLICT);
// MA approves: member stub; a member token then authorizes key enrollment // MA approves: member stub; a member token then authorizes key enrollment
commands::registry_approve(&dir, id, None).unwrap(); commands::registry_approve(&dir, id, None, None).unwrap();
let signed = registry::load_registry(&dir.join("registry.json")).unwrap(); let signed = registry::load_registry(&dir.join("registry.json")).unwrap();
assert!(signed.doc.members.iter().any(|member| member.id == id)); assert!(signed.doc.members.iter().any(|member| member.id == id));
let apps = registry::load_applications(&dir.join("applications.json")).unwrap(); let apps = registry::load_applications(&dir.join("applications.json")).unwrap();
@@ -166,7 +166,6 @@ async fn signup_stores_private_application_and_class() {
"email": "ops@keswick.example", "email": "ops@keswick.example",
"address": "1 Fell Road, Keswick", "address": "1 Fell Road, Keswick",
"domain": "keswick.example", "domain": "keswick.example",
"class": "enrichment",
"payment": "IBAN XX00 0000", "payment": "IBAN XX00 0000",
"privacy_link": "https://keswick.example/privacy", "privacy_link": "https://keswick.example/privacy",
"attestation": true, "attestation": true,
@@ -182,7 +181,8 @@ async fn signup_stores_private_application_and_class() {
Some("keswick-research.frx.invalid") Some("keswick-research.frx.invalid")
); );
// private application record holds the contract details // private application record holds the contract details; applicants do not
// self-declare a class — the MA assigns it at approval
let apps = registry::load_applications(&dir.join("applications.json")).unwrap(); let apps = registry::load_applications(&dir.join("applications.json")).unwrap();
assert_eq!(apps.len(), 1); assert_eq!(apps.len(), 1);
let app = &apps[0]; let app = &apps[0];
@@ -192,11 +192,17 @@ async fn signup_stores_private_application_and_class() {
assert_eq!(app.email, "ops@keswick.example"); assert_eq!(app.email, "ops@keswick.example");
assert_eq!(app.payment, "IBAN XX00 0000"); assert_eq!(app.payment, "IBAN XX00 0000");
assert_eq!(app.privacy_link, "https://keswick.example/privacy"); assert_eq!(app.privacy_link, "https://keswick.example/privacy");
assert_eq!(app.class, "enrichment"); assert_eq!(app.class, "source");
assert_eq!(app.status, "pending"); assert_eq!(app.status, "pending");
// approval creates the member entry with the declared class // approval with --class enrichment creates the member with that class
commands::registry_approve(&dir, "keswick-research.frx.invalid", None).unwrap(); commands::registry_approve(
&dir,
"keswick-research.frx.invalid",
None,
Some(frxd::config::CLASS_ENRICHMENT),
)
.unwrap();
let signed = registry::load_registry(&dir.join("registry.json")).unwrap(); let signed = registry::load_registry(&dir.join("registry.json")).unwrap();
let member = signed let member = signed
.doc .doc
@@ -223,7 +229,7 @@ async fn invite_reissues_token_per_node() {
.unwrap(); .unwrap();
submit_application(&http, &base, "Multi Node").await; submit_application(&http, &base, "Multi Node").await;
let id = "multi-node.frx.invalid"; let id = "multi-node.frx.invalid";
commands::registry_approve(&dir, id, None).unwrap(); commands::registry_approve(&dir, id, None, None).unwrap();
let enroll = |token: String, pubkey: String| { let enroll = |token: String, pubkey: String| {
let http = http.clone(); let http = http.clone();
@@ -276,7 +282,7 @@ async fn wizard_enrolls_and_writes_config() {
let http = reqwest::Client::new(); let http = reqwest::Client::new();
submit_application(&http, &base, "Wizard Test").await; submit_application(&http, &base, "Wizard Test").await;
let id = "wizard-test.frx.invalid"; let id = "wizard-test.frx.invalid";
commands::registry_approve(&dir, id, None).unwrap(); commands::registry_approve(&dir, id, None, None).unwrap();
let token = commands::registry_token(&dir, id, None).unwrap(); let token = commands::registry_token(&dir, id, None).unwrap();
let signed = registry::load_registry(&dir.join("registry.json")).unwrap(); let signed = registry::load_registry(&dir.join("registry.json")).unwrap();
let credentials = format!( let credentials = format!(