diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ea8c4bf --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +/target diff --git a/AGENTS.md b/AGENTS.md index ff30ca8..c5812d8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,8 +1,10 @@ # AGENTS.md ## Repo shape -- Spec-only repo: `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.3) is the entire content. No code, README, manifests, tests, CI, or build tooling — don't look for or invent build/lint/test commands. -- `frxd` (Rust: tokio, tantivy, axum, ed25519-dalek) is planned in §7 but does not exist here; treat it as design intent, not current structure. +- `rfc.txt` (FRX — Federated Retrieval Exchange, Draft 0.3) is the normative spec; `src/` is the Phase 1 `frxd` implementation (single crate, two binaries). +- `frxd` is the member node (init/add/index/serve/relay/query/status); `frx` is the thin client (search/query/status). Relay and node roles are separate subcommands. +- Commands: `cargo build`, `cargo test` (62 tests: unit in `src/`; e2e `tests/phase1.rs`; conformance `tests/conformance.rs`; concurrency/restart `tests/concurrency.rs`; real subprocess CLI `tests/cli.rs`; 1000-doc `tests/scale.rs`; purge-log absence `tests/purges.rs`; shared fixtures `tests/common/mod.rs`). No CI/lint config. +- E2E pattern: relay + nodes in-process on ephemeral ports with tempdir corpora; use `tests/common/mod.rs` helpers (`spawn_relay*`, `query_envelope`, `poll_messages`, `register`) for new coverage. Raw relay polls return envelopes (payload under `body`), not response bodies. ## Editing the spec - Read all of `rfc.txt` before editing; it is the sole source of truth and is deliberately terse. @@ -12,10 +14,26 @@ - §10 Open Issues are known gaps, not oversights (e.g., signature canonicalization blocks Phase-1 interop). Check it before "fixing" something. - Use the spec's vocabulary — member/querier/responder, citations/receipts/aggregates, source/enrichment members — not client/server or search-engine terms. +## Implementation notes +- Envelope signing is provisional (`src/crypto.rs`): `FRX/0.3` + fields + sorted-key canonical JSON body. §10's signature canonicalization open issue is unsolved — never present this scheme as interoperable. +- Phase 1 only: no aggregates (Phase 2), no receipts/lineage/delegation (Phase 3). Responses travel relay-mediated unicast; transport is HTTP long-poll, not SSE. +- Relay verifies signatures, carries only `query` broadcasts, holds no history (queue drained on poll), and returns 429 + Retry-After under backpressure — never silent drops. +- Responder searches only collections marked shared (I9), stays silent when nothing matches, and emits ordered results with honest `truncated`/`more_available` and no scores (I6). +- Index layout: Tantivy at `/index`, collections manifest at `/collections.toml`; `exposure` (metadata|full) gates whether `content` is returned. +- Egress checks live in the responder path (`src/node.rs` `respond`), not the relay — keep private collections unreachable there. +- Appendix B is executable policy: `tests/purges.rs` has one absence test per rejected mechanism (11 rows, not the RFC's informal "nine"). Add a negative test there before ever re-proposing one, and only if the rationale is addressed. +- `add`/`reindex` reset a collection (delete by manifest `name`) before re-adding, so deleted files don't linger; collection identity is its name, and same-named collections replace each other. +- Relay backpressure is global: any member's full queue 429s every publisher until drained (visible per §3, but one lagging member can stall the firehose — revisit before scale). + +## Known gaps (Phase 2/3, intentional — don't fake them) +- No aggregate serving/dashboard (RFC §9 conformance is partial without it), no directory watching (new files need `reindex`), no receipts/lineage/delegation, no TLS, no consumer admission/relay discovery. +- Node query dedup is by `qid` only; there is no envelope replay/nonce window (RFC doesn't require one). + ## Technical plans (deliberately not in the RFC) - Record plans here — not as spec edits — when they are implementation/demo choices rather than protocol surface. - Demo plan: build a useful end-to-end demo on GDELT and Common Crawl (CC-NEWS; sometimes called "OpenCrawl" in discussion) as enrichment members / backfill seeding. RFC §6 and Appendix A already name both as example derived corpora, so no new mechanisms are required; enrichment members are metadata-only exposure and sit outside the citation market. -- Phase 1 remains the minimal two-node query/response demo (§7); the enrichment demo layers on top of it. +- Phase 1 (two-node query/response) is built and tested; the enrichment demo layers on top of it. - Language: Rust (settled, matches §7). Decided by the engine requirement, not preference: Tantivy gives in-process Lucene-class BM25 + incremental indexing; C/C++ embedded alternatives are worse (Xapian GPL-2+, CLucene unmaintained, SQLite FTS5 thin), plus single static musl binaries for the install story and memory safety on the untrusted network/crypto path. Don't re-litigate. - frxd modes (one binary, config toggles, no code required of publishers): querier (broadcast/local-first search), responder (match incoming queries against shared collections, sign), local index (watch dirs, extract text, explicit shared marking per I9). Use RFC terms querier/responder, not "subscriber/publisher". +- Roles are not exclusive: a single node may issue queries and answer them concurrently (I5, §3 "any member"). Implement querier/responder as independent enable flags — never an exclusive mode enum or fixed deployment role. - Matching accuracy is a project-health concern: start lexical (Tantivy), plan a hybrid cheap lexical gate + optional local embedding rerank (two-stage ingestion, Appendix A); embedding model stays local and replaceable (I2/I5). diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 0000000..8e21718 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,3035 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aho-corasick" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba" +dependencies = [ + "memchr", +] + +[[package]] +name = "allocator-api2" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" + +[[package]] +name = "android_system_properties" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae221649c9976a6f6c56ae1facf410f3ddb33cc661c4b7b61020a912d4237fbc" +dependencies = [ + "libc", +] + +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "arc-swap" +version = "1.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b" +dependencies = [ + "rustversion", +] + +[[package]] +name = "async-trait" +version = "0.1.92" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "aws-lc-rs" +version = "1.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b281d307588d634de920874890732659e2e7672f72b5e10e81badc1a8a83621e" +dependencies = [ + "aws-lc-sys", + "zeroize", +] + +[[package]] +name = "aws-lc-sys" +version = "0.45.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9bff6c3b54fad79a2e60b8102caf565819711497c1f5f092f49508e2f5c31b27" +dependencies = [ + "cc", + "cmake", + "dunce", + "fs_extra", + "pkg-config", +] + +[[package]] +name = "axum" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90" +dependencies = [ + "axum-core", + "bytes", + "form_urlencoded", + "futures-util", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-util", + "itoa", + "matchit", + "memchr", + "mime", + "percent-encoding", + "pin-project-lite", + "serde_core", + "serde_json", + "serde_path_to_error", + "serde_urlencoded", + "sync_wrapper", + "tokio", + "tower", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "axum-core" +version = "0.5.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "http-body-util", + "mime", + "pin-project-lite", + "sync_wrapper", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "base64" +version = "0.22.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" + +[[package]] +name = "base64" +version = "0.23.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5" + +[[package]] +name = "bitflags" +version = "2.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" + +[[package]] +name = "bitpacking" +version = "0.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96a7139abd3d9cebf8cd6f920a389cf3dc9576172e32f4563f188cae3c3eb019" +dependencies = [ + "crunchy", +] + +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "bon" +version = "3.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60eafe0d77c3a2fc292c1d1346c3041b33c0a108085a2afabf672b70f69dbbc9" +dependencies = [ + "bon-macros", +] + +[[package]] +name = "bon-macros" +version = "3.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd0f9631d8aaaee112c41985d675ef269e02acbd4f33122836af4f0c5f699ff6" +dependencies = [ + "darling", + "ident_case", + "prettyplease", + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" + +[[package]] +name = "cc" +version = "1.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3eb0f42d6c360dc3f8a821f6bf2fdea7f72bfd36b3076eb0e6d1e9e0752fff4" +dependencies = [ + "find-msvc-tools", + "jobserver", + "libc", + "shlex", +] + +[[package]] +name = "census" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4f4c707c6a209cbe82d10abd08e1ea8995e9ea937d2550646e02798948992be0" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "chacha20" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" +dependencies = [ + "cfg-if", + "cpufeatures", + "rand_core 0.10.1", +] + +[[package]] +name = "chrono" +version = "0.4.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327" +dependencies = [ + "iana-time-zone", + "num-traits", + "windows-link", +] + +[[package]] +name = "clap" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aa8876b300ab35ba921adea3dfd70157a46249b33f95c9084ae5709785478946" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0797fb7aeb1406c84efac526901f7ec3ead2124f946b494e72879d4b54704d" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9c751b79415d4e559e3d1fcf128e09e720eb673a06d26cf6f392d37d75b66e0" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "clap_lex" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c133bc6a41be0d194c306b5506d15e6feeea7b1d6604bd3f8310dfb2ca96486" + +[[package]] +name = "cmake" +version = "0.1.58" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0f78a02292a74a88ac736019ab962ece0bc380e3f977bf72e376c5d78ff0678" +dependencies = [ + "cc", +] + +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + +[[package]] +name = "combine" +version = "4.6.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfc320937d09e6de266b31b9afb480f197d7a861be86be7cb2ea7e5d1bfffc5e" +dependencies = [ + "bytes", + "memchr", +] + +[[package]] +name = "core-foundation" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "cpufeatures" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" +dependencies = [ + "libc", +] + +[[package]] +name = "crc32fast" +version = "1.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "crossbeam-channel" +version = "0.5.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "98b0cc327b5bc766e7fda9c9260cc0fa81b43a8e240440422dff70788e3f9ef1" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-deque" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "622f3fc73690be383c7214310406f28a90e6edeadc3cea882f9d71e495b9711a" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc74980687109a3b14c72fd458107bf0baa1da1a1a805e178d15501ba9b86d9d" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" + +[[package]] +name = "crunchy" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" + +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", + "rand_core 0.10.1", +] + +[[package]] +name = "curve25519-dalek" +version = "5.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b5eed333089e2e1c1ac8c6c0398e5e2497b4c9926ca6d0365ed1e099afa5bc23" +dependencies = [ + "cfg-if", + "cpufeatures", + "curve25519-dalek-derive", + "digest", + "fiat-crypto", + "rand_core 0.10.1", + "rustc_version", + "subtle", + "zeroize", +] + +[[package]] +name = "curve25519-dalek-derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "darling" +version = "0.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed17f5901b6630b993ca003def43f2f8ef4014fc13b047b57aad617ff32bc2ec" +dependencies = [ + "darling_core", + "darling_macro", +] + +[[package]] +name = "darling_core" +version = "0.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6837e2cf7485aaae18f86181d2f0e9a7ed297a025e220aeabf63fdebd3a2ddff" +dependencies = [ + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 3.0.5", +] + +[[package]] +name = "darling_macro" +version = "0.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ac7135c3ef02b2f7833bbeb1be5ba7f966dcde8a87c6b87f65a778d71a02785" +dependencies = [ + "darling_core", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "datasketches" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c286de4e81ea2590afc24d754e0f83810c566f50a1388fa75ebd57928c0d9745" + +[[package]] +name = "deranged" +version = "0.5.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c" +dependencies = [ + "serde_core", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "displaydoc" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "downcast-rs" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "117240f60069e65410b3ae1bb213295bd828f707b5bec6596a1afc8793ce0cbc" + +[[package]] +name = "dunce" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813" + +[[package]] +name = "ed25519" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29fcf32e6c73d1079f83ab4d782de2d81620346a5f38c6237a86a22f8368980a" +dependencies = [ + "signature", +] + +[[package]] +name = "ed25519-dalek" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ebaa1a2bf1290ab3bfe5a7b771d050ebffab2711c19a81691c683a5144a25de" +dependencies = [ + "curve25519-dalek", + "ed25519", + "rand_core 0.10.1", + "sha2", + "signature", + "subtle", + "zeroize", +] + +[[package]] +name = "either" +version = "1.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34" + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "erased-serde" +version = "0.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2add8a07dd6a8d93ff627029c51de145e12686fbc36ecb298ac22e74cf02dec" +dependencies = [ + "serde", + "serde_core", + "typeid", +] + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "fastdivide" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9afc2bd4d5a73106dd53d10d73d3401c2f32730ba2c0b93ddb888a8983680471" + +[[package]] +name = "fastrand" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" + +[[package]] +name = "fiat-crypto" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "64cd1e32ddd350061ae6edb1b082d7c54915b5c672c389143b9a63403a109f24" + +[[package]] +name = "find-msvc-tools" +version = "0.1.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d" + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "form_urlencoded" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "frxd" +version = "0.1.0" +dependencies = [ + "anyhow", + "axum", + "chrono", + "clap", + "ed25519-dalek", + "hex", + "rand 0.8.8", + "reqwest", + "serde", + "serde_json", + "tantivy", + "tempfile", + "tokio", + "toml", +] + +[[package]] +name = "fs4" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8640e34b88f7652208ce9e88b1a37a2ae95227d84abec377ccd3c5cfeb141ed4" +dependencies = [ + "rustix", + "windows-sys 0.59.0", +] + +[[package]] +name = "fs_extra" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c" + +[[package]] +name = "futures-channel" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4" +dependencies = [ + "futures-core", +] + +[[package]] +name = "futures-core" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" + +[[package]] +name = "futures-macro" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "futures-sink" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" + +[[package]] +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" +dependencies = [ + "futures-core", + "futures-macro", + "futures-task", + "pin-project-lite", + "slab", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "wasi", + "wasm-bindgen", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "js-sys", + "libc", + "r-efi", + "rand_core 0.10.1", + "wasm-bindgen", +] + +[[package]] +name = "h2" +version = "0.4.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16" +dependencies = [ + "atomic-waker", + "bytes", + "fnv", + "futures-core", + "futures-sink", + "http", + "indexmap", + "slab", + "tokio", + "tokio-util", + "tracing", +] + +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" +dependencies = [ + "allocator-api2", + "equivalent", + "foldhash", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "htmlescape" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e9025058dae765dee5070ec375f591e2ba14638c63feff74f13805a72e523163" + +[[package]] +name = "http" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "918d3568bebf352712bc2ef3d46a8bcf1a75b373be6539de198e9105cbbf9ce0" +dependencies = [ + "bytes", + "itoa", +] + +[[package]] +name = "http-body" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ca2a8f2913ee65f60facd6a5905613afaa448497a0230cc41ce022d93290bc2c" +dependencies = [ + "bytes", + "http", +] + +[[package]] +name = "http-body-util" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "pin-project-lite", +] + +[[package]] +name = "httparse" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87" + +[[package]] +name = "httpdate" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9" + +[[package]] +name = "hybrid-array" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17" +dependencies = [ + "typenum", +] + +[[package]] +name = "hyper" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27b501faa50e7a26c3d3560ca625132f4078a17771f4810baf70475ae48cbe43" +dependencies = [ + "atomic-waker", + "bytes", + "futures-channel", + "futures-core", + "h2", + "http", + "http-body", + "httparse", + "httpdate", + "itoa", + "pin-project-lite", + "smallvec", + "tokio", + "want", +] + +[[package]] +name = "hyper-rustls" +version = "0.27.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f" +dependencies = [ + "http", + "hyper", + "hyper-util", + "rustls", + "tokio", + "tokio-rustls", + "tower-service", +] + +[[package]] +name = "hyper-util" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0" +dependencies = [ + "base64 0.22.1", + "bytes", + "futures-channel", + "futures-util", + "http", + "http-body", + "hyper", + "ipnet", + "libc", + "percent-encoding", + "pin-project-lite", + "socket2", + "tokio", + "tower-service", + "tracing", +] + +[[package]] +name = "iana-time-zone" +version = "0.1.65" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "log", + "wasm-bindgen", + "windows-core", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f" +dependencies = [ + "cc", +] + +[[package]] +name = "icu_collections" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fa68d21081c4a05d5a901a1c62add574c77048b6a1c67be3b50ce0b60d4ca513" +dependencies = [ + "displaydoc", + "potential_utf", + "utf8_iter", + "yoke", + "zerofrom", + "zerovec", +] + +[[package]] +name = "icu_locale_core" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d56e28588da92eee5c3201a6eff33fabdd49b62269c8938d4ff050ce4d900deb" +dependencies = [ + "displaydoc", + "litemap", + "tinystr", + "writeable", + "zerovec", +] + +[[package]] +name = "icu_normalizer" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12f9cf5f235641ed274641dd81c3f28d870e276763d0797aeeab72317b1c646f" +dependencies = [ + "icu_collections", + "icu_normalizer_data", + "icu_properties", + "icu_provider", + "smallvec", + "zerovec", +] + +[[package]] +name = "icu_normalizer_data" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1563da1ed3e0b3bf3d74c9b85917ac9c56464d2f57242270c09c9e752f8021a0" + +[[package]] +name = "icu_properties" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e7ca276ad3145661a65914e6daf131ca5120cd3dcee8f8f3214b8875184a148" +dependencies = [ + "displaydoc", + "icu_collections", + "icu_locale_core", + "icu_properties_data", + "icu_provider", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_properties_data" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e590f038c1464a96894fd6d10127e90a8be4509f56ff7ecef851b15cee0b7caa" + +[[package]] +name = "icu_provider" +version = "2.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d27bbb9d3abbefac45d55f647c9de1d44aafcd1186eb91879afef17c396c3e73" +dependencies = [ + "displaydoc", + "icu_locale_core", + "writeable", + "yoke", + "zerofrom", + "zerotrie", + "zerovec", +] + +[[package]] +name = "ident_case" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39" + +[[package]] +name = "idna" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" +dependencies = [ + "idna_adapter", + "smallvec", + "utf8_iter", +] + +[[package]] +name = "idna_adapter" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714" +dependencies = [ + "icu_normalizer", + "icu_properties", +] + +[[package]] +name = "indexmap" +version = "2.14.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855" +dependencies = [ + "equivalent", + "hashbrown 0.17.1", +] + +[[package]] +name = "inventory" +version = "0.3.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4f0c30c76f2f4ccee3fe55a2435f691ca00c0e4bd87abe4f4a851b1d4dac39b" +dependencies = [ + "rustversion", +] + +[[package]] +name = "ipnet" +version = "2.12.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0" + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itertools" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b192c782037fadd9cfa75548310488aabdbf3d2da73885b31bd0abd03351285" +dependencies = [ + "either", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "jni" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5efd9a482cf3a427f00d6b35f14332adc7902ce91efb778580e180ff90fa3498" +dependencies = [ + "cfg-if", + "combine", + "jni-macros", + "jni-sys", + "log", + "simd_cesu8", + "thiserror", + "walkdir", + "windows-link", +] + +[[package]] +name = "jni-macros" +version = "0.22.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a00109accc170f0bdb141fed3e393c565b6f5e072365c3bd58f5b062591560a3" +dependencies = [ + "proc-macro2", + "quote", + "rustc_version", + "simd_cesu8", + "syn 2.0.119", +] + +[[package]] +name = "jni-sys" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6377a88cb3910bee9b0fa88d4f42e1d2da8e79915598f65fb0c7ee14c878af2" +dependencies = [ + "jni-sys-macros", +] + +[[package]] +name = "jni-sys-macros" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38c0b942f458fe50cdac086d2f946512305e5631e720728f2a61aabcd47a6264" +dependencies = [ + "quote", + "syn 2.0.119", +] + +[[package]] +name = "jobserver" +version = "0.1.35" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c00acbd29eabad4a2392fa0e921c874934dbbf4194312ad20f04a0ed67a3cb3" +dependencies = [ + "getrandom 0.4.3", + "libc", +] + +[[package]] +name = "js-sys" +version = "0.3.105" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e" +dependencies = [ + "cfg-if", + "futures-util", + "wasm-bindgen", +] + +[[package]] +name = "levenshtein_automata" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c2cdeb66e45e9f36bfad5bbdb4d2384e70936afbee843c6f6543f0c551ebb25" + +[[package]] +name = "libc" +version = "0.2.189" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "litemap" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47d9d19d1d6efa0109d2f65ff4c85cddd50bd572e5a00127ab10987290bcefae" + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6" + +[[package]] +name = "lru" +version = "0.16.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f66e8d5d03f609abc3a39e6f08e4164ebf1447a732906d39eb9b99b7919ef39" +dependencies = [ + "hashbrown 0.16.1", +] + +[[package]] +name = "lru-slab" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4050469837a6ff301cd14c1f8f24f88549e6d548f24f64e2148eb0f72cebc51f" + +[[package]] +name = "lz4_flex" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ef0d4ed8669f8f8826eb00dc878084aa8f253506c4fd5e8f58f5bce72ddb97e" + +[[package]] +name = "matchit" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3" + +[[package]] +name = "measure_time" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "51c55d61e72fc3ab704396c5fa16f4c184db37978ae4e94ca8959693a235fc0e" +dependencies = [ + "log", +] + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "memmap2" +version = "0.9.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1219ed1b7f229ee7104d281dd01d6802fe28bb6e95d292942c4daacdeb798c0" +dependencies = [ + "libc", +] + +[[package]] +name = "mime" +version = "0.3.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a" + +[[package]] +name = "minimal-lexical" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" + +[[package]] +name = "mio" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + +[[package]] +name = "murmurhash32" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2195bf6aa996a481483b29d62a7663eed3fe39600c460e323f8ff41e90bdd89b" + +[[package]] +name = "nom" +version = "7.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a" +dependencies = [ + "memchr", + "minimal-lexical", +] + +[[package]] +name = "num-conv" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441" + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "oneshot" +version = "0.1.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "269bca4c2591a28585d6bf10d9ed0332b7d76900a1b02bec41bdc3a2cdcda107" + +[[package]] +name = "openssl-probe" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" + +[[package]] +name = "ordered-float" +version = "5.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8c7c9e0d9b23589f26070720bac724174bfec1083e82f7854cdd0267518343c0" +dependencies = [ + "num-traits", +] + +[[package]] +name = "ownedbytes" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2fbd56f7631767e61784dc43f8580f403f4475bd4aaa4da003e6295e1bab4a7e" +dependencies = [ + "stable_deref_trait", +] + +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link", +] + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "pkg-config" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548" + +[[package]] +name = "potential_utf" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d83eb9bc6d8e5cf568e7a1101d60ee05e81ed50ea106026f3d18deeb046d7661" +dependencies = [ + "zerovec", +] + +[[package]] +name = "powerfmt" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "prettyplease" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2bfe0f4c752e450fc2faf62654f1c134747922825d5b04ca717b8874f41a40c0" +dependencies = [ + "proc-macro2", + "syn 3.0.5", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quinn" +version = "0.11.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4051e23e9185c255a7e33ef59cdbca87a22d359052eecd22fc6b901fb37d9d11" +dependencies = [ + "bytes", + "cfg_aliases", + "pin-project-lite", + "quinn-proto", + "quinn-udp", + "rustc-hash", + "rustls", + "socket2", + "thiserror", + "tokio", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-proto" +version = "0.11.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9746dbde176634f4f2f1faf2404e30a31b2bc1e9cafb5329c95d8177a18c9fc" +dependencies = [ + "aws-lc-rs", + "bytes", + "getrandom 0.4.3", + "lru-slab", + "rand 0.10.2", + "rand_pcg", + "ring", + "rustc-hash", + "rustls", + "rustls-pki-types", + "slab", + "thiserror", + "tinyvec", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-udp" +version = "0.5.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694" +dependencies = [ + "cfg_aliases", + "libc", + "once_cell", + "socket2", + "tracing", + "windows-sys 0.61.2", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rand" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e058c7de0b26af77780c769414d6257830bb240f3c38477dbc2c16e5f54d6d4c" +dependencies = [ + "libc", + "rand_chacha", + "rand_core 0.6.4", +] + +[[package]] +name = "rand" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" +dependencies = [ + "chacha20", + "getrandom 0.4.3", + "rand_core 0.10.1", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core 0.6.4", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom 0.2.17", +] + +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + +[[package]] +name = "rand_pcg" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "caa0f4137e1c0a72f4c651489402276c8e8e1cf081f3b0ba156d2cbeef09e86a" +dependencies = [ + "rand_core 0.10.1", +] + +[[package]] +name = "rayon" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags", +] + +[[package]] +name = "regex" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "reqwest" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "16a1cfa75cc186dd73d5818e510e042e40927bccc9c236b061cea97e1eb08029" +dependencies = [ + "base64 0.23.1", + "bytes", + "futures-core", + "h2", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-rustls", + "hyper-util", + "js-sys", + "log", + "percent-encoding", + "pin-project-lite", + "quinn", + "rustls", + "rustls-pki-types", + "rustls-platform-verifier", + "serde", + "serde_json", + "sync_wrapper", + "tokio", + "tokio-rustls", + "tower", + "tower-http", + "tower-service", + "url", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", +] + +[[package]] +name = "ring" +version = "0.17.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7" +dependencies = [ + "cc", + "cfg-if", + "getrandom 0.2.17", + "libc", + "untrusted", + "windows-sys 0.52.0", +] + +[[package]] +name = "rust-stemmers" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e46a2036019fdb888131db7a4c847a1063a7493f971ed94ea82c67eada63ca54" +dependencies = [ + "serde", + "serde_derive", +] + +[[package]] +name = "rustc-hash" +version = "2.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d" + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustls" +version = "0.23.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" +dependencies = [ + "aws-lc-rs", + "once_cell", + "rustls-pki-types", + "rustls-webpki", + "subtle", + "zeroize", +] + +[[package]] +name = "rustls-native-certs" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dab5152771c58876a2146916e53e35057e1a4dfa2b9df0f0305b07f611fdea4d" +dependencies = [ + "openssl-probe", + "rustls-pki-types", + "schannel", + "security-framework", +] + +[[package]] +name = "rustls-pki-types" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2f4925028c7eb5d1fcdaf196971378ed9d2c1c4efc7dc5d011256f76c99c0a96" +dependencies = [ + "web-time", + "zeroize", +] + +[[package]] +name = "rustls-platform-verifier" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26d1e2536ce4f35f4846aa13bff16bd0ff40157cdb14cc056c7b14ba41233ba0" +dependencies = [ + "core-foundation", + "core-foundation-sys", + "jni", + "log", + "once_cell", + "rustls", + "rustls-native-certs", + "rustls-platform-verifier-android", + "rustls-webpki", + "security-framework", + "security-framework-sys", + "webpki-root-certs", + "windows-sys 0.61.2", +] + +[[package]] +name = "rustls-platform-verifier-android" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f" + +[[package]] +name = "rustls-webpki" +version = "0.103.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2" +dependencies = [ + "aws-lc-rs", + "ring", + "rustls-pki-types", + "untrusted", +] + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "schannel" +version = "0.1.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91c1b7e4904c873ef0710c1f407dde2e6287de2bebc1bbbf7d430bb7cbffd939" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "security-framework" +version = "3.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d" +dependencies = [ + "bitflags", + "core-foundation", + "core-foundation-sys", + "libc", + "security-framework-sys", +] + +[[package]] +name = "security-framework-sys" +version = "2.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_path_to_error" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457" +dependencies = [ + "itoa", + "serde", + "serde_core", +] + +[[package]] +name = "serde_spanned" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26" +dependencies = [ + "serde_core", +] + +[[package]] +name = "serde_urlencoded" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd" +dependencies = [ + "form_urlencoded", + "itoa", + "ryu", + "serde", +] + +[[package]] +name = "sha2" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "signature" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28d567dcbaf0049cb8ac2608a76cd95ff9e4412e1899d389ee400918ca7537f5" +dependencies = [ + "rand_core 0.10.1", +] + +[[package]] +name = "simd_cesu8" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11031e251abf8611c80f460e19dbdeb54a66db918e49c65a7065b46ac7aec520" +dependencies = [ + "rustc_version", + "simdutf8", +] + +[[package]] +name = "simdutf8" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e" + +[[package]] +name = "sketches-ddsketch" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "513c3f5f732bfd6fbb187619c2dfe9d2f25f1a2976f01d575f0fd329d565df56" +dependencies = [ + "serde", +] + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "smallvec" +version = "1.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba467056f1b547ed52077911161fc86985becbc60e8e1857c8a144dab0def891" + +[[package]] +name = "socket2" +version = "0.6.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "subtle" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "sync_wrapper" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263" +dependencies = [ + "futures-core", +] + +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tantivy" +version = "0.26.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "861facfabd71044968f364837f9a083b56464ba5a59079f88706ee5c451ca069" +dependencies = [ + "aho-corasick", + "arc-swap", + "base64 0.22.1", + "bitpacking", + "bon", + "byteorder", + "census", + "crc32fast", + "crossbeam-channel", + "datasketches", + "downcast-rs", + "fastdivide", + "fnv", + "fs4", + "htmlescape", + "itertools", + "levenshtein_automata", + "log", + "lru", + "lz4_flex", + "measure_time", + "memmap2", + "once_cell", + "oneshot", + "rayon", + "regex", + "rust-stemmers", + "rustc-hash", + "serde", + "serde_json", + "sketches-ddsketch", + "smallvec", + "tantivy-bitpacker", + "tantivy-columnar", + "tantivy-common", + "tantivy-fst", + "tantivy-query-grammar", + "tantivy-stacker", + "tantivy-tokenizer-api", + "tempfile", + "thiserror", + "time", + "typetag", + "uuid", + "winapi", +] + +[[package]] +name = "tantivy-bitpacker" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fed3d674429bcd2de5d0a6d1aa5495fed8afd9c5ecce993019caf7615f53fa4" +dependencies = [ + "bitpacking", +] + +[[package]] +name = "tantivy-columnar" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c57166f5bcfd478f370ab8445afb4678dce44801fa5ce5c451aaf8595583c5dc" +dependencies = [ + "downcast-rs", + "fastdivide", + "itertools", + "serde", + "tantivy-bitpacker", + "tantivy-common", + "tantivy-sstable", + "tantivy-stacker", +] + +[[package]] +name = "tantivy-common" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbf10915aa75da3c3b0d58b58853d2e889efbaf32d4982a4c3715dde6bba23e5" +dependencies = [ + "async-trait", + "byteorder", + "ownedbytes", + "serde", + "time", +] + +[[package]] +name = "tantivy-fst" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d60769b80ad7953d8a7b2c70cdfe722bbcdcac6bccc8ac934c40c034d866fc18" +dependencies = [ + "byteorder", + "regex-syntax", + "utf8-ranges", +] + +[[package]] +name = "tantivy-query-grammar" +version = "0.26.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfadb8526b6da90704feb293b0701a6aae62ea14983143344be2dc5ce30f1d82" +dependencies = [ + "fnv", + "nom", + "ordered-float", + "serde", + "serde_json", +] + +[[package]] +name = "tantivy-sstable" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a2cfc3ac5164cbadc28965ffb145a8f47582a60ae5897859ad8d4316596c606" +dependencies = [ + "futures-util", + "itertools", + "tantivy-bitpacker", + "tantivy-common", + "tantivy-fst", + "zstd", +] + +[[package]] +name = "tantivy-stacker" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6cbb051742da9d53ca9e8fff43a9b10e319338b24e2c0e15d0372df19ffeb951" +dependencies = [ + "murmurhash32", + "tantivy-common", +] + +[[package]] +name = "tantivy-tokenizer-api" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eac258c2c6390673f2685813afeeafcb8c4e0ee7de8dd3fc46838dcc37263f98" +dependencies = [ + "serde", +] + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom 0.4.3", + "once_cell", + "rustix", + "windows-sys 0.61.2", +] + +[[package]] +name = "thiserror" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "time" +version = "0.3.55" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134" +dependencies = [ + "deranged", + "num-conv", + "powerfmt", + "serde_core", + "time-core", + "time-macros", +] + +[[package]] +name = "time-core" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109" + +[[package]] +name = "time-macros" +version = "0.2.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e689342a48d2ea927c87ea50cabf8594854bf940e9310208848d680d668ed85" +dependencies = [ + "num-conv", + "time-core", +] + +[[package]] +name = "tinystr" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1e27c91459209c2986af3dcf603a5a74a4368754ce37414f59acc971167f643" +dependencies = [ + "displaydoc", + "zerovec", +] + +[[package]] +name = "tinyvec" +version = "1.13.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fd3ca314f692efd6c868f8408f53fe444634a845f96c028b97d35f6a1f79f0ee" + +[[package]] +name = "tokio" +version = "1.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed" +dependencies = [ + "bytes", + "libc", + "mio", + "parking_lot", + "pin-project-lite", + "signal-hook-registry", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "tokio-rustls" +version = "0.26.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b0c85f2c3ef0b1cd58b36682f4b17aaa995f0e5db534d85692b4903abce21f67" +dependencies = [ + "rustls", + "tokio", +] + +[[package]] +name = "tokio-util" +version = "0.7.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "libc", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "toml" +version = "1.1.6+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "920602543f0911ab71da12c50d59701da54c196d1a2bf5cb4b75667f137a406a" +dependencies = [ + "indexmap", + "serde_core", + "serde_spanned", + "toml_datetime", + "toml_parser", + "toml_writer", + "winnow", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_parser" +version = "1.1.3+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d38ac1cf9b95face32296c0a3ede1fdc270627c9d9c02a7274dd6d960dc4d56" +dependencies = [ + "winnow", +] + +[[package]] +name = "toml_writer" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7d56353a2a665ad0f41a421187180aab746c8c325620617ad883a99a1cbe66d2" + +[[package]] +name = "tower" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4" +dependencies = [ + "futures-core", + "futures-util", + "pin-project-lite", + "sync_wrapper", + "tokio", + "tower-layer", + "tower-service", + "tracing", +] + +[[package]] +name = "tower-http" +version = "0.6.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840" +dependencies = [ + "bitflags", + "bytes", + "futures-util", + "http", + "http-body", + "pin-project-lite", + "tower", + "tower-layer", + "tower-service", + "url", +] + +[[package]] +name = "tower-layer" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e" + +[[package]] +name = "tower-service" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3" + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "log", + "pin-project-lite", + "tracing-core", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "try-lock" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" + +[[package]] +name = "typeid" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc7d623258602320d5c55d1bc22793b57daff0ec7efc270ea7d55ce1d5f5471c" + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "typetag" +version = "0.2.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c90e86058a30d42a1a928dfb4b49bb33c98c3a2b4909492e6b0881cd94798ec2" +dependencies = [ + "erased-serde", + "inventory", + "once_cell", + "serde", + "typetag-impl", +] + +[[package]] +name = "typetag-impl" +version = "0.2.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f153acc4e99a5f2a5aefa09fb078be54e26271b2813f6041200b224c098d8328" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "untrusted" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1" + +[[package]] +name = "url" +version = "2.5.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed" +dependencies = [ + "form_urlencoded", + "idna", + "percent-encoding", + "serde", +] + +[[package]] +name = "utf8-ranges" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7fcfc827f90e53a02eaef5e535ee14266c1d569214c6aa70133a624d8a3164ba" + +[[package]] +name = "utf8_iter" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "uuid" +version = "1.26.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ef6dac1e96601b4fb3acccccff2139741fcb757cb9a36089bf5be91cfb285ce" +dependencies = [ + "getrandom 0.4.3", + "js-sys", + "serde_core", + "wasm-bindgen", +] + +[[package]] +name = "walkdir" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b" +dependencies = [ + "same-file", + "winapi-util", +] + +[[package]] +name = "want" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e" +dependencies = [ + "try-lock", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasm-bindgen" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-futures" +version = "0.4.78" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ef4c5d3d2cdf5c54f4231181768f5510842e350db025faf1f7163b1030ed928" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 3.0.5", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "web-sys" +version = "0.3.105" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fbddc4a036f00ec4f18c83445bd3115cb306a91da554919a099d9222fe4a7f8" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "web-time" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "webpki-root-certs" +version = "1.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b96554aa2acc8ccdb7e1c9a58a7a68dd5d13bccc69cd124cb09406db612a1c9b" +dependencies = [ + "rustls-pki-types", +] + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows-core" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" +dependencies = [ + "windows-implement", + "windows-interface", + "windows-link", + "windows-result", + "windows-strings", +] + +[[package]] +name = "windows-implement" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "windows-interface" +version = "0.59.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-result" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-strings" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-sys" +version = "0.52.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" +dependencies = [ + "windows-targets", +] + +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "winnow" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b97319f7b8343df12cc98938e5c3eb436064524c8d2b4e30a1d3a36eecdf81" + +[[package]] +name = "writeable" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc" + +[[package]] +name = "yoke" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5" +dependencies = [ + "stable_deref_trait", + "yoke-derive", + "zerofrom", +] + +[[package]] +name = "yoke-derive" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zerocopy" +version = "0.8.57" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d35102a9f36d089ccae9e4c6802bc118be4487b80aaffc0ab4e0cf5ce92d2873" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.57" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "146c01f5ab44258da43cf276c74a2763db2ff3969c9c652c3f2de07041d0b2bc" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "zerofrom" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272" +dependencies = [ + "zerofrom-derive", +] + +[[package]] +name = "zerofrom-derive" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zeroize" +version = "1.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e" + +[[package]] +name = "zerotrie" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ea269c3bd32f0a32c321907a2ae912ba6f4649bb0fc764a15627e99a7095a3f" +dependencies = [ + "displaydoc", + "yoke", + "zerofrom", +] + +[[package]] +name = "zerovec" +version = "0.11.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb0464e17806c1d976d5cba29399c7f08e516e279e2ba493f63123b5fca67dd8" +dependencies = [ + "yoke", + "zerofrom", + "zerovec-derive", +] + +[[package]] +name = "zerovec-derive" +version = "0.11.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" + +[[package]] +name = "zstd" +version = "0.13.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e91ee311a569c327171651566e07972200e76fcfe2242a4fa446149a3881c08a" +dependencies = [ + "zstd-safe", +] + +[[package]] +name = "zstd-safe" +version = "7.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "64d80649ab6db9d9f6f9c80a40becd948eda4714a0a5ac8c4d157a32231c7882" +dependencies = [ + "zstd-sys", +] + +[[package]] +name = "zstd-sys" +version = "2.1.0+zstd.1.5.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ef0a8027ec3ee71300ab3bcbcd0393f434aa72b91ca6d635a39941deae8eea0" +dependencies = [ + "cc", + "pkg-config", +] diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 0000000..a4851cf --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,22 @@ +[package] +name = "frxd" +version = "0.1.0" +edition = "2024" + +[dependencies] +anyhow = "1.0.104" +axum = "0.8.9" +chrono = { version = "0.4.45", default-features = false, features = ["clock", "std"] } +clap = { version = "4.6.7", features = ["derive"] } +ed25519-dalek = { version = "3.0.0", features = ["rand_core"] } +hex = "0.4.3" +rand = "0.8" +reqwest = { version = "0.13.5", default-features = false, features = ["json", "rustls", "http2"] } +serde = { version = "1.0.229", features = ["derive"] } +serde_json = "1.0.151" +tantivy = "0.26.2" +tokio = { version = "1.53.1", features = ["full"] } +toml = "1.1.6" + +[dev-dependencies] +tempfile = "3.27.0" diff --git a/src/bin/frx.rs b/src/bin/frx.rs new file mode 100644 index 0000000..a8a5ba9 --- /dev/null +++ b/src/bin/frx.rs @@ -0,0 +1,55 @@ +use std::path::PathBuf; + +use anyhow::Result; +use clap::{Parser, Subcommand}; +use frxd::commands; + +#[derive(Parser)] +#[command( + name = "frx", + version, + about = "FRX client — local search and broadcast queries" +)] +struct Cli { + #[arg(long, global = true, default_value = "frxd.toml")] + config: PathBuf, + #[command(subcommand)] + command: Command, +} + +#[derive(Subcommand)] +enum Command { + Search { + text: String, + #[arg(long, default_value_t = 10)] + limit: usize, + }, + Query { + text: String, + #[arg(long)] + max_results: Option, + #[arg(long)] + timeout_ms: Option, + #[arg(long)] + local_only: bool, + }, + Status, +} + +#[tokio::main] +async fn main() -> Result<()> { + let cli = Cli::parse(); + match cli.command { + Command::Search { text, limit } => commands::search(&cli.config, &text, limit).await?, + Command::Query { + text, + max_results, + timeout_ms, + local_only, + } => { + commands::query(&cli.config, &text, max_results, timeout_ms, !local_only).await?; + } + Command::Status => commands::status(&cli.config).await?, + } + Ok(()) +} diff --git a/src/commands.rs b/src/commands.rs new file mode 100644 index 0000000..39d6ad2 --- /dev/null +++ b/src/commands.rs @@ -0,0 +1,115 @@ +use std::path::Path; + +use anyhow::{Context, Result}; + +use crate::config::Config; +use crate::index::{Collection, LocalIndex, load_collections, save_collections}; +use crate::message::{EXPOSURE_FULL, EXPOSURE_METADATA}; +use crate::node; +use crate::render; + +pub fn add( + config_path: &Path, + path: &Path, + name: Option, + shared: bool, + exposure: &str, +) -> Result<()> { + let config = Config::load(config_path)?; + let index = LocalIndex::open(&config.index_dir())?; + let name = name.unwrap_or_else(|| { + path.file_name() + .and_then(|n| n.to_str()) + .unwrap_or("collection") + .to_string() + }); + let exposure = if exposure == EXPOSURE_FULL { + EXPOSURE_FULL + } else { + EXPOSURE_METADATA + }; + let collection = Collection { + name: name.clone(), + path: path + .canonicalize() + .with_context(|| format!("resolving {}", path.display()))? + .display() + .to_string(), + shared, + exposure: exposure.to_string(), + }; + let added = index.add_collection(&collection)?; + let manifest = config.collections_path(); + let mut collections = load_collections(&manifest)?; + collections.retain(|existing| existing.name != name); + collections.push(collection); + save_collections(&manifest, &collections)?; + println!( + "indexed {added} file(s) from {} as collection '{name}' (shared={shared}, exposure={exposure})", + path.display() + ); + Ok(()) +} + +pub fn reindex(config_path: &Path) -> Result<()> { + let config = Config::load(config_path)?; + let index = LocalIndex::open(&config.index_dir())?; + let collections = load_collections(&config.collections_path())?; + if collections.is_empty() { + println!("no collections registered; use `frxd add `"); + return Ok(()); + } + for collection in collections { + let added = index.add_collection(&collection)?; + println!( + "collection '{}' ({}): {added} file(s)", + collection.name, collection.path + ); + } + Ok(()) +} + +pub async fn search(config_path: &Path, text: &str, limit: usize) -> Result<()> { + let config = Config::load(config_path)?; + let index = LocalIndex::open(&config.index_dir())?; + let (hits, total) = index.search(text, limit, false)?; + render::local_hits(&hits, total); + Ok(()) +} + +pub async fn query( + config_path: &Path, + text: &str, + max_results: Option, + timeout_ms: Option, + network: bool, +) -> Result<()> { + let config = Config::load(config_path)?; + let base = format!("http://{}", config.node.listen); + let outcome = node::control_query(&base, text, max_results, timeout_ms, network).await?; + render::query_outcome(&outcome); + Ok(()) +} + +pub async fn status(config_path: &Path) -> Result<()> { + let config = Config::load(config_path)?; + let base = format!("http://{}", config.node.listen); + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(2)) + .build()?; + match client.get(format!("{base}/v1/local/status")).send().await { + Ok(response) if response.status().is_success() => { + let value: serde_json::Value = response.json().await?; + println!("{}", serde_json::to_string_pretty(&value)?); + } + Ok(response) => println!("node at {base} responded {}", response.status()), + Err(_) => { + println!("node not running at {base}"); + println!("config: {}", config_path.display()); + println!("name: {}", config.node.name); + println!("data dir: {}", config.data_dir().display()); + println!("relays: {:?}", config.node.relays); + } + } + Ok(()) +} diff --git a/src/config.rs b/src/config.rs new file mode 100644 index 0000000..0756dd4 --- /dev/null +++ b/src/config.rs @@ -0,0 +1,151 @@ +use std::fs; +use std::path::{Path, PathBuf}; + +use anyhow::{Context, Result}; +use serde::{Deserialize, Serialize}; + +use crate::crypto::Keypair; + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Config { + pub node: NodeSection, + #[serde(default)] + pub query: QuerySection, + #[serde(default)] + pub index: IndexSection, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct NodeSection { + pub name: String, + pub listen: String, + pub relays: Vec, + #[serde(default)] + pub trusted_keys: Vec, + #[serde(default = "default_true")] + pub responder: bool, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct QuerySection { + #[serde(default = "default_max_results")] + pub max_results: usize, + #[serde(default = "default_timeout_ms")] + pub timeout_ms: u64, +} + +impl Default for QuerySection { + fn default() -> Self { + Self { + max_results: default_max_results(), + timeout_ms: default_timeout_ms(), + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct IndexSection { + #[serde(default = "default_data_dir")] + pub data_dir: String, +} + +impl Default for IndexSection { + fn default() -> Self { + Self { + data_dir: default_data_dir(), + } + } +} + +fn default_true() -> bool { + true +} + +fn default_max_results() -> usize { + 5 +} + +fn default_timeout_ms() -> u64 { + 2000 +} + +fn default_data_dir() -> String { + "./frx-data".to_string() +} + +impl Config { + pub fn new(name: &str, listen: &str, relay: &str, data_dir: &str) -> Self { + Self { + node: NodeSection { + name: name.to_string(), + listen: listen.to_string(), + relays: vec![relay.to_string()], + trusted_keys: Vec::new(), + responder: true, + }, + query: QuerySection::default(), + index: IndexSection { + data_dir: data_dir.to_string(), + }, + } + } + + pub fn load(path: &Path) -> Result { + let raw = fs::read_to_string(path) + .with_context(|| format!("reading config {}", path.display()))?; + toml::from_str(&raw).with_context(|| format!("parsing config {}", path.display())) + } + + pub fn save(&self, path: &Path) -> Result<()> { + if let Some(parent) = path.parent() { + if !parent.as_os_str().is_empty() { + fs::create_dir_all(parent)?; + } + } + fs::write(path, toml::to_string_pretty(self)?)?; + Ok(()) + } + + pub fn data_dir(&self) -> PathBuf { + PathBuf::from(&self.index.data_dir) + } + + pub fn index_dir(&self) -> PathBuf { + self.data_dir().join("index") + } + + pub fn key_path(&self) -> PathBuf { + self.data_dir().join("key.hex") + } + + pub fn collections_path(&self) -> PathBuf { + self.data_dir().join("collections.toml") + } + + pub fn load_key(&self) -> Result { + let raw = fs::read_to_string(self.key_path()) + .with_context(|| format!("reading key {}", self.key_path().display()))?; + Keypair::from_hex(&raw) + } + + pub fn save_key(&self, key: &Keypair) -> Result<()> { + fs::create_dir_all(self.data_dir())?; + let path = self.key_path(); + fs::write(&path, key.to_hex())?; + set_private_permissions(&path)?; + Ok(()) + } +} + +fn set_private_permissions(path: &Path) -> Result<()> { + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(path, fs::Permissions::from_mode(0o600))?; + } + #[cfg(not(unix))] + { + let _ = path; + } + Ok(()) +} diff --git a/src/crypto.rs b/src/crypto.rs new file mode 100644 index 0000000..4e36789 --- /dev/null +++ b/src/crypto.rs @@ -0,0 +1,166 @@ +use crate::PROTOCOL; +use crate::message::Envelope; +use anyhow::{Context, Result, anyhow}; +use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey}; +use rand::RngCore; +use serde_json::Value; + +pub struct Keypair { + signing: SigningKey, +} + +impl Keypair { + pub fn generate() -> Self { + let mut bytes = [0u8; 32]; + rand::thread_rng().fill_bytes(&mut bytes); + Self { + signing: SigningKey::from_bytes(&bytes), + } + } + + pub fn from_hex(s: &str) -> Result { + let raw = hex::decode(s.trim()).context("key is not hex")?; + let bytes: [u8; 32] = raw + .as_slice() + .try_into() + .map_err(|_| anyhow!("key must be 32 bytes"))?; + Ok(Self { + signing: SigningKey::from_bytes(&bytes), + }) + } + + pub fn to_hex(&self) -> String { + hex::encode(self.signing.to_bytes()) + } + + pub fn public_hex(&self) -> String { + hex::encode(self.signing.verifying_key().to_bytes()) + } + + pub fn verifying_key(&self) -> VerifyingKey { + self.signing.verifying_key() + } + + pub fn sign(&self, bytes: &[u8]) -> String { + hex::encode(self.signing.sign(bytes).to_bytes()) + } +} + +pub fn random_nonce() -> String { + let mut bytes = [0u8; 16]; + rand::thread_rng().fill_bytes(&mut bytes); + hex::encode(bytes) +} + +pub fn random_id() -> String { + let mut bytes = [0u8; 8]; + rand::thread_rng().fill_bytes(&mut bytes); + hex::encode(bytes) +} + +pub fn canonical_json(value: &Value) -> String { + let mut out = String::new(); + write_canonical(value, &mut out); + out +} + +fn write_canonical(value: &Value, out: &mut String) { + match value { + Value::Null => out.push_str("null"), + Value::Bool(b) => out.push_str(if *b { "true" } else { "false" }), + Value::Number(n) => out.push_str(&n.to_string()), + Value::String(s) => out.push_str(&serde_json::to_string(s).expect("string serializes")), + Value::Array(items) => { + out.push('['); + for (i, item) in items.iter().enumerate() { + if i > 0 { + out.push(','); + } + write_canonical(item, out); + } + out.push(']'); + } + Value::Object(map) => { + let mut keys: Vec<&String> = map.keys().collect(); + keys.sort(); + out.push('{'); + for (i, key) in keys.iter().enumerate() { + if i > 0 { + out.push(','); + } + out.push_str(&serde_json::to_string(key).expect("key serializes")); + out.push(':'); + write_canonical(&map[*key], out); + } + out.push('}'); + } + } +} + +pub fn signing_bytes(envelope: &Envelope) -> Vec { + format!( + "{}\n{}\n{}\n{}\n{}\n{}", + PROTOCOL, + envelope.msg_type, + envelope.from, + envelope.ts, + envelope.nonce, + canonical_json(&envelope.body) + ) + .into_bytes() +} + +pub fn verify_envelope(envelope: &Envelope) -> Result<()> { + let key_bytes = hex::decode(&envelope.from).context("from is not hex")?; + let key_bytes: [u8; 32] = key_bytes + .as_slice() + .try_into() + .map_err(|_| anyhow!("from must be a 32-byte ed25519 public key"))?; + let verifying_key = + VerifyingKey::from_bytes(&key_bytes).map_err(|e| anyhow!("bad public key: {e}"))?; + let sig_bytes = hex::decode(&envelope.sig).context("sig is not hex")?; + let sig_bytes: [u8; 64] = sig_bytes + .as_slice() + .try_into() + .map_err(|_| anyhow!("sig must be 64 bytes"))?; + let signature = Signature::from_bytes(&sig_bytes); + verifying_key + .verify_strict(&signing_bytes(envelope), &signature) + .map_err(|_| anyhow!("signature verification failed")) +} + +pub fn now_ts() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn canonical_json_sorts_keys_recursively() { + let a = json!({"b": 1, "a": {"d": [3, 2], "c": "x"}}); + let b = json!({"a": {"c": "x", "d": [3, 2]}, "b": 1}); + assert_eq!(canonical_json(&a), canonical_json(&b)); + assert_eq!(canonical_json(&a), r#"{"a":{"c":"x","d":[3,2]},"b":1}"#); + } + + #[test] + fn envelope_sign_verify_roundtrip() { + let key = Keypair::generate(); + let env = Envelope::new(&key, crate::message::TYPE_QUERY, json!({"text": "hello"})); + verify_envelope(&env).unwrap(); + } + + #[test] + fn tampered_body_fails_verification() { + let key = Keypair::generate(); + let mut env = Envelope::new(&key, crate::message::TYPE_QUERY, json!({"text": "hello"})); + env.body = json!({"text": "hello", "extra": true}); + assert!(verify_envelope(&env).is_err()); + } +} diff --git a/src/extract.rs b/src/extract.rs new file mode 100644 index 0000000..cdfb3f3 --- /dev/null +++ b/src/extract.rs @@ -0,0 +1,242 @@ +use std::fs; +use std::path::Path; +use std::time::SystemTime; + +use chrono::{DateTime, SecondsFormat, Utc}; + +pub const MAX_BODY_BYTES: usize = 1_000_000; + +pub struct Extracted { + pub title: String, + pub body: String, + pub published: String, +} + +pub fn supported(path: &Path) -> bool { + match path.extension().and_then(|e| e.to_str()) { + Some(ext) => matches!( + ext.to_ascii_lowercase().as_str(), + "txt" | "md" | "markdown" | "html" | "htm" | "rst" | "log" + ), + None => false, + } +} + +pub fn extract_file(path: &Path) -> Option { + if !supported(path) { + return None; + } + let raw = fs::read_to_string(path).ok()?; + let ext = path + .extension() + .and_then(|e| e.to_str()) + .unwrap_or("") + .to_ascii_lowercase(); + let body = if ext == "html" || ext == "htm" { + strip_html(&raw) + } else { + collapse_blank_lines(&raw) + }; + let body = truncate_chars(&body, MAX_BODY_BYTES); + let title = if ext == "html" || ext == "htm" { + html_title(&raw).unwrap_or_else(|| fallback_title(path, &body)) + } else { + markdown_title(&raw).unwrap_or_else(|| fallback_title(path, &body)) + }; + let published = fs::metadata(path) + .and_then(|m| m.modified()) + .ok() + .map(format_time) + .unwrap_or_default(); + Some(Extracted { + title: truncate_chars(title.trim(), 200), + body, + published, + }) +} + +pub fn summary_of(body: &str) -> String { + truncate_chars(body.trim(), 300).replace('\n', " ") +} + +fn fallback_title(path: &Path, body: &str) -> String { + if let Some(line) = body.lines().map(str::trim).find(|l| !l.is_empty()) { + if line.chars().count() > 3 { + return truncate_chars(line, 120); + } + } + path.file_stem() + .and_then(|s| s.to_str()) + .unwrap_or("untitled") + .to_string() +} + +fn markdown_title(raw: &str) -> Option { + raw.lines() + .map(str::trim) + .find(|l| l.starts_with("# ")) + .map(|l| l.trim_start_matches('#').trim().to_string()) +} + +fn html_title(raw: &str) -> Option { + let lower = raw.to_ascii_lowercase(); + let start = lower.find("')? + start + 1; + let end = lower[open_end..].find("")? + open_end; + let title = decode_entities(raw[open_end..end].trim()); + if title.is_empty() { None } else { Some(title) } +} + +fn strip_html(raw: &str) -> String { + let mut out = String::with_capacity(raw.len() / 2); + let mut chars = raw.chars().peekable(); + let mut skipping: Option = None; + while let Some(c) = chars.next() { + if c == '<' { + let mut tag = String::new(); + for t in chars.by_ref() { + if t == '>' { + break; + } + tag.push(t); + } + let name = tag + .trim_start_matches('/') + .trim() + .split(|c: char| c.is_whitespace() || c == '/') + .next() + .unwrap_or("") + .to_ascii_lowercase(); + if skipping.is_none() && (name == "script" || name == "style") { + skipping = Some(name.clone()); + } else if skipping.as_deref() == Some(name.as_str()) + && tag.trim_start().starts_with('/') + { + skipping = None; + } + out.push(' '); + continue; + } + if skipping.is_none() { + out.push(c); + } + } + collapse_whitespace(&decode_entities(&out)) +} + +fn collapse_whitespace(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + let mut last_space = false; + for c in s.chars() { + if c.is_whitespace() { + if !last_space { + out.push(' '); + } + last_space = true; + } else { + out.push(c); + last_space = false; + } + } + out.trim().to_string() +} + +fn collapse_blank_lines(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + let mut blank_run = 0; + for line in s.lines() { + if line.trim().is_empty() { + blank_run += 1; + if blank_run > 1 { + continue; + } + } else { + blank_run = 0; + } + out.push_str(line); + out.push('\n'); + } + out.trim().to_string() +} + +fn decode_entities(s: &str) -> String { + s.replace(" ", " ") + .replace("&", "&") + .replace("<", "<") + .replace(">", ">") + .replace(""", "\"") + .replace("'", "'") + .replace("'", "'") +} + +fn truncate_chars(s: &str, max: usize) -> String { + if s.len() <= max { + return s.to_string(); + } + let end = s + .char_indices() + .take_while(|(i, _)| *i < max) + .last() + .map(|(i, c)| i + c.len_utf8()) + .unwrap_or(0); + s[..end].to_string() +} + +fn format_time(t: SystemTime) -> String { + let dt: DateTime = t.into(); + dt.to_rfc3339_opts(SecondsFormat::Secs, true) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Write; + + #[test] + fn html_is_stripped_and_title_extracted() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("page.html"); + let mut f = fs::File::create(&path).unwrap(); + write!( + f, + "Rust & Ownership

Hello world

" + ) + .unwrap(); + let extracted = extract_file(&path).unwrap(); + assert_eq!(extracted.title, "Rust & Ownership"); + assert!(extracted.body.contains("Hello world")); + assert!(!extracted.body.contains("alert")); + assert!(!extracted.body.contains("color:red")); + } + + #[test] + fn unsupported_extensions_are_ignored() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("archive.bin"); + fs::write(&path, "binary-ish").unwrap(); + assert!(!supported(&path)); + assert!(extract_file(&path).is_none()); + } + + #[test] + fn oversized_bodies_are_truncated_on_char_boundaries() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("big.txt"); + let mut body = "é".repeat(MAX_BODY_BYTES / 2 + 50); + body.push_str(" tail"); + fs::write(&path, &body).unwrap(); + let extracted = extract_file(&path).unwrap(); + assert!(extracted.body.len() <= MAX_BODY_BYTES); + assert!(extracted.body.chars().all(|c| c == 'é')); + } + + #[test] + fn markdown_heading_becomes_title() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("note.md"); + fs::write(&path, "# Borrowing\n\nRules of borrowing.\n").unwrap(); + let extracted = extract_file(&path).unwrap(); + assert_eq!(extracted.title, "Borrowing"); + assert!(extracted.body.contains("Rules of borrowing.")); + } +} diff --git a/src/index.rs b/src/index.rs new file mode 100644 index 0000000..09b1600 --- /dev/null +++ b/src/index.rs @@ -0,0 +1,359 @@ +use std::fs; +use std::path::{Path, PathBuf}; +use std::sync::Mutex; + +use anyhow::{Context, Result}; +use serde::{Deserialize, Serialize}; +use tantivy::collector::{Count, TopDocs}; +use tantivy::directory::MmapDirectory; +use tantivy::query::{AllQuery, BooleanQuery, Occur, Query, QueryParser, TermQuery}; +use tantivy::schema::{Field, IndexRecordOption, STORED, STRING, Schema, TEXT, Value}; +use tantivy::{Index, IndexReader, IndexWriter, TantivyDocument, Term, doc}; + +use crate::extract::{extract_file, summary_of, supported}; +use crate::message::{EXPOSURE_FULL, ResponseItem}; + +const WRITER_BUDGET: usize = 50_000_000; + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Collection { + pub name: String, + pub path: String, + pub shared: bool, + pub exposure: String, +} + +#[derive(Debug, Default, Serialize, Deserialize)] +struct CollectionsFile { + #[serde(default, rename = "collection")] + collections: Vec, +} + +pub fn load_collections(manifest: &Path) -> Result> { + if !manifest.exists() { + return Ok(Vec::new()); + } + let raw = fs::read_to_string(manifest).context("reading collections manifest")?; + let parsed: CollectionsFile = toml::from_str(&raw).context("parsing collections manifest")?; + Ok(parsed.collections) +} + +pub fn save_collections(manifest: &Path, collections: &[Collection]) -> Result<()> { + if let Some(parent) = manifest.parent() { + fs::create_dir_all(parent)?; + } + let file = CollectionsFile { + collections: collections.to_vec(), + }; + fs::write(manifest, toml::to_string_pretty(&file)?)?; + Ok(()) +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct SearchHit { + pub url: String, + pub title: String, + pub summary: String, + pub published: String, + pub exposure: String, + pub collection: String, + pub shared: bool, + pub content: Option, +} + +pub fn response_items(hits: &[SearchHit]) -> Vec { + hits.iter() + .map(|hit| ResponseItem { + url: hit.url.clone(), + title: hit.title.clone(), + summary: hit.summary.clone(), + published: hit.published.clone(), + exposure: hit.exposure.clone(), + content: if hit.exposure == EXPOSURE_FULL { + hit.content.clone() + } else { + None + }, + }) + .collect() +} + +struct Fields { + url: Field, + title: Field, + body: Field, + summary: Field, + published: Field, + exposure: Field, + collection: Field, + shared: Field, +} + +pub struct LocalIndex { + index: Index, + reader: IndexReader, + writer: Mutex, + fields: Fields, +} + +impl LocalIndex { + pub fn open(dir: &Path) -> Result { + fs::create_dir_all(dir)?; + let schema = build_schema(); + let index = Index::open_or_create(MmapDirectory::open(dir)?, schema) + .context("opening tantivy index")?; + let schema = index.schema(); + let fields = Fields { + url: schema.get_field("url")?, + title: schema.get_field("title")?, + body: schema.get_field("body")?, + summary: schema.get_field("summary")?, + published: schema.get_field("published")?, + exposure: schema.get_field("exposure")?, + collection: schema.get_field("collection")?, + shared: schema.get_field("shared")?, + }; + let reader = index.reader()?; + let writer = index.writer::(WRITER_BUDGET)?; + Ok(Self { + index, + reader, + writer: Mutex::new(writer), + fields, + }) + } + + pub fn add_collection(&self, collection: &Collection) -> Result { + let root = PathBuf::from(&collection.path); + { + let writer = self.writer.lock().expect("writer lock"); + writer.delete_term(Term::from_field_text( + self.fields.collection, + &collection.name, + )); + } + let mut files = Vec::new(); + walk(&root, &mut files); + let mut added = 0; + for file in files { + if self.add_file(collection, &file)? { + added += 1; + } + } + self.commit()?; + Ok(added) + } + + pub fn add_file(&self, collection: &Collection, path: &Path) -> Result { + let Some(extracted) = extract_file(path) else { + return Ok(false); + }; + let canonical = path.canonicalize().unwrap_or_else(|_| path.to_path_buf()); + let url = format!("file://{}", canonical.display()); + let summary = summary_of(&extracted.body); + let shared = if collection.shared { "true" } else { "false" }; + let writer = self.writer.lock().expect("writer lock"); + writer.delete_term(Term::from_field_text(self.fields.url, &url)); + writer.add_document(doc!( + self.fields.url => url.as_str(), + self.fields.title => extracted.title.as_str(), + self.fields.body => extracted.body.as_str(), + self.fields.summary => summary.as_str(), + self.fields.published => extracted.published.as_str(), + self.fields.exposure => collection.exposure.as_str(), + self.fields.collection => collection.name.as_str(), + self.fields.shared => shared, + ))?; + Ok(true) + } + + pub fn commit(&self) -> Result<()> { + self.writer.lock().expect("writer lock").commit()?; + self.reader.reload()?; + Ok(()) + } + + pub fn search( + &self, + text: &str, + limit: usize, + only_shared: bool, + ) -> Result<(Vec, u64)> { + let limit = limit.max(1); + self.reader.reload()?; + let searcher = self.reader.searcher(); + let user_query: Box = if text.trim().is_empty() { + Box::new(AllQuery) + } else { + let parser = + QueryParser::for_index(&self.index, vec![self.fields.title, self.fields.body]); + let (query, _errors) = parser.parse_query_lenient(text); + query + }; + let query: Box = if only_shared { + let shared_term = TermQuery::new( + Term::from_field_text(self.fields.shared, "true"), + IndexRecordOption::Basic, + ); + Box::new(BooleanQuery::new(vec![ + (Occur::Must, user_query), + (Occur::Must, Box::new(shared_term)), + ])) + } else { + user_query + }; + let total = searcher.search(&*query, &Count)? as u64; + let top = searcher.search(&*query, &TopDocs::with_limit(limit).order_by_score())?; + let mut hits = Vec::with_capacity(top.len()); + for (_score, address) in top { + let document: TantivyDocument = searcher.doc(address)?; + hits.push(SearchHit { + url: text_value(&document, self.fields.url), + title: text_value(&document, self.fields.title), + summary: text_value(&document, self.fields.summary), + published: text_value(&document, self.fields.published), + exposure: text_value(&document, self.fields.exposure), + collection: text_value(&document, self.fields.collection), + shared: text_value(&document, self.fields.shared) == "true", + content: Some(text_value(&document, self.fields.body)), + }); + } + Ok((hits, total)) + } + + pub fn doc_count(&self) -> u64 { + self.reader.searcher().num_docs() + } +} + +fn text_value(document: &TantivyDocument, field: Field) -> String { + document + .get_first(field) + .and_then(|value| value.as_str()) + .unwrap_or("") + .to_string() +} + +fn build_schema() -> Schema { + let mut builder = Schema::builder(); + builder.add_text_field("url", STRING | STORED); + builder.add_text_field("title", TEXT | STORED); + builder.add_text_field("body", TEXT | STORED); + builder.add_text_field("summary", STORED); + builder.add_text_field("published", STORED); + builder.add_text_field("exposure", STRING | STORED); + builder.add_text_field("collection", STRING | STORED); + builder.add_text_field("shared", STRING | STORED); + builder.build() +} + +fn walk(root: &Path, out: &mut Vec) { + let Ok(entries) = fs::read_dir(root) else { + return; + }; + for entry in entries.flatten() { + let Ok(file_type) = entry.file_type() else { + continue; + }; + if file_type.is_symlink() { + continue; + } + let name = entry.file_name(); + let name = name.to_string_lossy(); + if file_type.is_dir() { + if name.starts_with('.') { + continue; + } + walk(&entry.path(), out); + } else if file_type.is_file() && supported(&entry.path()) { + out.push(entry.path()); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::message::EXPOSURE_METADATA; + + fn collection(name: &str, root: &Path, shared: bool, exposure: &str) -> Collection { + Collection { + name: name.to_string(), + path: root.display().to_string(), + shared, + exposure: exposure.to_string(), + } + } + + #[test] + fn shared_filter_and_exposure_are_enforced() { + let temp = tempfile::tempdir().unwrap(); + let shared_dir = temp.path().join("shared"); + let private_dir = temp.path().join("private"); + fs::create_dir_all(&shared_dir).unwrap(); + fs::create_dir_all(&private_dir).unwrap(); + fs::write(shared_dir.join("a.txt"), "rust ownership rules").unwrap(); + fs::write(private_dir.join("b.txt"), "rust secret notes").unwrap(); + + let index = LocalIndex::open(&temp.path().join("index")).unwrap(); + index + .add_collection(&collection("shared", &shared_dir, true, EXPOSURE_FULL)) + .unwrap(); + index + .add_collection(&collection( + "private", + &private_dir, + false, + EXPOSURE_METADATA, + )) + .unwrap(); + + let (hits, total) = index.search("rust", 10, false).unwrap(); + assert_eq!(total, 2); + assert_eq!(hits.len(), 2); + + let (shared_hits, shared_total) = index.search("rust", 10, true).unwrap(); + assert_eq!(shared_total, 1); + assert_eq!(shared_hits.len(), 1); + assert!(shared_hits[0].url.contains("shared")); + + let items = response_items(&shared_hits); + assert!(items[0].content.is_some()); + } + + #[test] + fn reindex_removes_deleted_files() { + let temp = tempfile::tempdir().unwrap(); + let dir = temp.path().join("corpus"); + fs::create_dir_all(&dir).unwrap(); + fs::write(dir.join("a.txt"), "rust one").unwrap(); + fs::write(dir.join("b.txt"), "rust two").unwrap(); + let index = LocalIndex::open(&temp.path().join("index")).unwrap(); + index + .add_collection(&collection("test", &dir, true, EXPOSURE_FULL)) + .unwrap(); + assert_eq!(index.search("rust", 10, false).unwrap().1, 2); + + fs::remove_file(dir.join("b.txt")).unwrap(); + index + .add_collection(&collection("test", &dir, true, EXPOSURE_FULL)) + .unwrap(); + assert_eq!(index.search("rust", 10, false).unwrap().1, 1); + } + + #[test] + fn metadata_exposure_withholds_content() { + let temp = tempfile::tempdir().unwrap(); + let dir = temp.path().join("corpus"); + fs::create_dir_all(&dir).unwrap(); + fs::write(dir.join("a.txt"), "metadata only").unwrap(); + let index = LocalIndex::open(&temp.path().join("index")).unwrap(); + index + .add_collection(&collection("test", &dir, true, EXPOSURE_METADATA)) + .unwrap(); + let (hits, _) = index.search("metadata", 10, true).unwrap(); + let items = response_items(&hits); + assert!(items[0].content.is_none()); + assert_eq!(items[0].exposure, EXPOSURE_METADATA); + } +} diff --git a/src/lib.rs b/src/lib.rs new file mode 100644 index 0000000..2916636 --- /dev/null +++ b/src/lib.rs @@ -0,0 +1,12 @@ +pub mod commands; +pub mod config; +pub mod crypto; +pub mod extract; +pub mod index; +pub mod message; +pub mod node; +pub mod relay; +pub mod render; + +pub const VERSION: &str = env!("CARGO_PKG_VERSION"); +pub const PROTOCOL: &str = "FRX/0.3"; diff --git a/src/main.rs b/src/main.rs new file mode 100644 index 0000000..b20b4d5 --- /dev/null +++ b/src/main.rs @@ -0,0 +1,141 @@ +use std::path::PathBuf; + +use anyhow::{Result, bail}; +use clap::{Parser, Subcommand, ValueEnum}; +use frxd::config::Config; +use frxd::crypto::Keypair; +use frxd::message::{EXPOSURE_FULL, EXPOSURE_METADATA}; +use frxd::{commands, node, relay}; + +#[derive(Parser)] +#[command( + name = "frxd", + version, + about = "FRX member node — querier, responder, and local index (Draft 0.3)" +)] +struct Cli { + #[arg(long, global = true, default_value = "frxd.toml")] + config: PathBuf, + #[command(subcommand)] + command: Command, +} + +#[derive(Subcommand)] +enum Command { + Init { + #[arg(long, default_value = "member")] + name: String, + #[arg(long, default_value = "127.0.0.1:7701")] + listen: String, + #[arg(long, default_value = "http://127.0.0.1:7700")] + relay: String, + #[arg(long, default_value = "./frx-data")] + data_dir: String, + #[arg(long)] + force: bool, + }, + Add { + path: PathBuf, + #[arg(long)] + name: Option, + #[arg(long)] + shared: bool, + #[arg(long, value_enum, default_value_t = Exposure::Full)] + exposure: Exposure, + }, + Reindex, + Search { + text: String, + #[arg(long, default_value_t = 10)] + limit: usize, + }, + Query { + text: String, + #[arg(long)] + max_results: Option, + #[arg(long)] + timeout_ms: Option, + #[arg(long)] + local_only: bool, + }, + Serve, + Relay { + #[arg(long, default_value = "127.0.0.1:7700")] + listen: String, + #[arg(long, default_value_t = relay::DEFAULT_CAPACITY)] + capacity: usize, + }, + Status, +} + +#[derive(Clone, Copy, ValueEnum)] +enum Exposure { + Metadata, + Full, +} + +#[tokio::main] +async fn main() -> Result<()> { + let cli = Cli::parse(); + match cli.command { + Command::Init { + name, + listen, + relay, + data_dir, + force, + } => { + if cli.config.exists() && !force { + bail!( + "config {} already exists (use --force to overwrite)", + cli.config.display() + ); + } + let config = Config::new(&name, &listen, &relay, &data_dir); + let key = Keypair::generate(); + config.save_key(&key)?; + config.save(&cli.config)?; + println!("wrote config {}", cli.config.display()); + println!("wrote key {}", config.key_path().display()); + println!("pubkey {}", key.public_hex()); + println!("data dir {}", config.data_dir().display()); + } + Command::Add { + path, + name, + shared, + exposure, + } => { + let exposure = match exposure { + Exposure::Metadata => EXPOSURE_METADATA, + Exposure::Full => EXPOSURE_FULL, + }; + commands::add(&cli.config, &path, name, shared, exposure)?; + } + Command::Reindex => commands::reindex(&cli.config)?, + Command::Search { text, limit } => commands::search(&cli.config, &text, limit).await?, + Command::Query { + text, + max_results, + timeout_ms, + local_only, + } => { + commands::query(&cli.config, &text, max_results, timeout_ms, !local_only).await?; + } + Command::Serve => { + let config = Config::load(&cli.config)?; + let handle = node::Node::start(config).await?; + println!("frxd listening on http://{}", handle.addr); + println!("pubkey {}", handle.pubkey); + println!("control API POST http://{}/v1/local/query", handle.addr); + tokio::signal::ctrl_c().await?; + } + Command::Relay { listen, capacity } => { + let (listener, addr) = relay::bind(&listen).await?; + println!("relay listening on http://{addr} (capacity {capacity})"); + relay::run(listener, capacity).await?; + } + Command::Status => commands::status(&cli.config).await?, + } + Ok(()) +} diff --git a/src/message.rs b/src/message.rs new file mode 100644 index 0000000..30cb9e7 --- /dev/null +++ b/src/message.rs @@ -0,0 +1,184 @@ +use crate::crypto::{Keypair, now_ts, random_nonce, signing_bytes, verify_envelope}; +use anyhow::{Context, Result, anyhow}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use serde_json::Value; + +pub const TYPE_QUERY: &str = "query"; +pub const TYPE_RESPONSE: &str = "response"; +pub const TYPE_AGGREGATE: &str = "aggregate"; + +pub const EXPOSURE_METADATA: &str = "metadata"; +pub const EXPOSURE_FULL: &str = "full"; + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Envelope { + #[serde(rename = "type")] + pub msg_type: String, + pub from: String, + pub ts: u64, + pub nonce: String, + pub body: Value, + pub sig: String, +} + +impl Envelope { + pub fn new(key: &Keypair, msg_type: &str, body: Value) -> Self { + let mut envelope = Self { + msg_type: msg_type.to_string(), + from: key.public_hex(), + ts: now_ts(), + nonce: random_nonce(), + body, + sig: String::new(), + }; + envelope.sig = key.sign(&signing_bytes(&envelope)); + envelope + } + + pub fn verify(&self) -> Result<()> { + verify_envelope(self) + } + + pub fn parse_body(&self) -> Result { + serde_json::from_value(self.body.clone()).context("malformed body") + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Budget { + pub max_results: usize, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct QueryBody { + pub qid: String, + pub text: String, + #[serde(default)] + pub entities: Vec, + pub budget: Budget, +} + +impl QueryBody { + pub fn new(text: &str, max_results: usize) -> Self { + Self { + qid: crate::crypto::random_id(), + text: text.to_string(), + entities: Vec::new(), + budget: Budget { + max_results: max_results.clamp(1, 1000), + }, + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ResponseItem { + pub url: String, + pub title: String, + pub summary: String, + pub published: String, + pub exposure: String, + pub content: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ResponseBody { + pub qid: String, + pub results: Vec, + pub truncated: bool, + pub more_available: u64, + pub cursor: Option, +} + +pub fn build_response( + qid: &str, + hits: Vec, + total: u64, + max_results: usize, +) -> ResponseBody { + let max_results = max_results.max(1); + let mut results = hits; + results.truncate(max_results); + let more_available = total.saturating_sub(results.len() as u64); + ResponseBody { + qid: qid.to_string(), + results, + truncated: more_available > 0, + more_available, + cursor: None, + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AggregateBody { + pub period: String, + pub sent: u64, + pub passed: u64, + pub cited: u64, +} + +pub fn require_type(envelope: &Envelope, expected: &str) -> Result<()> { + if envelope.msg_type != expected { + return Err(anyhow!( + "expected message type {expected}, got {}", + envelope.msg_type + )); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn item(n: usize) -> ResponseItem { + ResponseItem { + url: format!("file:///doc{n}.txt"), + title: format!("doc {n}"), + summary: String::new(), + published: String::new(), + exposure: EXPOSURE_METADATA.to_string(), + content: None, + } + } + + #[test] + fn max_results_is_respected() { + let response = build_response("q1", vec![item(1), item(2), item(3)], 10, 2); + assert_eq!(response.results.len(), 2); + } + + #[test] + fn truncation_is_honest() { + let response = build_response("q1", vec![item(1), item(2), item(3)], 10, 2); + assert!(response.truncated); + assert_eq!(response.more_available, 8); + + let response = build_response("q2", vec![item(1), item(2)], 2, 5); + assert!(!response.truncated); + assert_eq!(response.more_available, 0); + } + + #[test] + fn response_carries_no_scores() { + let response = build_response("q1", vec![item(1)], 1, 5); + let json = serde_json::to_string(&response).unwrap(); + assert!(!json.contains("score")); + assert!(!json.contains("relevance")); + } + + #[test] + fn zero_budget_is_clamped() { + let query = QueryBody::new("anything", 0); + assert_eq!(query.budget.max_results, 1); + } + + #[test] + fn envelope_rejects_wrong_type() { + let key = Keypair::generate(); + let envelope = Envelope::new(&key, TYPE_RESPONSE, json!({})); + assert!(require_type(&envelope, TYPE_QUERY).is_err()); + } +} diff --git a/src/node.rs b/src/node.rs new file mode 100644 index 0000000..1470f19 --- /dev/null +++ b/src/node.rs @@ -0,0 +1,434 @@ +use std::collections::{HashMap, HashSet}; +use std::net::SocketAddr; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +use anyhow::{Context, Result, anyhow}; +use axum::extract::State; +use axum::http::StatusCode; +use axum::response::{IntoResponse, Response}; +use axum::routing::{get, post}; +use axum::{Json, Router}; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; +use tokio::net::TcpListener; +use tokio::task::JoinHandle; + +use crate::config::Config; +use crate::crypto::Keypair; +use crate::index::{LocalIndex, SearchHit, response_items}; +use crate::message::{ + Envelope, QueryBody, ResponseBody, ResponseItem, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE, + build_response, +}; + +pub struct Node { + pub config: Config, + pub key: Keypair, + index: LocalIndex, + seen: Mutex>, + pending: Mutex>>, + client: reqwest::Client, + sent: AtomicU64, + received: AtomicU64, +} + +pub struct NodeHandle { + pub addr: SocketAddr, + pub pubkey: String, + pub node: Arc, + tasks: Vec>, +} + +impl Drop for NodeHandle { + fn drop(&mut self) { + for task in &self.tasks { + task.abort(); + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RemoteResponse { + pub member: String, + pub results: Vec, + pub truncated: bool, + pub more_available: u64, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct MergedItem { + pub provenance: String, + #[serde(flatten)] + pub item: ResponseItem, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct LocalQueryOutcome { + pub qid: String, + pub text: String, + pub local: LocalPart, + pub responses: Vec, + pub merged: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct LocalPart { + pub results: Vec, + pub total: u64, +} + +impl Node { + pub fn open(config: Config) -> Result> { + let key = config.load_key()?; + let index = LocalIndex::open(&config.index_dir())?; + let client = reqwest::Client::builder() + .timeout(Duration::from_secs(15)) + .build() + .context("building http client")?; + Ok(Arc::new(Self { + config, + key, + index, + seen: Mutex::new(HashSet::new()), + pending: Mutex::new(HashMap::new()), + client, + sent: AtomicU64::new(0), + received: AtomicU64::new(0), + })) + } + + pub fn doc_count(&self) -> u64 { + self.index.doc_count() + } + + pub fn sent(&self) -> u64 { + self.sent.load(Ordering::SeqCst) + } + + pub fn received(&self) -> u64 { + self.received.load(Ordering::SeqCst) + } + + pub fn local_search(&self, text: &str, limit: usize) -> Result<(Vec, u64)> { + self.index.search(text, limit, false) + } + + pub async fn start(config: Config) -> Result { + let node = Node::open(config)?; + let listener = TcpListener::bind(&node.config.node.listen) + .await + .with_context(|| format!("binding {}", node.config.node.listen))?; + let addr = listener.local_addr()?; + let mut tasks = Vec::new(); + for relay in node.config.node.relays.clone() { + tasks.push(tokio::spawn(poll_relay(node.clone(), relay))); + } + let app = router(node.clone()); + tasks.push(tokio::spawn(async move { + if let Err(error) = axum::serve(listener, app).await { + eprintln!("control server stopped: {error}"); + } + })); + Ok(NodeHandle { + addr, + pubkey: node.key.public_hex(), + node, + tasks, + }) + } + + pub async fn local_query( + &self, + text: &str, + max_results: Option, + timeout_ms: Option, + network: bool, + ) -> Result { + let max = max_results.unwrap_or(self.config.query.max_results).max(1); + let query = QueryBody::new(text, max); + let qid = query.qid.clone(); + let (local_hits, local_total) = self.index.search(text, max, false)?; + let local_items = response_items(&local_hits); + let mut responses = Vec::new(); + if network { + self.pending + .lock() + .expect("pending lock") + .insert(qid.clone(), Vec::new()); + self.sent.fetch_add(1, Ordering::SeqCst); + let envelope = Envelope::new(&self.key, TYPE_QUERY, serde_json::to_value(&query)?); + self.publish(&envelope).await; + let timeout = Duration::from_millis(timeout_ms.unwrap_or(self.config.query.timeout_ms)); + let deadline = Instant::now() + timeout; + while Instant::now() < deadline { + tokio::time::sleep(Duration::from_millis(25)).await; + } + let collected = self + .pending + .lock() + .expect("pending lock") + .remove(&qid) + .unwrap_or_default(); + self.received + .fetch_add(collected.len() as u64, Ordering::SeqCst); + for (member, body) in collected { + responses.push(RemoteResponse { + member, + results: body.results, + truncated: body.truncated, + more_available: body.more_available, + }); + } + } + let mut merged = Vec::new(); + let mut seen_urls = HashSet::new(); + for item in &local_items { + if seen_urls.insert(item.url.clone()) { + merged.push(MergedItem { + provenance: "local".to_string(), + item: item.clone(), + }); + } + } + for response in &responses { + for item in &response.results { + if seen_urls.insert(item.url.clone()) { + merged.push(MergedItem { + provenance: response.member.clone(), + item: item.clone(), + }); + } + } + } + Ok(LocalQueryOutcome { + qid, + text: text.to_string(), + local: LocalPart { + results: local_items, + total: local_total, + }, + responses, + merged, + }) + } + + async fn publish(&self, envelope: &Envelope) -> usize { + let mut delivered = 0; + for relay in &self.config.node.relays { + let url = format!("{}/v1/publish", relay.trim_end_matches('/')); + match self.client.post(&url).json(envelope).send().await { + Ok(response) if response.status().is_success() => delivered += 1, + Ok(response) => eprintln!("relay {relay} rejected query: {}", response.status()), + Err(error) => eprintln!("relay {relay} unreachable: {error}"), + } + } + delivered + } + + async fn dispatch(self: &Arc, envelope: Envelope, relay: &str) { + if envelope.verify().is_err() { + return; + } + if !self.config.node.trusted_keys.is_empty() + && !self.config.node.trusted_keys.contains(&envelope.from) + { + return; + } + match envelope.msg_type.as_str() { + TYPE_QUERY => { + if envelope.from == self.key.public_hex() || !self.config.node.responder { + return; + } + let Ok(query) = envelope.parse_body::() else { + return; + }; + if query.text.trim().is_empty() || query.qid.is_empty() { + return; + } + { + let mut seen = self.seen.lock().expect("seen lock"); + if !seen.insert(query.qid.clone()) { + return; + } + if seen.len() > 10_000 { + seen.clear(); + } + } + let node = self.clone(); + let relay = relay.to_string(); + let querier = envelope.from.clone(); + tokio::spawn(async move { + if let Err(error) = node.respond(&query, &querier, &relay).await { + eprintln!("responder failed: {error}"); + } + }); + } + TYPE_RESPONSE => { + let Ok(body) = envelope.parse_body::() else { + return; + }; + let mut pending = self.pending.lock().expect("pending lock"); + if let Some(list) = pending.get_mut(&body.qid) { + list.push((envelope.from.clone(), body)); + } + } + TYPE_AGGREGATE => {} + _ => {} + } + } + + async fn respond(&self, query: &QueryBody, querier: &str, relay: &str) -> Result<()> { + let max = query.budget.max_results.clamp(1, 1000); + let (hits, total) = self.index.search(&query.text, max, true)?; + if hits.is_empty() { + return Ok(()); + } + let body = build_response(&query.qid, response_items(&hits), total, max); + let envelope = Envelope::new(&self.key, TYPE_RESPONSE, serde_json::to_value(&body)?); + let mut relays = vec![relay.to_string()]; + for configured in &self.config.node.relays { + if !relays.contains(configured) { + relays.push(configured.clone()); + } + } + let mut last_error: Option = None; + for candidate in relays { + let url = format!( + "{}/v1/unicast?to={}", + candidate.trim_end_matches('/'), + querier + ); + match self.client.post(&url).json(&envelope).send().await { + Ok(response) if response.status().is_success() => return Ok(()), + Ok(response) => { + last_error = Some(anyhow!( + "relay {candidate} rejected response: {}", + response.status() + )) + } + Err(error) => last_error = Some(anyhow!("relay {candidate} unreachable: {error}")), + } + } + Err(last_error.unwrap_or_else(|| anyhow!("no relays configured"))) + } +} + +async fn poll_relay(node: Arc, relay: String) { + let base = relay.trim_end_matches('/').to_string(); + loop { + let url = format!( + "{}/v1/poll?member={}&timeout_ms=20000", + base, + node.key.public_hex() + ); + match node.client.get(&url).send().await { + Ok(response) if response.status() == reqwest::StatusCode::NO_CONTENT => continue, + Ok(response) if response.status().is_success() => { + match response.json::().await { + Ok(payload) => { + if let Some(messages) = payload.get("messages").and_then(Value::as_array) { + for message in messages { + if let Ok(envelope) = + serde_json::from_value::(message.clone()) + { + node.dispatch(envelope, &base).await; + } + } + } + } + Err(_) => tokio::time::sleep(Duration::from_secs(1)).await, + } + } + Ok(_) => tokio::time::sleep(Duration::from_secs(1)).await, + Err(_) => tokio::time::sleep(Duration::from_secs(1)).await, + } + } +} + +pub fn router(node: Arc) -> Router { + Router::new() + .route("/v1/local/query", post(local_query)) + .route("/v1/local/status", get(local_status)) + .with_state(node) +} + +#[derive(Deserialize)] +pub struct LocalQueryRequest { + pub text: String, + pub max_results: Option, + pub timeout_ms: Option, + #[serde(default = "default_network")] + pub network: bool, +} + +fn default_network() -> bool { + true +} + +async fn local_query( + State(node): State>, + Json(request): Json, +) -> Response { + match node + .local_query( + &request.text, + request.max_results, + request.timeout_ms, + request.network, + ) + .await + { + Ok(outcome) => (StatusCode::OK, Json(outcome)).into_response(), + Err(error) => ( + StatusCode::INTERNAL_SERVER_ERROR, + Json(json!({ "error": error.to_string() })), + ) + .into_response(), + } +} + +async fn local_status(State(node): State>) -> Response { + Json(json!({ + "name": node.config.node.name, + "pubkey": node.key.public_hex(), + "listen": node.config.node.listen, + "relays": node.config.node.relays, + "responder": node.config.node.responder, + "doc_count": node.doc_count(), + "sent": node.sent(), + "received": node.received(), + })) + .into_response() +} + +pub async fn control_query( + base: &str, + text: &str, + max_results: Option, + timeout_ms: Option, + network: bool, +) -> Result { + let client = reqwest::Client::builder() + .timeout(Duration::from_secs(30)) + .build()?; + let url = format!("{}/v1/local/query", base.trim_end_matches('/')); + let response = client + .post(&url) + .json(&json!({ + "text": text, + "max_results": max_results, + "timeout_ms": timeout_ms, + "network": network, + })) + .send() + .await + .with_context(|| format!("calling local node at {url} (is `frxd serve` running?)"))?; + let status = response.status(); + let value: Value = response.json().await.context("parsing node response")?; + if !status.is_success() { + return Err(anyhow!("local node error {status}: {value}")); + } + Ok(value) +} diff --git a/src/relay.rs b/src/relay.rs new file mode 100644 index 0000000..e97cde8 --- /dev/null +++ b/src/relay.rs @@ -0,0 +1,191 @@ +use std::collections::{HashMap, VecDeque}; +use std::net::SocketAddr; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +use anyhow::Result; +use axum::extract::{Query, State}; +use axum::http::{StatusCode, header}; +use axum::response::{IntoResponse, Response}; +use axum::routing::{get, post}; +use axum::{Json, Router}; +use serde::Deserialize; +use serde_json::json; +use tokio::net::TcpListener; + +use crate::message::{Envelope, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE}; + +pub const DEFAULT_CAPACITY: usize = 256; + +#[derive(Default)] +struct MemberQueue { + items: VecDeque<(u64, Envelope)>, +} + +struct Inner { + members: HashMap, + seq: u64, +} + +pub struct Relay { + inner: Mutex, + capacity: usize, +} + +impl Relay { + pub fn new(capacity: usize) -> Arc { + Arc::new(Self { + inner: Mutex::new(Inner { + members: HashMap::new(), + seq: 0, + }), + capacity, + }) + } + + fn push(&self, targets: Option<&str>, envelope: Envelope) -> Result { + let mut inner = self.inner.lock().expect("relay lock"); + if inner + .members + .values() + .any(|q| q.items.len() >= self.capacity) + { + return Err(StatusCode::TOO_MANY_REQUESTS); + } + inner.seq += 1; + let seq = inner.seq; + match targets { + Some(member) => { + let Some(queue) = inner.members.get_mut(member) else { + return Err(StatusCode::NOT_FOUND); + }; + queue.items.push_back((seq, envelope)); + Ok(1) + } + None => { + let from = envelope.from.clone(); + inner.members.entry(from).or_default(); + let mut delivered = 0; + for queue in inner.members.values_mut() { + queue.items.push_back((seq, envelope.clone())); + delivered += 1; + } + Ok(delivered) + } + } + } +} + +pub fn router(relay: Arc) -> Router { + Router::new() + .route("/health", get(health)) + .route("/v1/publish", post(publish)) + .route("/v1/unicast", post(unicast)) + .route("/v1/poll", get(poll)) + .with_state(relay) +} + +pub async fn run(listener: TcpListener, capacity: usize) -> Result<()> { + let relay = Relay::new(capacity); + axum::serve(listener, router(relay)).await?; + Ok(()) +} + +pub async fn bind(addr: &str) -> Result<(TcpListener, SocketAddr)> { + let listener = TcpListener::bind(addr).await?; + let local = listener.local_addr()?; + Ok((listener, local)) +} + +async fn health() -> &'static str { + "ok" +} + +async fn publish(State(relay): State>, Json(envelope): Json) -> Response { + if envelope.verify().is_err() { + return bad_request("invalid signature"); + } + if envelope.msg_type != TYPE_QUERY { + return bad_request("relay carries broadcast queries only"); + } + match relay.push(None, envelope) { + Ok(delivered) => ( + StatusCode::ACCEPTED, + Json(json!({ "delivered": delivered })), + ) + .into_response(), + Err(status) => backpressure(status), + } +} + +#[derive(Deserialize)] +struct UnicastParams { + to: String, +} + +async fn unicast( + State(relay): State>, + Query(params): Query, + Json(envelope): Json, +) -> Response { + if envelope.verify().is_err() { + return bad_request("invalid signature"); + } + if envelope.msg_type != TYPE_RESPONSE && envelope.msg_type != TYPE_AGGREGATE { + return bad_request("unicast carries responses and aggregates only"); + } + match relay.push(Some(¶ms.to), envelope) { + Ok(_) => (StatusCode::OK, Json(json!({ "delivered": true }))).into_response(), + Err(StatusCode::NOT_FOUND) => ( + StatusCode::NOT_FOUND, + Json(json!({ "error": "member not connected" })), + ) + .into_response(), + Err(status) => backpressure(status), + } +} + +#[derive(Deserialize)] +struct PollParams { + member: String, + timeout_ms: Option, +} + +async fn poll(State(relay): State>, Query(params): Query) -> Response { + if params.member.is_empty() || hex::decode(¶ms.member).is_err() { + return bad_request("valid member key required"); + } + let timeout = Duration::from_millis(params.timeout_ms.unwrap_or(25_000).min(60_000)); + let deadline = Instant::now() + timeout; + loop { + let batch: Vec = { + let mut inner = relay.inner.lock().expect("relay lock"); + let queue = inner.members.entry(params.member.clone()).or_default(); + queue + .items + .drain(..) + .map(|(_, envelope)| envelope) + .collect() + }; + if !batch.is_empty() { + return (StatusCode::OK, Json(json!({ "messages": batch }))).into_response(); + } + if Instant::now() >= deadline { + return StatusCode::NO_CONTENT.into_response(); + } + tokio::time::sleep(Duration::from_millis(25)).await; + } +} + +fn bad_request(message: &str) -> Response { + (StatusCode::BAD_REQUEST, Json(json!({ "error": message }))).into_response() +} + +fn backpressure(status: StatusCode) -> Response { + ( + status, + [(header::RETRY_AFTER, "1")], + Json(json!({ "error": "transport backpressure" })), + ) + .into_response() +} diff --git a/src/render.rs b/src/render.rs new file mode 100644 index 0000000..23fde9c --- /dev/null +++ b/src/render.rs @@ -0,0 +1,63 @@ +use serde_json::Value; + +use crate::index::SearchHit; + +pub fn local_hits(hits: &[SearchHit], total: u64) { + println!("{total} local match(es), showing {}", hits.len()); + for hit in hits { + let scope = if hit.shared { "shared" } else { "private" }; + println!("- [{} / {}] {}", hit.collection, scope, hit.title); + println!(" {}", hit.url); + if !hit.summary.is_empty() { + println!(" {}", hit.summary); + } + } +} + +pub fn query_outcome(value: &Value) { + if let Some(qid) = value.get("qid").and_then(Value::as_str) { + println!("qid {qid}"); + } + let local_total = value + .pointer("/local/total") + .and_then(Value::as_u64) + .unwrap_or(0); + println!("local: {local_total} match(es)"); + if let Some(items) = value.pointer("/local/results").and_then(Value::as_array) { + for item in items { + print_item("local", item); + } + } + if let Some(responses) = value.get("responses").and_then(Value::as_array) { + println!("remote: {} response(s)", responses.len()); + for response in responses { + let member = response + .get("member") + .and_then(Value::as_str) + .unwrap_or("unknown"); + let short = member.get(..16).unwrap_or(member); + let truncated = response + .get("truncated") + .and_then(Value::as_bool) + .unwrap_or(false); + let more = response + .get("more_available") + .and_then(Value::as_u64) + .unwrap_or(0); + println!(" from member {short}... (truncated={truncated}, more_available={more})"); + if let Some(results) = response.get("results").and_then(Value::as_array) { + for item in results { + print_item(short, item); + } + } + } + } +} + +fn print_item(provenance: &str, item: &Value) { + let title = item.get("title").and_then(Value::as_str).unwrap_or(""); + let url = item.get("url").and_then(Value::as_str).unwrap_or(""); + let exposure = item.get("exposure").and_then(Value::as_str).unwrap_or(""); + println!("- [{provenance}] {title} ({exposure})"); + println!(" {url}"); +} diff --git a/tests/cli.rs b/tests/cli.rs new file mode 100644 index 0000000..35c10f9 --- /dev/null +++ b/tests/cli.rs @@ -0,0 +1,250 @@ +mod common; + +use std::fs; +use std::io::{BufRead, BufReader, Read}; +use std::path::Path; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +fn frxd() -> Command { + Command::new(env!("CARGO_BIN_EXE_frxd")) +} + +fn frx() -> Command { + Command::new(env!("CARGO_BIN_EXE_frx")) +} + +fn init_args( + config: &Path, + name: &str, + listen_port: u16, + relay_port: u16, + data: &Path, +) -> Vec { + vec![ + "--config".to_string(), + config.display().to_string(), + "init".to_string(), + "--name".to_string(), + name.to_string(), + "--listen".to_string(), + format!("127.0.0.1:{listen_port}"), + "--relay".to_string(), + format!("http://127.0.0.1:{relay_port}"), + "--data-dir".to_string(), + data.display().to_string(), + ] +} + +fn add_args(config: &Path, docs: &Path, name: &str, shared: bool) -> Vec { + let mut args = vec![ + "--config".to_string(), + config.display().to_string(), + "add".to_string(), + docs.display().to_string(), + "--name".to_string(), + name.to_string(), + "--exposure".to_string(), + "full".to_string(), + ]; + if shared { + args.push("--shared".to_string()); + } + args +} + +fn run_ok(args: &[String]) -> String { + let output = frxd().args(args).output().unwrap(); + assert!( + output.status.success(), + "command failed: {}\n{}", + args.join(" "), + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8_lossy(&output.stdout).to_string() +} + +struct Service { + child: Child, +} + +impl Drop for Service { + fn drop(&mut self) { + let _ = self.child.kill(); + let _ = self.child.wait(); + } +} + +fn spawn_service(args: &[String], needle: &str) -> Service { + let mut child = frxd() + .args(args) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .unwrap(); + let reader = BufReader::new(child.stdout.take().unwrap()); + let mut reader = reader; + let deadline = Instant::now() + Duration::from_secs(10); + loop { + let mut line = String::new(); + let read = reader.read_line(&mut line).unwrap(); + if read == 0 { + panic!("service exited before printing '{needle}'"); + } + if line.contains(needle) { + break; + } + assert!(Instant::now() < deadline, "timeout waiting for '{needle}'"); + } + std::thread::spawn(move || { + let mut sink = Vec::new(); + let _ = reader.read_to_end(&mut sink); + }); + Service { child } +} + +#[test] +fn cli_init_add_search_status() { + let root = tempfile::tempdir().unwrap(); + let config = root.path().join("frxd.toml"); + let data = root.path().join("data"); + let docs = root.path().join("docs"); + fs::create_dir_all(&docs).unwrap(); + fs::write(docs.join("note.txt"), "cli smoke rust document").unwrap(); + + let config_arg = config.display().to_string(); + let stdout = run_ok(&init_args(&config, "alice", 0, 1, &data)); + assert!(stdout.contains("pubkey")); + assert!(config.exists()); + let key = data.join("key.hex"); + assert!(key.exists()); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = fs::metadata(&key).unwrap().permissions().mode() & 0o777; + assert_eq!(mode, 0o600, "key file must not be world readable"); + } + + let stdout = run_ok(&add_args(&config, &docs, "docs", true)); + assert!(stdout.contains("indexed 1 file(s)")); + + let stdout = run_ok(&[ + "--config".to_string(), + config_arg.clone(), + "search".to_string(), + "rust".to_string(), + ]); + assert!(stdout.contains("cli smoke rust document")); + + let output = frx() + .args(["--config", &config_arg, "search", "rust"]) + .output() + .unwrap(); + assert!(output.status.success()); + assert!(String::from_utf8_lossy(&output.stdout).contains("cli smoke rust document")); + + let stdout = run_ok(&["--config".to_string(), config_arg, "status".to_string()]); + assert!(stdout.contains("node not running")); +} + +#[test] +fn cli_init_refuses_overwrite_without_force() { + let root = tempfile::tempdir().unwrap(); + let config = root.path().join("frxd.toml"); + let data = root.path().join("data"); + let args = init_args(&config, "alice", 0, 1, &data); + run_ok(&args); + + let output = frxd().args(&args).output().unwrap(); + assert!(!output.status.success()); + assert!(String::from_utf8_lossy(&output.stderr).contains("already exists")); + + let mut forced = args.clone(); + forced.push("--force".to_string()); + run_ok(&forced); +} + +#[test] +fn cli_full_network_pipeline() { + let root = tempfile::tempdir().unwrap(); + let relay_port = common::free_port(); + let relay_args = vec![ + "relay".to_string(), + "--listen".to_string(), + format!("127.0.0.1:{relay_port}"), + ]; + let _relay = spawn_service(&relay_args, "relay listening"); + + let bob_port = common::free_port(); + let alice_port = common::free_port(); + let bob_config = root.path().join("bob.toml"); + let alice_config = root.path().join("alice.toml"); + let bob_docs = root.path().join("bob-docs"); + let alice_docs = root.path().join("alice-docs"); + fs::create_dir_all(&bob_docs).unwrap(); + fs::create_dir_all(&alice_docs).unwrap(); + fs::write( + bob_docs.join("tantivy.md"), + "# Tantivy BM25\nTantivy indexes rust documents.", + ) + .unwrap(); + fs::write(alice_docs.join("private.txt"), "alice private rust note").unwrap(); + + run_ok(&init_args( + &bob_config, + "bob", + bob_port, + relay_port, + &root.path().join("bob-data"), + )); + run_ok(&init_args( + &alice_config, + "alice", + alice_port, + relay_port, + &root.path().join("alice-data"), + )); + run_ok(&add_args(&bob_config, &bob_docs, "shared", true)); + run_ok(&add_args(&alice_config, &alice_docs, "mine", false)); + + let bob_serve = vec![ + "--config".to_string(), + bob_config.display().to_string(), + "serve".to_string(), + ]; + let alice_serve = vec![ + "--config".to_string(), + alice_config.display().to_string(), + "serve".to_string(), + ]; + let _bob = spawn_service(&bob_serve, "frxd listening"); + let _alice = spawn_service(&alice_serve, "frxd listening"); + std::thread::sleep(Duration::from_millis(400)); + + let alice_config_arg = alice_config.display().to_string(); + let stdout = run_ok(&[ + "--config".to_string(), + alice_config_arg.clone(), + "query".to_string(), + "tantivy".to_string(), + ]); + assert!(stdout.contains("Tantivy BM25"), "{stdout}"); + assert!(stdout.contains("remote: 1 response(s)"), "{stdout}"); + + let stdout = run_ok(&[ + "--config".to_string(), + alice_config_arg.clone(), + "query".to_string(), + "rust".to_string(), + "--local-only".to_string(), + ]); + assert!(stdout.contains("alice private rust note"), "{stdout}"); + assert!(stdout.contains("remote: 0 response(s)"), "{stdout}"); + + let output = frx() + .args(["--config", &alice_config_arg, "query", "tantivy"]) + .output() + .unwrap(); + assert!(output.status.success()); + assert!(String::from_utf8_lossy(&output.stdout).contains("Tantivy BM25")); +} diff --git a/tests/common/mod.rs b/tests/common/mod.rs new file mode 100644 index 0000000..f07dfdf --- /dev/null +++ b/tests/common/mod.rs @@ -0,0 +1,163 @@ +#![allow(dead_code)] +use std::path::Path; +use std::time::Duration; + +use frxd::config::{Config, IndexSection, NodeSection, QuerySection}; +use frxd::crypto::Keypair; +use frxd::index::Collection; +use frxd::message::{Envelope, QueryBody, TYPE_QUERY}; +use frxd::relay; +use serde_json::Value; + +pub fn config_for(dir: &Path, name: &str, relay_url: &str) -> Config { + let config = Config { + node: NodeSection { + name: name.to_string(), + listen: "127.0.0.1:0".to_string(), + relays: vec![relay_url.to_string()], + trusted_keys: Vec::new(), + responder: true, + }, + query: QuerySection { + max_results: 5, + timeout_ms: 700, + }, + index: IndexSection { + data_dir: dir.join("data").display().to_string(), + }, + }; + config.save_key(&Keypair::generate()).unwrap(); + config +} + +pub fn collection(name: &str, path: &Path, shared: bool, exposure: &str) -> Collection { + Collection { + name: name.to_string(), + path: path.display().to_string(), + shared, + exposure: exposure.to_string(), + } +} + +pub async fn spawn_relay() -> String { + spawn_relay_with_capacity(256).await +} + +pub async fn spawn_relay_with_capacity(capacity: usize) -> String { + let (listener, addr) = relay::bind("127.0.0.1:0").await.unwrap(); + tokio::spawn(async move { + let _ = relay::run(listener, capacity).await; + }); + format!("http://{addr}") +} + +pub fn free_port() -> u16 { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + listener.local_addr().unwrap().port() +} + +pub fn query_envelope(key: &Keypair, text: &str, max_results: usize) -> Envelope { + let body = QueryBody::new(text, max_results); + Envelope::new(key, TYPE_QUERY, serde_json::to_value(&body).unwrap()) +} + +pub async fn register(client: &reqwest::Client, relay_url: &str, member: &str) { + poll(client, relay_url, member, 30).await; +} + +pub async fn poll_messages( + client: &reqwest::Client, + relay_url: &str, + member: &str, + timeout_ms: u64, +) -> Vec { + let response = poll(client, relay_url, member, timeout_ms).await; + if response.status() == reqwest::StatusCode::NO_CONTENT { + return Vec::new(); + } + let payload: Value = response.json().await.unwrap(); + payload + .get("messages") + .and_then(Value::as_array) + .cloned() + .unwrap_or_default() +} + +pub fn messages_of_type(messages: &[Value], msg_type: &str) -> Vec { + messages + .iter() + .filter(|m| m.get("type").and_then(Value::as_str) == Some(msg_type)) + .cloned() + .collect() +} + +pub fn client() -> reqwest::Client { + reqwest::Client::builder() + .timeout(Duration::from_secs(10)) + .build() + .unwrap() +} + +pub async fn ask( + client: &reqwest::Client, + addr: &str, + text: &str, + max: usize, +) -> (reqwest::StatusCode, String, Value) { + let response = client + .post(format!("http://{addr}/v1/local/query")) + .json(&serde_json::json!({ + "text": text, + "max_results": max, + "timeout_ms": 700, + })) + .send() + .await + .unwrap(); + let status = response.status(); + let raw = response.text().await.unwrap(); + let value: Value = serde_json::from_str(&raw).unwrap(); + (status, raw, value) +} + +pub async fn publish( + client: &reqwest::Client, + relay_url: &str, + envelope: &Envelope, +) -> reqwest::Response { + client + .post(format!("{relay_url}/v1/publish")) + .json(envelope) + .send() + .await + .unwrap() +} + +pub async fn unicast( + client: &reqwest::Client, + relay_url: &str, + to: &str, + envelope: &Envelope, +) -> reqwest::Response { + client + .post(format!("{relay_url}/v1/unicast?to={to}")) + .json(envelope) + .send() + .await + .unwrap() +} + +pub async fn poll( + client: &reqwest::Client, + relay_url: &str, + member: &str, + timeout_ms: u64, +) -> reqwest::Response { + client + .get(format!( + "{relay_url}/v1/poll?member={member}&timeout_ms={timeout_ms}" + )) + .send() + .await + .unwrap() +} diff --git a/tests/concurrency.rs b/tests/concurrency.rs new file mode 100644 index 0000000..7e2a662 --- /dev/null +++ b/tests/concurrency.rs @@ -0,0 +1,210 @@ +mod common; + +use std::collections::BTreeSet; +use std::fs; +use std::path::Path; +use std::time::{Duration, Instant}; + +use common::{ + ask, client, collection, config_for, poll_messages, publish, query_envelope, register, + spawn_relay, +}; +use frxd::crypto::Keypair; +use frxd::index::LocalIndex; +use frxd::message::{EXPOSURE_FULL, TYPE_RESPONSE}; +use frxd::node::{self, Node, NodeHandle}; +use serde_json::Value; + +fn corpus_dir(root: &Path, name: &str, files: &[(&str, &str)]) -> std::path::PathBuf { + let dir = root.join(format!("{name}-docs")); + fs::create_dir_all(&dir).unwrap(); + for (file, text) in files { + fs::write(dir.join(file), text).unwrap(); + } + dir +} + +async fn start_node_with_corpus( + root: &Path, + name: &str, + relays: Vec, + files: &[(&str, &str)], +) -> NodeHandle { + let docs = corpus_dir(root, name, files); + let mut config = config_for(&root.join(name), name, relays.first().unwrap()); + config.node.relays = relays; + { + let index = LocalIndex::open(&config.index_dir()).unwrap(); + index + .add_collection(&collection("docs", &docs, true, EXPOSURE_FULL)) + .unwrap(); + } + Node::start(config).await.unwrap() +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn concurrent_queries_each_get_their_response() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + vec![relay_url.clone()], + &[("doc.txt", "concurrent rust document")], + ) + .await; + let alice = start_node_with_corpus( + root.path(), + "alice", + vec![relay_url.clone()], + &[("mine.txt", "alice notes")], + ) + .await; + + let http = client(); + let addr = alice.addr.to_string(); + let mut tasks = Vec::new(); + for _ in 0..4 { + let http = http.clone(); + let addr = addr.clone(); + tasks.push(tokio::spawn( + async move { ask(&http, &addr, "rust", 5).await }, + )); + } + let mut qids = BTreeSet::new(); + for task in tasks { + let (_status, _raw, value) = task.await.unwrap(); + let qid = value + .get("qid") + .and_then(Value::as_str) + .unwrap() + .to_string(); + qids.insert(qid); + let responses = value.get("responses").and_then(Value::as_array).unwrap(); + assert_eq!(responses.len(), 1, "query lost its response: {value}"); + } + assert_eq!(qids.len(), 4); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn multi_relay_deduplicates_and_responds_once() { + let root = tempfile::tempdir().unwrap(); + let relay_one = spawn_relay().await; + let relay_two = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + vec![relay_one.clone(), relay_two.clone()], + &[("doc.txt", "dual relay rust document")], + ) + .await; + + let http = client(); + let alice = Keypair::generate(); + register(&http, &relay_one, &alice.public_hex()).await; + register(&http, &relay_two, &alice.public_hex()).await; + let envelope = query_envelope(&alice, "rust", 5); + assert!( + publish(&http, &relay_one, &envelope) + .await + .status() + .is_success() + ); + assert!( + publish(&http, &relay_two, &envelope) + .await + .status() + .is_success() + ); + + let deadline = Instant::now() + Duration::from_millis(1200); + let mut responses = Vec::new(); + while responses.is_empty() && Instant::now() < deadline { + for relay in [&relay_one, &relay_two] { + let messages = poll_messages(&http, relay, &alice.public_hex(), 200).await; + responses.extend( + messages + .iter() + .filter(|m| m.get("type").and_then(Value::as_str) == Some(TYPE_RESPONSE)) + .cloned(), + ); + } + } + assert_eq!( + responses.len(), + 1, + "duplicate relay delivery was answered twice" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn index_persists_across_node_restart() { + let root = tempfile::tempdir().unwrap(); + let docs = corpus_dir( + root.path(), + "bob", + &[("doc.txt", "persistent rust document")], + ); + let config = config_for(&root.path().join("bob"), "bob", "http://127.0.0.1:1"); + { + let index = LocalIndex::open(&config.index_dir()).unwrap(); + index + .add_collection(&collection("docs", &docs, true, EXPOSURE_FULL)) + .unwrap(); + } + + let first = Node::start(config.clone()).await.unwrap(); + let outcome = node::control_query( + &format!("http://{}", first.addr), + "rust", + Some(5), + Some(100), + false, + ) + .await + .unwrap(); + assert_eq!( + outcome.pointer("/local/total").and_then(Value::as_u64), + Some(1) + ); + drop(first); + + let second = Node::start(config).await.unwrap(); + let outcome = node::control_query( + &format!("http://{}", second.addr), + "rust", + Some(5), + Some(100), + false, + ) + .await + .unwrap(); + assert_eq!( + outcome.pointer("/local/total").and_then(Value::as_u64), + Some(1) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn poll_returns_queued_batch_in_one_call() { + let relay_url = spawn_relay().await; + let http = client(); + let member = Keypair::generate(); + register(&http, &relay_url, &member.public_hex()).await; + for index in 0..3 { + let envelope = query_envelope(&member, &format!("query {index}"), 5); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + } + let messages = poll_messages(&http, &relay_url, &member.public_hex(), 300).await; + assert_eq!(messages.len(), 3); + assert!( + poll_messages(&http, &relay_url, &member.public_hex(), 50) + .await + .is_empty() + ); +} diff --git a/tests/conformance.rs b/tests/conformance.rs new file mode 100644 index 0000000..48888fa --- /dev/null +++ b/tests/conformance.rs @@ -0,0 +1,924 @@ +mod common; + +use std::collections::BTreeSet; +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::{Duration, Instant}; + +use common::{ + ask, client, collection, config_for, messages_of_type, poll, poll_messages, publish, + query_envelope, register, spawn_relay, spawn_relay_with_capacity, unicast, +}; +use frxd::crypto::Keypair; +use frxd::index::LocalIndex; +use frxd::message::{ + EXPOSURE_FULL, EXPOSURE_METADATA, Envelope, QueryBody, TYPE_AGGREGATE, TYPE_QUERY, + TYPE_RESPONSE, +}; +use frxd::node::{Node, NodeHandle}; +use serde_json::{Value, json}; + +fn keys(value: &Value) -> BTreeSet { + value.as_object().expect("object").keys().cloned().collect() +} + +async fn status(http: &reqwest::Client, addr: &str) -> Value { + http.get(format!("http://{addr}/v1/local/status")) + .send() + .await + .unwrap() + .json() + .await + .unwrap() +} + +fn corpus_dir(root: &Path, name: &str, files: &[(&str, &str)]) -> PathBuf { + let dir = root.join(format!("{name}-docs")); + fs::create_dir_all(&dir).unwrap(); + for (file, text) in files { + fs::write(dir.join(file), text).unwrap(); + } + dir +} + +async fn start_node_with_corpus( + root: &Path, + name: &str, + relay_url: &str, + shared: bool, + exposure: &str, + files: &[(&str, &str)], +) -> NodeHandle { + let docs = corpus_dir(root, name, files); + let config = config_for(&root.join(name), name, relay_url); + { + let index = LocalIndex::open(&config.index_dir()).unwrap(); + index + .add_collection(&collection("docs", &docs, shared, exposure)) + .unwrap(); + } + Node::start(config).await.unwrap() +} + +async fn collect_responses( + http: &reqwest::Client, + relay_url: &str, + member: &str, + timeout_ms: u64, +) -> Vec { + let start = Instant::now(); + let deadline = Duration::from_millis(timeout_ms); + let mut responses = Vec::new(); + while start.elapsed() < deadline { + let messages = poll_messages(http, relay_url, member, 200).await; + responses.extend(messages_of_type(&messages, TYPE_RESPONSE)); + if !responses.is_empty() { + break; + } + } + responses +} + +async fn raw_query( + http: &reqwest::Client, + relay_url: &str, + asker: &Keypair, + text: &str, + timeout_ms: u64, +) -> Vec { + register(http, relay_url, &asker.public_hex()).await; + let envelope = query_envelope(asker, text, 5); + assert!( + publish(http, relay_url, &envelope) + .await + .status() + .is_success() + ); + collect_responses(http, relay_url, &asker.public_hex(), timeout_ms).await +} + +#[test] +fn envelope_field_set_is_fixed() { + let key = Keypair::generate(); + let envelope = Envelope::new(&key, TYPE_QUERY, json!({"qid": "q", "text": "t"})); + let value = serde_json::to_value(&envelope).unwrap(); + assert_eq!( + keys(&value), + BTreeSet::from([ + "body".to_string(), + "from".to_string(), + "nonce".to_string(), + "sig".to_string(), + "ts".to_string(), + "type".to_string(), + ]) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn ordering_travels_scores_dont() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[ + ("strong.txt", "rust rust rust rust search engine"), + ("weak.txt", "rust notes"), + ], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + let responses = raw_query(&http, &relay_url, &alice, "rust", 700).await; + assert_eq!(responses.len(), 1); + let results = responses[0] + .pointer("/body/results") + .and_then(Value::as_array) + .unwrap(); + assert_eq!(results.len(), 2); + let first_url = results[0].get("url").and_then(Value::as_str).unwrap(); + assert!( + first_url.contains("strong"), + "best match should lead the ordering: {first_url}" + ); + for result in results { + assert!(!keys(result).contains("score")); + assert!(!keys(result).contains("rank")); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn metadata_exposure_withholds_content() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_METADATA, + &[("corpus.txt", "enrichment corpus about rust")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + let responses = raw_query(&http, &relay_url, &alice, "rust", 700).await; + assert_eq!(responses.len(), 1); + let result = &responses[0] + .pointer("/body/results") + .and_then(Value::as_array) + .unwrap()[0]; + assert_eq!( + result.get("exposure").and_then(Value::as_str), + Some(EXPOSURE_METADATA) + ); + assert!(result.get("content").unwrap().is_null()); + assert!( + !result + .get("summary") + .and_then(Value::as_str) + .unwrap() + .is_empty() + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn full_exposure_carries_content() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("corpus.txt", "full exposure rust content body")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + let responses = raw_query(&http, &relay_url, &alice, "rust", 700).await; + let result = &responses[0] + .pointer("/body/results") + .and_then(Value::as_array) + .unwrap()[0]; + assert_eq!( + result.get("exposure").and_then(Value::as_str), + Some(EXPOSURE_FULL) + ); + assert!( + result + .get("content") + .and_then(Value::as_str) + .unwrap() + .contains("full exposure rust content body") + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn private_collections_are_silent_egress() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + false, + EXPOSURE_FULL, + &[("private.txt", "rust private data")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + let responses = raw_query(&http, &relay_url, &alice, "rust", 500).await; + assert!(responses.is_empty(), "private collection answered"); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn one_result_is_a_conformant_response() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("only.txt", "solitary rust document")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + let responses = raw_query(&http, &relay_url, &alice, "solitary", 700).await; + assert_eq!(responses.len(), 1); + assert_eq!( + responses[0] + .pointer("/body/results") + .and_then(Value::as_array) + .unwrap() + .len(), + 1 + ); + assert_eq!( + responses[0] + .pointer("/body/truncated") + .and_then(Value::as_bool), + Some(false) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn empty_query_earns_silence() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "rust content")], + ) + .await; + let alice = start_node_with_corpus( + root.path(), + "alice", + &relay_url, + false, + EXPOSURE_FULL, + &[("mine.txt", "alice rust")], + ) + .await; + let response = client() + .post(format!("http://{}/v1/local/query", alice.addr)) + .json(&json!({"text": "", "max_results": 5, "timeout_ms": 500})) + .send() + .await + .unwrap() + .json::() + .await + .unwrap(); + assert!( + response + .get("responses") + .and_then(Value::as_array) + .unwrap() + .is_empty() + ); + assert!(bob.node.doc_count() > 0); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn entities_are_optional_hints() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "entity tagged rust document")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + register(&http, &relay_url, &alice.public_hex()).await; + let body = json!({ + "qid": "entities-test", + "text": "rust", + "entities": ["Q999999", "not-a-real-entity"], + "budget": {"max_results": 5} + }); + let envelope = Envelope::new(&alice, TYPE_QUERY, body); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await; + assert_eq!(responses.len(), 1); + assert_eq!( + responses[0].pointer("/body/qid").and_then(Value::as_str), + Some("entities-test") + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn trusted_keys_allowlist_filters_senders() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let alice = Keypair::generate(); + let untrusted = Keypair::generate(); + + let docs = corpus_dir(root.path(), "bob", &[("doc.txt", "rust document")]); + let mut config = config_for(&root.path().join("bob"), "bob", &relay_url); + config.node.trusted_keys = vec![alice.public_hex()]; + { + let index = LocalIndex::open(&config.index_dir()).unwrap(); + index + .add_collection(&collection("docs", &docs, true, EXPOSURE_FULL)) + .unwrap(); + } + let _bob = Node::start(config).await.unwrap(); + + let http = client(); + let trusted = raw_query(&http, &relay_url, &alice, "rust", 700).await; + assert_eq!(trusted.len(), 1, "trusted member got no response"); + + register(&http, &relay_url, &untrusted.public_hex()).await; + let envelope = query_envelope(&untrusted, "rust", 5); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + let denied = collect_responses(&http, &relay_url, &untrusted.public_hex(), 400).await; + assert!(denied.is_empty(), "untrusted member received a response"); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn backpressure_is_visible_and_recoverable() { + let relay_url = spawn_relay_with_capacity(1).await; + let http = client(); + let member = Keypair::generate(); + let publisher = Keypair::generate(); + register(&http, &relay_url, &member.public_hex()).await; + + let first = query_envelope(&publisher, "one", 5); + assert_eq!( + publish(&http, &relay_url, &first).await.status(), + reqwest::StatusCode::ACCEPTED + ); + let second = query_envelope(&publisher, "two", 5); + let response = publish(&http, &relay_url, &second).await; + assert_eq!(response.status(), reqwest::StatusCode::TOO_MANY_REQUESTS); + assert_eq!( + response + .headers() + .get("retry-after") + .and_then(|v| v.to_str().ok()), + Some("1") + ); + + assert_eq!( + poll_messages(&http, &relay_url, &member.public_hex(), 300) + .await + .len(), + 1 + ); + assert_eq!( + poll_messages(&http, &relay_url, &publisher.public_hex(), 300) + .await + .len(), + 1 + ); + + let third = query_envelope(&publisher, "three", 5); + assert_eq!( + publish(&http, &relay_url, &third).await.status(), + reqwest::StatusCode::ACCEPTED + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn unicast_is_need_to_know() { + let relay_url = spawn_relay().await; + let http = client(); + let alice = Keypair::generate(); + let bob = Keypair::generate(); + register(&http, &relay_url, &alice.public_hex()).await; + register(&http, &relay_url, &bob.public_hex()).await; + + let query = query_envelope(&alice, "rust", 5); + assert!( + publish(&http, &relay_url, &query) + .await + .status() + .is_success() + ); + + let response = Envelope::new( + &bob, + TYPE_RESPONSE, + json!({"qid": "q1", "results": [], "truncated": false, "more_available": 0, "cursor": null}), + ); + assert_eq!( + unicast(&http, &relay_url, &alice.public_hex(), &response) + .await + .status(), + reqwest::StatusCode::OK + ); + + let alice_messages = poll_messages(&http, &relay_url, &alice.public_hex(), 300).await; + assert_eq!(messages_of_type(&alice_messages, TYPE_RESPONSE).len(), 1); + let bob_messages = poll_messages(&http, &relay_url, &bob.public_hex(), 300).await; + assert!( + messages_of_type(&bob_messages, TYPE_RESPONSE).is_empty(), + "unicast response leaked to another member" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn unicast_to_unknown_member_is_visible() { + let relay_url = spawn_relay().await; + let http = client(); + let sender = Keypair::generate(); + let response = Envelope::new( + &sender, + TYPE_RESPONSE, + json!({"qid": "q1", "results": [], "truncated": false, "more_available": 0, "cursor": null}), + ); + let stranger = Keypair::generate().public_hex(); + assert_eq!( + unicast(&http, &relay_url, &stranger, &response) + .await + .status(), + reqwest::StatusCode::NOT_FOUND + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn tampered_envelopes_are_rejected() { + let relay_url = spawn_relay().await; + let http = client(); + let key = Keypair::generate(); + let mut envelope = query_envelope(&key, "rust", 5); + envelope.body = json!({"qid": "q1", "text": "forged", "budget": {"max_results": 5}}); + let publish_response = publish(&http, &relay_url, &envelope).await; + assert_eq!(publish_response.status(), reqwest::StatusCode::BAD_REQUEST); + + let mut response = Envelope::new( + &key, + TYPE_RESPONSE, + json!({"qid": "q1", "results": [], "truncated": false, "more_available": 0, "cursor": null}), + ); + response.sig = "00".repeat(64); + let unicast_response = unicast(&http, &relay_url, &key.public_hex(), &response).await; + assert_eq!(unicast_response.status(), reqwest::StatusCode::BAD_REQUEST); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn invalid_poll_member_is_rejected() { + let relay_url = spawn_relay().await; + let response = poll(&client(), &relay_url, "not-hex-at-all", 30).await; + assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn malformed_json_is_rejected() { + let relay_url = spawn_relay().await; + let http = client(); + let response = http + .post(format!("{relay_url}/v1/publish")) + .header("content-type", "application/json") + .body("{not json") + .send() + .await + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); + + let root = tempfile::tempdir().unwrap(); + let config = config_for(&root.path().join("alice"), "alice", &relay_url); + let node = Node::start(config).await.unwrap(); + let response = http + .post(format!("http://{}/v1/local/query", node.addr)) + .header("content-type", "application/json") + .body("{not json") + .send() + .await + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn node_does_not_answer_its_own_queries() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "self query rust")], + ) + .await; + let (_status, _raw, value) = ask(&client(), &bob.addr.to_string(), "rust", 5).await; + assert!( + value + .get("responses") + .and_then(Value::as_array) + .unwrap() + .is_empty(), + "node responded to itself: {value}" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn ingress_happens_only_locally() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "rust served by bob")], + ) + .await; + let alice = start_node_with_corpus( + root.path(), + "alice", + &relay_url, + false, + EXPOSURE_FULL, + &[("mine.txt", "alice rust notes")], + ) + .await; + let http = client(); + let bob_before = status(&http, &bob.addr.to_string()).await; + let alice_before = status(&http, &alice.addr.to_string()).await; + + let (_s, _r, value) = ask(&http, &alice.addr.to_string(), "rust", 5).await; + assert_eq!( + value + .get("responses") + .and_then(Value::as_array) + .unwrap() + .len(), + 1 + ); + + let bob_after = status(&http, &bob.addr.to_string()).await; + let alice_after = status(&http, &alice.addr.to_string()).await; + assert_eq!(bob_before.get("doc_count"), bob_after.get("doc_count")); + assert_eq!(alice_before.get("doc_count"), alice_after.get("doc_count")); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn querier_and_responder_roles_coexist() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let alice = start_node_with_corpus( + root.path(), + "alice", + &relay_url, + true, + EXPOSURE_FULL, + &[("a.txt", "alice rust document")], + ) + .await; + let bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("b.txt", "bob rust document")], + ) + .await; + + let http = client(); + let alice_addr = alice.addr.to_string(); + let bob_addr = bob.addr.to_string(); + let alice_ask = async { common::ask(&http, &alice_addr, "rust", 5).await }; + let bob_ask = async { common::ask(&http, &bob_addr, "rust", 5).await }; + let (alice_result, bob_result) = tokio::join!(alice_ask, bob_ask); + + for (name, (_status, _raw, value)) in [("alice", alice_result), ("bob", bob_result)] { + let responses = value.get("responses").and_then(Value::as_array).unwrap(); + assert_eq!(responses.len(), 1, "{name} got {value}"); + let member = responses[0].get("member").and_then(Value::as_str).unwrap(); + assert!(!member.is_empty()); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn budget_is_clamped_and_never_exceeded() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[ + ("one.txt", "rust one"), + ("two.txt", "rust two"), + ("three.txt", "rust three"), + ], + ) + .await; + let alice = start_node_with_corpus( + root.path(), + "alice", + &relay_url, + false, + EXPOSURE_FULL, + &[("mine.txt", "alice doc")], + ) + .await; + let http = client(); + + let (_s, _r, value) = ask(&http, &alice.addr.to_string(), "rust", 0).await; + let responses = value.get("responses").and_then(Value::as_array).unwrap(); + assert_eq!(responses.len(), 1); + assert_eq!( + responses[0] + .get("results") + .and_then(Value::as_array) + .unwrap() + .len(), + 1, + "zero budget must clamp to one, never send nothing or more" + ); + + let (_s, _r, value) = ask(&http, &alice.addr.to_string(), "rust", 1000).await; + let responses = value.get("responses").and_then(Value::as_array).unwrap(); + assert_eq!( + responses[0] + .get("results") + .and_then(Value::as_array) + .unwrap() + .len(), + 3 + ); + assert_eq!( + responses[0].get("truncated").and_then(Value::as_bool), + Some(false) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn aggregates_are_unicast_only_and_ignored_by_nodes() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "aggregate test rust")], + ) + .await; + let http = client(); + let sender = Keypair::generate(); + register(&http, &relay_url, &bob.pubkey).await; + + let broadcast = Envelope::new( + &sender, + TYPE_AGGREGATE, + json!({"period": "2026-03", "sent": 1, "passed": 0, "cited": 0}), + ); + assert_eq!( + publish(&http, &relay_url, &broadcast).await.status(), + reqwest::StatusCode::BAD_REQUEST, + "aggregates must not travel the broadcast channel" + ); + + let unicast_envelope = Envelope::new( + &sender, + TYPE_AGGREGATE, + json!({"period": "2026-03", "sent": 1, "passed": 0, "cited": 0}), + ); + assert_eq!( + unicast(&http, &relay_url, &bob.pubkey, &unicast_envelope) + .await + .status(), + reqwest::StatusCode::OK + ); + tokio::time::sleep(Duration::from_millis(200)).await; + let (_s, _r, value) = ask(&http, &bob.addr.to_string(), "ignored", 5).await; + assert_eq!( + value.pointer("/local/total").and_then(Value::as_u64), + Some(0) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn node_survives_unreachable_relay() { + let root = tempfile::tempdir().unwrap(); + let alice = start_node_with_corpus( + root.path(), + "alice", + "http://127.0.0.1:1", + false, + EXPOSURE_FULL, + &[("mine.txt", "offline rust note")], + ) + .await; + let (status_code, _raw, value) = ask(&client(), &alice.addr.to_string(), "rust", 5).await; + assert!(status_code.is_success()); + assert_eq!( + value.pointer("/local/total").and_then(Value::as_u64), + Some(1) + ); + assert!( + value + .get("responses") + .and_then(Value::as_array) + .unwrap() + .is_empty() + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn duplicate_delivery_is_answered_once() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "duplicate rust doc")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + register(&http, &relay_url, &alice.public_hex()).await; + let envelope = query_envelope(&alice, "rust", 5); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await; + assert_eq!( + responses.len(), + 1, + "duplicate query produced duplicate responses" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn multiple_responders_merge_with_provenance() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("bob.txt", "bob rust contribution")], + ) + .await; + let _carol = start_node_with_corpus( + root.path(), + "carol", + &relay_url, + true, + EXPOSURE_FULL, + &[("carol.txt", "carol rust contribution")], + ) + .await; + let alice = start_node_with_corpus( + root.path(), + "alice", + &relay_url, + false, + EXPOSURE_FULL, + &[("mine.txt", "alice rust local")], + ) + .await; + let (_s, _r, value) = ask(&client(), &alice.addr.to_string(), "rust", 5).await; + let responses = value.get("responses").and_then(Value::as_array).unwrap(); + assert_eq!(responses.len(), 2, "expected both responders: {value}"); + let members: BTreeSet<&str> = responses + .iter() + .map(|r| r.get("member").and_then(Value::as_str).unwrap()) + .collect(); + assert_eq!(members.len(), 2); + let merged = value.get("merged").and_then(Value::as_array).unwrap(); + let provenances: BTreeSet<&str> = merged + .iter() + .map(|m| m.get("provenance").and_then(Value::as_str).unwrap()) + .collect(); + assert!(provenances.contains("local")); + assert_eq!(provenances.len(), 3); +} + +#[test] +fn canonical_signing_bytes_are_stable() { + let mut envelope = Envelope { + msg_type: TYPE_QUERY.to_string(), + from: "aa".repeat(32), + ts: 1_700_000_000, + nonce: "00112233445566778899aabbccddeeff".to_string(), + body: json!({"text": "rust", "qid": "q1", "budget": {"max_results": 5}, "entities": []}), + sig: String::new(), + }; + let first = frxd::crypto::signing_bytes(&envelope); + envelope.body = + json!({"entities": [], "budget": {"max_results": 5}, "qid": "q1", "text": "rust"}); + let second = frxd::crypto::signing_bytes(&envelope); + assert_eq!(first, second); + assert_eq!( + String::from_utf8(first).unwrap(), + "FRX/0.3\nquery\naaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n1700000000\n00112233445566778899aabbccddeeff\n{\"budget\":{\"max_results\":5},\"entities\":[],\"qid\":\"q1\",\"text\":\"rust\"}" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn query_body_without_entities_parses() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + let _bob = start_node_with_corpus( + root.path(), + "bob", + &relay_url, + true, + EXPOSURE_FULL, + &[("doc.txt", "minimal body rust")], + ) + .await; + let alice = Keypair::generate(); + let http = client(); + register(&http, &relay_url, &alice.public_hex()).await; + let envelope = Envelope::new( + &alice, + TYPE_QUERY, + json!({"qid": "min", "text": "rust", "budget": {"max_results": 5}}), + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + let responses = collect_responses(&http, &relay_url, &alice.public_hex(), 700).await; + assert_eq!(responses.len(), 1); +} + +#[test] +fn query_body_budget_is_the_only_quantity_knob() { + let value = serde_json::to_value(QueryBody::new("rust", 5)).unwrap(); + assert_eq!( + keys(&value), + BTreeSet::from([ + "qid".to_string(), + "text".to_string(), + "entities".to_string(), + "budget".to_string() + ]) + ); +} diff --git a/tests/phase1.rs b/tests/phase1.rs new file mode 100644 index 0000000..a639b0f --- /dev/null +++ b/tests/phase1.rs @@ -0,0 +1,116 @@ +mod common; + +use std::fs; + +use common::{ask, client, collection, config_for, publish, spawn_relay}; +use frxd::index::LocalIndex; +use frxd::message::{EXPOSURE_FULL, Envelope, TYPE_RESPONSE}; +use frxd::node::Node; + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn two_node_broadcast_query_flow() { + let root = tempfile::tempdir().unwrap(); + let relay_url = spawn_relay().await; + + let shared_dir = root.path().join("bob/shared"); + let private_dir = root.path().join("bob/private"); + fs::create_dir_all(&shared_dir).unwrap(); + fs::create_dir_all(&private_dir).unwrap(); + fs::write( + shared_dir.join("alpha.txt"), + "Tantivy provides BM25 relevance scoring for rust search", + ) + .unwrap(); + fs::write( + shared_dir.join("beta.txt"), + "Rust lifetimes and borrowing explained", + ) + .unwrap(); + fs::write(private_dir.join("gamma.txt"), "rust secret launch codes").unwrap(); + + let bob_config = config_for(&root.path().join("bob"), "bob", &relay_url); + { + let index = LocalIndex::open(&bob_config.index_dir()).unwrap(); + index + .add_collection(&collection("shared", &shared_dir, true, EXPOSURE_FULL)) + .unwrap(); + index + .add_collection(&collection("private", &private_dir, false, "metadata")) + .unwrap(); + } + let _bob = Node::start(bob_config).await.unwrap(); + + let alice_dir = root.path().join("alice"); + let alice_local = root.path().join("alice_mine"); + fs::create_dir_all(&alice_local).unwrap(); + fs::write(alice_local.join("notes.txt"), "my rust notes").unwrap(); + let alice_config = config_for(&alice_dir, "alice", &relay_url); + { + let index = LocalIndex::open(&alice_config.index_dir()).unwrap(); + index + .add_collection(&collection("mine", &alice_local, false, "metadata")) + .unwrap(); + } + let alice = Node::start(alice_config).await.unwrap(); + + let http = client(); + let addr = alice.addr.to_string(); + + let (status, raw, value) = ask(&http, &addr, "rust", 1).await; + assert!(status.is_success()); + assert_eq!( + value.pointer("/local/total").and_then(|v| v.as_u64()), + Some(1) + ); + let responses = value.get("responses").and_then(|v| v.as_array()).unwrap(); + assert_eq!(responses.len(), 1, "expected one responder: {value}"); + let response = &responses[0]; + let results = response.get("results").and_then(|v| v.as_array()).unwrap(); + assert_eq!(results.len(), 1); + assert_eq!( + response.get("truncated").and_then(|v| v.as_bool()), + Some(true) + ); + assert_eq!( + response.get("more_available").and_then(|v| v.as_u64()), + Some(1) + ); + assert!(!raw.contains("gamma"), "unshared collection leaked: {raw}"); + assert!(!raw.contains("\"score\""), "response carries scores: {raw}"); + + let (_status, _raw, value) = ask(&http, &addr, "rust", 5).await; + let responses = value.get("responses").and_then(|v| v.as_array()).unwrap(); + assert_eq!(responses.len(), 1); + assert_eq!( + responses[0] + .get("results") + .and_then(|v| v.as_array()) + .unwrap() + .len(), + 2 + ); + assert_eq!( + responses[0].get("truncated").and_then(|v| v.as_bool()), + Some(false) + ); + + let (_status, _raw, value) = ask(&http, &addr, "secret launch codes", 5).await; + let responses = value.get("responses").and_then(|v| v.as_array()).unwrap(); + assert!( + responses.is_empty(), + "private collection was served: {value}" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn relay_rejects_non_query_broadcasts() { + let relay_url = spawn_relay().await; + let key = frxd::crypto::Keypair::generate(); + let envelope = Envelope::new( + &key, + TYPE_RESPONSE, + serde_json::json!({"qid": "x", "results": []}), + ); + let response = publish(&client(), &relay_url, &envelope).await; + assert_eq!(response.status(), reqwest::StatusCode::BAD_REQUEST); +} diff --git a/tests/purges.rs b/tests/purges.rs new file mode 100644 index 0000000..9122c3c --- /dev/null +++ b/tests/purges.rs @@ -0,0 +1,389 @@ +mod common; + +use std::collections::BTreeSet; +use std::fs; + +use common::{client, collection, config_for, poll, publish, spawn_relay, unicast}; +use frxd::crypto::Keypair; +use frxd::index::LocalIndex; +use frxd::message::{ + AggregateBody, Envelope, QueryBody, ResponseItem, TYPE_AGGREGATE, TYPE_QUERY, TYPE_RESPONSE, + build_response, +}; +use frxd::node::{self, Node}; +use serde_json::Value; + +fn assert_exact_keys(value: &Value, expected: &[&str]) { + let actual: BTreeSet<&str> = value + .as_object() + .expect("object") + .keys() + .map(String::as_str) + .collect(); + let expected: BTreeSet<&str> = expected.iter().copied().collect(); + assert_eq!(actual, expected, "fields drifted: {value}"); +} + +fn assert_absent_fields(value: &Value, banned: &[&str]) { + let raw = serde_json::to_string(value).unwrap(); + for field in banned { + assert!( + !raw.contains(&format!("\"{field}\"")), + "purged field '{field}' present: {raw}" + ); + } +} + +#[test] +fn no_bounty_winner_selection_or_slashing() { + let query = serde_json::to_value(QueryBody::new("rust", 5)).unwrap(); + assert_exact_keys(&query, &["qid", "text", "entities", "budget"]); + assert_exact_keys(query.get("budget").unwrap(), &["max_results"]); + let envelope = Envelope::new(&Keypair::generate(), TYPE_QUERY, query.clone()); + let envelope = serde_json::to_value(&envelope).unwrap(); + for value in [&query, &envelope] { + assert_absent_fields( + value, + &[ + "bounty", + "winner", + "winner_selection", + "slashing", + "stake", + "reward", + "payment", + ], + ); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_protocol_query_dedup() { + let relay_url = spawn_relay().await; + let http = client(); + let key = Keypair::generate(); + let envelope = Envelope::new( + &key, + TYPE_QUERY, + serde_json::json!({"qid": "q1", "text": "rust"}), + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + let response = poll(&http, &relay_url, &key.public_hex(), 300).await; + let payload: Value = response.json().await.unwrap(); + assert_eq!( + payload + .get("messages") + .and_then(Value::as_array) + .map(Vec::len), + Some(2), + "relay deduplicated identical queries: {payload}" + ); +} + +#[test] +fn no_k_fetch_ingestion_attestations() { + let response = build_response("q1", Vec::new(), 0, 5); + let value = serde_json::to_value(&response).unwrap(); + assert_exact_keys( + &value, + &["qid", "results", "truncated", "more_available", "cursor"], + ); + assert_absent_fields( + &value, + &[ + "attestation", + "attestations", + "k_fetch", + "fetch_proof", + "ingestion_proof", + "fetch_count", + ], + ); +} + +#[test] +fn no_result_count_etiquette() { + let query = serde_json::to_value(QueryBody::new("rust", 5)).unwrap(); + assert_absent_fields( + &query, + &["min_results", "results_count", "serp", "count_floor"], + ); + let response = serde_json::to_value(build_response("q1", Vec::new(), 0, 5)).unwrap(); + assert_absent_fields( + &response, + &["min_results", "results_count", "serp", "count_floor"], + ); +} + +#[test] +fn no_global_reputation_score() { + let item = ResponseItem { + url: "file:///x".to_string(), + title: "t".to_string(), + summary: String::new(), + published: String::new(), + exposure: "metadata".to_string(), + content: None, + }; + let item_value = serde_json::to_value(&item).unwrap(); + assert_exact_keys( + &item_value, + &[ + "url", + "title", + "summary", + "published", + "exposure", + "content", + ], + ); + let response_value = serde_json::to_value(build_response("q1", vec![item], 1, 5)).unwrap(); + for value in [&item_value, &response_value] { + assert_absent_fields(value, &["score", "rank", "reputation", "weight", "rating"]); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_aggregate_appeals() { + let relay_url = spawn_relay().await; + let http = client(); + let sender = Keypair::generate(); + let member = Keypair::generate(); + + let dispute = Envelope::new( + &sender, + "dispute", + serde_json::json!({"about": "counter", "reason": "unfair"}), + ); + assert_eq!( + publish(&http, &relay_url, &dispute).await.status(), + reqwest::StatusCode::BAD_REQUEST + ); + assert_eq!( + unicast(&http, &relay_url, &member.public_hex(), &dispute) + .await + .status(), + reqwest::StatusCode::BAD_REQUEST + ); + + assert_eq!( + poll(&http, &relay_url, &member.public_hex(), 30) + .await + .status(), + reqwest::StatusCode::NO_CONTENT + ); + let aggregate = Envelope::new( + &sender, + TYPE_AGGREGATE, + serde_json::to_value(AggregateBody { + period: "2026-03".to_string(), + sent: 1, + passed: 1, + cited: 0, + }) + .unwrap(), + ); + assert_eq!( + unicast(&http, &relay_url, &member.public_hex(), &aggregate) + .await + .status(), + reqwest::StatusCode::OK + ); + + let body = serde_json::to_value(AggregateBody { + period: "2026-03".to_string(), + sent: 1, + passed: 1, + cited: 0, + }) + .unwrap(); + assert_exact_keys(&body, &["period", "sent", "passed", "cited"]); + assert_absent_fields(&body, &["appeal", "dispute", "complaint", "sanction"]); + + for route in ["/v1/appeal", "/v1/dispute"] { + let response = http + .get(format!("{relay_url}{route}")) + .send() + .await + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::NOT_FOUND); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_topic_channels() { + let query = serde_json::to_value(QueryBody::new("rust", 5)).unwrap(); + assert_exact_keys(&query, &["qid", "text", "entities", "budget"]); + assert_absent_fields( + &query, + &[ + "topic", "channel", "taxonomy", "category", "routing", "topic_id", + ], + ); + + let relay_url = spawn_relay().await; + let http = client(); + let alice = Keypair::generate(); + let bob = Keypair::generate(); + assert_eq!( + poll(&http, &relay_url, &alice.public_hex(), 30) + .await + .status(), + reqwest::StatusCode::NO_CONTENT + ); + assert_eq!( + poll(&http, &relay_url, &bob.public_hex(), 30) + .await + .status(), + reqwest::StatusCode::NO_CONTENT + ); + let envelope = Envelope::new( + &alice, + TYPE_QUERY, + serde_json::json!({"qid": "q1", "text": "rust"}), + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + for member in [&alice, &bob] { + let response = poll(&http, &relay_url, &member.public_hex(), 300).await; + let payload: Value = response.json().await.unwrap(); + assert_eq!( + payload + .get("messages") + .and_then(Value::as_array) + .map(Vec::len), + Some(1), + "receiver-side filtering expected, got {payload}" + ); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_broadcast_responses() { + let relay_url = spawn_relay().await; + let http = client(); + let key = Keypair::generate(); + for msg_type in [TYPE_RESPONSE, "evidence", "commons", "broadcast_response"] { + let envelope = Envelope::new(&key, msg_type, serde_json::json!({})); + assert_eq!( + publish(&http, &relay_url, &envelope).await.status(), + reqwest::StatusCode::BAD_REQUEST, + "relay accepted broadcast {msg_type}" + ); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_normative_query_canonical_form() { + let root = tempfile::tempdir().unwrap(); + let docs = root.path().join("docs"); + fs::create_dir_all(&docs).unwrap(); + fs::write(docs.join("alpha.txt"), "rust ownership and borrowing").unwrap(); + let config = config_for(&root.path().join("alice"), "alice", "http://127.0.0.1:1"); + { + let index = LocalIndex::open(&config.index_dir()).unwrap(); + index + .add_collection(&collection("docs", &docs, true, "full")) + .unwrap(); + } + let node = Node::start(config).await.unwrap(); + let text = "rust?! ownership/borrowing: \"quotes\" ünïcode {braces} #tag"; + let outcome = node::control_query( + &format!("http://{}", node.addr), + text, + Some(5), + Some(100), + false, + ) + .await + .unwrap(); + assert_eq!(outcome.get("text").and_then(Value::as_str), Some(text)); + assert_eq!( + outcome.pointer("/local/total").and_then(Value::as_u64), + Some(1), + "arbitrary phrasing should match lexically: {outcome}" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_supply_announce_firehose() { + let relay_url = spawn_relay().await; + let http = client(); + let key = Keypair::generate(); + for msg_type in ["announce", "supply", "documents", "publish"] { + let envelope = Envelope::new(&key, msg_type, serde_json::json!({})); + assert_eq!( + publish(&http, &relay_url, &envelope).await.status(), + reqwest::StatusCode::BAD_REQUEST, + "relay accepted supply announcement {msg_type}" + ); + } + for route in ["/v1/announce", "/v1/supply"] { + let response = http + .get(format!("{relay_url}{route}")) + .send() + .await + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::NOT_FOUND); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn no_durable_replayable_broadcast_stream() { + let relay_url = spawn_relay().await; + let http = client(); + let key = Keypair::generate(); + let envelope = Envelope::new( + &key, + TYPE_QUERY, + serde_json::json!({"qid": "q1", "text": "rust"}), + ); + assert!( + publish(&http, &relay_url, &envelope) + .await + .status() + .is_success() + ); + + let response = poll(&http, &relay_url, &key.public_hex(), 300).await; + let payload: Value = response.json().await.unwrap(); + assert_eq!( + payload + .get("messages") + .and_then(Value::as_array) + .map(Vec::len), + Some(1) + ); + + assert_eq!( + poll(&http, &relay_url, &key.public_hex(), 50) + .await + .status(), + reqwest::StatusCode::NO_CONTENT, + "relay replayed a drained message" + ); + + for route in ["/v1/history", "/v1/replay"] { + let response = http + .get(format!("{relay_url}{route}")) + .send() + .await + .unwrap(); + assert_eq!(response.status(), reqwest::StatusCode::NOT_FOUND); + } +} diff --git a/tests/scale.rs b/tests/scale.rs new file mode 100644 index 0000000..eb7af52 --- /dev/null +++ b/tests/scale.rs @@ -0,0 +1,51 @@ +use std::fs; +use std::time::Instant; + +use frxd::index::{Collection, LocalIndex, response_items}; +use frxd::message::{EXPOSURE_FULL, build_response}; + +#[test] +fn thousand_file_corpus_is_searchable_and_honest() { + let temp = tempfile::tempdir().unwrap(); + let docs = temp.path().join("corpus"); + fs::create_dir_all(&docs).unwrap(); + let total = 1000; + for i in 0..total { + let part = docs.join(format!("part{:02}", i / 100)); + fs::create_dir_all(&part).unwrap(); + fs::write( + part.join(format!("doc{i:04}.txt")), + format!("commonneedle unique{i} rust document number {i}"), + ) + .unwrap(); + } + + let index = LocalIndex::open(&temp.path().join("index")).unwrap(); + let start = Instant::now(); + let added = index + .add_collection(&Collection { + name: "corpus".to_string(), + path: docs.display().to_string(), + shared: true, + exposure: EXPOSURE_FULL.to_string(), + }) + .unwrap(); + println!("indexed {added} documents in {:?}", start.elapsed()); + assert_eq!(added, total); + assert_eq!(index.doc_count(), total as u64); + + let start = Instant::now(); + let (hits, matches) = index.search("commonneedle", 10, true).unwrap(); + println!("query matched {matches} documents in {:?}", start.elapsed()); + assert_eq!(matches, total as u64); + assert_eq!(hits.len(), 10); + + let response = build_response("scale", response_items(&hits), matches, 10); + assert!(response.truncated); + assert_eq!(response.more_available, (total - 10) as u64); + assert!(serde_json::to_string(&response).unwrap().len() > 0); + + let (hits, matches) = index.search("unique777", 10, true).unwrap(); + assert_eq!(matches, 1); + assert!(hits[0].summary.contains("unique777")); +}