TRuST targets a specific gap: how well retrieval systems and search-based agents find concise, verifiable evidence on the Russian web where sources are fragmented, inconsistently structured, or archived. The benchmark forces retrieval systems to solve real Runet failure modes rather than just answerability on clean datasets.
What Sets It Apart
- Focused on Russian-language, Runet-specific information-seeking problems rather than general multilingual QA, so models must handle regional sites, transliterations, and local registries.
- Compositional short-answer tasks (324 items) written and verified by human annotators with explicit gold supporting documents, emphasizing retrieval correctness over fluent generation.
- Five retrieval challenge types (multihop, structured evidence, temporal tracking, entity disambiguation, comparative reasoning) emphasize diverse failure modes that break simple single-pass retrieval.
- Provided as a fixed-index evaluation: documents are parsed and embedded (Qwen3-Embedding-8B) into a FAISS-style index so systems are measured on retrieval precision and evidence selection rather than on crawling or indexing variability.
Who It’s For and Trade-offs
Great fit if you need a focused retrieval benchmark for Russian: retrieval researchers, teams building LLM agents operating on Runet, or groups validating dense retrievers and rerankers under realistic noise. Look elsewhere if your goal is large-scale multilingual benchmarking, end-to-end web crawling evaluation, or training datasets—the preview is encrypted and the full evaluation files require repository mapping and decryption, so TRuST is intentionally set up for controlled fixed-index experiments.
Where It Fits
Use TRuST when you want to stress-test retrievers and agent search strategies on regional, temporally sensitive, and structured-evidence tasks; combine it with a downstream judging pipeline (BrowseComp-Plus or adapted harness) to isolate retrieval from generation errors.
