Finding Needles in Document Haystacks
Augmenting Serendipitous Claim Retrieval Workflows
NEEDLE lets researchers search a large collection of papers one sentence at a time, read each claim in its context, and check which other claims support or contradict it.
@inproceedings{dueck2025needles,
title = {Finding Needles in Document Haystacks: Augmenting Serendipitous Claim Retrieval Workflows},
author = {D{\"u}ck, Moritz and Holter, Steffen and Chan, Robin Shing Moon and Sevastjanova, Rita and El-Assady, Mennatallah},
booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
series = {CHI '25},
year = {2025},
publisher = {ACM},
address = {Yokohama, Japan},
doi = {10.1145/3706598.3713715}
}
The question
Early in a research project, people skim a field to form and test hunches. With thousands of papers that means reading titles, abstracts and summaries, which hide the specific statements that matter. Reading everything closely does not scale either.
We asked whether search could work on individual claims instead, while keeping each claim tied to the document it came from.
What we built
NEEDLE treats every sentence in a corpus as a claim that can be found, read in context and kept. It supports three ways in, which came out of a year of workshops with medical researchers and, later, political scientists1The two domains read differently. Medical researchers care about outcomes and study design, political analysts about wording and tone, but both struggled with the same early, exploratory phase..
- Keyword search retrieves single claims across the whole corpus, matching entities through domain ontologies rather than exact words.
- Hypothesis search takes a full sentence and returns related claims.
- Consistency checking ranks claims that support or contradict a statement, using natural language inference.
A close-reading view shows each claim inside its paper, a corpus view shows where results sit across documents and years, and a canvas lets people arrange and link claims into an argument.
How well it works
Ten researchers, five from medicine and five from political science, each solved two comparable 15-minute tasks, one with NEEDLE and one with their usual tools. The medical corpus held 3,000 open-access articles, the political one UN General Debate speeches from 2008 to 2022.
With their own tools, most people skimmed titles and abstracts and copied passages into a text editor. With NEEDLE, 9 of 10 started from keyword search and most first collected many claims on the canvas before linking them. Participants singled out the canvas for structuring an argument. Some with prior knowledge of a topic felt abstracts were enough for short tasks, and pointed to NEEDLE for more specific questions.
Read more
The paper covers the requirements, the retrieval pipeline, two use cases and the expert interviews.
Citation
@inproceedings{dueck2025needles,
title = {Finding Needles in Document Haystacks: Augmenting Serendipitous Claim Retrieval Workflows},
author = {D{\"u}ck, Moritz and Holter, Steffen and Chan, Robin Shing Moon and Sevastjanova, Rita and El-Assady, Mennatallah},
booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
series = {CHI '25},
year = {2025},
publisher = {ACM},
address = {Yokohama, Japan},
doi = {10.1145/3706598.3713715}
}