Finding Needles in Document Haystacks

Augmenting Serendipitous Claim Retrieval Workflows

NEEDLE lets researchers search a large collection of papers one sentence at a time, read each claim in its context, and check which other claims support or contradict it.

The NEEDLE interface, with a corpus overview, a close-reading view of a paper with highlighted claims, a search panel of ranked claims and a canvas where claims are linked

The question

Early in a research project, people skim a field to form and test hunches. With thousands of papers that means reading titles, abstracts and summaries, which hide the specific statements that matter. Reading everything closely does not scale either.

We asked whether search could work on individual claims instead, while keeping each claim tied to the document it came from.

What we built

NEEDLE treats every sentence in a corpus as a claim that can be found, read in context and kept. It supports three ways in, which came out of a year of workshops with medical researchers and, later, political scientists1The two domains read differently. Medical researchers care about outcomes and study design, political analysts about wording and tone, but both struggled with the same early, exploratory phase..

A close-reading view shows each claim inside its paper, a corpus view shows where results sit across documents and years, and a canvas lets people arrange and link claims into an argument.

How well it works

Ten researchers, five from medicine and five from political science, each solved two comparable 15-minute tasks, one with NEEDLE and one with their usual tools. The medical corpus held 3,000 open-access articles, the political one UN General Debate speeches from 2008 to 2022.

With their own tools, most people skimmed titles and abstracts and copied passages into a text editor. With NEEDLE, 9 of 10 started from keyword search and most first collected many claims on the canvas before linking them. Participants singled out the canvas for structuring an argument. Some with prior knowledge of a topic felt abstracts were enough for short tasks, and pointed to NEEDLE for more specific questions.

Read more

The paper covers the requirements, the retrieval pipeline, two use cases and the expert interviews.

Citation

@inproceedings{dueck2025needles,
  title     = {Finding Needles in Document Haystacks: Augmenting Serendipitous Claim Retrieval Workflows},
  author    = {D{\"u}ck, Moritz and Holter, Steffen and Chan, Robin Shing Moon and Sevastjanova, Rita and El-Assady, Mennatallah},
  booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
  series    = {CHI '25},
  year      = {2025},
  publisher = {ACM},
  address   = {Yokohama, Japan},
  doi       = {10.1145/3706598.3713715}
}