How Search Decides What You See · One-hour workshop · Page 1 of 2
Why did this search miss a paper?
Name (optional): ____________________ Date: ____________________
Prepared teaching scenario. These observations are stipulated for practice, not findings from a test of a named product.
A researcher asks an academic search assistant, “Is there an open access citation advantage?” Its answer cites five papers. The researcher identifies another relevant paper that is not cited. An exact-title search in the service’s ordinary discovery mode finds that paper.
The documentation for this assistant mode says it transforms the question into a lexical query, retrieves up to 30 candidates, reranks them, and supplies up to five to the answer writer. Initially, only the answer and its citations are visible.
1. Your first response · before discussion
Where might the paper have been lost, and what would you try next?
2. What the evidence can establish · with a partner
Separate what was observed from what the documentation describes. Give two plausible explanations. What remains unknown about the two search modes?
3. Choose a check
What additional evidence would distinguish your explanations? State what that evidence could establish.
Continue after the facilitator supplies the saved trace. No live search or subscription is needed.
How Search Decides What You See · One-hour workshop · Page 2 of 2
Revise, explain and apply
4. After the trace is revealed
Record the new evidence. Which boundary excluded the paper? Choose one next action and explain what it would test or accomplish.
5. Explain it to the researcher
Write three to five sentences: what happened, what to try next, and what would remain uncertain even if the paper were recovered.
6. A different case · work individually first
A second assistant’s saved trace shows that a known relevant paper was retrieved, survived reranking, and was included in the text supplied to the answer writer. The final answer does not cite it.
Where would you investigate now, and what evidence would you seek? What does this tell you about the completeness of the overall search?
7. Take it back to work
What would you change in your first response? Name one consultation, class or evaluation where you could use this method.
Adapted from Aaron C. H. Tay, How Search Decides What You See, version 1.2.1. Workshop handout · CC BY 4.0. Source: aarontaycheehsien.github.io/Information-retrieval-crashcourse/teaching-notes.html