Kelsey Urgo - Academia.edu (original) (raw)

Papers by Kelsey Urgo

arXiv (Cornell University), Aug 15, 2022

Search systems are often used to support learning-oriented goals. This trend has given rise to th... more Search systems are often used to support learning-oriented goals. This trend has given rise to the "search-as-learning" movement, which proposes that search systems should be designed to support learning. To this end, an important research question is: How does a searcher's type of learning objective influence their trajectory (or pathway) towards that objective? We report on a lab study (N = 36) in which participants gathered information to meet a specific type of learning objective. To characterize learning objectives and pathways, we leveraged Anderson and Krathwohl's (A&K's) taxonomy [1]. A&K's taxonomy situates learning objectives at the intersection of two orthogonal dimensions: (1) cognitive process (remember, understand, apply, analyze, evaluate, and create) and (2) knowledge type (factual, conceptual, procedural, and metacognitive knowledge). Participants completed learning-oriented search tasks that varied along three cognitive processes (apply, evaluate, create) and three knowledge types (factual, conceptual, procedural knowledge). A pathway is defined as a sequence of learning instances (e.g., subgoals) that were also each classified into cells from A&K's taxonomy. Our study used a think-aloud protocol, and pathways were generated through a qualitative analysis of participants' think-aloud comments and recorded screen activities. We investigate three research questions. First, in RQ1, we study the impact of the learning objective on pathway characteristics (e.g., pathway length). Second, in RQ2, we study the impact of the learning objective on the types of A&K cells traversed along the pathway. Third, in RQ3, we study common and uncommon transitions between A&K cells along pathways conditioned on the knowledge type of the objective. We discuss implications of our results for designing search systems to support learning.

Information Processing & Management

Proceedings of the Annual Conference of CAIS / Actes du congrès annuel de l'ACSI, 2021

Capturing information behaviours and attitudes that occur in natural settings is a challenge. Obs... more Capturing information behaviours and attitudes that occur in natural settings is a challenge. Observational methods are often intrusive or retrospective proxies, which may change behaviour or misrepresent attitudes. Technology enables novel approaches to in-situ quantitative data collection but rarely explores qualitative reflections; informing researchers on what happened, but not necessarily why. Recent work uses multi-method approaches that combine quantitative data, tracking experiences, feelings, and behaviours over time, with qualitative data to gain deeper insights into subjective experiences. This paper introduces information and library scientists to a multi-method approach to the data collection of subjective experiences over time.

ACM SIGIR Conference on Human Information Interaction and Retrieval, 2022

Modern search systems are largely designed and optimized for simple navigational or fact-finding ... more Modern search systems are largely designed and optimized for simple navigational or fact-finding tasks, with little support for complex tasks involving comprehension and learning. In response, the search-as-learning research community has undertaken a wide range of research questions focused on understanding how various types of learning outcomes are affected by searcher characteristics, the search task, and the search system. Typically, these views embed learning within a search system. In this paper we take a different view, embedding search within a framework for an end-to-end learning system designed to support learning in a formal educational context. Our central goal is to motivate research questions aligned to advance progress on techniques for active support of comprehension and formal learning. Thus we intentionally set aside goals for informal and surface learning. We argue that to be effective, such a search-centric learning system must model four key components: individual students (searcher factors), the educational domain (topic factors), academic assignments (task factors), and progress toward learning goals (the objective function of the end-to-end system). In modeling these components, our hypothetical system makes inferences about students' learning histories, knowledge states, comprehension, and the utilities of different types of information resources. We present examples of possible techniques and data sources for each model. We also introduce the novel concept of leveraging school assignments as rich task context. Our intention is not to propose a functional system, but to frame search-as-learning in the context of comprehension and to inspire research questions arising from an end-to-end view of this important research domain. CCS CONCEPTS • Information systems → Users and interactive retrieval.

In recent years, the "search as learning" community has argued that search systems shou... more In recent years, the "search as learning" community has argued that search systems should be designed to support learning. We report on a lab study in which we manipulated the learning objectives associated with search tasks assigned to participants. We manipulated learning objectives by leveraging Anderson and Krathwohl's taxonomy of learning (A&K's taxonomy) [2], which situates learn-ing objectives at the intersection of two orthogonal dimensions: the cognitive process and the knowledge type dimension. Participants in our study completed tasks with learning objectives that varied across three cognitive processes (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). We focus on the effects of the task's cognitive process and knowledge type on participants' pre-/post-task perceptions and search behaviors. Our results found that the three knowledge types considered in our study had a greater effect than the ...

This is a replication and extension of findings from the SWELL Knowledge Work dataset, files incl... more This is a replication and extension of findings from the SWELL Knowledge Work dataset, files include the replication and extension paper, r code, and relevant data.

ACM Transactions on Information Systems, 2022

Information Processing & Management, 2022

2017 IEEE High Performance Extreme Computing Conference (HPEC), 2017

This paper introduces xDCI, a Data Science Cyber-infrastructure to support research in a number o... more This paper introduces xDCI, a Data Science Cyber-infrastructure to support research in a number of scientific domains including genomics, environmental science, biomedical and health science, and social science. xDCI leverages open-source software packages such as the integrated Rule Oriented Data System and the CyVerse Discovery Environment to address significant challenges in data storage, sharing, analysis and visualization. We provide three example applications to evaluate xDCI for different domains: analysis of 3D images of mice brains, videos analysis of neonatal resuscitation, and risk analytics. Finally, we conclude with a discussion of potential improvements to xDCI.

Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, 2019

Search tasks play an important role in the study and development of interactive information retri... more Search tasks play an important role in the study and development of interactive information retrieval (IIR) systems. Prior work has examined how search tasks vary along dimensions such as the task's main activity, end goal, structure, and complexity. Recently, researchers have been exploring task complexity from the perspective of cognitive complexity-related to the types (and variety) of mental activities required by the task. Anderson & Krathwohl's two-dimensional taxonomy of learning has been a commonly used framework for investigating tasks from the perspective of cognitive complexity [1]. A&K's 2D taxonomy involves a cognitive process dimension and an orthogonal knowledge dimension. Prior IIR research has successfully leveraged the cognitive process dimension of this 2D taxonomy to develop search tasks and investigate their effects on searchers' needs, perceptions, and behaviors. However, the knowledge dimension of the taxonomy has been largely ignored. In this conceptual paper, we argue that future IIR research should consider both dimensions of A&K's taxonomy. Specifically, we discuss related work, present details on both dimensions of A&K's taxonomy, and explain how to use the taxonomy to develop search tasks and learning assessment materials. Additionally, we discuss how considering both dimensions of A&K's taxonomy has important implications for future IIR research.

Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval, 2020

In recent years, the "search as learning" community has argued that search systems should be desi... more In recent years, the "search as learning" community has argued that search systems should be designed to support learning. We report on a lab study in which we manipulated the learning objectives associated with search tasks assigned to participants. We manipulated learning objectives by leveraging Anderson and Krathwohl's taxonomy of learning (A&K's taxonomy) [2], which situates learning objectives at the intersection of two orthogonal dimensions: the cognitive process and the knowledge type dimension. Participants in our study completed tasks with learning objectives that varied across three cognitive processes (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). We focus on the effects of the task's cognitive process and knowledge type on participants' pre-/post-task perceptions and search behaviors. Our results found that the three knowledge types considered in our study had a greater effect than the three cognitive processes. Specifically, conceptual knowledge tasks were perceived to be more difficult and required more search activity. We discuss implications for designing search systems that support learning.

Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, Mar 14, 2020

There is a growing body of research in the Search as Learning community that recognizes the need ... more There is a growing body of research in the Search as Learning community that recognizes the need for users to learn during search, but modern search systems have yet to adapt to support this need. Our research proposes three research goals toward addressing the support of user learning during search. Research goal 1 (RG1) introduces a more precise and reliable metric of assessing user learning. Anderson & Krathwohl's 2-dimensional taxonomy is used as a framework to develop learning objectives and assessment questions to measure user learning during search. Additionally, Anderson & Krathwohl's taxonomy is used as a coding scheme to outline the pathways users traverse along the way to a particular learning objective. Research goal 2 (RG2) investigates the prediction of learning objectives using behavioral measures. Finally, research goal 3 (RG3) proposes a search system that presents information relevant to the user based on their current learning sub-goal and scaffolds information based on the pathways they are likely to traverse given a particular learning objective.