Evaluating user interfaces to information retrieval systems (original) (raw)

Evaluation of user interface designs for information retrieval systems: a computer-based experiment

Decision Support Systems, 1999

In this study, we conducted a computer-based experiment to evaluate and compare the effectiveness of six different interface designs, graphical or list-based, in supporting communication of an object's ''relevance'' from an information Ž. retrieval IR system to its users. We adopted the Model Human Processor to provide a necessary framework to incorporate relevant cognitive psychology theories and user-centered design principles in the development of different interfaces. The study had a well-researched theoretical foundation, complied with relevant design principles, and included a large-scale empirical evaluation. Our results suggest that interface design may have a significant effect on system-user concept communication, regardless of users' familiarity with the search task, and that a graphical user interface may be more effective in supporting such communication than a list-based design. Furthermore, we also examined the cognitive load and user satisfaction resulting from each investigated interface design. Findings of the study have important implications for the Ž. design of IR systems including online library systems and Internet-based search systems as well as for the information representation and visualization of knowledge management systems, which ordinarily depend on text-based display methods to support system-user concept communication.

Strategic help in user interfaces for information retrieval

Journal of the American Society for Information Science and Technology, 2002

Although no unified definition of the concept of search strategy in Information Retrieval (IR) exists so far, its importance is manifest: nonexpert users, directly interacting with an IR system, apply a limited portfolio of simple actions; they do not know how to react in critical situations; and they often do not even realize that their difficulties are due to strategic problems. A user interface to an IR system should therefore provide some strategic help, focusing user's attention on strategic issues and providing tools to generate better strategies. Because neither the user nor the system can autonomously solve the information problem, but they complement each other, we propose a collaborative coaching approach, in which the two partners cooperate: the user retains the control of the session and the system provides suggestions. The effectiveness of the approach is demonstrated by a conceptual analysis, a prototype knowledge-based system named FIRE, and its evaluation through informal laboratory experiments.

A Framework to Evaluate Interface Suitability for a Given Scenario of Textual Information Retrieval

J. Univers. Comput. Sci., 2011

Visualization of search results is an essential step in the textual Information Retrieval (IR) process. Indeed, Information Retrieval Interfaces (IRIs) are used as a link between users and IR systems, a simple example being the ranked list proposed by common search engines. Due to the importance that takes visualization of search results, many interfaces have been proposed in the last decade (which can be textual, 2D or 3D IRIs). Two kinds of evaluation methods have been developed: (1) various evaluation methods of these interfaces were proposed aiming at validating ergonomic and cognitive aspects; (2) various evaluation methods were applied on information retrieval systems (IRS) aiming at measuring their effectiveness. However, as far as we know, these two kinds of evaluation methods are disjoint. Indeed, considering a given IRI associated to a given IRS, what happens if we associate this IRI to another IRS not having the same effectiveness. In this context, we propose an IRI evalu...

A comparison of usability techniques for evaluating information retrieval system interfaces

Performance Measurement and Metrics, 2008

We report on a series of experiments designed to compare usability testing methods in a novel information retrieval interface. The purpose of this ongoing work is to investigate the problems people encounter while performing information retrieval tasks, and to assess evaluation methods by looking at the problem focus, the quality of the results and the cost effectiveness of each method. This ftrst communication compares expett evaluation using heuristics [15] with end user testing [24].

Designing an information retrieval interface based on user characteristics

Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '85, 1985

Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission.

A Survey of Information Retrieval Interfaces

Information Retrieval has become common-place with the advent of the Internet with most Internet users being familiar with the use of search engines such as Google, Yahoo, and the like. Indeed, with the amount of information on the Internet growing so rapidly, search engines are crucial in finding what we want.

Subjective Relevance: Implications on Interface Design for Information Retrieval Systems

Lecture Notes in Computer Science, 2005

Information retrieval (IR) systems are traditionally developed using the objective relevance approach based on the "best match" principle assuming that users can specify their needs in queries and that the documents retrieved are relevant to them. This paper advocates a subjective relevance (SR) approach to value-add objective relevance and address its limitations by considering relevance in terms of users' needs and contexts. A pilot study was conducted to elicit features on SR from experts and novices. Elicited features were then analyzed using characteristics of SR types and stages in information seeking to inform the design of an IR interface supporting SR. The paper presents initial work towards the design and development of user-centered IR systems that prompt features supporting the four main types of SR.

User-Oriented Evaluation Methods for Interactive Web Search Interfaces

2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops, 2007

Although significant efforts have been devoted to the study and evaluation of information retrieval systems from an algorithmic perspective, far less work has been performed on the evaluation of these systems from the user's perspective. This is certainly the case for Web information retrieval, where the major search engines continue to utilise interfaces that have not changed substantially since their introduction. One of the challenges in developing new Web search interfaces is the evaluation of these systems in comparison to one another, as well as in comparison to the popular Web search engines. In this paper, we highlight some of the methods used in the literature for evaluating Web search systems, and present a summary of the methods that we have found to be effective in dealing with the challenges of evaluating intelligent and interactive Web search interfaces.