Marianne Lykke | Aalborg, Denmark (original) (raw)

Papers by Marianne Lykke

Research paper thumbnail of The Role of Automated Categorisation in e-Government Information Retrieval

Zenodo (CERN European Organization for Nuclear Research), Jul 7, 2013

High-precision search results are essential for supporting e-government employees' information ta... more High-precision search results are essential for supporting e-government employees' information tasks. Prior studies have shown that existing features of e-government retrieval systems need improvement in terms of search facilities, navigation and metadata. This paper investigates how automated categorisation can enhance information organisation and retrieval and presents the results of a realistic evaluation that compared automated categorisation and free text indexing of the government intranet used by Danish tax authorities. Thirty-two individuals participated in the evaluation, which conducted a combination of simulated searches and genuine search tasks. Searching behaviour and search outcome was documented by search logs, relevance assessments, and post search interviews. The evaluation demonstrates a high potential for automated categorisation in a government context. Overall, the categorised organisation generated more reformulations at session level and less success at query level. Session success was found to be fairly even between the two systems. Categorised overviews were useful if the participant did not possess much knowledge of the task at hand. When task knowledge was present, categorisation was used to support the assumptions of a correct search. Participants avoided automated categorisation if high-precision documents were among the top results or if few documents were retrieved. The findings emphasise the importance of simultaneously providing different search options for e-government IR systems, and they reveal that automated categorisation is a valuable candidate for improving search facilities within this domain.

Research paper thumbnail of Simulated work tasks

Proceedings of the 5th Information Interaction in Context Symposium, 2014

ABSTRACT This paper investigates simulated work tasks as a tool for information retrieval (IR) ev... more ABSTRACT This paper investigates simulated work tasks as a tool for information retrieval (IR) evaluation in a work-based, specialized setting. It has been shown that simulated work tasks must be tailored toward the group of study participant to ensure that the depicted situations are realistic and interesting from the participants' point of view [3]. Specifically, we investigate what characterizes an effective simulated work task in a professional government setting and how to design workable tasks for the evaluation of in-house information systems such as a corporate Intranet. The findings reveal that the test participants adopt the tasks. To understand and apply the simulated work tasks, the participants draw on different types of experience and knowledge: Topical, related, structural, and common knowledge. The study also shows that the knowledge types identified are more important for successful retrieval of information than similarity of the simulated work tasks with the participants' daily work tasks.

Research paper thumbnail of Searching for people in the workplace: aims, behaviour, and challenges

Information research, Jun 18, 2024

Introduction. People have been shown to play an important role in solving search tasks in profess... more Introduction. People have been shown to play an important role in solving search tasks in professional settings. Searching for a person covers a large part of the information retrieval carried out in the workplace. However, it can be challenging to identify relevant people through a query. Thus, to support workplace users, knowledge of the characteristics of people searches is needed to identify which steps can be taken to ease the retrieval process. Method. This study examined the characteristics of people searching in a biotechnology company based on a mixed-methods process consisting of search log data from an in-house enterprise system, a questionnaire survey, and semistructured interviews with users of the enterprise system. Results. The findings showed that people search accounted for 59.5% of all queries in the search log. Log data and questionnaire data were analysed to determine how the queries were distributed across organisational areas and information sources. The interview data provided insight into how people search was used for locating people and expertise and as part of exploratory, complex subject searches. Challenges in searching for people were also discussed in the interviews. The findings were used to discuss the implications for future workplace information systems.

Research paper thumbnail of Semantic components enhance retrieval of domain-specific documents

Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007

Susan L. Price Computer Science Dept. Portland State University Portland, OR, USA 1-503-725-2419 ... more Susan L. Price Computer Science Dept. Portland State University Portland, OR, USA 1-503-725-2419 prices@cs.pdx.edu ... Marianne Lykke Nielsen Royal School of Library & Information Sciences Aalborg, Denmark 45-98-15-79-22 mln@db.dk

Research paper thumbnail of Supporting Domain-Specific Digital Libraries in Government: Two Case Studies

Integrated Series In Information Systems, 2008

... deployment and accessibility advantages of a web application using techniques described byGle... more ... deployment and accessibility advantages of a web application using techniques described byGlenwright 11 and ... to an existing vendor of products used by the agency, and (e) hiring a ... of participants in our current project including Vibeke Luk, Peter Vedsted, Jens M. Rubak, and ...

Research paper thumbnail of Indexing and retrieval challenges in digital government systems–summary of an empirical research project

2nd Scandinavian Workshop on …, 2005

Research paper thumbnail of A framework for work task based thesaurus design

Journal of Documentation, 2001

Design and construction of indexing languages require thorough knowledge and understanding of the... more Design and construction of indexing languages require thorough knowledge and understanding of the information environment. This empirical study investigated a mixed set of methods (group interviews, recollection of information needs and word association tests to collect data; content analysis and discourse analysis to analyse data) to evaluate whether these methods collected the data needed for work domain oriented thesaurus design. The findings showed that the study methods together provided the domain knowledge needed to define the role of the thesaurus and design its content and structure. The study was carried out from a person‐insituation perspective. The findings reflected the information environment and made it possible to develop a thesaurus according to the characteristics of the work domain. It seemed more difficult to capture the needs of the individual user and adapt the thesaurus to individual characteristics.

Research paper thumbnail of Design considerations for introducing PBL in computer engineering

2012 XXXVIII Conferencia Latinoamericana En Informatica (CLEI), 2012

Nowadays, universities are driven to adopt pedagogical approaches that enable them to better meet... more Nowadays, universities are driven to adopt pedagogical approaches that enable them to better meet the needs of the new generation of students and industry. This article presents an ongoing experience in innovating the computer engineering curriculum through a problem-based and projectoriented learning approach to develop skills such as problem solving, effective communication, teamwork and lifelong learning, which are increasingly demanded by employers and society. The investigation determined an initial set of design principles that can be used to guide an implementation strategy of the approach. The methodology allows continuous evaluation and refinement of the design. The findings so far point to a number of obstacles, from student, teachers and management, which must be overcome to achieve an effective result.

Research paper thumbnail of Motivating Students Through Positive Learning Experiences: A Comparison of Three Learning Designs for Computer Programming Courses

Based on the assumption that wellbeing influences learning, the aim of this paper is to provide i... more Based on the assumption that wellbeing influences learning, the aim of this paper is to provide insight into students’ emotional responses to different learning designs. By comparing three different learning designs their respective influence on students’ feelings about and motivation for learning is discussed. This experimental, controlled comparison study was conducted in an introductory computer programming course. The three learning designs were: 1. A classical teacher-lead course; 2. A problem based learning (PBL) course; and 3. A PBL course combined with the use of LEGO Mindstorms Robots. Three different studies were used for data collection: 1. A questionnaire survey with 229 students from groups exposed to the three different learning designs; 2. Six qualitative walk-alongs collecting data from these groups by informal interviews and observations; 3. Class room observations. Findings from the three studies were discussed in three focus group interviews with 10 students from...

Research paper thumbnail of Intelligent Behavior-Based Techniques: do they have a role in enterprise search?

The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organization, 2019

Research paper thumbnail of Experiences with Automated Categorization in E-Government Information Retrieval

KNOWLEDGE ORGANIZATION, 2014

Research paper thumbnail of Task-based evaluation of associative thesaurus in real-life environment

Proceedings of the American Society for Information Science and Technology, 2005

The paper concerns the development and design of retrieval thesauri. It explores the possibilitie... more The paper concerns the development and design of retrieval thesauri. It explores the possibilities of improving user interaction by enriching retrieval thesauri by word associations. An experimental retrieval test is carried out in a real-life environment in order to explore and evaluate how associatively derived thesaurus terms affect user interactions and user satisfaction. The study shows that word associations may result in a usable and workable thesaurus. The method provides current, relevant and domain-specific information, making it possible to develop thesauri reflecting the knowledge domain in which they are acting.

Research paper thumbnail of Using semantic components to facilitate access to domain-specific documents in government settings

Proceedings of the 2006 national conference on Digital government research - dg.o '06, 2006

Page 1. Using Semantic Components to Facilitate Access to Domain-Specific Documents in Government... more Page 1. Using Semantic Components to Facilitate Access to Domain-Specific Documents in Government Settings 1Susan Price, 1Lois Delcambre, 2Marianne Lykke Nielsen, 3Timothy Tolle, 4Vibeke Luk, 5Mathew Weaver 1Computer Science Dept. ...

Research paper thumbnail of Panel: Perspectives on adaptivity in information retrieval interaction (PAIRI)

Proceedings of the American Society for Information Science and Technology, 2010

Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users... more Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users adapting to the systems. System adaption entails dynamic user modeling, effective information architecture and enhanced search features such as search integration and relevance feedback; user adaptation through interactions entails mental model building and modification towards a coherent state of knowledge and learning. The panel is structured as follows. Initially we provide an overview of the panel contents, consisting of four central dimensions of adaptivity in IR interaction. These are adaption 1) through integration of information objects; 2) of information system to searcher; 3) of searcher to information system; and 4) to context and practice. The sequence follows the order of the panellists, i.e., each panellist is the prime mover of a particular dimension.

Research paper thumbnail of Automated classification of web pages in hierarchical browsing

Journal of Documentation, 2009

PurposeThe purpose of this study is twofold: to investigate whether it is meaningful to use the E... more PurposeThe purpose of this study is twofold: to investigate whether it is meaningful to use the Engineering Index (Ei) classification scheme for browsing, and then, if proven useful, to investigate the performance of an automated classification algorithm based on the Ei classification scheme.Design/methodology/approachA user study was conducted in which users solved four controlled searching tasks. The users browsed the Ei classification scheme in order to examine the suitability of the classification systems for browsing. The classification algorithm was evaluated by the users who judged the correctness of the automatically assigned classes.FindingsThe study showed that the Ei classification scheme is suited for browsing. Automatically assigned classes were on average partly correct, with some classes working better than others. Success of browsing showed to be correlated and dependent on classification correctness.Research limitations/implicationsFurther research should address pr...

Research paper thumbnail of NSF project: harvesting information to sustain our forests

Proceedings of the …, 2005

Lois Delcambre Computer Science Department Portland State University Portland, OR 97225 USA +1 50... more Lois Delcambre Computer Science Department Portland State University Portland, OR 97225 USA +1 503 725-2405 lmd@cs.pdx.edu ... Mathew Weaver Computer Science & Eng. Department OGI School of Science & Eng. OHSU; Portland, OR 97006 +1 435 245-9354 ...

Research paper thumbnail of The word association method: a gateway to work-task based retrieval

Research paper thumbnail of Perspectives on adaptivity in information retrieval interaction (PAIRI): panel

Adaptivity in IR interactions requires the IR systems adapting to users' situations and the ... more Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users adapting to the systems. System adaption entails dynamic user modeling, effective information architecture and enhanced search features such as search integration and relevance feedback; user adaptation through interactions entails mental model building and modification towards a coherent state of knowledge and learning.

Research paper thumbnail of Professional e-government seeking behavior

Proceedings of the American Society for Information Science and Technology, 2013

The present paper is concerned with professional egovernment seeking behavior. With the digitaliz... more The present paper is concerned with professional egovernment seeking behavior. With the digitalization of governments, expectations have been raised with regard to changes in the composition of employee work tasks. The purpose of our study is to determine whether these changes affect seeking behavior and if so how they change it. We focus on the status of current seeking behavior in a Danish government administration. The results showed that information needs are commonly verificative and consciously topical and that information seeking most often takes the form of mere "look-ups." Employees experience many search problems, such as the lack of specificity, differences in the interpretation of the topics of documents, and unwieldy and irrelevant search results. These problems can be solved by a combination of improved indexing practices and search features.

Research paper thumbnail of Discounted Cumulated Gain Based Evaluation of Multiple-Query IR Sessions

Lecture Notes in Computer Science

IR research has a strong tradition of laboratory evaluation of systems. Such research is based on... more IR research has a strong tradition of laboratory evaluation of systems. Such research is based on test collections, pre-defined test topics, and standard evaluation metrics. While recent research has emphasized the user viewpoint by proposing user-based metrics and non-binary relevance assessments, the methods are insufficient for truly user-based evaluation. The common assumption of a single query per topic and session poorly represents real life. On the other hand, one well-known metric for multiple queries per session, instance recall, does not capture early (within session) retrieval of (highly) relevant documents. We propose an extension to the Discounted Cumulated Gain (DCG) metric, the Session-based DCG (sDCG) metric for evaluation scenarios involving multiple query sessions, graded relevance assessments, and open-ended user effort including decisions to stop searching. The sDCG metric discounts relevant results from later queries within a session. We exemplify the sDCG metric with data from an interactive experiment, discuss how the metric might be applied, and present research questions for which the metric is helpful.

Research paper thumbnail of The Role of Automated Categorisation in e-Government Information Retrieval

Zenodo (CERN European Organization for Nuclear Research), Jul 7, 2013

High-precision search results are essential for supporting e-government employees' information ta... more High-precision search results are essential for supporting e-government employees' information tasks. Prior studies have shown that existing features of e-government retrieval systems need improvement in terms of search facilities, navigation and metadata. This paper investigates how automated categorisation can enhance information organisation and retrieval and presents the results of a realistic evaluation that compared automated categorisation and free text indexing of the government intranet used by Danish tax authorities. Thirty-two individuals participated in the evaluation, which conducted a combination of simulated searches and genuine search tasks. Searching behaviour and search outcome was documented by search logs, relevance assessments, and post search interviews. The evaluation demonstrates a high potential for automated categorisation in a government context. Overall, the categorised organisation generated more reformulations at session level and less success at query level. Session success was found to be fairly even between the two systems. Categorised overviews were useful if the participant did not possess much knowledge of the task at hand. When task knowledge was present, categorisation was used to support the assumptions of a correct search. Participants avoided automated categorisation if high-precision documents were among the top results or if few documents were retrieved. The findings emphasise the importance of simultaneously providing different search options for e-government IR systems, and they reveal that automated categorisation is a valuable candidate for improving search facilities within this domain.

Research paper thumbnail of Simulated work tasks

Proceedings of the 5th Information Interaction in Context Symposium, 2014

ABSTRACT This paper investigates simulated work tasks as a tool for information retrieval (IR) ev... more ABSTRACT This paper investigates simulated work tasks as a tool for information retrieval (IR) evaluation in a work-based, specialized setting. It has been shown that simulated work tasks must be tailored toward the group of study participant to ensure that the depicted situations are realistic and interesting from the participants' point of view [3]. Specifically, we investigate what characterizes an effective simulated work task in a professional government setting and how to design workable tasks for the evaluation of in-house information systems such as a corporate Intranet. The findings reveal that the test participants adopt the tasks. To understand and apply the simulated work tasks, the participants draw on different types of experience and knowledge: Topical, related, structural, and common knowledge. The study also shows that the knowledge types identified are more important for successful retrieval of information than similarity of the simulated work tasks with the participants' daily work tasks.

Research paper thumbnail of Searching for people in the workplace: aims, behaviour, and challenges

Information research, Jun 18, 2024

Introduction. People have been shown to play an important role in solving search tasks in profess... more Introduction. People have been shown to play an important role in solving search tasks in professional settings. Searching for a person covers a large part of the information retrieval carried out in the workplace. However, it can be challenging to identify relevant people through a query. Thus, to support workplace users, knowledge of the characteristics of people searches is needed to identify which steps can be taken to ease the retrieval process. Method. This study examined the characteristics of people searching in a biotechnology company based on a mixed-methods process consisting of search log data from an in-house enterprise system, a questionnaire survey, and semistructured interviews with users of the enterprise system. Results. The findings showed that people search accounted for 59.5% of all queries in the search log. Log data and questionnaire data were analysed to determine how the queries were distributed across organisational areas and information sources. The interview data provided insight into how people search was used for locating people and expertise and as part of exploratory, complex subject searches. Challenges in searching for people were also discussed in the interviews. The findings were used to discuss the implications for future workplace information systems.

Research paper thumbnail of Semantic components enhance retrieval of domain-specific documents

Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007

Susan L. Price Computer Science Dept. Portland State University Portland, OR, USA 1-503-725-2419 ... more Susan L. Price Computer Science Dept. Portland State University Portland, OR, USA 1-503-725-2419 prices@cs.pdx.edu ... Marianne Lykke Nielsen Royal School of Library & Information Sciences Aalborg, Denmark 45-98-15-79-22 mln@db.dk

Research paper thumbnail of Supporting Domain-Specific Digital Libraries in Government: Two Case Studies

Integrated Series In Information Systems, 2008

... deployment and accessibility advantages of a web application using techniques described byGle... more ... deployment and accessibility advantages of a web application using techniques described byGlenwright 11 and ... to an existing vendor of products used by the agency, and (e) hiring a ... of participants in our current project including Vibeke Luk, Peter Vedsted, Jens M. Rubak, and ...

Research paper thumbnail of Indexing and retrieval challenges in digital government systems–summary of an empirical research project

2nd Scandinavian Workshop on …, 2005

Research paper thumbnail of A framework for work task based thesaurus design

Journal of Documentation, 2001

Design and construction of indexing languages require thorough knowledge and understanding of the... more Design and construction of indexing languages require thorough knowledge and understanding of the information environment. This empirical study investigated a mixed set of methods (group interviews, recollection of information needs and word association tests to collect data; content analysis and discourse analysis to analyse data) to evaluate whether these methods collected the data needed for work domain oriented thesaurus design. The findings showed that the study methods together provided the domain knowledge needed to define the role of the thesaurus and design its content and structure. The study was carried out from a person‐insituation perspective. The findings reflected the information environment and made it possible to develop a thesaurus according to the characteristics of the work domain. It seemed more difficult to capture the needs of the individual user and adapt the thesaurus to individual characteristics.

Research paper thumbnail of Design considerations for introducing PBL in computer engineering

2012 XXXVIII Conferencia Latinoamericana En Informatica (CLEI), 2012

Nowadays, universities are driven to adopt pedagogical approaches that enable them to better meet... more Nowadays, universities are driven to adopt pedagogical approaches that enable them to better meet the needs of the new generation of students and industry. This article presents an ongoing experience in innovating the computer engineering curriculum through a problem-based and projectoriented learning approach to develop skills such as problem solving, effective communication, teamwork and lifelong learning, which are increasingly demanded by employers and society. The investigation determined an initial set of design principles that can be used to guide an implementation strategy of the approach. The methodology allows continuous evaluation and refinement of the design. The findings so far point to a number of obstacles, from student, teachers and management, which must be overcome to achieve an effective result.

Research paper thumbnail of Motivating Students Through Positive Learning Experiences: A Comparison of Three Learning Designs for Computer Programming Courses

Based on the assumption that wellbeing influences learning, the aim of this paper is to provide i... more Based on the assumption that wellbeing influences learning, the aim of this paper is to provide insight into students’ emotional responses to different learning designs. By comparing three different learning designs their respective influence on students’ feelings about and motivation for learning is discussed. This experimental, controlled comparison study was conducted in an introductory computer programming course. The three learning designs were: 1. A classical teacher-lead course; 2. A problem based learning (PBL) course; and 3. A PBL course combined with the use of LEGO Mindstorms Robots. Three different studies were used for data collection: 1. A questionnaire survey with 229 students from groups exposed to the three different learning designs; 2. Six qualitative walk-alongs collecting data from these groups by informal interviews and observations; 3. Class room observations. Findings from the three studies were discussed in three focus group interviews with 10 students from...

Research paper thumbnail of Intelligent Behavior-Based Techniques: do they have a role in enterprise search?

The Human Position in an Artificial World: Creativity, Ethics and AI in Knowledge Organization, 2019

Research paper thumbnail of Experiences with Automated Categorization in E-Government Information Retrieval

KNOWLEDGE ORGANIZATION, 2014

Research paper thumbnail of Task-based evaluation of associative thesaurus in real-life environment

Proceedings of the American Society for Information Science and Technology, 2005

The paper concerns the development and design of retrieval thesauri. It explores the possibilitie... more The paper concerns the development and design of retrieval thesauri. It explores the possibilities of improving user interaction by enriching retrieval thesauri by word associations. An experimental retrieval test is carried out in a real-life environment in order to explore and evaluate how associatively derived thesaurus terms affect user interactions and user satisfaction. The study shows that word associations may result in a usable and workable thesaurus. The method provides current, relevant and domain-specific information, making it possible to develop thesauri reflecting the knowledge domain in which they are acting.

Research paper thumbnail of Using semantic components to facilitate access to domain-specific documents in government settings

Proceedings of the 2006 national conference on Digital government research - dg.o '06, 2006

Page 1. Using Semantic Components to Facilitate Access to Domain-Specific Documents in Government... more Page 1. Using Semantic Components to Facilitate Access to Domain-Specific Documents in Government Settings 1Susan Price, 1Lois Delcambre, 2Marianne Lykke Nielsen, 3Timothy Tolle, 4Vibeke Luk, 5Mathew Weaver 1Computer Science Dept. ...

Research paper thumbnail of Panel: Perspectives on adaptivity in information retrieval interaction (PAIRI)

Proceedings of the American Society for Information Science and Technology, 2010

Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users... more Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users adapting to the systems. System adaption entails dynamic user modeling, effective information architecture and enhanced search features such as search integration and relevance feedback; user adaptation through interactions entails mental model building and modification towards a coherent state of knowledge and learning. The panel is structured as follows. Initially we provide an overview of the panel contents, consisting of four central dimensions of adaptivity in IR interaction. These are adaption 1) through integration of information objects; 2) of information system to searcher; 3) of searcher to information system; and 4) to context and practice. The sequence follows the order of the panellists, i.e., each panellist is the prime mover of a particular dimension.

Research paper thumbnail of Automated classification of web pages in hierarchical browsing

Journal of Documentation, 2009

PurposeThe purpose of this study is twofold: to investigate whether it is meaningful to use the E... more PurposeThe purpose of this study is twofold: to investigate whether it is meaningful to use the Engineering Index (Ei) classification scheme for browsing, and then, if proven useful, to investigate the performance of an automated classification algorithm based on the Ei classification scheme.Design/methodology/approachA user study was conducted in which users solved four controlled searching tasks. The users browsed the Ei classification scheme in order to examine the suitability of the classification systems for browsing. The classification algorithm was evaluated by the users who judged the correctness of the automatically assigned classes.FindingsThe study showed that the Ei classification scheme is suited for browsing. Automatically assigned classes were on average partly correct, with some classes working better than others. Success of browsing showed to be correlated and dependent on classification correctness.Research limitations/implicationsFurther research should address pr...

Research paper thumbnail of NSF project: harvesting information to sustain our forests

Proceedings of the …, 2005

Lois Delcambre Computer Science Department Portland State University Portland, OR 97225 USA +1 50... more Lois Delcambre Computer Science Department Portland State University Portland, OR 97225 USA +1 503 725-2405 lmd@cs.pdx.edu ... Mathew Weaver Computer Science & Eng. Department OGI School of Science & Eng. OHSU; Portland, OR 97006 +1 435 245-9354 ...

Research paper thumbnail of The word association method: a gateway to work-task based retrieval

Research paper thumbnail of Perspectives on adaptivity in information retrieval interaction (PAIRI): panel

Adaptivity in IR interactions requires the IR systems adapting to users' situations and the ... more Adaptivity in IR interactions requires the IR systems adapting to users' situations and the users adapting to the systems. System adaption entails dynamic user modeling, effective information architecture and enhanced search features such as search integration and relevance feedback; user adaptation through interactions entails mental model building and modification towards a coherent state of knowledge and learning.

Research paper thumbnail of Professional e-government seeking behavior

Proceedings of the American Society for Information Science and Technology, 2013

The present paper is concerned with professional egovernment seeking behavior. With the digitaliz... more The present paper is concerned with professional egovernment seeking behavior. With the digitalization of governments, expectations have been raised with regard to changes in the composition of employee work tasks. The purpose of our study is to determine whether these changes affect seeking behavior and if so how they change it. We focus on the status of current seeking behavior in a Danish government administration. The results showed that information needs are commonly verificative and consciously topical and that information seeking most often takes the form of mere "look-ups." Employees experience many search problems, such as the lack of specificity, differences in the interpretation of the topics of documents, and unwieldy and irrelevant search results. These problems can be solved by a combination of improved indexing practices and search features.

Research paper thumbnail of Discounted Cumulated Gain Based Evaluation of Multiple-Query IR Sessions

Lecture Notes in Computer Science

IR research has a strong tradition of laboratory evaluation of systems. Such research is based on... more IR research has a strong tradition of laboratory evaluation of systems. Such research is based on test collections, pre-defined test topics, and standard evaluation metrics. While recent research has emphasized the user viewpoint by proposing user-based metrics and non-binary relevance assessments, the methods are insufficient for truly user-based evaluation. The common assumption of a single query per topic and session poorly represents real life. On the other hand, one well-known metric for multiple queries per session, instance recall, does not capture early (within session) retrieval of (highly) relevant documents. We propose an extension to the Discounted Cumulated Gain (DCG) metric, the Session-based DCG (sDCG) metric for evaluation scenarios involving multiple query sessions, graded relevance assessments, and open-ended user effort including decisions to stop searching. The sDCG metric discounts relevant results from later queries within a session. We exemplify the sDCG metric with data from an interactive experiment, discuss how the metric might be applied, and present research questions for which the metric is helpful.