Towards the Layered Evaluation of Interactive Adaptive Systems using ELECTRE TRI Method (original) (raw)
Related papers
Proposed Evaluation Framework for Adaptive Hypermedia Systems
Although a number of frameworks exist for the evaluation of Adaptive Hypermedia Systems (AHS), recently suggested layered frameworks have proved useful in identifying the exact cause of the adaptation failure or any other error in the system. This paper presents an evaluation framework for AHS for internet which is an extension of the layered frameworks and adds new dimensions to them. It treats evaluation as an integral part of development process of AHS and also evaluates the successful access of AHS on the internet. The framework has four dimensions which are orthogonal to each other -Environment -the environment in which AHS is accessed, Adaptation -the type of adaptation used, Development Process -software engineering life cycle steps used for developing AHS and the Evaluation Modules -the layers of AHS which are evaluated in context of other dimensions.
Layered evaluation of interactive adaptive systems: Framework and formative methods
2010
The evaluation of interactive adaptive systems has long been acknowledged to be a complicated and demanding endeavour. Some promising approaches in the recent past have attempted tackling the problem of evaluating adaptivity by "decomposing" and evaluating it in a "piece-wise" manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the "layered" evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature. 123 384 A. Paramythis et al. A. Paramythis et al.
On the Evaluation of Adaptive Web Systems
2004
Adaptive Hypermedia Systems (AHS) affect the way most of the Web-based applications are developed and used. One of the application domains affected by AHS is the domain of Web-based Support Systems (WSS), which is part of the more general online information systems domain. Since the very beginning of the research on AHS, different systems have been developed for different domains and application areas (such as educational systems, online information systems, etc.). Besides some general features, which are expected to be present in any software system, AHSs have their own extra features that make them different from non-adaptive systems. Through a careful study of a large number of adaptive hypermedia systems, a hierarchy of primary features based on which one can evaluate different adaptive web systems is proposed. An evaluation weighting scheme is also proposed for the given features. A comparative analysis is carried out for 4 major adaptive hypermedia systems (AHA!, InterBook, SETA and SeAN). The results of our detailed evaluation put AHA! at the top followed by SETA, SeAN and InterBook, respectively.
EvalCHOICE: A Decision Support Approach for the Usability Evaluation of Interactive Adaptive Systems
Procedia Computer Science, 2017
Successful usability evaluation of interactive adaptive systems requires the use of appropriate methods. Different usability evaluation methods exist in the literature. These methods can be applied in association with the individual layers and with the whole adaptive system. In each situation, it is essential to apply the appropriate evaluation methods. In this paper, we propose a novel decision support approach called EvalCHOICE. It guides evaluators in the choice of appropriate usability evaluation methods for individual layers and the adaptive system as a whole. Our proposal is applied to determine the suitable evaluation methods for a given Web-based adaptive system in the field of transport.
A Decomposition Model for the Layered Evaluation of Interactive Adaptive Systems
Lecture Notes in Computer Science, 2005
A promising approach towards evaluating adaptive systems is to decompose the adaptation process and evaluate the system in a "piece-wise" manner. This paper presents a decomposition model that integrates two previous proposals. The main "stages" identified are: (a) collection of input data, (b) interpretation of the collected data, (c) modeling of the current state of the "world", (d) deciding upon adaptation, and (e) applying adaptation.
UCEAS: User-centred Evaluations of Adaptive Systems
scss.tcd.ie
Adaptive Hypermedia (AH) research is concerned with the dynamic composition and personalisation of hypermedia documents in order to provide more context sensitive retrieval and reuse of digital content. The evaluation of Adaptive Hypermedia Systems (AHS) is difficult due to the complexity of such systems . Several Problems and pitfalls are encountered when evaluating these systems . Very little research has been carried out to address this problem. This PhD work proposes a user-centred approach to the evaluation of the adaptive mechanism of AHS. The proposed approach will be validated using personalised systems developed by the Centre for Next Generation Localisation (CNGL). The framework developed by this research will help to standardise current approaches, offer hints regarding the identification of failures and misconceptions of the adaptive mechanism. It will be applicable to all adaptive systems with no limitations of domain or inference mechanism. A review of approaches, methodologies and techniques adopted by existing systems was conducted and the results analysed. An architectural design of the framework has been designed and currently implementation work is going on.
Utility-Based Evaluation of Adaptive Systems
Theoretical Computer Science, 2003
The variety of user-adaptive hypermedia systems available calls for methods of comparison. Layered evaluation techniques appear to be useful for this purpose. In this paper we present a utility-based evaluation approach that is based on these techniques. Issues that arise when putting utility-based evaluation into practice are dealt with. We also explain the need for interpretative user models and common
Decision Models in the Design of Adaptive Educational Hypermedia Systems
Several efforts have been reported in literature aiming to support the Adaptation Model (AM) design in Adaptive Educational Hypermedia Systems (AEHS) with either guidance for the direct definition of adaptation rules, or semi-automated mechanisms which generate the AM via the implicit definition of such rules. The main drawback of the direct definition of adaptation rules is that there can be cases during the run-time execution of AEHS where no adaptation decision can be made, due to insufficiency and/or inconsistency of the defined adaptation rule sets. The goal of the semi-automated, decision-based approaches is to generate a continuous decision function that estimates the desired AEHS response, aiming to overcome the above mentioned problem. However, such approaches still miss a commonly accepted framework for evaluating their performance. In this chapter, we review the design approaches for the definition of the AM in AEHS and discuss a set of performance evaluation metrics prop...
Adaptive Hypermedia Systems design
2003
Adaptive Hypermedia Systems represent a great potential for e-learning; nevertheless instructors and designers find it difficult to develop adaptive application within their framework. This paper presents a map-based visual design method, developed at the AHS atelier in the University of Lugano and tailored for non-technical people, for designing adaptive courseware according to different instructional strategies.
Adaptive Hypermedia System Design: A Method From Practice
2003
Adaptive Hypermedia Systems represent a great potential for e-learning; nevertheless instructors and designers find it difficult to develop adaptive application within their framework. This paper presents a map-based visual design method, developed at the AHS atelier in the University of Lugano and tailored for non-technical people, for designing adaptive courseware according to different instructional strategies.