Testing the Usability of Visual Languages: A Web-Based Methodology (original) (raw)
Related papers
Improving the Design of Visual Programming Language Experiments Using Cognitive Walkthroughs
Journal of Visual Languages & Computing, 2002
Visual programming languages aim to promote usability, but their usability is rarely examined scienti¢cally. One reason is the di⁄culty of designing successful experimental evaluations.We propose the Cognitive Walkthrough (CW) as an aid to improve experiments' designs. The CW is an HCI technique designed for evaluating interfaces. The technique focuses on the potential di⁄culties of novice users and is therefore particularly suited for evaluating programming situations that arise in visual programming language experiments performed in laboratory settings.We ¢rst describe an empirical study performed without bene¢t of a walkthrough and show how the study was improved by a series of walkthroughs.We also describe two other empirical studies and how they were improved with the help of the CW.We found the method to be quick to use, e¡ective at improving the experimental design, and usable by non-specialists.
Usability issues and design principles for visual programming languages
2003
Despite two decades of empirical studies focusing on programmers and the problems with programming, usability of textual programming languages is still hard to achieve. Its younger relation, visual programming languages (VPLs) also share the same problem of poor usability. This research explores and investigates the usability issues relating to VPLs in order to suggest a set of design principles that emphasise usability. The approach adopted focuses on issues arising from the interaction and communication between the human (programmers), the computer (user interface), and the program. Being exploratory in nature, this PhD reviews the literature as a starting point for stimulating and developing research questions and hypotheses that experimental studies were conducted to investigate. However, the literature alone cannot provide a fully comprehensive list of possible usability problems in VPLs so that design principles can be confidently recommended. A commercial VPL was, therefore, holistically evaluated and a comprehensive list of usability problems was obtained from the research. Six empirical studies employing both quantitative and qualitative methodology were undertaken as dictated by the nature of the research. Five of these were controlled experiments and one was qualitative-naturalistic. The experiments studied the effect of a programming paradigm and of representation of program flow on novices' performances. The results indicated superiority of control-flow programs in relation to data-flow programs; a control-flow preference among novices; and in addition that directional representation does not affect performance while traversal direction does-due to cognitive demands imposed upon programmers. Results of the qualitative study included a list of 145 usability problems and these were further categorised into ten problem areas. These findings were integrated with other analytical work based upon the review of the literature in a structured fashion to form a checklist and a set of design principles for VPLs that are empirically grounded and evaluated against existing research in the literature. Furthermore, an extended framework for Cognitive Dimensions of Notations is also discussed and proposed as an evaluation method for diagrammatic VPLs on the basis of the qualitative study. The above consists of the major findings and deliverables of this research. Nevertheless, there are several other findings identified on the basis of the substantial amount of data obtained in the series of experiments carried out, which have made a novel contribution to knowledge in the fields of Human-Computer Interaction, Psychology of Programming, and Visual Programming Languages. 19. Chattratichart, J. & Kuljis, 1. (2000b). A comprehensibility comparison of three visual representations and a textual program in two paradigms. In Proceedings of the Visual End User Workshop (pp. 104-119).
Experience report: Visual programming in the real world
1994
This paper reports direct experience with two commercial, widely used visual programming environments. While neither of these systems is object oriented, the tools have transformed the development process and indicate a direction for visual object oriented tools to proceed. This paper reports on real world applications of visual tools that have exposed a perhaps unexpected effect of the systems development environment: The most dramatic gains in productivity are attributed to the communication among the customer, developer and computer that are facilitated by the visual syntax of the tools. If a similar level of communications support can be achieved in the visual object oriented programming environment, even greater productivity gains then now available can be expected. In our environment of test and measurement, visual programming currently provides productivity improvements of from four to ten times compared to conventional text-based programming.
Using the Cognitive Walkthrough to Improve the Design of a Visual Programming Experiment
Visual Languages/Human-Centric Computing Languages and Environments, 2000
Visual programming languages aim to promote usability, but are rarely examined for it. One reason is the difficulty of designing successful experimental evaluations. We propose the Cognitive Walkthrough as an aid to improve experimental designs. This is a novel application of an HCI-derived technique designed for evaluating interfaces rather than experiments. The technique focuses on the potential difficulties of novice users and is therefore particularly suited for evaluating the programming situation, which is knowledge-based and non-routine. We describe an empirical study performed without benefit of a walkthrough and show how the study was improved by a series of walkthroughs. We found the method to be quick to use, effective at improving the experimental design, and usable by non-specialists.
A Visual Programming Language for Novices
Approved for public release; distribution unlimited. S B U aO 03 12 01A Unc. ass if ied SrCUj"VY CLASSiwFIcON Of 145PG REPORT DOCUMENTATION PAGE Ia. REPORT-SECURITY CASWICATION lb. RESTRICTIVE MARKINGS unclassified 2a SECURtITy CLASSIFICATION AUTHORITY 3. DISTRIBUTION / AVAILAILITY OF REPORT-1Approved for public release; 2b. DECLASSIORWAIONiIDW1OOIIANG SCEDL Dis tribut ion uni i.ni ted 4 PERFORMING ORGANIZATION REPORT NUMBER(S) S. MONITORING ORGANIZATION REPORT NUMIER(S) AIP-22 64. NAME OF PERFORMING ORGANIZATION 6b. OFFICE SYMBOL 7a NAME OF MONITORING ORGANIZATION Car-legie-iM-ellon University (if sabiloei) Computer Sciences Division 3 Office of Naval Research (Code iL13 Sc. ADORESS (City, Stat. and ZIP Code) 7b ADDRESS (City, State. and ZIP Code) D eparti-ent of Ps;.'hoigy 800 N. Quincy Street Pittsburgi, Pennsylvania 13213 rlington, Virginia 22zd7-5000 U4. 14AME OF PFNOINGI SPONSORING 8 b OFFICE SYMBOL 9 PROCUREMENT INSTRUMENT IDENTIFICATION NUMBER ORGANIZATION '9 A8STRACT Continue on reverse it mecessary and identif'y by block number) , We present BridgeTalk,,,a new approach to visual languages for novice programmers. The design of BridgeTalk is based on data about how novices learn to program. BridgeTalk allows novices to program with programming plansframa-like oijects that capture essential program components like Pkeep a running total' and ifterate down a data stri.cture. Novices are focused on the interactions between plans, not on the implementation details for a particular plan. Beginning with plans as a basis for a novice programming language, we were forced to develop a programming formalism that can deal with multiple levels of detail, merged plan implementations, and interrelationships between plans. The actual visual presentation for the language is based on six 4design, implement, test with students, and redesign# cycles. "e , 20 DISTRIBUTION IAVAILAILITY OF ABSTRACT 21 AIsTRACT SECURITY CL.ASSIFICATION O3UNCLASSiFIEO1UNLIMITEO Z SAME AS RPT C O"C USERS 22a NAME OF RESPONSIBLE NOIVIDUAL 22b T'ELEPHONE (Include Ared Code) T22c. OFFICE SYKASO!. ,)r. A-lan L. Meyrowitz (2QGZ' 69t)-.64 I ,U0014 uo FORM 1473. 84 MAR 53 APR Ilidtior Miay 00 YSd ufll Onrldu1tCd SECURITY CL.
Visual programming in a visual domain: a case study of cognitive dimensions
People and Computers, 1994
We present a new visual programming language and environment that serves as a form of feedback and representation in a Programming by Demonstration system. The language di ers from existing visual languages because it explicitly represents data objects and implicitly represents operations by c hanges in data objects. The system was designed to provide non-programmers with programming support for common, repetitive tasks and incorporates some principles of cognition to assist these users in learning to use it. With this in mind, we analyzed the language and its editor along cognitive dimensions. The assessment p r o vided insight into both strengths and weaknesses of the system, prompting a numberofdesign changes. This demonstrates how useful such an analysis can be.
2007 37th annual frontiers in education conference - global engineering: knowledge without borders, opportunities without passports, 2007
Programming has traditionally been taught using a text-based language where the students use an editor to type in language statements. The program is then compiled or interpreted using system commands. In these courses there are programming concepts that we want students to learn, such as declarations, math operations, loops, conditional statements, arrays, file manipulation, functions, etc. This project will attempt to determine the effectiveness of a graphical programming language (icon based) where a computer program is constructed with graphical icons. The test bed for this project will be Engineering H192 (Engineering Fundamentals and Laboratory II) at Ohio State. This course is the second course of a three-course sequence for first-year honors engineering students. The languages for this course are C/C++ and MATLAB. The course is taught to eight sections of 36 students. For a pilot section of H192, the C/C++ problems and the instructor notes will be re-written for the graphical icon development environment. The remaining sections will be the control group. A concept inventory will be completed by all students in the pilot and control sections at the beginning and at the end of the quarter. This work will begin in Winter Quarter 2007 and will be completed at the end of Winter Quarter 2008. This project is being funded by the National Instruments Foundation.
Usability testing of a graphical programming system
Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI '91, 1991
Traditional programming language design has focussed on efficiency and expressiveness, with minimal attention to the ease with which a programmer can translate task requirements into statements in the language, a characteristic we call "facility." The programming walkthrough is a method for assessing the facility of a language design before implementation. We describe the method and its predictions for a graphical programming language, ChemTrains. These predictions are contrasted with protocols of subjects attempting to write their first ChemTrains program. We conclude that the walkthrough is a valuable aid at the design stage, but it is not infallible. Our results also suggest that it may not be enough for programmers to know how to solve a problem; they must also understand why the solution will succeed.
Criteria for Evaluation of Visual Programming Languages
Journal of Visual Languages & Computing, 1997
Interest in visual programming languages has increased as graphic support in hardware and software has made display and manipulation of visual images, icons, diagrams and forms reasonable to consider. In this paper, we present a set of evaluation criteria and associated metrics to judge visual programming languages. The five criteria, visual nature, functionality, ease of comprehension, paradigm support and scalability, are intended to capture the essence of a general purpose visual programming language. These criteria are supplemented with a set of subjective metrics, resulting in an evaluation method that can be used to assess the quality of an individual visual programming language or to compare among elements of a set of such languages.