Effects of Experience on Gaze Behavior during Program Animation (original) (raw)
Gaze Evidence for Different Activities in Program Understanding
2012
We present an empirical study that illustrates the potential of dual eye-tracking to detect successful understanding and social processes during pair-programming. The gaze of forty pairs of programmers was recorded during a program understanding task. An analysis of the gaze transitions between structural elements of the code, declarations of identifiers and expressions shows that pairs with better understanding do less systematic execution of the code and more "tracing" of the data flow by alternating between identifiers and expressions. Interaction consists of moments where partners' attention converges on the same same part of the code and moments where it diverges. Moments of convergence are accompanied by more systematic execution of the code and less transitions among identifiers and expressions.
User-adaptive explanatory program visualization: evaluation and insights from eye movements
User Modeling and User-adapted Interaction, 2010
User-adaptive visualization and explanatory visualization have been suggested to increase educational effectiveness of program visualization. This paper presents an attempt to assess the value of these two approaches. The results of a controlled experiment indicate that explanatory visualization allows students to substantially increase the understanding of a new programming topic. Furthermore, an educational application that features explanatory visualization and employs a user model to track users’ progress allows students to interact with a larger amount of material than an application which does not follow users’ activity. However, no support for the difference in short-term knowledge gain between the two applications is found. Nevertheless, students admit that they prefer the version that estimates and visualizes their progress and adapts the learning content to their level of understanding. They also use the application’s estimation to pace their work. The differences in eye movement patterns between the applications employing adaptive and non-adaptive explanatory visualizations are investigated as well. Gaze-based measures show that adaptive visualization captivates attention more than its non-personalized counterpart and is more interesting to students. Natural language explanations also accumulate a big portion of students’ attention. Furthermore, the results indicate that working memory span can mediate the perception of adaptation. It is possible that user-adaptation in an educational context provides a different service to people with different mental processing capabilities.
2021
The use of eye tracking in the study of program comprehension in software engineering allows researchers to gain a better understanding of the strategies and processes applied by programmers. Despite the large number of eye tracking studies in software engineering, very few datasets are publicly available. The existence of the large Eye Movements in Programming Dataset (EMIP) opens the door for new studies and makes reproducibility of existing research easier. In this paper, a Python library (the EMIP Toolkit) for customized post-processing of the EMIP dataset is presented. The toolkit is specifically designed to make using the EMIP dataset easier and more accessible. It implements features for fixation detection and correction, trial visualization, source code lexical data enrichment, and mapping fixation data over areas of interest. In addition to the toolkit, a filtered token-level dataset with scored recording quality is presented for all Java trials (accounting for 95.8% of the...
Eye Movements in Code Reading: Relaxing the Linear Order
2015 IEEE 23rd International Conference on Program Comprehension, 2015
Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs. Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators.
Software for an eye tracking device enabling analysis of a student’s interaction with program code
2018
Evaluating a student’s programming knowledge is not always precise or objective. Students may give a correct answer to a question without understanding the underlying code. One approach to solving this problem is to track the student’s gaze during the examination by using an eye tracking device. Data acquired from the eye tracker can show if students are actually looking at the code they are explaining by providing the x and y coordinates of their gaze. These coordinates can then be mapped to the code displayed on the screen, allowing the comparison between the code the students were looking at and the code which they were expected to look at, thereby confirming or refuting their knowledge. This paper presents a solution to the aforementioned problem. The solution includes establishing communication with the Gazepoint eye tracking device, using a separately created Java library. Coordinates received from the device are transformed using the iTrace Eclipse plugin, modified to support...
Investigating Eye Movements in Natural Language and C++ Source Code - A Replication Experiment
Lecture Notes in Computer Science
Natural language text and source code are very different in their structure and semantics. Source code uses words from natural language such as English mainly in comments and identifier names. Is there an inherent difference in the way programmers read natural language text compared to source code? Does expertise play a role in the reading behavior of programmers? In order to start answering these questions, we conduct a controlled experiment with novice and non-novice programmers while they read small short snippets of natural language text and C++ source code. This study is a replication of an earlier study by Busjahn et al. [1] but uses C++ instead of Java source code. The study was conducted with 33 students, who were each given ten tasks: a set of seven programs, and three natural language texts. They were asked one of three random comprehension questions after each task. Using several linearity metrics presented in an earlier study [1], we analyze the eye movements on source code and natural language. The results indicate that novices and non-novices both read source code less linearly than natural language text. We did not find any differences between novices and non-novices between natural language text and source code. We compare our results to the Busjahn study and provide directions for future work.
Assisting comprehension of animation programs through interactive code visualization
Journal of Visual Languages & Computing, 2013
Visual languages have been widely used to help people create animation programs. However, current programming environments lack features supporting efficient code exploration and program comprehension, particularly for understanding relationships among parts of animation programs. In this paper, we present novel interactive visualizations aimed at helping people to understand animation programs. We conducted an empirical study to evaluate the impact of these visualizations on programmer comprehension of the code, showing that our approach enabled programmers to comprehend more information with less effort and in less time. This result is potentially significant because it demonstrates an approach for helping users to explore and understand animation code. We anticipate that this approach could be applied in a wide variety of animation programming tools, which could ease common animation programming tasks that require understanding code.