Automated gaze-based mind wandering detection during computerized learning in classrooms (original) (raw)
References
Ahmidi, N., Hager, G. D., Ishii, L., Fichtinger, G., Gallia, G. L., & Ishii, M.: Surgical task and skill classification from eye tracking and tool motion in minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J. P. W., Viergever, M. A. (eds.) 13th International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 295–302. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15711-0_37 (2010) Chapter Google Scholar
Allen, I.E., Seaman, J.: Online Report Card: Tracking Online Education in the United States. Babson Survey Research Group, Babson (2016) Google Scholar
Baird, B., Smallwood, J., Lutz, A., Schooler, J.W.: The decoupled mind: mind-wandering disrupts cortical phase-locking to perceptual events. J. Cognit. Neurosci. 26(11), 2596–2607 (2014). https://doi.org/10.1162/jocn_a_00656 Article Google Scholar
Baltrusaitis, T., Robinson, P., Morency, L.P.: OpenFace: an open source facial behavior analysis toolkit. In: 2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016, pp. 1–10. https://doi.org/10.1109/WACV.2016.7477553 (2016)
Barron, E., Riby, L.M., Greer, J., Smallwood, J.: Absorbed in thought: the effect of mind wandering on the processing of relevant and irrelevant events. Psychol. Sci. 22(5), 596–601 (2011). https://doi.org/10.1177/0956797611404083 Article Google Scholar
Bates, A.T.: Technology, E-Learning and Distance Education. Routledge, Abingdon (2005) Book Google Scholar
Berliner, D.C.: What’s all the fuss about instructional time? In: Bromme, R., Ben-Peretz, M. (eds.) The Nature of Time in School. Theoretical Concepts, Practitioners Perceptions, pp. 3–35. Teachers College Press, New York (1990). https://doi.org/10.1108/EUM0000000002517 Chapter Google Scholar
Bixler, R., D’Mello, S.K.: Toward fully automated person-independent detection of mind wandering. In: Dimitrova, V., Kuflik, T., Chin, D., Ricci, F., Dolog, P., Houben, G.-J. (eds.) User Modeling Adaptation and Personalization, pp. 37–48. Springer, Aalborg (2014). https://doi.org/10.1007/978-3-319-08786-3_4 Chapter Google Scholar
Blanchard, N., Bixler, R., Joyce, T., D’Mello, S.K.: Automated physiological based detection of mind wandering during learning. In: Trausan-Matu, S., Boyer, K., Crosby, M., Panourgia, K. (eds.) Intelligent Tutoring Systems, vol. 8474, pp. 55–60. Springer, Cham (2014) Chapter Google Scholar
Bosch, N., D’Mello, S. K.: Automatic detection of mind wandering from video in the lab and in the classroom. IEEE Trans. Affect. Comput. (n.d.) (in review)
Bosch, N., D’Mello, S.K., Baker, R.S.J.D., Ocumpaugh, J., Shute, V., Ventura, M., et al.: Automatic detection of learning-centered affective states in the wild. In: Proceedings of the 20th International Conference on Intelligent User Interfaces, pp. 379–388. ACM, New York. https://doi.org/10.1145/2678025.2701397 (2015)
D’Mello, S.K.: What do we think about when we learn? In: Mills, K.K., Long, D., Magliano, J., Wierner, K. (eds.) Deep comprehension, pp. 52–67. Routledge, Abingdon (2018) Google Scholar
D’Mello, S.K., Hays, P., Williams, C., Cade, W., Brown, J., Olney, A.: Collaborative lecturing by human and computer tutors. In: Aleven, V., Kay, J., Mostow, J. (eds.) Intelligent Tutoring Systems, pp. 178–187. Springer, Berlin, Heidelberg (2010a). https://doi.org/10.1007/978-3-642-13437-1_18 Chapter Google Scholar
Drummond, J., Litman, D.: In the zone: towards detecting student zoning out using supervised machine learning. In: Aleven, V., Kay, J., Mostow, J. (eds.) Intelligent Tutoring Systems, pp. 306–308. Springer, Pittsburgh (2010). https://doi.org/10.1007/978-3-642-13437-1_53 Chapter Google Scholar
Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978) Google Scholar
Forbes-Riley, K., Litman, D.: When does disengagement correlate with learning in spoken dialog computer tutoring? In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) Artificial Intelligence in Education, pp. 81–89. Springer, Auckland (2011). https://doi.org/10.1007/978-3-642-21869-9_13 Chapter Google Scholar
Franklin, M.S., Broadway, J.M., Mrazek, M.D., Smallwood, J., Schooler, J.W.: Window to the wandering mind: pupillometry of spontaneous thought while reading. Q. J. Exp. Psychol. 66(12), 2289–2294 (2013). https://doi.org/10.1080/17470218.2013.858170 Article Google Scholar
Girn, M., Mills, C., Laycock, E., Ellamil, M., Ward, L., Christoff, K.: Neural dynamics of spontaneous thought: an electroencephalographic study. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) Augmented Cognition. Neurocognition and Machine Learning: 11th International Conference. https://doi.org/10.1007/978-3-319-58628-1_3 (2017) Google Scholar
Hild, J., Kühnle, C., & Beyerer, J.: Gaze-based moving target acquisition in real-time full motion video. In: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, pp. 241–244. ACM, New York. https://doi.org/10.1145/2857491.2857525 (2016)
Huey, E.B.: The Psychology and Pedagogy of Reading: With a Review of the History of Reading and Writing and of Methods, Texts, and Hygiene in Reading. The Macmillan company, London (1908) Google Scholar
Hutt, S., Mills, C., White, S., Donnelly, P.J., D’Mello, S.K.: The eyes have it: gaze-based detection of mind wandering during learning with an intelligent tutoring system. In: Barnes, T., Chi, M., Feng, M. (eds.) The 9th International Conference on Educational Data Mining, Raleigh, pp. 86–93, (2016)
Hutt, S., Hardey, J., Bixler, R., Stewart, A., Risko, E., D’Mello, S.K.: Gaze-based detection of mind wandering during lecture viewing. In: Xiangen, H., Tiffany, B., Arnon, H. Paquette, L. (eds.) Proceedings of the 10th International Conference on Educational Data Mining (EDM 2017), pp. 226–231 (2017a)
Hutt, S., Mills, C., Bosch, N., Krasich, K., Brockmole, J., D’Mello, S.K.: “Out of the fr-eye-ing pan”: towards gaze-based models of attention during learning with technology in the classroom. In: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 94–103. ACM, New York. https://doi.org/10.1145/3079628.3079669 (2017b)
Javel, É.: Essai sur la physiologie de la lecture. Annales d’Oculistique 80, 61–73 (1878) Google Scholar
Kam, J.W.Y., Dao, E., Farley, J., Fitzpatrick, K., Smallwood, J., Schooler, J.W., Handy, T.C.: Slow fluctuations in attentional control of sensory cortex. J. Cognit. Neurosci. 23(2), 460–470 (2011). https://doi.org/10.1162/jocn.2010.21443 Article Google Scholar
Krafka, K., Khosla, A., Kellnhofer, P., Kannan, H., Bhandarkar, S., Matusik, W., Torralba, A.: Eye tracking for everyone. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, pp. 2176–2184. https://doi.org/10.1109/CVPR.2016.239 (2016)
Krasich, K., McManus, R., Hutt, S., Faber, M., D’Mello, S.K., Brockmole, J.: Gaze-based signatures of mind wandering during real-world scene processing. J. Exp. Psychol. Gen. 147(8), 1111 (2018). https://doi.org/10.1037/xge0000411 Article Google Scholar
Maurer, B., Krischkowsky, A., & Tscheligi, M.: Exploring gaze and hand gestures for non-verbal in-game communication. In: Extended Abstracts Publication of the Annual Symposium on Computer–Human Interaction in Play, pp. 315–322. ACM. https://doi.org/10.1145/3130859.3131296 (2017)
McVay, J.C., Kane, M.J.: Conducting the train of thought: working memory capacity, goal neglect, and mind wandering in an executive-control task. J. Exp. Psychol. Learn. Mem. Cognit. 35(1), 196–204 (2009). https://doi.org/10.1037/a0014104 Article Google Scholar
McVay, J.C., Kane, M.J.: Does mind wandering reflect executive function or executive failure? Comment on Smallwood and Schooler (2006) and Watkins (2008). Psychol. Bull. 136(2), 188–197 (2010). https://doi.org/10.1037/a0018298 Article Google Scholar
McVay, J.C., Kane, M.J.: Drifting from slow to “D’oh!”: working memory capacity and mind wandering predict extreme reaction times and executive control errors. J. Exp. Psychol. Learn. Mem. Cognit. 38(3), 525–549 (2012). https://doi.org/10.1037/a0025896 Article Google Scholar
McVay, J.C., Kane, M.J., Kwapil, T.R.: Tracking the train of thought from the laboratory into everyday life: an experience-sampling study of mind wandering across controlled and ecological contexts. Psychon. Bull. Rev. 16(5), 857–863 (2009). https://doi.org/10.3758/PBR.16.5.857 Article Google Scholar
Mills, C., D’Mello, S.K. Toward a real-time (day) dreamcatcher: sensor-free detection of mind wandering during online reading. In: Santos, O.C., Boticario, J.G., Romero, C., Pechenizkiy, M., Merceron, A., Mitros, P., et al. (eds.) Proceedings of the 8th International Conference on Educational Data Mining, pp. 69–76 (2015a)
Mills, C., D’Mello, S.K., Bosch, N., Olney, A.M.: Mind wandering during learning with an intelligent tutoring system. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, F.M. (eds.) Artificial Intelligence in Education, pp. 267–276. Springer, Madrid (2015b). https://doi.org/10.1007/978-3-319-19773-9_27 Chapter Google Scholar
Mills, C., Bixler, R., Wang, X., D’Mello, S.K.: Automatic gaze-based detection of mind wandering during film viewing. In: Barnes, T., Chi, M., Feng, M. (eds.) The 9th International Conference on Educational Data Mining, Raleigh, pp. 30–37 (2016)
Navarro, D., Sundstedt, V.: Simplifying game mechanics: gaze as an implicit interaction method. In: SIGGRAPH Asia 2017 Technical Briefs, p. 4. ACM. https://doi.org/10.1145/3145749.3149446 (2017)
Olney, A.M., D’Mello, S.K., Person, N., Cade, W., Hays, P., Williams, C., et al.: Guru: a computer tutor that models expert human tutors. In: Cerri, S., Clancey, W., Papadourakis, G., Panourgia, K. (eds.) Intelligent Tutoring Systems, pp. 256–261. Springer, Chania (2012). https://doi.org/10.1007/978-3-642-30950-2_32 Chapter Google Scholar
Olney, A.M., Risko, E.F., D’Mello, S.K., Graesser, A.C.: Attention in educational contexts: the role of the learning task in guiding attention. In: Fawcett, J., Risko, E.F., Kingstone, A. (eds.) The Handbook of Attention. MIT Press, Cambridge (2015) Google Scholar
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., Hays, J.: WebGazer : scalable webcam eye tracking using user interactions. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 3839–3845 (2016)
Person, N.K., Olney, A., D’Mello, S.K., Lehman, B.: Interactive concept maps and learning outcomes in guru. In: Florida Association for Institutional Research Conference, Marco Island, pp. 456–461 (2012)
Pham, P., Wang, J.: AttentiveLearner: improving mobile MOOC learning via implicit heart rate tracking. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) Artificial Intelligence in Education, pp. 367–376. Springer, Madrid (2015). https://doi.org/10.1007/978-3-319-19773-9_37 Chapter Google Scholar
Randall, J.G., Oswald, F.L., Beier, M.E.: Mind-wandering, cognition, and performance: a theory-driven meta-analysis of attention regulation. Psychol. Bull. 140(6), 1411–1431 (2014). https://doi.org/10.1037/a0037428 Article Google Scholar
Rasor, T., Olney, A., D’Mello, S.: Student speech act classification using machine learning. In: FLAIRS Conference (2011)
Risko, E.F., Anderson, N., Sarwal, A., Engelhardt, M., Kingstone, A.: Everyday attention: variation in mind wandering and memory in a lecture. Appl. Cognit. Psychol. 26(2), 234–242 (2012). https://doi.org/10.1002/acp.1814 Article Google Scholar
Robertson, I.H., Manly, T., Andrade, J., Baddeley, B.T., Yiend, J.: “Oops!”: performance correlates of everyday attentional failures in traumatic brain injured and normal subjects. Neuropsychologia 35(6), 747–758 (1997). https://doi.org/10.1016/S0028-3932(97)00015-8 Article Google Scholar
Schooler, J.W., Reichle, E.D., Halpern, D.V.: Zoning out while reading: evidence for dissociations between experience and metaconsciousness. In: Levin, D.T. (ed.) Thinking and Seeing: Visual Metacognition in Adults and Children, pp. 203–226. MIT Press, Cambridge (2004) Google Scholar
Schooler, J.W., Smallwood, J., Christoff, K., Handy, T.C., Reichle, E.D., Sayette, M.A.: Meta-awareness, perceptual decoupling and the wandering mind. Trends Cognit. Sci. 15(7), 319–326 (2011). https://doi.org/10.1016/j.tics.2011.05.006 Article Google Scholar
Sewell, W., Komogortsev, O.: Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network. In: Proceedings of the 28th of the International Conference Extended Abstracts on Human Factors in Computing Systems—CHI EA’10, p. 3739. https://doi.org/10.1145/1753846.1754048 (2010)
Smallwood, J., Fishman, D.J., Schooler, J.W.: Counting the cost of an absent mind: mind wandering as an underrecognized influence on educational performance. Psychol. Bull. Rev. 14(2), 230–236 (2007). https://doi.org/10.3758/BF03194057 Article Google Scholar
Smallwood, J., Beach, E., Schooler, J.W., Handy, T.C.: Going AWOL in the brain: mind wandering reduces cortical analysis of external events. J. Cognit. Neurosci. 20(3), 458–469 (2008a). https://doi.org/10.1162/jocn.2008.20037 Article Google Scholar
Sottilare, R.A., Graesser, A., Hu, X., Holden, H.: Design Recommendations for Intelligent Tutoring Systems—Volume 1: Learner Modeling, vol. 1. US Army Research Laboratory, Adelphi (2013) Google Scholar
Stawarczyk, D., Majerus, S., Maj, M., Van der Linden, M., D’Argembeau, A.: Mind-wandering: phenomenology and function as assessed with a novel experience sampling method. Acta Physiol. (Oxf) 136(3), 370–381 (2011). https://doi.org/10.1016/j.actpsy.2011.01.002 Article Google Scholar
Stewart, A., Bosch, N., Chen, H., Donnelly, P., D’Mello, S.K.: Face forward: detecting mind wandering from video during narrative film comprehension. In: André, E., Baker, R., Hu, X., Rodrigo, M.M.T., du Boulay, B. (eds.) Artificial Intelligence in Education, pp. 359–370. Springer, Berlin (2017a). https://doi.org/10.1007/978-3-319-61425-0_30 Chapter Google Scholar
Stewart, A., Bosch, N., D’Mello, S.K.: Generalizability of face-based mind wandering detection across task contexts. In: Proceedings of the 10th International Conference on Educational Data Mining (EDM 2017), Wuhan, pp. 88–95 (2017b)
Szpunar, K.K., Khan, N.Y., Schacter, D.L.: Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proc. Natl. Acad. Sci. 110(16), 6313–6317 (2013a). https://doi.org/10.1073/pnas.1221764110 Article Google Scholar
Vosskuhler, A., Nordmeier, V., Kuchinke, L., Jacobs, A.M.: OGAMA (Open Gaze and Mouse Analyzer): open-source software designed to analyze eye and mouse movements in slideshow study designs. Behav. Res. Methods 40(4), 1150–1162 (2008). https://doi.org/10.3758/BRM.40.4.1150 Article Google Scholar
Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E.: Let’s look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 107–114. ACM. https://doi.org/10.1145/2168556.2168573 (2012)
Yarbus, A.L.: Eye Movements and Vision. Plenum Publishing Corp, New York (1967) Book Google Scholar
Yonetani, R., Kawashima, H., Matsuyama, T.: Multi-mode saliency dynamics model for analyzing gaze and attention. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 115–122. ACM, New York. https://doi.org/10.1145/2168556.2168574 (2012)