Vitomir Kovanovic | University of South Australia (original) (raw)

Book chapters by Vitomir Kovanovic

Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015

This report forms one part in a series of articles offering an overview of the state of distance,... more This report forms one part in a series of articles offering an overview of the state of distance, online, and blended learning, and positioning them in relation to the emerging domain of digital learning. This particular report focuses on blended learning (BL), referring to the practices that combine (or blend) traditional face-to-face (f2f) learning with online learning (OL). as the concept of BL continues to gain traction in educational settings, researchers are attempting to establish and verify the learning gains it brings. This report seeks to outline the debate regarding BL definitions, pedagogical benefits, and deficiencies that arise in academic studies, and reflect on the future direction for BL. Our critical overview of the state and development of BL is structured to reflect the dominant themes of twenty systematically selected second-order academic studies of BL. This report reviews main findings around such dominant themes as the effectiveness of BL, recommended instructional practices in BL delivery and design, as well as the state of research into BL. The findings suggest that advances in technology have fueled the development of BL from a grassroots practice to an emerging research field. The implementation of BL practices by including both online and f2f modes of delivery positively influence student performance, making BL an attractive educational provision. at present, the field of BL is still dependent on the modes of delivery it is derived from, drawing heavily on OL in both theory and in practice. The field of BL is a dynamically changing area, and much of the critique of the existing research noted here is likely to be rapidly addressed in future work. That being said, a critical overview of the field suggests that it can further mature by adopting a digital learning perspective in its own activities.

Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015

This report is one of a series of reports describing the historical developments and current stat... more This report is one of a series of reports describing the historical developments and current state of distance education, online learning, and blended learning. with the intent of informing future research and practice in the emerging discipline of digital learning, this tertiary study focuses on the history and state of distance education, and the understanding of the large body of empirical research as captured by secondary studies (i.e., meta-analyses and systematic literature reviews). we conducted an automated search for secondary studies in several online digital libraries, and a manual search through Google Scholar and the ten most relevant academic journals. Our search identified 339 secondary studies in the domains of distance education, online learning, and blended learning. of those, 37 secondary studies on distance education research and practice met the selection criteria for final inclusion in our study. Based on the analysis of these secondary sources, three main themes emerged: i) comparison of distance education and traditional classroom instruction, ii) identification of important factors of distance education delivery, and iii) factors of institutional adoption of distance education. our results indicate that distance education, when properly planned, designed, and supported by the appropriate mix of technology and pedagogy, is equivalent to, or in certain scenarios more effective than, traditional face-to-face classroom instruction. This highlights the importance of instructional design and the active role of institutions play in providing support structures for instructors and learners. The implications for future research and practice are discussed.

Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015

This report analyzes findings from research into online learning in order to provide guidelines f... more This report analyzes findings from research into online learning in order to provide guidelines for further research and practice. within this tertiary study, we performed a systematic review of thirty-two second-order studies that address issues of teaching and learning in online settings. From the examination of the studies included in the review, four prominent topics emerged: i) comparison of online learning with the traditional classroom, ii) comparison of various instructional practices within two or more online courses, iii) perspectives of students and instructors regarding learning and teaching in online settings, and iv) adoption of online learning in institutions of higher and adult education. except for showing no significant difference in effectiveness of online learning compared to traditional face-to-face settings, the studies within the first theme also provided directions for further research, necessary to better understand what practices work best in online settings. Our findings further indicate that contemporary research into online learning almost univocally agrees that structured online discussions with clear guidelines and expectations, well-designed courses with interactive content and flexible deadlines, and continuous instructor involvement that includes the provision of individualized, timely, and formative feedback are the most promising approaches to fostering learning in online environments. however, this also implies a more complex role for the instructor in online settings, and a need for research on instructional strategies that would allow for the development of student self-regulatory skills. implications for future research and practice, as well as the position of online learning within the broader aspect of digital learning are further discussed.

Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds) Handbook of Learning Analytics and Educational Data Mining, 2017

The field of learning analytics recently attracted attention from educational practitioners and r... more The field of learning analytics recently attracted attention from educational practitioners and researchers interested in the use of large amounts of learning data for understanding learning process and improving learning and teaching practices. In this chapter, we introduce content analytics – a particular form of learning analytics focused on the analysis of different forms of content related to learning. While several publications provided brief overviews of content analytics, the goal of this chapter is to define content analytics and provide a comprehensive overview of the most important studies in the published literature to date. Given the early stage of the learning analytics field, the focus of this chapter is on the important problems and challenges for which existing content analytics approaches are suitable and have been successfully used in the past. We also reflect on the current trends in content analytics and their position within a broader domain of educational research.

Encyclopedia of Information Science and Technology, Second Edition, 2009

Agent-based systems are one of the most important and exciting areas of research and development ... more Agent-based systems are one of the most important and exciting areas of research and development that emerged in information technology (IT) in the past two decades. In a nutshell, an agent is a computer program that is capable of performing a flexible, autonomous action in typically dynamic and unpredictable domains (Luck, McBurney, Shehory, & Willmott, 2005).

Encyclopedia of Information Communication Technology, 2009

Since the AAAI (http://www.aaai.org) Spring Symposium in 1994, intelligent software agents and ag... more Since the AAAI (http://www.aaai.org) Spring Symposium in 1994, intelligent software agents and agent-based systems became one of the most significant and exciting areas of research and development (R&D) that inspired many scientific and commercial projects. In a nutshell, an agent is a computer program that is capable of performing a flexible, autonomous action in typically dynamic and unpredictable domains (Luck, McBurney, Shehory, & Willmott, 2005). Agents emerged as a response of the IT research community to the new data-processing requirements that traditional computing models and paradigms were increasingly incapable to deal with (e.g., the huge and ever-increasing quantities of available data). Agent-oriented R&D has its roots in different disciplines. Undoubtedly, the main contribution to the field of autonomous agents came from artificial intelligence (AI) which is focused on building intelligent artifacts; and if these artifacts sense and act in some environment, then they can be considered agents (Russell & Norvig, 1995). Also, object-oriented programming (Booch, 2004), concurrent object-based systems (Agha, Wegner, & Yonezawa, 1993), and human-computer interaction (Maes, 1994) are fields that have constantly driven forward the agent R&D in the last few decades.

Journal papers by Vitomir Kovanovic

HERDSA Review of Higher Education, 2019

It has been almost a decade since the emergence of learning analytics, a bricolage field of resea... more It has been almost a decade since the emergence of learning analytics, a bricolage field of research and practice that focuses on understanding and optimising learning and learning environments. Since the initial efforts to make sense of large learning-related datasets, learning analytics has come a long way in developing sophisticated methods for capturing various proxies of learning. Researchers in the field also quickly recognised the necessity to tackle complex and often controversial issues of privacy and ethics when dealing with learner-generated data. Finally, despite huge interests in analytics across various stakeholders – governments, educational institutions, teachers, and learners – learning analytics is still facing many challenges when it comes to broader adoption. This article provides an overview of this journey, critically reflecting on the existing research, providing insights into the recent advances, and discussing the future of the field, positioning learning analytics within the broader agenda of systems thinking as means of advancing its institutional adoption.

IEEE Transactions on Learning Technologies, 2019

 Abstract-Learning in computer-mediated setting represents a complex, multidimensional process. ... more  Abstract-Learning in computer-mediated setting represents a complex, multidimensional process. This complexity calls for a comprehensive analytical approach that would allow for understanding of various dimensions of learner generated discourse and the structure of the underlying social interactions. Current research, however, primarily focuses on manual or, more recently, supervised methods for discourse analysis. Moreover, discourse and social structures are typically analyzed separately without the use of computational methods that can offer a holistic perspective. This paper proposes an approach that addresses these two challenges i) by using an unsupervised machine learning approach to extract speech acts as representations of knowledge construction processes and finds transition probabilities between speech acts across different messages; and ii) by integrating the use of discovered speech acts to explain the formation of social ties and predicting course outcomes. We extracted six categories of speech acts from messages exchanged in discussion forums of two MOOCs and each category corresponded to knowledge construction processes from well-established theoretical models. We further showed how measures derived from discourse analysis explained the ways how social ties were created that framed emerging social networks. Multiple regression models showed that the combined use of measures derived from discourse analysis and social ties predicted learning outcomes.

Learning and Instruction, 2019

Recent developments in educational technologies have provided a viable solution to the challenges... more Recent developments in educational technologies have provided a viable solution to the challenges associated with scaling personalised feedback to students. However, there is currently little empirical evidence about the impact such scaled feedback has on student learning progress and study behaviour. This paper presents the findings of a study that looked at the impact of a learning analytics (LA)-based feedback system on students' self-regulated learning and academic achievement in a large, first-year undergraduate course. Using the COPES model of self-regulated learning (SRL), we analysed the learning operations of students, by way of log data from the learning management system and e-book, as well as the products of SRL, namely, performance on course assessments, from three years of course offerings. The latest course offering involved an intervention condition that made use of an educational technology to provide LA-based process feedback. Propensity score matching was employed to match a control group to the student cohort enrolled in the latest course offering, creating two equal-sized groups of students who received the feedback (the experimental group) and those who did not (the control group). Growth mixture modelling and mixed between-within ANOVA were also employed to identify differences in the patterns of online self-regulated learning operations over the course of the semester. The results showed that the experimental group showed significantly different patterns in their learning operations and performed better in terms of final grades. Moreover, there was no difference in the effect of feedback on final grades among students with different prior academic achievement scores, indicating that the LA-based feedback deployed in this course is able to support students’ learning, regardless of prior academic standing.

The Internet and Higher Education, 2019

This paper examines the discrete learning strategies employed within a massive open online course... more This paper examines the discrete learning strategies employed within a massive open online course and their relationship to the student learning experience. The theoretical framework centered on the Community of Inquiry model of online education, which outlines the three critical dimensions (presences) of student learning experience: teaching, social, and cognitive presence. The Community of Inquiry survey instrument, administered as the part of the post-course survey, was used to measure student perceived levels of the three presences. Cluster analysis revealed three different groups of students with unique study strategies: limited users, selective users, and broad users. The strategies adopted significantly differed in student use of available tools and resources as well as the perceived levels of cognitive presence. The results also indicate there were significant differences regarding student commitment to learning, motivations and goals for enrolling in a MOOC, as well as goal orientation, approaches to learning, and the use of different study strategies. Implications for research and practice of online learning are further discussed.

Distance Education in China, 2017

In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-in-progress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system’s perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications.

Computers & Education, 2018

This study presents an evaluation of the Community of Inquiry (CoI) survey instrument developed b... more This study presents an evaluation of the Community of Inquiry (CoI) survey instrument developed by Arbaugh et al. (2008) within the context of Massive Open Online Courses (MOOCs). The study reports the results of a reliability analysis and exploratory factor analysis of the CoI survey instrument using the data of 1487 students from five MOOC courses. The findings confirmed the reliability and validity of the CoI survey instrument for the assessment of the key dimensions of the CoI model: teaching presence, social presence, and cognitive presence. Although the CoI survey instrument captured the same latent constructs within the MOOC context as in the Garrison's three-factor model (Garrison et al., 1999), analyses suggested a six-factor model with additional three factors as a better fit to the data. These additional factors were 1) course organization and design (a sub-component of teaching presence), 2) group affectivity (a sub-component of social presence), and 3) resolution phase of inquiry learning (a sub-component of cognitive presence). The emergence of these additional factors revealed that the discrepancies between the dynamics of the traditional online courses and MOOCs affect the student perceptions of the three CoI presences. Based on the results of our analysis, we provide an update to the famous CoI model which captures the distinctive characteristics of the CoI model within the MOOC setting. The results of the study and their implications are further discussed.

Online Learning, 2018

Dual-layer MOOCs are an educational framework designed to create customizable modality pathways t... more Dual-layer MOOCs are an educational framework designed to create customizable modality pathways through a learning experience. The basic premise is to design two framework choices through a course - one that is instructor guided and the other that is student-determined and open. Learners have the option to create their own customized pathway by choosing or combining both modalities as they see fit at any given time in the course. This mixed-methods study sought to understand the patterns that learners engaged in during a course designed with this pathway framework. The results of the quantitative examination of the course activity are presented, as well as the categories and themes that arose from the qualitative research. The results of the analysis indicates that learners value the ability to choose the pathway that they engage the course in. Additional research is needed to improve the technical and design aspects of the framework.

Review of Educational Research, 2018

Despite a surge of empirical work on student participation in online learning environments, the c... more Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis of learning-related constructs used in the prediction and measurement of student engagement and learning outcome. Based on our literature review, we identify current gaps in the research, including a lack of solid frameworks to explain learning in open online setting. Finally, we put forward a novel framework suitable for open online contexts based on a well-established model of student engagement. Our model is intended to guide future work studying the association between contextual factors (i.e., demographic, classroom, and individual needs), student engagement (i.e., academic, behavioral, cognitive, and affective engagement metrics), and learning outcomes (i.e., academic, social, and affective). The proposed model affords further interstudy comparisons as well as comparative studies with more traditional education models.

The International Review of Research in Open and Distributed Learning, 2018

The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has freq... more The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has frequently been contested, particularly when learner interactions are limited to MOOC forums. The establishment of social presence—a perceived sense of somebody being present and " real " —is among the strategies to tackle the challenges of online learning and could be applied in MOOCs. Thus far, social presence in MOOCs has been under-researched. Studies that previously examined social presence in MOOCs did not account for the peculiar nature of open online learning. In contrast to the existing work, this study seeks to understand how learners perceive social presence, and the different nuances of social presence in diverse MOOC populations. In particular, we compare perceptions of social presence across the groups of learners with different patterns of forum participation in three edX MOOCs. The findings reveal substantial differences in how learners with varying forum activity perceive social presence. Perceptions of social presence also differed in courses with the varying volume of forum interaction and duration. Finally, learners with sustained forum activity generally reported higher social presence scores that included low affectivity and strong group cohesion perceptions. With this in mind, this study is significant because of the insights into brings to the current body of knowledge around social presence in MOOCs. The study's findings also raise questions about the effectiveness of transferring existing socio-constructivist constructs into the MOOC contexts.

The Internet and Higher Education, 2018

Interactions between learners are fundamental when implementing connectivist pedagogy. Essentiall... more Interactions between learners are fundamental when implementing connectivist pedagogy. Essentially, the establishment of “connections” among students underpins the learning process. These connections can be interpreted as learners’ available pool of social capital, access to which is leveraged through a distributed learning environment such as a MOOC. This study applies linear mixed models to explore the factors associated with social capital for learners in two connectivist MOOCs. Using Coh-Metrix, a computational linguistics tool, we analyzed interactions distributed via Twitter, blogs and Facebook, and examined how learners’ linguistic characteristics are associated with their social capital distributed within a network. Our analyses show that the language used by learners is related to the creation of ties between them. We also observed the role of media, time and learner activity on the development of social capital. The findings suggest that pedagogical considerations are essential to help learners leverage access to potential social capital in a networked learning context.

Learning: Research and Practice, 2017

The field of learning analytics was founded with the goal to harness vast amounts of data about l... more The field of learning analytics was founded with the goal to harness vast amounts of data about learning collected by the extensive use of technology. After the early formation, the field has now entered the next phase of maturation with a growing community who has an evident impact on research, practice, policy, and decision-making. Although learning analytics is a bricolage field borrowing from many related other disciplines, there is still no systematized model that shows how these different disciplines are pieced together. Existing models and frameworks of learning analytics are valuable in identifying elements and processes of learning analytics, but they insufficiently elaborate on the links with foundational disciplines. With this in mind, this paper proposes a consolidated model of the field of research and practice that is composed of three mutually connected dimensions – theory, design, and data science. The paper defines why and how each of the three dimensions along with their mutual relations is critical for research and practice of learning analytics. Finally, the paper stresses the importance of multi-perspective approaches to learning analytics based on its three core dimensions for a healthy development of the field and a sustainable impact on research and practice.

Journal of Educational and Behavioral Statistics, 2016

In recent years, a wide array of tools have emerged for the purposes of conducting educational da... more In recent years, a wide array of tools have emerged for the purposes of conducting educational data mining (EDM) and/or learning analytics (LA) research. In this article, we hope to highlight some of the most widely used, most accessible, and most powerful tools available for the researcher interested in conducting EDM/LA research. We will highlight the utility that these tools have with respect to common data preprocessing and analysis steps in a typical research project as well as more descriptive information such as price point and user-friendliness. We will also highlight niche tools in the field, such as those used for Bayesian knowledge tracing (BKT), data visualization, text analysis, and social network analysis. Finally, we will discuss the importance of familiarizing oneself with multiple tools—a data analysis toolbox—for the practice of EDM/LA research.

Journal of Learning Analytics, 2015

With the widespread adoption of Learning Management Systems (LMS) and other learning technology, ... more With the widespread adoption of Learning Management Systems (LMS) and other learning technology, large amounts of data – commonly known as trace data – are being recorded and are readily accessible to educational researchers. Among different uses of trace data, it has been extensively used to calculate time that students spent on different learning activities – commonly referred to as student time-on-task. Extracted time-on-task measures are then used to build predictive models of student learning in order to understand and improve learning processes. While time-on-task measures have been extensively used in Learning Analytics research, the details of their estimation are rarely described and the consequences that this process entails are not fully examined. This paper presents findings from two experiments that looked at the different time-on-task estimation methods and how they influence the final research findings. Based on modeling different student performance measures with popular statistical methods in two datasets (one online and one blended), our findings indicate that time-on-task estimation methods play an important role in shaping the final study results. This is particularly true for online setting where the amount of interaction with LMS is typically higher. The primary goal of this paper is to raise awareness and initiate a debate on the important issue of time-on-task estimation within a broader learning analytics community. Finally, the paper provides an overview of commonly adopted time-on-task estimation methods in educational and related research fields.

This paper describes a study that looked at the effects of different teaching presence approaches... more This paper describes a study that looked at the effects of different teaching presence approaches in communities of inquiry, and ways in which student–student online discussions with high levels of cognitive presence can be designed. Specifically, this paper proposes that high-levels of cognitive presence can be facilitated in online courses, based on the community of inquiry model, by building upon existing research in i) self-regulated learning through externally-facilitated regulation scaffolding and ii) computer-supported collaborative learning through role assignment. We conducted a quasi-experimental study in a fully-online course (N = 82) using six offerings of the course. After performing a quantitative content analysis of online discussion transcripts, a multilevel linear modeling analysis showed the significant positive effects of both externally-facilitated regulation scaffolding and role assignment on the level of cognitive presence. Specifically, the results showed that externally-facilitated regulation scaffolding had a higher effect on cognitive presence than extrinsically induced motivation through grades. The results showed the effectiveness of role assignment to facilitate a high-level of cognitive presence. More importantly, the results showed a significant effect of the interaction between externally-facilitated regulation scaffolding and role assignment on cognitive presence. The paper concludes with a discussion of practical and theoretical implications.

Siemens, G., Gašević, D., & Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, 2015

Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds) Handbook of Learning Analytics and Educational Data Mining, 2017

Encyclopedia of Information Science and Technology, Second Edition, 2009

Encyclopedia of Information Communication Technology, 2009

HERDSA Review of Higher Education, 2019

IEEE Transactions on Learning Technologies, 2019

Learning and Instruction, 2019

The Internet and Higher Education, 2019

Distance Education in China, 2017

Computers & Education, 2018

Online Learning, 2018

Review of Educational Research, 2018

The International Review of Research in Open and Distributed Learning, 2018

The Internet and Higher Education, 2018

Learning: Research and Practice, 2017

Journal of Educational and Behavioral Statistics, 2016

Journal of Learning Analytics, 2015

Journal of Computer Assisted Learning, 2015

With the steady development of online education and online learning environments, possibilities t... more With the steady development of online education and online learning environments, possibilities to support social interactions between students have advanced significantly. This study examined the relationship between indicators of social presence and academic performance. Social presence is defined as students' ability to engage socially with an online learning community. The results of a multiple regression analysis showed that certain indicators of social presence were significant predictors of final grades in a master's level computer science online course. Moreover, the study also revealed that teaching presence moderated the association between social presence and academic performance, indicating that a course design that increased the level of meaningful interactions between students had a significant impact on the development of social presence, and thus could positively affect students' academic performance. This is especially important in situations when discussions are introduced to promote the development of learning outcomes assessed in courses. Another implication of our results is that indicators of social presence can be used for early detection of students at risk of failing a course. Findings inform research and practice in the emerging field of learning analytics by prompting the opportunities to offer actionable insights into the reasons why certain students are lagging behind.

Computers & Education, 2015

Contemporary literature on online and distance education almost unequivocally argues for the impo... more Contemporary literature on online and distance education almost unequivocally argues for the importance of interactions in online learning settings. Nevertheless, the relationship between different types of interactions and learning outcomes is rather complex. Analyzing 204 offerings of 29 courses, over the period of six years, this study aimed at expanding the current understanding of the nature of this relationship. Specifically, with the use of trace data about interactions and utilizing the multilevel linear mixed modeling techniques, the study examined whether frequency and duration of student–student, student–instructor, student–system, and student–content interactions had an effect of learning outcomes, measured as final course grades. The findings show that the time spent on student–system interactions had a consistent and positive effect on the learning outcome, while the quantity of student–content interactions was negatively associated with the final course grades. The study also showed the importance of the educational level and the context of individual courses for the interaction types supported. Our findings further confirmed the potential of the use of trace data and learning analytics for studying learning and teaching in online settings. However, further research should account for various qualitative aspects of the interactions used while learning, different pedagogical/media features, as well as for the course design and delivery conditions in order to better explain the association between interaction types and the learning achievement. Finally, the results might imply the need for the development of the institutional and program-level strategies for learning and teaching that would promote effective pedagogical approaches to designing and guiding interactions in online and distance learning settings.

The Internet and Higher Education, 2015

This paper describes a study that looked at the effects of different technology-use profiles on e... more This paper describes a study that looked at the effects of different technology-use profiles on educational experience within communities of inquiry, and how they are related to the students' levels of cognitive presence in asynchronous online discussions. Through clustering of students (N = 81) in a graduate distance education engineering course, we identified six different profiles: 1) task-focused users, 2) content-focused no-users, 3) no-users, 4) highly intensive users, 5) content-focused intensive users, and 6) socially-focused intensive users. Identified profiles significantly differ in terms of their use of learning platform and their levels of cognitive presence, with large effect sizes of 0.54 and 0.19 multivariate η2, respectively. Given that several profiles are associated with higher levels of cognitive presence, our results suggest multiple ways for students to be successful within communities of inquiry. Our results also emphasize a need for a different instructional support and pedagogical interventions for different technology-use profiles.

British Journal of Educational Technology, 2015

One of the striking differences between massive open online courses (MOOCs) and previous innovati... more One of the striking differences between massive open online courses (MOOCs) and previous innovations in the education technology field is the unprecedented interest and involvement of the general public. As MOOCs address pressing problems in higher education and the broader educational practice, awareness of the general public debate around MOOCs is essential. Understanding the public discourse around MOOCs can provide insights into important social and public problems, thus enabling the MOOC research community to better focus their research endeavors. While there have been some reports looking at the state of the MOOC-related research, the analysis of the public debate surrounding MOOCs is still largely missing.

In this paper, we present the results of a study that looked at the content of the public discourse related to MOOCs. We identified the most important themes and topics in MOOC-related mainstream news reports. Our results indicate that coverage of MOOCs in public media is rapidly decreasing: by the middle of 2014, it decreased by almost 50% from the highest activity during 2013. In addition, the focus of those discussions is also changing. While the majority of discussions during 2012 and 2013 were focused on MOOC providers, the announcements of their partnerships, and million dollar investments, the current focus of MOOC discourse seems to be moving toward more productive topics focused on the overall position of MOOCs in the global educational landscape. Among different topics that this study discovered, government-related issues and the use of data and analytics are some of the topics that seem to be growing in popularity during the first half of 2014.

The International Review of Research in Open and Distributed Learning, 2015

Distributed Massive Open Online Courses (MOOCs) are based on the premise that online learning occ... more Distributed Massive Open Online Courses (MOOCs) are based on the premise that online learning occurs through a network of interconnected learners. The teachers’ role in distributed courses extends to forming such a network by facilitating communication that connects learners and their separate personal learning environments scattered around the Internet. The study reported in this paper examined who fulfilled such an influential role in a particular distributed MOOC – a connectivist course (cMOOC) offered in 2011. Social network analysis was conducted over a socio-technical network of the Twitter-based course interactions, comprising both human course participants and hashtags; where the latter represented technological affordances for scaling course communication. The results of the week-by-week analysis of the network of interactions suggest that the teaching function becomes distributed among influential actors in the network. As the course progressed, both human and technological actors comprising the network subsumed the teaching functions, and exerted influence over the network formation. Regardless, the official course facilitators preserved a high level of influence over the flow of information in the investigated cMOOC.

Journal of Learning Analytics, 2014

This paper describes doctoral research that focuses on the development of a learning analytics fr... more This paper describes doctoral research that focuses on the development of a learning analytics framework for inquiry-based digital learning. Building on the Community of Inquiry model (CoI) — a foundation commonly used in the research and practice of digital learning and teaching — this research builds on the existing body of knowledge in two important ways. First, given that the CoI model requires substantial manual coding of student discourse, its potential for guiding pedagogical interventions are limited. Thus, the first contribution is the development of a learning analytics system that automates this coding process by means of a novel text classification algorithm that takes into the account the process nature of inquiry-based learning and the specifics of communication through asynchronous discussions. Furthermore, it is equally important to investigate how learning processes unfold over time through student interactions with information, technology, and other course participants. With this in mind, the second contribution of this research focuses on the development of analytical models that provide insight into these important aspects of inquiry-based learning.

Proceedings of the 19th IEEE International Conference on Advanced Learning Technologies (ICALT'19), 2019

This paper presents an approach to studying relationships between students' social presence and c... more This paper presents an approach to studying relationships between students' social presence and course topics from transcripts of asynchronous discussions in online learning environments. Specifically, the paper uses topic modeling and epistemic network analysis to investigate how students' social presence is expressed across different course topics. Finally, we show how this method can be adopted to examine how students' social presence changed due to an instructional intervention. The results of this study and its implications are further discussed.

Proceedings of the First International Conference for Quantitative Ethnography (ICQE'19), 2019

The knowledge required by teachers has long been a focus of public and academic attention. Follow... more The knowledge required by teachers has long been a focus of public and academic attention. Following a period of intense research interest in teachers’ knowledge in the 1980s and 1990s, many researchers have adopted Shulman’s suggestion that expert teaching practice is based on seven forms of knowledge which collectively are referred to as a knowledge base for teaching. Shulman’s work also offered a decision-making framework known as pedagogical reasoning and action, which allows teachers to use their seven forms of knowledge to make effective pedagogical decisions. Despite the widespread acceptance of these ideas, no empirical evidence exploring the connections between knowledge and decision-making is evident in the research literature. This paper reports on a pilot study in which the connections between knowledge and decisions in science, mathematics and information technology teachers’ lesson plans are quantified and represented using epistemic network analysis. Findings reveal and levels of complexity that have been intimated but, until now, not supported with empirical evidence.

Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 2019

Student engagement is often considered an overarching construct in educational research and pract... more Student engagement is often considered an overarching construct in educational research and practice. Though frequently employed in the learning analytics literature, engagement has been subjected to a variety of interpretations and there is little consensus regarding the very definition of the construct. This raises grave concerns with regards to construct validity: namely, do these varied metrics measure the same thing? To address such concerns, this paper proposes, quantifies, and validates a model of engagement which is both grounded in the theoretical literature and described by common metrics drawn from the field of learning analytics. To identify a latent variable structure in our data we used exploratory factor analysis and validated the derived model on a separate sub-sample of our data using confirmatory factor analysis. To analyze the associations between our latent variables and student outcomes, a structural equation model was fitted, and the validity of this model across different course settings was assessed using MIMIC modeling. Across different domains, the broad consistency of our model with the theoretical literature suggest a mechanism that may be used to inform both interventions and course design.

Artificial Intelligence in Education, 2018

This paper presents a novel method for the evaluation of students’ use of asynchronous discussion... more This paper presents a novel method for the evaluation of students’ use of asynchronous discussions in online learning environments. In particular, the paper shows how students’ cognitive development across different course topics can be examined using the combination of natural language processing and graph-based analysis techniques. Drawing on the theoretical foundation of the community of inquiry model, we show how topic modeling and epistemic network analysis can provide qualitatively new insight into students’ development of critical and deep thinking skills. We also show how the same method can be used to investigate the effectiveness of instructional interventions and its effect on student learning. The results of this study and its practical implications are further discussed.

Proceedings ASCILITE 2018: 35th International Conference on Innovation, Practice and Research in the Use of Educational Technologies in Tertiary Education, 2018

Although self-regulated learning (SRL) is essential part of learning, students often commence stu... more Although self-regulated learning (SRL) is essential part of learning, students often commence studies with poor SRL skills. This places much emphasis on course design to foster SRL. In online education, this is a complex undertaking. The present study examines how online technologies can be harnessed to promote SRL. This study of an online first year course (N=138) investigates how student use of a video annotation tool incorporating in-video quizzes can predict learning outcomes and foster SRL. The study found that students were more likely to complete the in-quiz self-assessment questions than contribute to socially-shared resources such as annotations or summaries. This finding may be a result of the higher cognitive load associated with writing tasks versus responses to in-video questions. The findings also revealed a strong positive association (R 2 =0.45) between student completion of the in-video quizzes and course grade. It is not surprising that quiz attempts reflect performance. However, it is important to consider the interaction between the correct and incorrect responses. Above a certain threshold of positive answers, the association between incorrect in-video quiz submissions and final grade becomes negative. The study has implications on how analytics are interpreted and how instructors can frame feedback to foster SRL skills.

Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK'18), 2018

Reflective writing has been widely recognized as one of the most effective activities for fosteri... more Reflective writing has been widely recognized as one of the most effective activities for fostering students' reflective and critical thinking. The analysis of students' reflective writings has been the focus of many research studies. However, to date this has been typically a very labor-intensive manual process involving content analysis of student writings. With recent advancements in the field of learning analytics, there have been several attempts to use text analytics to examine student reflective writings. This paper presents the results of a study examining the use of theoretically-sound linguistic indicators of different psychological processes for the development of an analytics system for assessment of reflective writing. More precisely, we developed a random-forest classification system using linguistic indicators provided by the LIWC and Coh-Metrix tools. We also examined what particular indicators are representative of the different types of student reflective writings.

Lifelong Technology-Enhanced Learning, 2018

This paper presents a method for automated content analysis of students’ messages in asynchronous... more This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s κκ\kappa of .72, which represents a “substantial” agreement, and it is above the Cohen’s κκ\kappa threshold of .70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

Proceedings of the Seventh National Conference on Work-integrated Learning (ACEN’18), 2018

Among different approaches for increasing students' workplace readiness, Work-integrated Learning... more Among different approaches for increasing students' workplace readiness, Work-integrated Learning received significant attention by both industry and academia. However, despite its benefits, recent reports highlight many obstacles and challenges associated with the adoption of Work-integrated learning, which negatively affect students' skills development. However, increased availability of educational data and recent developments within Learning Analytics field highlighted the potentials of using analytical methods and collected data to improve Work-integrated learning adoption. In this paper, we briefly review Learning Analytics and ways in which it can be used to support Work-integrated learning adoption and practices.

Proceedings of the 26th International Conference on World Wide Web Companion

Online learning has become very popular over the last decade. However, there are still many detai... more Online learning has become very popular over the last decade. However, there are still many details that remain unknown about the strategies that students follow while studying online. In this study, we focus on the direction of detecting 'invisible' collaboration ties between students in online learning environments. Specifically, the paper presents a method developed to detect student ties based on temporal proximity of their assignment submissions. The paper reports on findings of a study that made use of the proposed method to investigate the presence of close submitters in two different massive open online courses. The results show that most of the students (i.e., student user accounts) were grouped as couples, though some bigger communities were also detected. The study also compared the population detected by the algorithm with the rest of user accounts and found that close submitters needed a statistically significant lower amount of activity with the platform to achieve a certificate of completion in a MOOC. These results confirm that the detected close submitters were performing some collaboration or even engaged in unethical behaviors, which facilitates their way into a certificate. However, more work is required in the future to specify various strategies adopted by close submitters and possible associations between the user accounts.

Proceedings of the Seventh International Conference on Learning Analytics and Knowledge (LAK'17), 2017

In 2011, the phenomenon of MOOCs had swept the world of education and put online education in the... more In 2011, the phenomenon of MOOCs had swept the world of education and put online education in the focus of the public discourse around the world. Although researchers were excited with the vast amounts of MOOC data being collected, the benefits of this data did not stand to the expectations due to several challenges. The analyses of MOOC data are very time-consuming and labor-intensive, and require a highly advanced set of technical skills, often not available to the education researchers. Because of this MOOC data analyses are rarely done before the courses end, limiting the potential of data to impact the student learning outcomes and experience. In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-in-progress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system’s perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications.

Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK'16), 2016

In this paper, we present the results of an exploratory study that examined the problem of automa... more In this paper, we present the results of an exploratory study that examined the problem of automating content analysis of student online discussion transcripts. We looked at the problem of coding discussion transcripts for the levels of cognitive presence, one of the three main constructs in the Community of Inquiry (CoI) model of distance education. Using Coh-Metrix and LIWC features, together with a set of custom features developed to capture discussion context, we developed a random forest classification system that achieved 70.3% classification accuracy and 0.63 Cohen’s kappa, which is significantly higher than values reported in the previous studies. Besides improvement in classification accuracy, the developed system is also less sensitive to overfitting as it uses only 205 classification features, which is around 100 times less features than in similar systems based on bag-of-words features. We also provide an overview of the classification features most indicative of the different phases of cognitive presence that gives an additional insights into the nature of cognitive presence learning cycle. Overall, our results show great potential of the proposed approach, with an added benefit of providing further characterization of the cognitive presence coding scheme.

Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK'16), 2016

As the field of learning analytics continues to mature, there is a corresponding evolution and so... more As the field of learning analytics continues to mature, there is a corresponding evolution and sophistication of the associated analytical methods and techniques. In this regard social network analysis (SNA) has emerged as one of the cornerstones of learning analytics methodologies. However, despite the noted importance of social networks for facilitating the learning process, it remains unclear how and to what extent such network measures are associated with specific learning outcomes. Motivated by Simmel’s theory of social interactions and building on the argument that social centrality does not always imply benefits, this study aimed to further contribute to the understanding of the association between students’ social centrality and their academic performance. The study reveals that learning analytics research drawing on SNA should incorporate both – descriptive and statistical methods to provide a more comprehensive and holistic understanding of a students’ network position. In so doing researchers can undertake more nuanced and contextually salient inferences about learning in network settings. Specifically, we show how differences in the factors framing students’ interactions within two instances of a MOOC affect the association between the three social network centrality measures (i.e., degree, closeness, and betweenness) and the final course outcome.

Proceedings of the 2015 HOME Conference, 2015

Recent development of Massive Open Online Courses (MOOcs) commenced unprescendented interest of t... more Recent development of Massive Open Online Courses (MOOcs) commenced unprescendented interest of the general public. To leverage from attention given to MOOCs, understanding of public discourse is essential, as it can give critical insights into the important domains of biggest societal interests. Prevoius research showed the great need for understanding specifics of MOOC adoption around the world and the necessity to better cater to the needs of different markets. With this in mind, this paper presents study that looked specificially at the Europe-related MOOC discourse between 2008 and 2015. We identified important themes in the MOOC public discourse and evaluated their changes over time. Further implications of our findings are also discussed.

Information Retrieval Technology, Lecture Notes in Computer Science, 2015

Within online learning communities, receiving timely and meaningful insights into the quality of ... more Within online learning communities, receiving timely and meaningful insights into the quality of learning activities is an important part of an effective educational experience. Commonly adopted methods – such as the Community of Inquiry framework – rely on manual coding of online discussion transcripts, which is a costly and time consuming process. There are several efforts underway to enable the automated classification of online discussion messages using supervised ma- chine learning, which would enable the real-time analysis of interactions occurring within online learning communities. This paper investigates the importance of incorporating features that utilise the structure of on- line discussions for the classification of “cognitive presence” – the central dimension of the Community of Inquiry framework focusing on the quality of students’ critical thinking within online learning communities. We implemented a Conditional Random Field classification solution, which incorporates structural features that may be useful in increasing classification performance over other implementations. Our approach leads to an improvement in classification accuracy of 5.8% over current existing techniques when tested on the same dataset, with a precision and recall of 0.630 and 0.504 respectively.

Proceedings of 2015 HERDSA conference, 2015

Higher Education Institutions are increasingly called upon to provide more flexible student learn... more Higher Education Institutions are increasingly called upon to provide more flexible student learning pathways – in degree programs as well as through the introduction of methods for micro-credentialing. However, the rhetoric of establishing such open and personalised learning pathways is far easier than the reality of implementation and organisational change. For instance, universities have long struggled to break away from the “credit hour” even while learners are being challenged to be more independent in their learning choices and education needs. The intent of this paper is to explore new models of education that embrace open learning pathways for lifelong learning and productive participation in the information age. The paper draws on the recent research and experiences gained from running a simultaneous xMOOC and cMOOC (a dual layer MOOC) using newly developed software based on the principles of student self-regulated learning. The software, ProSOLO, links a user’s nominated learning goals and experiences directly with their achievement of stated competencies. This process provides learners with greater autonomy in their study by removing the rigidity of more traditional education programs and offers new models of micro-credentialing.

There is an emerging trend in higher education for the adoption of massive open online courses (M... more There is an emerging trend in higher education for the adoption of massive open online courses (MOOCs). However, despite this interest in learning at scale, there has been limited work investigating the impact MOOCs can play on student learning. In this study, we adopt a novel approach, using language and discourse as a tool to explore its association with two established measures of learning: traditional academic performance and social centrality. We demonstrate how characteristics of language diagnostically reveal the performance and social position of learners as they interact in a MOOC. We use Coh-Metrix, a theoretically grounded, computational linguistic modeling tool, to explore students’ forum postings across five potent discourse dimensions. Using a Social Network Analysis (SNA) methodology, we determine learners’ social centrality. Linear mixed-effect modeling is used for all other analyses to control for individual learner and text characteristics. The results indicate that learners performed significantly better when they engaged in more expository style discourse, with surface and deep level cohesive integration, abstract language, and simple syntactic structures. However, measures of social centrality revealed a different picture. Learners garnered a more significant and central position in their social network when they engaged with more narrative style discourse with less overlap between words and ideas, simpler syntactic structures and abstract words. Implications for further research and practice are discussed regarding the misalignment between these two learning-related outcomes.

Proceedings of the Fifth International Conference on Learning Analytics & Knowledge (LAK'15), 2015

Connections established between learners via interactions are seen as fundamental for connectivis... more Connections established between learners via interactions are seen as fundamental for connectivist pedagogy. Connections can also be viewed as learning outcomes, i.e. learners’ social capital accumulated through distributed learning environments. We applied linear mixed effects modeling to investigate whether the social capital accumulation interpreted through learners’ centrality to course interaction networks, is influenced by the language learners use to express and communicate in two connectivist MOOCs. Interactions were distributed across the three social media, namely Twitter, blog and Facebook. Results showed that learners in a cMOOC connect easier with the individuals who use a more informal, narrative style, but still maintain a deeper cohesive structure to their communication.

Proceedings of the Fifth International Conference on Learning Analytics & Knowledge (LAK'15), 2015

Creating meaning from a wide variety of available information and being able to choose what to le... more Creating meaning from a wide variety of available information and being able to choose what to learn are highly relevant skills for learning in a connectivist setting. In this work, various approaches have been utilized to gain insights into learning processes occurring within a network of learners and understand the factors that shape learners’ interests and the topics to which learners devote a significant attention. This study combines different methods to develop a scalable analytic approach for a comprehensive analysis of learners’ discourse in a connectivist massive open online course (cMOOC). By linking techniques for semantic annotation and graph analysis with a qualitative analysis of learner-generated discourse, we examined how social media platforms (blogs, Twitter, and Facebook) and course recommendations influence content creation and topics discussed within a cMOOC. Our findings indicate that learners tend to focus on several prominent topics that emerge very quickly in the course. They maintain that focus, with some exceptions, throughout the course, regardless of readings suggested by the instructor. Moreover, the topics discussed across different social media differ, which can likely be attributed to the affordances of different media. Finally, our results indicate a relatively low level of cohesion in the topics discussed which might be an indicator of a diversity of the conceptual coverage discussed by the course participants.

Proceedings of the Fifth International Conference on Learning Analytics & Knowledge (LAK'15), 2015

All forms of learning take time. There is a large body of research suggesting that the amount of ... more All forms of learning take time. There is a large body of research suggesting that the amount of time spent on learning can improve the quality of learning, as represented by academic performance. The wide-spread adoption of learning technologies such as learning management systems (LMSs), has resulted in large amounts of data about student learning being readily accessible to educational researchers. One common use of this data is to measure time that students have spent on different learning tasks (i.e., time-on-task). Given that LMS systems typically only capture times when students executed various actions, time-on-task measures are estimated based on the recorded trace data. LMS trace data has been extensively used in many studies in the field of learning analytics, yet the problem of time-on-task estimation is rarely described in detail and the consequences that it entails are not fully examined. This paper presents the results of a study that examined the effects of different time-on-task estimation methods on the results of commonly adopted analytical models. The primary goal of this paper is to raise awareness of the issue of accuracy and appropriateness surrounding time-estimation within the broader learning analytics community, and to initiate a debate about the challenges of this process. Furthermore, the paper provides an overview of time-on-task estimation methods in educational and related research fields.

Proceedings of the 5th India Software Engineering Conference (ISEC '12), 2012

Highway is a domain-specific language for implementing enterprise application integration solutio... more Highway is a domain-specific language for implementing enterprise application integration solutions in a technology independent and functional manner. As an internal DSL developed on top of Clojure programming language, Highway uses functional programming techniques in order to simplify enterprise application integration development.

In this paper we focus on abstractions and language constructs that define Highway’s approach to integration. We also cover implementation of enterprise integration patterns using Highway since they represent various common situations in enterprise application integration development.

Joint Proceedings of the Workshop on Methodology in Learning Analytics (MLA) and the Workshop on Building the Learning Analytics Curriculum (BLAC) co-located with 7th International Learning Analytics and Knowledge Conference (LAK 2017), 2017

The recent technological advancements provide many opportunities for improvement of learners’ exp... more The recent technological advancements provide many opportunities for improvement of learners’ experience. The social nature of modern educational systems and the blending of formal and informal learning enable for more situated and personalized learning experiences. Moreover, the vast amount of data about learning activities can be utilized in a proactive manner to enable data-informed instructional interventions and attainment of learning outcomes. However, the present instructional and learning design approaches do not take into the account the potentials of digital data and analytics. In this paper, we introduce the Digital Learning Design framework which enables the development of course learning designs in a manner that incorporates evidence-driven nature of modern analytical systems with the sound pedagogical underpinnings of learning design research.

Proceedings of the Workshops at the LAK 2014 Conference co-located with 4th International Conference on Learning Analytics and Knowledge (LAK'14), 2014

In this paper we present the results of an exploratory study that examined the use of text mining... more In this paper we present the results of an exploratory study that examined the use of text mining and text classification for the automation of the content analysis of discussion transcripts within the context of distance education. We used Community of Inquiry (CoI) framework and focused on the content analysis of the cognitive presence construct given its central position within the CoI model. Our results demonstrate the potentials of proposed approach; The developed classifier achieved 58.4% accuracy and Cohen’s Kappa of 0.41 for the 5-category classification task. In this paper we analyze different classification features and describe the main problems and lessons learned from the development of such a system. Furthermore, we analyzed the use of several novel classification features that are based on the specifics of cognitive presence construct and our results indicate that some of them significantly improve classification accuracy.

Proceedings of the Workshops held at Educational Data Mining 2014, co-located with 7th International Conference on Educational Data Mining (EDM 2014), Jul 4, 2014

It is widely accepted that the social capital of students - developed through their participation... more It is widely accepted that the social capital of students - developed through their participation in learning communities – has a significant impact on many aspects of the students’ learning outcomes, such as academic performance, persistence, retention, program satisfaction and sense of community. However, the underlying social processes that contribute to the development of social capital are not well understood. By using the well-known Community of Inquiry (CoI) model of distance and online education, we looked into the nature of the underlying social processes, and how they relate to the development of the students’ social capital. The results of our study indicate that the affective, cohesive and interactive facets of social presence significantly predict the network centrality measures commonly used for measurement of social capital.

Analytics4Learning report series, 2017

In online learning, the most widely used model which outlines students’ learning experience is th... more In online learning, the most widely used model which outlines students’ learning experience is the community of inquiry (CoI) model. Central to the CoI model is the construct of cognitive presence, which focuses on students’ development of critical and deep thinking skills and is essential for the attainment of learning outcomes. Given the latent nature of cognitive presence, there are significant challenges related to its assessment, which currently requires manual coding of online discussion transcripts or reliance on self-reported measures using survey instruments. In this paper, we present a novel model for assessing students’ development of cognitive presence using automated learning analytics techniques. Building on the foundations of evidence-centered design, we developed a flexible model for assessing students’ cognitive presence based on educational trace data that can be used in variety of learning contexts (e.g., traditional for-credit online courses, massive open online courses, and blended courses). We used the model to develop two analytics systems for real-time monitoring of cognitive presence development and for delivering valuable feedback for instructors, enabling them to use different instructional interventions during a course.

Companion Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK'19), 2019

The poster presents the change of prioritised approaches to learning analytics (LA) among higher ... more The poster presents the change of prioritised approaches to learning analytics (LA) among higher education as their experience of adoption increases. The study examined 27 UK and European higher education institutions using the Epistemic Network Analysis technique. Results show that institutions with one or more years of experience with LA put more emphasis on understanding learning or teaching phenomena, whereas institutions with less experience of LA focused more on measuring the phenomena. This implicates a change of conceptualisation among institutions as their experience with LA increases.

Proceedings of the 2018 International Conference on Teaching, Assessment and Learning for Engineering (TALE'18), 2018

Online learning and massive open online courses are widely used in engineering and technology edu... more Online learning and massive open online courses are widely used in engineering and technology education. Engineering next-generation learning requires overcoming the potential constraints of online learning environments which necessitate higher levels of self-regulation than traditional classroom settings. This particular requirement demands that learners allocate their cognitive, metacognitive, affective and motivational resources to meet this need. Lack of self-regulation can affect learners’ engagement with the course content, resulting in sub-optimal learning outcomes or failure to complete the course. This paper reports on the design of a virtual learning assistant and its implementation in online learning activities. This paper outlines the virtual assistant’s use as a data collection tool and, further, proposes that the virtual learning assistant has the potential to be used as an assessment tool for self-regulatory skills, and as an intervention tool to support online learners’ self-regulation in online learning.

Proceedings of the Forth ACM Conference on Learning @ Scale (L@S'17), 2017

Proceedings of the Third ACM Conference on Learning @ Scale (L@S'16), 2016

Massive Open Online Courses represent a fertile ground for examining student behavior. However, d... more Massive Open Online Courses represent a fertile ground for examining student behavior. However, due to their openness MOOC attract a diverse body of students, for the most part, unknown to the course instructors. However, a certain number of students enroll in the same course multiple times, and there are records of their previous learning activities which might provide some useful information to course organizers before the start of the course. In this study, we examined how student behavior changes between subsequent course offerings. We identified profiles of returning students and also interesting changes in their behavior between two enrollments to the same course. Results and their implications are further discussed.

Proceedings of the Seventh International Conference on Learning Analytics and Knowledge (LAK'17), 2017

In this poster, we present the results of the study which examined the relationship between stude... more In this poster, we present the results of the study which examined the relationship between student differences in their use of the available technology and their perceived levels of cognitive presence within the MOOC context. The cognitive presence is a construct used to measure the level of practical inquiry in the Communities of Inquiry model. Our results revealed the existence of three clusters based on student technology use. The clusters significantly differed in terms of their levels of cognitive presence, most notably they differed on the levels of problem resolution.

With the increasing pace of technological changes in the modern society, there has been a growing... more With the increasing pace of technological changes in the modern society, there has been a growing interest from educators, business leaders, and policymakers in teaching important higher-order skills which were identified as necessary for thriving in the present-day globalized economy. In this regard, one of the most widely discussed higher order skills is critical thinking, whose importance in shaping problem solving, decision making, and logical thinking has been recognized. Within the domain of distance and online education, the Community of Inquiry (CoI) model provides a pedagogical framework for understanding the critical dimensions of student learning and factors which impact the development of student critical thinking. The CoI model follows the social-constructivist perspective on learning in which learning is seen as happening in both individual minds of learners and through the discourse within the group of learners. Central to the CoI model is the construct of cognitive presence, which captures the student cognitive engagement and the development of critical thinking and deep thinking skills. However, the assessment of cognitive presence is challenging task, particularly given its latent nature and the inherent physical and time separation between students and instructors in distance education settings. One way to address this problem is to make use of the vast amounts of learning data being collected by learning systems.

This thesis presents novel methods for understanding and assessing the levels of cognitive presence based on learning analytics techniques and the data collected by learning environments. We first outline a comprehensive model for cognitive presence assessment which builds on the well-established evidence-cantered design (ECD) assessment framework. The proposed assessment model provides a foundation of the thesis, showing how the developed analytical models and their components fit together and how they can be adjusted for new learning contexts. The thesis shows two distinct and complementary analytical methods for assessing students’ cognitive presence and its development. The first method is based on the automated classification of student discussion messages and captures learning as it is observed in the student dialogue. The second analytics method relies on the analysis of log data of students’ use of the learning platform and captures the individual dimension of the learning process. The developed analytics also extend current theoretical understanding of the cognitive presence construct through data-informed operationalization of cognitive presence with different quantitative measures extracted from the student use of online discussions.

We also examine methodological challenges of assessing cognitive presence and other forms of cognitive engagement through the analysis of trace data. Finally, with the intent of enabling for the wider adoption of the CoI model for new online learning modalities, the last two chapters examine the use of developed analytics within the context of Massive Open Online Courses (MOOCs). Given the substantial differences between traditional online and MOOC contexts, we first evaluate the suitability of the CoI model for MOOC settings and then assess students’ cognitive presence using the data collected by the MOOC platform. We conclude the thesis with the discussion of practical application and impact of the present work and the directions for the future research.

The Internet and Higher Education

Feedback is a major factor of student success within higher education learning. However, recent c... more Feedback is a major factor of student success within higher education learning. However, recent changes-such as increased class sizes and socioeconomic diversity of the student population-challenged the provision of effective student feedback. Although the use of educational technology for personalised feedback to diverse students has gained traction, the feedback gap still exists: educators wonder which students respond to feedback and which do not. In this study, a set of trackable Call to Action (CTA) links was embedded in two sets of feedback messages focusing on students' time management, with the goal of (1) examining the association between feedback engagement and course success and (2), to predict students' reaction to provided feedback. We also conducted two focus groups to further examine students' perception of provided feedback messages. Our results revealed that early engagement with the feedback was associated with higher chances of succeeding in the course...

The Internet and Higher Education

Learning and Instruction

The Internet and Higher Education

Connectivist pedagogies are geared towards building a network of learners that actively employ te... more Connectivist pedagogies are geared towards building a network of learners that actively employ technologies to establish interpersonal connections in open online settings. In this context, as course participants increasingly establish interpersonal relationships among peers they have greater opportunity to draw on and leverage the latent social capital that resides in such a distributed learning environment. However, to date there have been a limited number of studies exploring how learners build their social capital in open large-scale courses. To inform the facilitation of learner networks in open online settings and beyond, this study analyzed factors associated with how learners accumulate social capital in the form of learner connections over time. The study was conducted in two massive open online course offerings (Connectivism and Connective Knowledge) that were designed on the principles of connectivist pedagogy and that made use of data about social interaction from Twitter, blogs, and Facebook. For this purpose, linear mixed modelling was used to understand the associations between learner social capital, linguistic and discourse patterns, media used for interaction, as well as the time in the course when interaction took place. The results highlight the association between the language used by the learners and the creation of ties between them. Analyses on the accumulation of connections over time have implications for the pedagogical choices that would be expected to help learners leverage access to potential social capital in a networked context.

Proceedings of the Seventh International Learning Analytics & Knowledge Conference on - LAK '17, 2017

Proceedings of the Sixth International Conference on Learning Analytics & Knowledge - LAK '16, 2016

Proceedings of the Third (2016) ACM Conference on Learning @ Scale - L@S '16, 2016

Journal of Learning Analytics, 2015