Dmytro Babik | James Madison University (original) (raw)

Papers by Dmytro Babik

Educational technology research and development, Mar 19, 2024

Journal of Information Systems Education, 2022

Agile system development approaches, such as Scrum, have become popular with a wide spectrum of o... more Agile system development approaches, such as Scrum, have become popular with a wide spectrum of organizations from start-ups to government agencies. Recruiters and executives have been seeking graduates with Scrum competency beyond cursory familiarity. CIS programs responded with a variety of activities and exercises to address this need. This teaching tip demonstrates a hands-on active-learning experience designed to advance students' understanding of Scrum values, roles, artifacts, and events; to build confidence of using it in practice; to expose students to the experience of using versatile software as a service (SaaS) tools for project management; to enable students' greater agency via self-forming teams and ability to control their workload; and to stimulate interest in Scrum and encourage students to seek professional certifications. The activity may be applied within a constrained time frame of a conventional systems analysis and design (SAD) or project management (PM) course and fits in the niche between a single-class activity and a semester project. Our evidence suggests that this learning experience achieves its pedagogical objectives, is effective in both face-to-face and synchronous online courses, and results in vastly positive student response.

Educational Technology Research And Development, 2024

Over the past two decades, there has been an explosion of innovation in software tools that encap... more Over the past two decades, there has been an explosion of innovation in software tools that encapsulate and expand the capabilities of the widely used student peer assessment. While the affordances and pedagogical impacts of traditional in-person, "paper-and-pencil" peer assessment have been studied extensively and are relatively well understood, computerized (online) peer assessment introduced not only shifts in scalability and efficiency, but also entirely new capabilities and forms of social learning interactions, instructor leverage, and distributed cognition, that still need to be researched and systematized. Despite the ample research on traditional peer assessment and evidence of its efficacy, common vocabulary and shared understanding of online peer-assessment system design, including the variety of methods, techniques, and implementations, is still missing. We present key findings of a comprehensive survey based on a systematic research framework for examining and generalizing affordances and constraints of online peer-assessment systems. This framework (a) provides a foundation of a design-science metatheory of online peer assessment, (b) helps structure the discussion of user needs and design options, and (c) informs educators and system design practitioners. We identified two major themes in existing and potential research -orientation towards scaffolded learning vs. exploratory learning and system maturity. We also outlined an agenda for future studies.

The Journal of information and systems in education, Dec 1, 2020

Course and learning objectives are important tools for setting goals, navigating the course, and ... more Course and learning objectives are important tools for setting goals, navigating the course, and measuring performance. Unfortunately, when multiple interrelated objectives are presented as a list of statements, students perceive them as having little utility and tend to misunderstand or ignore them. To increase students' attention to course objectives, to help them understand the arc, structure, and valuable outcomes of the course, and to engage in active learning, we propose an approach to presenting course objectives in a visual form. The evidence suggests that visualized objectives increased students' interest in understanding them and provided aids to instructors to better explain how various components of the course fit together and translate into marketable skills. We recommend practical steps for visualizing objectives in any course and present examples of visualizations in two IS courses-"Enterprise Architecture" and "Systems Analysis and Design."

Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 2019

This study examines fidelity of ranking and rating scales in the context of online peer review an... more This study examines fidelity of ranking and rating scales in the context of online peer review and assessment. Using the Monte-Carlo simulation technique, we demonstrated that rating scales outperform ranking scales in revealing the relative "true" latent quality of the peer-assessed artifacts via the observed aggregate peer assessment scores. Our analysis focused on a simple, single-round peer assessment process and took into account peer assessment network topology, network size, the number of assessments per artifact, and the correlation statistics used. This methodology allows to separate the effects of structural components of peer assessment from cognitive effects.

We present the research framework for a taxonomy of online educational peer-assessment systems. T... more We present the research framework for a taxonomy of online educational peer-assessment systems. This framework enables researchers in technology-supported peer assessment to understand the current landscape of technologies supporting student peer review and assessment, specifically, its affordances and constraints. The framework helps identify the major themes in existing and potential research and formulate an agenda for future studies. It also informs educators and system design practitioners about use cases and design options.

IT-enabled peer-based creation, review, and evaluation systems are widely spread in multiple area... more IT-enabled peer-based creation, review, and evaluation systems are widely spread in multiple areas of open innovation and knowledge management. Despite a noticeable variety of designs, particularly, in the structure of peer networks (that is, how participants are linked to each other as creators and reviewers), these design choices are hardly ever grounded in design research. Characteristics of peer network structure, such as reciprocity and clustering, may affect how well such systems reveal participants’ competencies and their products’ qualities. Designing peer review systems that produce valid and reliable evaluations is, therefore, among the most fundamental concerns. Using a simulation approach, we show that reciprocity and clustering indeed have an effect, but its direction and magnitude depend on the evaluation scale used. So far, we have found no evidence that transitional networks have superior efficacy in comparison with “pure” networks. We outlined directions for further investigation

Mobius SLIP is a web-based tool that allows students to submit writing samples and receive anonym... more Mobius SLIP is a web-based tool that allows students to submit writing samples and receive anonymous critiques from other students in the class. Students submit an essay, and the Mobius SLIP system randomly assigns each student to an anonymous peer group. Students then rank the quality of each submission and provide actionable feedback. Students also rank the quality of the feedback for each assignment. Mobius SLIP produces performance analytics and recommends a grade based on these rankings, and instructors provide feedback (if desired). This session will demonstrate the system and present evidence of learning from three business classes.

In peer-based knowledge creation domains, problem complexity and subjectivity of individual under... more In peer-based knowledge creation domains, problem complexity and subjectivity of individual understanding impedes development of actors' competencies. Prior research remains ambivalent on whether interactions between peers lead to the development of shared, intersubjective, understanding about one's own and peers' competencies. On the one hand, actors may develop this shared understanding through social learning. On the other hand, due to the Dunning-Kruger effect, both less and more competent actors may persistently miscalibrate their own performance relative to peers. This dissertation examines how creation and evaluation competencies in peer-based social knowledge creation communities, where complex-problem social knowledge artifacts are produced, change and interact over time. It hypothesizes the existence of latent classes of longitudinal trajectories of creation and evaluation competency development, and convergence of these trajectories over multiple interactions,...

J. Inf. Syst. Educ., 2020

Posted Online: 8 September 2020 Published: 10 December 2020 Full terms and conditions of access a... more Posted Online: 8 September 2020 Published: 10 December 2020 Full terms and conditions of access and use, archived papers, submission instructions, a search tool, and much more can be found on the JISE website: http://jise.org ISSN: 2574-3872 (Online) 1055-3096 (Print) Teaching Tip: Visualizing IS Course Objectives and Marketable Skills Dmytro Babik Diane Lending Department of Computer Information Systems and Business Analytics James Madison University Harrisonburg, VA 22807, USA babikdx@jmu.edu, lendindc@jmu.edu

We propose a framework for research and development of Online Peer Review and Assessment (OPRA) s... more We propose a framework for research and development of Online Peer Review and Assessment (OPRA) systems that combines a common research-oriented data model labeled Peer Review Domain Model (PRDM) and an open-source data exchange specification labeled Peer Review Meta-Language (PRML). This framework received the approbation of several OPRA systems and have been used to amass research data via the Educational Peer Review Data Warehouse. We invite feedback on the structure of the framework and the use of the data warehouse from researchers interested in educational and academic peer review.

Peer assessment is widely used at all levels of education. Students give and receive feedback fro... more Peer assessment is widely used at all levels of education. Students give and receive feedback from their classmates, and thereby produce a wealth of information that can potentially be used to improve the assessment process. But thus far, each online peerassessment system has been an entity unto itself. There has been no attempt to compare the approaches taken by such systems, for example, the rubrics or the structuring of the assessment process. Our PeerLogic project is an attempt to change that. We are constructing a data warehouse of millions of peer reviews, from at least half-a-dozen systems, that can be mined to determine how differences in the assessment processes translate into differences in peer assessments. This paper reports on some of the issues that arise in the construction of the warehouse, and how we have resolved them in a way that will work for all constituent systems. We also presented an example of comparing data coming from two systems that are based on rating ...

Academy of Management Proceedings, 2012

Health care management review, Jan 8, 2016

The relationship between Chief Executive Officer (CEO) succession and hospitals' competitive ... more The relationship between Chief Executive Officer (CEO) succession and hospitals' competitive performance is an area of interest for health services researchers. Of particular interest is the impact on overall strategic direction and health system performance that results from selecting a CEO from inside the firm as opposed to seeking outside leadership. Empirical work-to-date has yielded mixed results. Much of this variability has been attributed to design flaws; however, in the absence of a clear message from the evidence, the preference for hiring "outsiders" continues to grow. This paper investigates on the extent to which insider CEO succession versus outsider succession impacts hospitals' competitive advantage vis-à-vis a sample of organizations that compete in the same sector. A hospital matching protocol based on propensity scores is used to control for endogeneity and makes comparisons of productivity across organizations through the use of stochastic front...

IEEE Transactions on Learning Technologies

Information Systems Frontiers, 2015

Sustainable development and corporate environmental responsibility are becoming closely tied with... more Sustainable development and corporate environmental responsibility are becoming closely tied with business competitiveness in modern economy. Given the impetus organizations have to enhance their sustainability image, there is a lack of research that examines the relationship between corporations’ messages to shareholders and their environmental and social responsibility. The current paper presents a methodology and preliminary findings of the pilot study of 104 IT and non-IT companies in the USA. Content analysis of letters to shareholders was done to check for significant differences in correlations between sustainability-related content and environmental performance in the sample. The paper also outlines methodology and expected results of the proposed research-in-progress. A similar methodology will be used to study a larger panel dataset of North American, European and Asian IT corporations to assess the dynamics of sustainability-related content over time and across companies ...