Comparison of performance on multiple-choice questions and open-ended questions in an introductory astronomy laboratory (original) (raw)
Related papers
Journal of College Science Teaching, 2016
This paper reports on an investigation into the correlations between students' understandings of introductory astronomy concepts and the correctness and coherency of their written responses to targeted Lecture-Tutorial questions. We assessed the correctness and coherency of responses from 454 college-level, general education, introductory astronomy students enrolled in courses taught in the spring of 2010, 2011, and 2012. We hypothesized that students who consistently provided responses with high levels of correctness and coherency would outperform students who did not on multiple measures of astronomy content knowledge. We compared students' correctness and coherency scores to their normalized gains on the Light and Spectroscopy Concept Inventory (LSCI) and to their scores on closely-related exam items. Our analysis revealed that no significant correlations exist between students' correctness and coherency scores and their LSCI gain scores or exam item scores. However, the participant group in our study did achieve high scores on the LSCI and on closely-related exam items. We hypothesized that these differences are due to the discussions that take place between students which suggests that instructors who teach with active engagement instructional strategies should focus their implementation on ensuring that their students fully engage in the richest possible discourse.
What is assessment? Why do it? Why do it in a particular way? This document addresses these important questions and provides a practical "how-to" guide for doing assessment. Assessment drives student learning; it is thus imperative that instructors conduct assessment in a manner that is well aligned with the instructor's goals for the course. This requires (a) that course goals be formalized, and (b) that the instructor have knowledge of various classroom assessment techniques and the kinds of course goals to which each of these assessment techniques is best suited. We briefly present several Classroom Assessment Techniques (CATs) that can be used to help instructors evaluate the extent to which course goals are being achieved, to help guide students toward desired learning outcomes, and to improve student self-evaluation of understanding. In addition, we outline a practical, generalized model for course development with which we demonstrate how to do assessment. For an on-line, user-friendly guide and resource to classroom assessment in college science courses, the reader is invited to visit the Field-Tested Learning Assessment Guide (FLAG) developed by the National Institute for Science Education
Analysis of Individual Test of Astronomy STandards (TOAST) Item Responses
The development of valid and reliable strategies to efficiently determine the knowledge landscape of introductory astronomy college students is an effort of great interest to the astronomy education community. This study examines individual item response rates from the Test Of Astronomy Standards (TOAST). The TOAST, a widely used 27-item, multiple-choice format, criterion-referenced test addresses both the full range of topics commonly taught in a one- or two-semester undergraduate introductory astronomy survey courses, and concepts described in various national science education standards, frameworks, and reform documents. The present study involves an examination of responses by 1104 participants, allowing for a rigorous item-by-item and distractor-by-distractor analysis of students’ responses. The results suggest that each individual TOAST item is functioning appropriately across a broad range of students, and is able to identify notable student misconceptions. These results also provide an opportunity to identify target areas of opportunity for astronomy education researcher that remain largely unstudied.
Journal of Astronomy & Earth Sciences Education (JAESE)
As discipline-based astronomy education researchers become more interested in experimentally testing innovative teaching strategies to enhance learning in undergraduate introductory astronomy survey courses ("ASTRO 101”), scholars are placing increased attention toward better understanding factors impacting student gain scores on the widely used Test Of Astronomy STandards (TOAST). Usually used in a pre-test and post-test study design, college faculty might naturally assume that the pre-course differences observed between high- and low-scoring college students might be due in large part to their pre-existing motivation, interest, experience in science, and attitudes about astronomy. To explore this notion, 11 non-science majoring undergraduates taking ASTRO 101 at west coast community colleges were interviewed in the first few weeks of the course after taking the TOAST as a pre-test to better understand students' pre-existing affect toward learning astronomy with an eye tow...
2014
In this dissertation, I examine the persistence of common astronomy misconceptions by the administration of a retrospective survey. The survey is a new instrument in that it permits the student to indicate either endorsement or rejection of each misconception at various stages in the student's life. I analyze data from a total of 639 students over six semesters. I compare the survey data to the results of exams taken by the students and additional instruments that assess students' misconceptions prior to instruction. I show that the consistency of the students' recollection of their own misconceptions is on par with the consistency of responses between prelims and the final exam. I also find that students who report higher increased childhood interest in astronomy are more likely to have accurate recalls of their own past recollections. I then discuss the use of principal components analysis as a technique for describing the extent to which misconceptions are correlated with each other. The analysis yields logical groupings of subtopics from which to teach. I then present a brief overview of item response theory, the methodology of which calculates relative difficulties of the items. My analysis reveals orders to teach the associated topics in ways that are most effective at dispelling misconceptions during instruction. I also find that the best order to teach the associated concepts is often different for high school and college level courses.
American Journal of Physics, 2009
We present the results of a national study on the teaching and learning of astronomy as taught in general education, non-science-major, introductory astronomy courses. Nearly 4000 students enrolled in 69 sections of courses taught by 36 different instructors at 31 institutions completed ͑preand post-instruction͒ the Light and Spectroscopy Concept Inventory ͑LSCI͒ from Fall 2006 to Fall 2007. The classes varied in size and were from all types of institutions, including 2-and 4-year colleges and universities. Normalized gain scores for each class were calculated. Pre-instruction LSCI scores were clustered around ϳ25%, independent of class size and institution type, and normalized gain scores varied from about −0.07 to 0.50. To estimate the fraction of classroom time spent on learner-centered, active-engagement instruction we developed and administered an Interactivity Assessment Instrument ͑IAI͒. Our results suggest that the differences in gains were due to instruction in the classroom, not the type of class or institution. We also found that higher interactivity classes had the highest gains, confirming that interactive learning strategies are capable of increasing student conceptual understanding. However, the wide range of gain scores seen for both lower and higher interactivity classes suggests that the use of interactive learning strategies is not sufficient by itself to achieve high student gain.
1997
The link between instruction in middle school science and assessment in ninth grade science in a small rural midwestern school district was investigated. Science students, in middle school grades 6-8 (917) and their 13 science teachers responded to surveys used to characterize science learning opportunities in the middle school classes. Ninth grade students (309) responded to a set of open-ended science questions that were developed from a standardized science test. The survey data provided a contextual framework for interpretation of ninth grade student performance on the open-ended science questions. The open-ended questions included questions for which students were required to graph and interpret data, write conclusions, identify control variables, and judge the validity of information. These open-ended questions were scored by three raters according to a scoring rubric. Middle school students did report having science learning opportunities to practice science as inquiry, and their teachers reported that much of the science inquiry practice was contextual. At least 90% of the ninth graders were from the same school district, but when practice in controlling data and making predictions was reported, student data from the open-ended questions suggested minimal transfer of these skills. The interpretation and use of graphic information and the actual graphing of data were areas that were problematic for ninth graders, as was the ability to provide reasons in support of answers. Recommendations based on this study include using a variety of contextual settings for framing science inquiry practice and more attention on discussing, reading, and writing in the content area. (Contains 8 tables and 12 references.)
Assessing astronomy students’ views about the nature of scientific inquiry
Physical Review Physics Education Research
This paper is part of the Focused Collection on Astronomy Education Research.] Students taking a second astronomy course for nonscientists were asked to reflect on the nature of scientific inquiry three times during the first half of the semester. First, they were assigned a short paper in which they were asked to argue for or against the thesis that observers of the night sky in ancient civilizations were scientists, and nearly all of the students argued that yes, they were. Second, they were asked to write a second paper in which they argued the opposite of their first thesis. Lastly, they were asked, on a midterm exam, to write an essay about whether the Ancient Greek astronomers were good scientists. In writing these papers and essays, many students included definitions of science or scientists, and these definitions progressed from paper to paper. In addition, the evidence students used to argue that ancient observers were scientists changed, with students becoming more expertlike.