Vanessa Ralph - Academia.edu (original) (raw)

Papers by Vanessa Ralph

Research paper thumbnail of You Are What You Assess: The Case for Emphasizing Chemistry on Chemistry Assessments

Journal of Chemical Education, 2021

What we emphasize and reward on assessments signals to students what matters to us. Accordingly, ... more What we emphasize and reward on assessments signals to students what matters to us. Accordingly, a great deal of scholarship in chemistry education has focused on defining the sorts of performances worth assessing. Here, we unpack observations we made while analyzing what "success" meant across three largeenrollment general chemistry environments. We observed that students enrolled in two of the three environments could succeed without ever connecting atomic/molecular behavior to how and why phenomena happen. These environments, we argue, were not really "chemistry classes" but rather opportunities for students to gain proficiency with a jumble of skills and factual recall. However, one of the three environments dedicated 14−57% of points on exams to items with the potential to engage students in using core ideas (e.g., energy, bonding interactions) to predict, explain, or model observable events. This course, we argue, is more aligned with the intellectual work of the chemical sciences than the other two. If our courses assess solely (or largely) decontextualized skills and factual recall we risk (1) gating access to STEM careers on the basis of facility with skills most students will never use outside the classroom and (2) never allowing students to experience the tremendous predictive and explanatory power of atomic/molecular models. We implore the community to reflect on whether "what counts" in the courses we teach aligns with the performances we actually value.

Research paper thumbnail of Pipeline or Funnel: The inequality we create with assessments; Tubería o Embudo: La desigualdad que creamos con las pruebas

Research paper thumbnail of Impact of Maintaining Assessment Emphasis on Three-Dimensional Learning as Organic Chemistry Moved Online

Journal of Chemical Education

Research paper thumbnail of Impact of Representations in Assessments on Student Performance and Equity

Journal of Chemical Education

Research paper thumbnail of Introducing randomization tests via an evaluation of peer-led team learning in undergraduate chemistry courses

Chemistry Education Research and Practice

The methodological limitations education researchers face in the evaluation of reformed instructi... more The methodological limitations education researchers face in the evaluation of reformed instruction have led to debates as to the evidence advancing evidence-based practices. To conduct more effective research, methodological pluralism in the evaluation of educational reforms can be used to complement the strengths and limitations of a corpus of literature informing the impact of an evidence-based practice. This study seeks to introduce randomization tests, a nonparametric statistical analysis incorporating a random-assignment component that can be applied to a single-subject (N = 1) research design, as a methodology to be counted amongst evaluations of instructional reforms. To demonstrate the utility of this approach, an evaluation of peer-led team learning (PLTL) for classes of second-semester general chemistry spanning 7 semesters was conducted using randomization tests. The design contributes novel understandings of PLTL including differences in effectiveness across instructors...

Research paper thumbnail of An explanative basis for the differential performance of students with low math aptitude in general chemistry

Chemistry Education Research and Practice

Students who score within the bottom quartile on cognitive measures of math aptitude have been id... more Students who score within the bottom quartile on cognitive measures of math aptitude have been identified as at-risk for low performance in chemistry courses, with less attention as to why...

Research paper thumbnail of Chemistry topics posing incommensurate difficulty to students with low math aptitude scores

Chemistry Education Research and Practice

The identification of students at risk for academic failure in undergraduate chemistry courses ha... more The identification of students at risk for academic failure in undergraduate chemistry courses has been heavily addressed in the literature. Arguably one of the strongest and most well-supported predictors of undergraduate success in chemistry is the mathematics portion of the SAT (SAT-M), a college-entrance, standardized test administered by the College Board. While students scoring in the bottom quartile of the SAT-M (herein referred to as at-risk) perform significantly worse on first-semester chemistry assessments, little is known of the topics on which these students differentially struggle. The purpose of this study is to provide insight as to which first-semester chemistry topics present an incommensurate challenge to at-risk students. Students were identified as either at-risk or not at-risk via SAT-M scores. Students’ assessment responses were collected across four semesters of first-semester chemistry courses at a large, public university (N = 5636). At-risk students strugg...

Research paper thumbnail of Chemistry topics posing incommensurate difficulty to students with low math aptitude scores

Chemistry Education Research and Practice

The identification of students at risk for academic failure in undergraduate chemistry courses ha... more The identification of students at risk for academic failure in undergraduate chemistry courses has been heavily addressed in the literature. Arguably one of the strongest and most well-supported predictors of undergraduate success in chemistry is the mathematics portion of the SAT (SAT-M), a college-entrance, standardized test administered by the College Board. While students scoring in the bottom quartile of the SAT-M (herein referred to as at-risk) perform significantly worse on first-semester chemistry assessments, little is known of the topics on which these students differentially struggle. The purpose of this study is to provide insight as to which first-semester chemistry topics present an incommensurate challenge to at-risk students. Students were identified as either at-risk or not at-risk via SAT-M scores. Students’ assessment responses were collected across four semesters of first-semester chemistry courses at a large, public university (N = 5636). At-risk students strugg...

Research paper thumbnail of You Are What You Assess: The Case for Emphasizing Chemistry on Chemistry Assessments

Journal of Chemical Education, 2021

What we emphasize and reward on assessments signals to students what matters to us. Accordingly, ... more What we emphasize and reward on assessments signals to students what matters to us. Accordingly, a great deal of scholarship in chemistry education has focused on defining the sorts of performances worth assessing. Here, we unpack observations we made while analyzing what "success" meant across three largeenrollment general chemistry environments. We observed that students enrolled in two of the three environments could succeed without ever connecting atomic/molecular behavior to how and why phenomena happen. These environments, we argue, were not really "chemistry classes" but rather opportunities for students to gain proficiency with a jumble of skills and factual recall. However, one of the three environments dedicated 14−57% of points on exams to items with the potential to engage students in using core ideas (e.g., energy, bonding interactions) to predict, explain, or model observable events. This course, we argue, is more aligned with the intellectual work of the chemical sciences than the other two. If our courses assess solely (or largely) decontextualized skills and factual recall we risk (1) gating access to STEM careers on the basis of facility with skills most students will never use outside the classroom and (2) never allowing students to experience the tremendous predictive and explanatory power of atomic/molecular models. We implore the community to reflect on whether "what counts" in the courses we teach aligns with the performances we actually value.

Research paper thumbnail of Pipeline or Funnel: The inequality we create with assessments; Tubería o Embudo: La desigualdad que creamos con las pruebas

Research paper thumbnail of Impact of Maintaining Assessment Emphasis on Three-Dimensional Learning as Organic Chemistry Moved Online

Journal of Chemical Education

Research paper thumbnail of Impact of Representations in Assessments on Student Performance and Equity

Journal of Chemical Education

Research paper thumbnail of Introducing randomization tests via an evaluation of peer-led team learning in undergraduate chemistry courses

Chemistry Education Research and Practice

The methodological limitations education researchers face in the evaluation of reformed instructi... more The methodological limitations education researchers face in the evaluation of reformed instruction have led to debates as to the evidence advancing evidence-based practices. To conduct more effective research, methodological pluralism in the evaluation of educational reforms can be used to complement the strengths and limitations of a corpus of literature informing the impact of an evidence-based practice. This study seeks to introduce randomization tests, a nonparametric statistical analysis incorporating a random-assignment component that can be applied to a single-subject (N = 1) research design, as a methodology to be counted amongst evaluations of instructional reforms. To demonstrate the utility of this approach, an evaluation of peer-led team learning (PLTL) for classes of second-semester general chemistry spanning 7 semesters was conducted using randomization tests. The design contributes novel understandings of PLTL including differences in effectiveness across instructors...

Research paper thumbnail of An explanative basis for the differential performance of students with low math aptitude in general chemistry

Chemistry Education Research and Practice

Students who score within the bottom quartile on cognitive measures of math aptitude have been id... more Students who score within the bottom quartile on cognitive measures of math aptitude have been identified as at-risk for low performance in chemistry courses, with less attention as to why...

Research paper thumbnail of Chemistry topics posing incommensurate difficulty to students with low math aptitude scores

Chemistry Education Research and Practice

The identification of students at risk for academic failure in undergraduate chemistry courses ha... more The identification of students at risk for academic failure in undergraduate chemistry courses has been heavily addressed in the literature. Arguably one of the strongest and most well-supported predictors of undergraduate success in chemistry is the mathematics portion of the SAT (SAT-M), a college-entrance, standardized test administered by the College Board. While students scoring in the bottom quartile of the SAT-M (herein referred to as at-risk) perform significantly worse on first-semester chemistry assessments, little is known of the topics on which these students differentially struggle. The purpose of this study is to provide insight as to which first-semester chemistry topics present an incommensurate challenge to at-risk students. Students were identified as either at-risk or not at-risk via SAT-M scores. Students’ assessment responses were collected across four semesters of first-semester chemistry courses at a large, public university (N = 5636). At-risk students strugg...

Research paper thumbnail of Chemistry topics posing incommensurate difficulty to students with low math aptitude scores

Chemistry Education Research and Practice

The identification of students at risk for academic failure in undergraduate chemistry courses ha... more The identification of students at risk for academic failure in undergraduate chemistry courses has been heavily addressed in the literature. Arguably one of the strongest and most well-supported predictors of undergraduate success in chemistry is the mathematics portion of the SAT (SAT-M), a college-entrance, standardized test administered by the College Board. While students scoring in the bottom quartile of the SAT-M (herein referred to as at-risk) perform significantly worse on first-semester chemistry assessments, little is known of the topics on which these students differentially struggle. The purpose of this study is to provide insight as to which first-semester chemistry topics present an incommensurate challenge to at-risk students. Students were identified as either at-risk or not at-risk via SAT-M scores. Students’ assessment responses were collected across four semesters of first-semester chemistry courses at a large, public university (N = 5636). At-risk students strugg...