The Tail Wagging the Dog; An Overdue Examination of Student Teaching Evaluations (original) (raw)
Related papers
2020
This study sought to determine what constitutes a minimally meaningful difference in student evaluations of their professors, when students are asked to rate their professors on the traditional 5-point teaching effectiveness item commonly used in higher education. A minimally meaningful difference is the smallest difference between two ratings that: 1) exceeds chance variation and 2) corresponds to a difference deemed meaningful using some external anchor or standard. Data was obtained through a series of surveys given to students at Butler University and to an online nationwide sample. Analysis occurred through both an anchor-based approach, using data obtained from a single survey, and a distribution-based method, using data obtained from two surveys administered two weeks apart. Both methods were used to find the minimal meaningful difference in student evaluations of professors. A meaningful difference of .84 was found when participants were asked to distinguish between two professors of higher quality. A meaningful difference of .75 was found when participants were asked to distinguish between two professors of lower quality. Both differences exceeded chance variation.
Professor , Student , and Course Attributes that Contribute to Successful Teaching Evaluations
Michael J. Seiler is an Assistant Professor of Finance and Vicky L. Seiler is an Assistant Professor of Marketing at Hawaii Pacific University, Honolulu, HI 96813. Dalen Chiang is a Professor at Cleveland State University. This study examines eight professor, student, and course attributes that affect four specific areas of teaching evaluations. All eight attributes significantly affect at least one of the four groups of student evaluation of teaching (SET) questions. The extant literature has previously ignored the fact that more than one factor exists. This has resulted in contradictory or inconclusive findings. Our study uses a more sophisticated methodology that allows for the delineation of all these intricate relationships. As a result, more clear and robust results emerge.[JEL: I20, I22, A00]
Student evaluations of teaching: teaching quantitative courses can be hazardous to one’s career
PeerJ
Anonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures of professors’ teaching effectiveness. However, previous research has generally used inappropriate parametric statistics and effect sizes to examine and to evaluate the significance of TEIFs on personnel decisions. Accordingly, we examined the influence of quantitative vs. non-quantitative courses on SET ratings and SET based personnel decisions using 14,872 publicly posted class evaluations where each evaluation represents a summary of SET ratings provided by individual students responding ...
Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness
ScienceOpen Research
Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. We show: SET are biased against female instructors by an amount that is large and statistically significant the bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded the bias varies by discipline and by student gender, among other things it is not possible to adjust for the bias, because it depends on so many factors SET are more sensitive to students' gender bias and grade expectations than they are to teaching effectiveness gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors.These findings are based on nonparametric statistical tests applied to two datasets: 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sec...
Student evaluations of teaching are ubiquitous and impactful on the careers of college teachers. However, there is limited empirical research documenting the accuracy of people's efforts in interpreting teaching evaluations. The current research consisted of three studies documenting the effect of small mean differences in teaching evaluations on judgements about teachers. Differences in means small enough to be within the margin of error significantly impacted faculty members' assignment of merit-based rewards (Study 1), department heads' evaluation of teaching techniques (Study 2) and faculty members' evaluation of specific teaching skills (Study 3). The results suggest that faculty and administrators do not apply appropriate statistical principles when evaluating teaching evaluations and instead use a general heuristic that higher evaluations are better.
William & Mary Journal of Women and the …, 2006
Research on student teaching evaluations is vast. An examination of this research demonstrates wide disagreements but also substantial consensus of authority for the proposition that student evaluations should be used only with extreme care, if at all, in making personnel decisions. A number of reasons cause administrators to use teaching evaluations for personnel decisions. The literature, however, is virtually unanimous in its condemnation of norming student evaluations in order to rank classroom performances. Current cases on academic freedom indicate some retrenchment by the Circuits from broader pronouncements in earlier Supreme Court cases. This paper concludes that the use of non-validated student evaluations alone without any other criteria for teaching effectiveness raises substantial problems in faculty retention and promotion decisions. It also suggests that such an approach in the right case might violate academic freedom and the First Amendment.
1975
Much has occurred in higher education to lead to expectation for change in the process of evaluation. The academic deans of all accredited private liberal arts colleges were asked to report on the procedures used in rating both overall and teaching performance, with 83.5 percent replying. Purposes of the questionnaire were to: (1) determine the relative level of importance placed on classroom teaching in the evaluation of overall performance of faculty members; (2) determine the types of information upon which evaluation of teaching performance is based; and (3) compare faculty evaluation policies and procedures during the contractions of the 1970's with the expansion of the mid-1960's. Important findings included: (1) significant declines in importance between 1966 and 1973 were recorded for research and publication; (2) campus committee work and student advising increased sharply as "major factors" in faculty evaluations; (3) in evaluating teaching performance, the deans' reliance on "systematic student ratings" increased significantly; and (4) whereas the importance of chairman's and dean's evaluations retained a prominent position in teaching evaluation, committee evaluation increased in prominence. Specific recommendations and an extensive bibliography are included in the publication. (Author)