The Gold Standard for Assessing Creativity (original) (raw)
Related papers
Extension of the Consensual Assessment Technique to Nonparallel Creative Products
Creativity Research Journal, 2004
The consensual technique for assessing creativity is widely used in research, but its validation has been limited to assessing the creativity of artifacts produced under tightly constrained experimental conditions. Typically, only artifacts produced in response to very similar instructions have been compared. This has allowed researchers to compare such things as the effects of different motivational conditions on creative performance, but it has not allowed many other kinds of comparisons. It has also limited the use of the technique to artifacts gathered for specific experimental purposes, as opposed to already-existing artifacts produced under less controlled conditions. For this study, samples of writings collected by the National Assessment of Educational Progress that were written in response to a very wide variety of assignments and under varying conditions were rated for creativity by 13 expert judges. Judges compared the creativity of 103 stories, 103 personal narratives, and 102 poems, all written by 8th-grade students. Very high levels of interrater reliability were obtained, demonstrating that the consensual method can be validly extended to such samples. New avenues for future research made possible by these findings are then discussed.
Assessment of Creativity: Theories and Methods
Creativity - A Force to Innovation, 2021
The history of creativity assessment is as old as the concept itself. Researchers from various cultures and disciplines attempted to define the concept of creativity and offer a valid way to assess it. Creativity is generally defined as the ability to produce work that is novel and appropriate. Researchers in the field attempted to measure creativity from different perspectives and tried to answer the question like “What are the mental processes involved in creative thought?, Which personality traits are associated with creativity?, How can a product can be judged to be creative? and, What are the external forces that affect creativity?”. The answers of these questions constitute the most commonly used creativity assessment instruments. This chapter presents a brief overview on assessment of creativity through the eyes of the psychometric perspective and discusses the strengths and weaknesses of various instruments used in the field.
The purpose of this study was to explore the reliability of measures of both individual and group creative work using the consensual assessment technique (CAT). CAT was used to measure individual and group creativity among a population of pre-service music teachers enrolled in a secondary general music class (n = 23) and was evaluated from multiple perspectives for reliability. Consistency was calculated using Cronbach’s alpha. Judges were found to be highly consistent for individual creativity (α = .90), individual craftsmanship (α = .87), group creativity (α = .86) and group craftsmanship (α = 81). Judges were much less consistent with their ratings of aesthetic sensitivity for individual compositions (α = .67) or group performances (α = .69). Absolute agreement was calculated by using intraclass correlation coefficient (ICC). Judges were found to be highly in agreement for individual creativity (α = .79), individual craftsman- ship (α = .83), group creativity (α = .87) and group craftsmanship (α = 83). Judges were much less in agreement with their ratings of aesthetic sensitivity for individual compositions (α = .57) or group performances (α = .71). Judges ratings for individual creativity were consistent over time, as evidenced by test– retest reliabilities of .89 (creativity), .83 (craftsmanship) and .79 (aesthetic sensibility). Results indicate, in agreement with prior research, that CAT is a reliable measure of creativity. The researchers introduce the idea that absolute agreement might be a worthwhile construct to explore in future work in the measurement of creativity in music education.
Evaluation and Testing in Creativity
2003
Creativity is guided by metaphors that sometimes are Perceptually based. Normal standards of evaluation like truth, parsimony, and simplicity do not apply to the outcomes of creative thought. It is argued that different criteria are at work at different stages of the creative process, and that the former may be implicit and non-conceptual as well as explicit and symbolic.
Roeper Review, 1998
Current work on creativity is based on methodologies which either are psychometric in nature or were developed in response to perceived weaknesses of creativity measurement. However, psychometric perspectives on creativity are still a vibrant and vital area of study. Considerable evidence of validity (or lack thereof) has been gathered for a diverse set of instruments and assessment techniques, and the resulting improvement in measurement quality has opened the door to several exciting areas of research. We review some of the current issues, describe recent advances, and suggest future directions for psychometric approaches to creativity research.
Characterising Creativity: a holistic methodology for the assessment of creative works
This paper reports on the early stages of a design experiment in educational assessment that challenges the dichotomous legacy evident in many assessment activities. Combining social networking technologies with the sociology of education the paper proposes that assessment activities are best understood as a negotiable field of exchange. In this design experiment students, peers and experts engage in explicit, "front-end" assessment to translate holistic judgments into institutional, and potentiality economic capital without adhering to long lists of pre-set criteria. This approach invites participants to use social networking technologies to judge creative works using scatter graphs, keywords and tag clouds. In doing so assessors will refine their evaluative expertise and negotiate the characteristics of creative works from which criteria will emerge (Sadler, 2008). The real-time advantages of web-based technologies will aggregate, externalise and democratise this transparent method of assessment for most, if not all, creative works that can be represented in a digital format.