Evaluating portfolio assessment systems: what are the appropriate criteria? (original) (raw)

New Directions in Portfolio Assessment: Assessing the Assessors

1993

Abstract: The plan to implement a professional portfolio policy in the Seven Oaks School Division, Winnipeg, Canada, will be perceived and lived by teachers in various ways. Their experiences will be colored by their understandings of the theories behind the use of ...

Portfolio Assessment: A Theoretical Prediction of Measurement Properties

1993

In this non-experimental study, a model was developed for portfolio assessment based on definitions and applications in the assessment literature. This model describes portfolio components, scores to be computed, and uses to he made of 'ale scores. The literature was then reviewed to find examples of actual applications that would provide realistic estimates of statistical characteristics of assessment results. Estimates of statistical characteristics were also obtained from performance assessments similar t, portfolios. The model and estimates were then used to estimate characteristics of an operational large-scale portfolio assessment program. Estimates were obtained of reliability of results, score distribution characteristics, and the validity of the procedure for writing evaluations for 12th graders. The analysis suggests that a well-structured and carefully scored portfolio assessment has the potential to provide scores that meet standards of reliability required for use with individual students. By developing a score that is the sum of the scores on the various portfolio entries, useful discriminations can be made among students. A factor that has not been considered is the cost of obtaining the expected results, which would undoubtedly be high. One table presents analysis results, and one graph shows composite score reliabilities. (SLD)

Knowledge, Attitudes, and Concerns About Portfolio Assessment: An Exploratory Study

Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 1998

In order to contribute to the limited empirical knowledge about portfolios, we surveyed 209 educators taking graduate classes in general and special education about their knowledge, attitudes, and concerns about portfolios. Respondents had a substantial amount of knowledge about some but not all aspects of portfolios and held favorable attitudes toward portfolios. They viewed inadequate training and support, the time to design and implement portfolio assessment, and evaluating portfolios as the most serious concerns with portfolios. General education teachers were more knowledgeable about portfolios than special educators. Women were both more knowledgeable and more positive about portfolios than men. Viewed together, the findings suggest that demographic characteristics of educators may be related to their knowledge, attitudes, and concerns about portfolios and might be considered in developing programs and support services for them.

Validity of portfolio assessment: which qualities determine ratings?

Medical Education, 2006

CONTEXT The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence.

Developing evaluation instrument based on CIPP models on the implementation of portfolio assessment

This study aimed to develop an evaluation instrument constructed by CIPP model on the implementation of portfolio assessment in science learning. This study used research and development (R & D) method; adapting 4-D by the development of non-test instrument, and the evaluation instrument constructed by CIPP model. CIPP is the abbreviation of Context, Input, Process, and Product. The techniques of data collection were interviews, questionnaires, and observations. Data collection instruments were: 1) the interview guidelines for the analysis of the problems and the needs, 2) questionnaire to see level of accomplishment of portfolio assessment instrument, and 3) observation sheets for teacher and student to dig up responses to the portfolio assessment instrument. The data obtained was quantitative data obtained from several validators. The validators consist of two lecturers as the evaluation experts, two practitioners (science teachers), and three colleagues. This paper shows the results of content validity obtained from the validators and the analysis result of the data obtained by using Aikens' V formula. The results of this study shows that the evaluation instrument based on CIPP models is proper to evaluate the implementation of portfolio assessment instruments. Based on the experts' judgments, practitioners, and colleagues, the Aikens' V coefficient was between 0.86-1,00 which means that it is valid and can be used in the limited trial and operational field trial.

Student perceptions of a portfolio assessment process

Medical Education, 2009

OBJECTIVES The objectives of this study were to identify and analyse students' attitudes to the portfolio assessment process over time. METHODS A questionnaire containing statements and open questions was used to obtain feedback from students at the

A Washback Study on Portfolio Assessment

ELT in Focus, 2020

The latest developments in the field of learning English as a foreign language have encouraged language teachers to use more authentic and realistic assessments. This situation makes portfolios as alternative assessments increasingly popular. As the popularity of the portfolio grows, questions among assessment developers have also arisen. They question whether the portfolio really is a proper assessment. In this study, the researcher investigated the wash-back effect of portfolio-based assessment. She observed the instructional process in a translation and a writing class to see how the two types of portfolios were used. This qualitative analysis study also studies by triangulating the data with the support of data from interviews and questionnaires. Researcher then classified the opinions and reflections of students and lecturers obtained from the interview and questionnaire responses. The findings suggest the use of a portfolio is indeed appropriate for use in writing and translat...