Jaekyung Lee - Academia.edu (original) (raw)

Papers by Jaekyung Lee

Brookings Papers on Education Policy

Journal of Research in Rural Education, 2000

... Despite these state policy efforts, it has not been clear whether schooling conditions and st... more ... Despite these state policy efforts, it has not been clear whether schooling conditions and student ... the cross-sectional nature of the NAEP data, it is cautioned against making causal infer ... signs of the implementation of many recent rec-ommendations for the reform of teaching ...

National Education Policy Center, Oct 1, 2011

National math assessment data from 3,112 eighth-grade students in 123 schools were used to determ... more National math assessment data from 3,112 eighth-grade students in 123 schools were used to determine whether location (rural versus nonrural) affects student achievement when related student and school-level factors are taken into account. Findings indicate that rural schools outperformed nonrural schools in math achievement and that the competitive edge of rural schools arose from school effects beyond the influences of school composition and individual student characteristics. Rural schools tended to have a better organizational context (teacher training, safe/orderly climate, collective support) conducive to higher performance, but suffered from poor instructional conditions (in terms of classroom resources, advanced courses, and progressive instruction), which balanced out achievement gains. Rural schools should capitalize on existing organizational and contextual strengths and also focus more effort on improving curricular and instructional weaknesses. Given that many rural students are poor and attend schools whose instructional resources and course offerings are limited, the level of their academic performance relative to their nonrural counterparts is encouraging. This study shows that rural schools can provide a model of strength worth studying and emulating. Contains 20 references and hierarchical linear regression formulas. (Author/TD)

Education and the Public Interest Center, Apr 7, 2010

The Education Trust research report Stuck Schools suggests a framework for identifying chronicall... more The Education Trust research report Stuck Schools suggests a framework for identifying chronically low-performing schools in need of turnaround. The study uses Maryland and Indiana to show that some low-performing schools make progress while others remain stagnant. The report has four serious problems of reliability and validity, however. First, the norm-referenced methodology guarantees "failed" schools independent of any true performance or improvement level by the school. Second, the report's reliance on state assessment data is misleading, and some schools' reported growth may be an artifact of regression to the mean and ceiling effects as well as instructional and testing practices. Third, the use of a linear growth model is questionable, since schools may not follow a strictly linear pattern of improvement. Fourth, the label of "stuck" becomes problematic given that there is no research-based guidance on how to improve schools other than vague prescriptions. In conclusion, the report's methods are so simplistic, arbitrary and illfitting with its own assumptions that it is more harmful to sound policymaking than helpful. There remains an outstanding question of how to help struggling schools after identification, but we need to know first whether the identification is based on reliable and valid measures, and if so, what school factors account for these differences.

Education Policy Analysis Archives, 2001

Iejll International Electronic Journal For Leadership in Learning, 1999

Content-driven systemic school reform emerged in recent years as a major policy alternative in th... more Content-driven systemic school reform emerged in recent years as a major policy alternative in the United States. Major reforms include the establishment of state curriculum frameworks, the development of student assessments, and the adoption of new textbooks tied to the curriculum frameworks. Because successful implementation of content-driven reform depends on the linkages between state-level policymaking and school-level policy implementation, the central question is how state policies have changed the ways in which classroom activitieg are organized and managed by teachers. This paper presents findings of a study that investigated the multilevel linkages of state education reform to instructional practices in mathematics. Four stages of empirical analyses were conducted with a combination of the following survey data: the 1992 Council of Chief State School Officers (CCSSO) state survey, the 1990 and 1992 National Assessment of Educational Progress Trial State Assessment (NAEP TSA) eighth-grade school survey, and the 1990-91 Schools and Staffing Survey (SASS) public school administrator survey. The data suggest that current state curriculum frameworks need to be aligned with instructional materials as well as with student assessments. The.findings also imply that policy implementation is a dynamic organizational process of interactions between policy characteristics and the institutional setting in a multilayered school system. Thus, systemic school reform needs to formulate desired connections between organizational levels in normative, structural, and functional arenas. The findings further suggest that the impacts of state curriculum and testing mandates on pedagogical practices between 1990 and 1992 were not substantial. Lastly, a comparison of educational outcomes in California and Minnesota showed that California's content-driven reforms have led to more successful instructional change than have Minnesota's outcome-based reforms. Four figures and 10 tables are included. Appendices contain descriptions of school-and state-level predictors. (Contains 35 references.) (LMI)

This study explores an alternative approach to educational program and policy evaluation by using... more This study explores an alternative approach to educational program and policy evaluation by using two major educational measurement/analysis methods, and illustrates their integrated applications to evaluating state reform policies. Most evaluations have been done one program at a time, but it is desirable to design evaluation research in a way that compares the effectiveness of several programs that have the same objectives but different content or function on the same set of outcome measures. Applying item response theory to policy and practice survey provides an innovative solution to objective measurement of policies and practices. In addition, multilevel analysis methods would not only provide a means for formulating school and state-level regression models simultaneously but also provide more precise estimates of the extent to which state policies affect school practices. An illustrative study of state policy examines the multilevel linkages between state policies and educational outcomes. First, objective measures of state policies are created through application of the Rasch model. Then the multilevel education policy-practice linkages are examined through the application of the hierarchical linking model. As the results illustrate, the idea of comparing two groups of states on their policy outcome measures is similar to the nonequivalent control group design. However, the research design proposed in this paper differs from the nonequivalent control group design in some significant ways: (1) treatment is not a single program, but a set of programs; (2) group exposure is a matter of degree; (3) all of the programs that constitute treatment do not have to occur between pretest and posttest; and (4) subjects examined on pretest and posttest do not have to be the same, but can be sampled independently. The proposed approach should give more flexibility for evaluation design in real-life settings, but at the same time more difficulties for interpretation of evaluation results. Some concerns are reviewed. (Contains three tables, one figure, and eight references.) (SLD) *

The Third International Mathematics and Science Study (TIMSS) is the largest, most comprehensive,... more The Third International Mathematics and Science Study (TIMSS) is the largest, most comprehensive, and most rigorous international comparison of education ever undertaken. TIMSS findings show similarities and differences in the processes and outcomes of schooling between the United States and East Asian countries, particularly Japan. This report reviews the initial findings from the TIMSS eighth-grade report, with a special focus on the United States and Japan, and discusses missing or incomplete links in international education studies. The initial findings highlight similarities between students' learning experiences at home and critical differences between those at school. These patterns imply that Japanese and U.S. students do not differ in out-of-school learning experiences, but there are differences neglected in TIMSS highlights, such as private tutoring. On the other hand, TIMSS researchers found substantial differences in the content and process of instruction at school and differences in instructional organization. While initial TIMSS findings show that the United States is still far from achieving the national goal of being first in the world in mathematics and science, these aggregate national patterns ignore the enormous local variation in the U.S. It is not clear whether federated countries like the United States are really comparable to other countries. For a valid comparison of the United States with highly centralized East Asian countries, it is suggested that the American states be treated as comparable national units. For a reliable comparison of the countries over time, it is suggested that current practices and outcomes be compared to past counterparts. something that will take into account the possibility of educational convergence between the United States and East Asian countries. To make fair, valid, and reliable comparisons, future international education studies need to take into account the linkages between formal and informal learning, consider local variation within a country, and compare current and past practices and outcomes. (Contains 1 figure, 1 table, and 33 references.)

Teachers College Record, 2007

This report is the product of the first year of study of the Statewide Systemic Initiatives (SSI)... more This report is the product of the first year of study of the Statewide Systemic Initiatives (SSI) Study on exploring data and methods to assess and understand the performance of states participating in the SSI. The report describes three sets of studies. The first studied valid and reliable measures of student achievement, and how to combine multiple measures of achievement, as well as how results of certification based on a single-measure compare with those based on a multimeasure analysis. The study addressed these questions through analyses of state and local data from two sites in Maine. The second study focused on valid and reliable ways to estimate school performance and identify value-added school effects on student achievement, as well as how the results of school evaluation based on a unilevel analysis compare with those based on a multilevel analysis. The study addressed these questions through multilevel analyses of National Assessment of Educational Progress assessment data in Maine and Kentucky. Results suggest that schoollevel compositional effects and student-level background characteristics need to be taken into account for fair evaluation of school performance. The third study explored ways to estimate academic progress and how to identify true achievement gains without statistical artifacts. These questions were addressed through the analysis of Maine Educational Assessment and Kentucky Instructional Results Information System assessment data. Results show that to set realistic expectations about schools' academic progress, past trends need to be taken into account, and that combining multiple years of data improves the reliability of school performance measures. (Contains 24 tables and 28 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document.

Online Submission, Apr 1, 2011

Through a synthesis of test publisher norms and national longitudinal datasets, this study provid... more Through a synthesis of test publisher norms and national longitudinal datasets, this study provides new national norms of academic growth in K-12 reading and math that can be used to reinterpret conventional effect sizes in time units. We propose d΄, a time-indexed effect size metric to estimate how long it would take for an "untreated" control group to reach the treatment group outcome in terms familiar to educators-years/months of schooling. It serves as a supplement to conventional effect size metrics such as Cohen's d by taking into account different amounts of time needed for learning at different age or grade levels. Through applications to Project STAR small class effects and NAEP racial achievement gaps, we demonstrate how to interpret and use d΄. It is expected to provide a more developmentally appropriate context for interpreting the size of an effect, a step toward bridging the gap between educational research and practice.

Given state activism in education reform during the 1980s, a major concern in the development of ... more Given state activism in education reform during the 1980s, a major concern in the development of education policy is whether reform states have increased their financial support for education in order to achieve desired objectives. This paper describes findings of a study that explored the distributive nature of state and local policy choices in education through comparative analyses of their long-term fiscal effort for three distinctive public services-welfare, highways, and police. The study built upon the unitary model of taxation and expenditure policies. The findings imply that policy shifts may have led to changes in educational expenditure patterns. Providing the general structures of state-level distributive effects, multilevel analyses suggest that the first wave of state reform affected not only fiscal effort for education but also distributive tendencies in education expenditures. While state reform appeared to increase the level of resources allocated to public education versus other social services, the distributive effects of state education reform tend to vary among states with different racial compositions. Redistributive tendencies in education expenditures tend to be accompanied by a low level of fiscal effort for education. In addition, developmental versus redistributive tendencies in education expenditures tend to be highly conflicting. One figure, 5 tables, and 13 endnotes are included. (Contains 36 references.) (LMI)

This study considered two questions about the use of national and state assessment databases: (1)... more This study considered two questions about the use of national and state assessment databases: (1) Do state and national assessments provide the same information on the performance of an educational system? and (2) What are the factors that might affect the discrepancies between national and state assessment results? Kentucky and Maine were chosen for a case study. Four categories in the assessments of these states were compared with the same four categories of the National Assessment of Educational Progress (NAEP). While there were close similarities between the corresponding categories, it was risky to make direct comparisons without understanding how the NAEP and state assessments defined performance standards and how each state arrived at its own proficiency category labels. The percentage of students performing at or above high proficiency levels in the Maine and Kentucky assessments were not substantially different from the national assessment results. However, results were not entirely consistent across grades and years, 'a finding attributed to the fact that the definitions of performance standards and the methods of standard setting were different. The sizes of achievement gains from the state's own computations were greater than counterpart gains from the NAEP, something attributed to the high-stakes nature of the state assessments. These findings suggest that policymakers and educators need to become more aware of the uses and limitations of current national and state assessments as education information databases. (Contains 2 figures, 13 tables, and 12 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document.

education policy analysis archives, 2013

This study examines the efficacy and moderators of New York State interventions for schools in ne... more This study examines the efficacy and moderators of New York State interventions for schools in need of improvement under NCLB, including: (1) school transfer, (2) supplementary education service (3) corrective action, (4) planning for restructuring, and (5) restructuring. Despite the fact that schools in increasingly aggressive treatment groups had higher performance gains relative to schools in good standing, propensity score matching analysis results reveal negative or null effects of the interventions. There are indications of treatment effect heterogeneity and the effects varied by the year of implementation and the propensity of treatment assignment (schooling conditions prior to interventions). The findings of our study have implications for both theory of action and program implementation.

Research in Middle Level Education Quarterly, 1998

This study explored the extent to which current adequate yearly progress (AYP) indices reflect re... more This study explored the extent to which current adequate yearly progress (AYP) indices reflect real change or a statistical artifact. The objective, in more specific terms, was to investigate regression artifacts in evaluating schools' academic progress and explore methods to overcome their effects in developing AYP measures. Schools in Kentucky and Maine were studied, using eighth-grade achievement data from the two states' student assessments, the Kentucky Instructional Results Information System and the Maine Educational Assessment. Three different types of school achievement gain estimates were computed: (1) the 1-year gain measure; (2) a 2-year gain measure; and (3) a 3-year gain measure. Time-reversed analyses were conducted to determine whether gain score really was a regression artifact. Findings for both states show that higher performing schools tend to gain less while lower performing schools gain more. This illustrates the well-known regression to the mean status phenomenon. Results also reveal regression to the mean growth phenomenon. Schools that gained more in the past tend to gain relatively less. The first force tends to make higher and lower performing schools appear convergent in their status, and the latter force may make more and less improving schools appear convergent in their growth. These two forces as statistical artifacts may confound AYP measures and need to be addressed. (Contains 4 figures, 11 tables, and 9 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document.

Education reform during the last 2 decades has been shaped by the forces of growing public distru... more Education reform during the last 2 decades has been shaped by the forces of growing public distrust of educational bureaucracies in a climate of rapid political change and growing international competition in the context of a global economy. Major school reforms in foui selected industrial countries that differ significantly in terms of educational institutions and cultures are examined using school reform literature, related government reports, and newspaper articles. Japan and Korea have highly centralized school governance systems and homogeneous educational values. Conversely, in the United States and England, education governance is decentralized, and educational values are relatively heterogeneous. In the latter two countries lack of focus and accountability were identified as major deficiencies of their educational systems, and efforts were made to standardize curriculum, tighten assessment practices, and introduce market-like competition. Similar political and economic challenges in Japan and Korea, on the other hand, resulted in policies to differentiate curriculum, diversify assessment, decentralize school governance, and make the system more diverse and democratic-enhancing whole-person education.

Society For Research on Educational Effectiveness, Mar 9, 2012

Background / Context: While there has been much discussion of the role and function of effect siz... more Background / Context: While there has been much discussion of the role and function of effect sizes in social and behavioral research, there is general agreement that effect sizes are valuable tools to help evaluate the magnitude of a difference or relationship, particularly, whether a statistically significant difference is a difference of practical concern (