Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Methodology, measurement and data

David Grissmer, Thomas White, Richard Buddin, Mark Berends, Daniel Willingham, Jamie DeCoster, Chelsea Duran, Chris Hulleman, William Murrah, Tanya Evans.

The Core Knowledge curriculum is a K-8 curriculum focused on building students General Knowledge about the world they live in that is hypothesized to increase reading comprehension and Reading/English-LA achievement. This study utilizes an experimental design to evaluate the long term effects of attending Charter schools teaching the Core Knowledge curriculum. Fourteen oversubscribed kindergarten lotteries for enrollment in nine Core Knowledge Charter schools using the curriculum had 2310 students applying from parents in predominately middle/high income school districts. State achievement data was collected at 3rd- 6th grade in Reading/English-LA and Mathematics and at 5th Grade in Science. A new methodology addresses two previously undiscovered sources of bias inherent in kindergarten lotteries that include middle/high income families. The unbiased confirmatory Reading-English-LA results show statistically significant ITT (0.241***) and TOT (0.473***) effects for 3rd-6th grade achievement with statistically significant ITT and TOT effects at each grade. Exploratory analyses also showed significant ITT (0.15*) and TOT (0.300*) unbiased effects at 5th grade in Science. A CK-Charter school in a low income school district also had statistically significant, moderate to large unbiased ITT and TOT effects in English Language Arts (ITT= 0.944**; TOT = 1.299**), Mathematics (ITT= 0.735*; TOT = 0.997*) and positive, but insignificant Science effects (ITT= 0.468; TOT = 0.622) that eliminated achievement gaps in all subjects.

More →


Shirin A. Hashim, Thomas Kelley-Kemple, Mary E. Laski.

We propose a new method for estimating school-level characteristics from publicly available census data. We use a school’s location to impute its catchment area by aggregating the nearest n census block groups such that the number of school-aged children in those n block groups is just over the number of students enrolled in that school. We then weight census data by the number of school-aged children in the block-group to estimate school-level measures. We conduct several robustness checks to assess the quality of our estimates and find that our method is broadly successful in replicating known school-level characteristics and producing unbiased estimates for school-level income. This method expands the available set of school-level variables to the broader and richer set of characteristics measured in the census, which can then be used to conduct descriptive and observational research across a long time horizon.

More →


Mengyuan Liang.

Even though women have continuously caught up with men in education attainment and labor market participation since the 1970s, the wage gap between men and women still universally exists today. Do female college graduates still earn less than their male counterparts if men’s and women’s “profiles” of observed productivity-related characteristics are statistically adjusted to be equivalent? To answer this research question and better understand the current gender wage gap, I introduce a novel propensity score stratification method for gender wage gap decomposition. This new method overcomes certain limitations of the traditional Blinder-Oaxaca decomposition method, and provides an example of validly applying propensity score-based methods (mostly used in causal settings) to gender wage gap decomposition, a non-causal setting. Making use of this new method, I analyze a nationally representative sample from the Baccalaureate and Beyond Longitudinal Study, which represents the 1993 Cohort of U.S. college graduates. Through propensity score stratification, the observed productivity-related characteristics between men and women in the sample are statistically adjusted to be equivalent within each stratum of propensity score. After “equalizing” these characteristics, evidence shows the women-to-men wage ratio among this college educated population is still 87.4% at the tenth year after they graduated from college. This remaining gender gap cannot be explained by the observed gender differences in productivity-related characteristics, and is the evidence of a discriminatory wage gap possibly existing in the labor market. Additionally, the unexplained gender wage gap universally exists regardless whether these “profiles” of qualifications and labor market experience are stereotypically female or male. Even acknowledging that this research cannot account for all the gender differences in productivity due to data limitation, the results of this research will add to the empirical evidence of measuring the discriminatory wage gap that possibly exists in the labor market.

More →


Jinyong Hahn, John D. Singleton, Nese Yildiz.

Panel or grouped data are often used to allow for unobserved individual heterogeneity in econometric models via fixed effects. In this paper, we discuss identification of a panel data model in which the unobserved heterogeneity both enters additively and interacts with treatment variables. We present identification and estimation methods for parameters of interest in this model under both strict and weak exogeneity assumptions. The key identification insight is that other periods' treatment variables are instruments for the unobserved fixed effects. We apply our proposed estimator to matched student-teacher data used to estimate value-added models of teacher quality. We show that the common assumption that the return to unobserved teacher quality is the same for all students is rejected by the data. We also present evidence that No Child Left Behind-era school accountability increased the effectiveness of teacher quality for lower performing students.

More →


Brendan Bartanen, Andrew Kwok, Andrew Avitabile, Brian Heseung Kim.

Heightened concerns about the health of the teaching profession highlight the importance of studying the early teacher pipeline. This exploratory, descriptive paper examines preservice teachers' (PST) expressed motivation for pursuing a teaching career and its relationship with PST characteristics and outcomes. Using data from one of the largest teacher education programs in Texas, we use a natural language processing algorithm to categorize into topical groups roughly 2,800 essay responses to the prompt, "Explain why you decided to become a teacher.'' We identify 11 topics that largely reflect altruistic and intrinsic (though not extrinsic) reasons for teaching. The frequency of motivation topics varied substantially by PST gender, race/ethnicity, and certification area. While topics collectively explained little of the variance in PST outcomes, we found preliminary evidence that intrinsic enjoyment of teaching and prior experiences with adversity predicted higher performance during clinical teaching and lower attrition as a full-time K–12 teacher.

More →


Emma R. Hart, Drew H. Bailey, Sha Luo, Pritha Sengupta, Tyler W. Watts.

Fadeout is a pervasive phenomenon: post-test impacts on cognitive skills commonly decrease in the years following an educational intervention. Less is known, although much is theorized, about social-emotional skill persistence. The current meta-analysis investigated whether educational RCT impacts on social-emotional skills demonstrated greater persistence than impacts on cognitive skills among 87 interventions involving 59,237 participants and 443 outcomes measured at post-test and at least one follow-up. For post-test impacts of the same magnitude, persistence rates were similar (43% of post-test magnitude) across skill types for follow-ups occurring 6 to 12 months after post-test. At 1- to 2-year follow-ups, persistence rates were larger for cognitive skills (37%) than for social-emotional skills. Interestingly, smaller posttest impacts persisted at proportionately higher rates than larger impacts, which may benefit interventions measuring social-emotional outcomes given their smaller post-test impacts. Considered in whole, social-emotional and cognitive skills demonstrated similar patterns of fadeout.

More →


Matthew Naven.

Low-socioeconomic status (SES), minority, and male students perform worse than their high-SES, non-minority, and female peers on standardized tests. This paper investigates how within-school differences in school quality contribute to these educational achievement gaps. Using individual-level data on the universe of public-school students in California, I estimate school quality using a value added methodology that accounts for the fact that students sort to schools on observable characteristics. I allow for within-school heterogeneity by estimating a distinct value added for each school's low-/high-SES, minority/non-minority, and male/female students. Standard value added models suggest that on average schools provide less value added to their low-SES, minority, and male students, particularly on postsecondary enrollment. However, value added models that control for neighborhood, older-sibling, and peer characteristics suggest that schools provide similar value added to low-/high-SES students and minority/non-minority students but more value added to female students. Within-school heterogeneity accounts for 6% of the test-score achievement gap and 22% of the difference in postsecondary enrollment between men and women.

More →


Meghan McCormick, Cullen MacDowell, Christina Weiland, JoAnn Hsueh, Michelle Maier, Mirjana Pralica, Samuel Maves, Catherine Snow, Jason Sachs.

This study uses implementation fidelity data from PreK to 1st grade in the Boston Public Schools (BPS) to measure instructional alignment and examine whether stronger alignment is associated with sustained benefits of BPS PreK on children’s language, literacy, and math skills through first grade. The study includes N = 498 students (mean age = 5.47, SD = 0.30 in K fall). Children who experienced strong instructional alignment across grades had faster gains in literacy (SD = .47) and math (SD = .28) skills through the spring of first grade compared with non-BPS PreK attenders. Mis-alignment predicted faster convergence in literacy skills. Results highlight that instructional alignment may help to sustain the initial benefits of PreK programs through first grade in a subset of outcome domains. Implications for further research measuring alignment in a broader range of settings and implications for practice are discussed.

More →


Ariana Audisio, Rebecca Taylor-Perryman, Tim Tasker, Matthew P. Steinberg.

Teachers are the most important school-specific factor in student learning. Yet, little evidence exists linking teacher professional development programs and the strategies or activities that comprise them to student achievement. In this paper, we examine a fellowship model for professional development designed and implemented by Leading Educators, a national nonprofit organization that aims to bridge research and practice to improve instructional quality and accelerate learning across school systems. During the 2015-16 and 2016-17 school years, Leading Educators conducted its fellowship program for two cohorts of teachers and school leaders to provide these educators ongoing, collaborative, job-embedded professional development and to improve student achievement. Relying on quasi-experimental methods, we find that a school’s participation in the fellowship program significantly increased student proficiency rates in English language arts and math on state achievement exams. Student achievement benefitted from a more sustained duration of participation in the fellowship program, varied depending on the share of a school’s educators who participated in the fellowship, and differed based on whether fellows independently selected into the program or were appointed to participate by their school leaders. Taken together, findings from this paper should inform professional learning organizations, schools, and policymakers on the design, implementation, and impact of educator professional development.

More →


Wes Austin, Bengie Chen, Dan Goldhaber, Eric A. Hanushek, Kris Holden, Cory Koedel, Helen F. Ladd, Jin Luo, Eric Parsons, Gregory Phelan, Steven G. Rivkin, Tim Sass, Mavzuna Tureava.

Anecdotal evidence points to the importance of school principals, but the limited existing research has neither provided consistent results nor indicated any set of essential characteristics of effective principals. This paper exploits extensive student-level panel data across six states to investigate both variations in principal performance and the relationship between effectiveness and key certification factors. While principal effectiveness varies widely across states, there is little indication that regulation of the background and training of principals yields consistently effective performance. Having prior teaching or management experience is not related to our estimates of principal value-added.

More →