Search for EdWorkingPapers here by author, title, or keywords.
Multiple outcomes of education
This paper describes and evaluates a web-based coaching program designed to support teachers in implementing Common Core-aligned math instruction. Web-based coaching programs can be operated at relatively lower costs, are scalable, and make it more feasible to pair teachers with coaches who have expertise in their content area and grade level. Results from our randomized field trial document sizable and sustained effects on both teachers’ ability to analyze instruction and on their instructional practice, as measured the Mathematical Quality of Instruction (MQI) instrument and student surveys. However, these improvements in instruction did not result in corresponding increases in math test scores as measured by state standardized tests or interim assessments. We discuss several possible explanations for this pattern of results.
Researchers commonly interpret effect sizes by applying benchmarks proposed by Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this paper, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, cost, and scalability into the process of interpreting the policy importance of effect sizes.
The Milwaukee Parental Choice Program (MPCP), the first modern private school choice program in the United States, has grown from 341 students attending 7 private schools in 1990 to 27,857 students attending 126 private schools in 2019. The MPCP has been subject to extensive study focused largely on student performance on standardized tests. This study presents new data on the college enrollment, persistence, and graduation of MPCP and MPS students who were tracked over 12 years beginning in 2006. MPCP participants are compared with a matched sample of MPS students who lived in the same neighborhood and had similar demographic characteristics and test scores at the beginning of the study. The collective evidence in this paper indicates that students in the MPCP program have greater educational attainment than the comparison group, as measured by college experience and outcomes. Most of the college attainment benefits of the MPCP are clear for both students who were in ninth grade at the beginning of the study, for whom positive attainment effects have previously been reported, and students who were initially enrolled in grades three through eight, who we examine here for the first time. As of 2018, MPCP students have spent more total years in a four-year college than their MPS peers. The MPCP students in the grade three through eight sample attained college degrees at rates that are statistically significantly higher than their matched MPS peers.
We examine the effect of admission to 16 stand-alone technical high schools within the Connecticut Technical High School System (CTHSS) on student educational and labor market outcomes. To identify the causal effect of admission on student outcomes, we exploit the fact that CTHSS utilizes a score-based admissions system and identify the effect of admission using a regression discontinuity approach. We find that male students attending one of the technical high schools are approximately 10 percentage points more likely to graduate from high school and 8 percentage points less likely to attend college, although there is some evidence that the negative effects on college attendance fade over time. We also find that male students attending a technical high school have quarterly earnings that are approximately 31% higher. Analyses of potential mechanisms behind these results reveal that male students that attend a technical high school have higher 9th grade attendance rates and higher 10th grade test scores. We find little evidence that attending a technical high school affects the educational or labor outcomes of women. These effects appear relatively broad based across different types of students in that we find little evidence of heterogeneity in these effects over student attributes like race and ethnicity, free lunch eligibility or residence in a poor, central city school district. However, when distinguishing between students based on the Career and Technical Education (CTE) offerings of the high school that these students likely would have attended, we find that the effects of admission to a CTHSS school are noticeably larger when the counterfactual high school has less CTE offerings.
Using novel variation in special education and English Language Learner classification from admissions lotteries, I find that students can achieve large academic gains without specialized services. Enrolling in a Boston charter school doubles the likelihood that students lose their special education or English Language Learner status, but exposes students to a high-performing general education program that includes high intensity tutoring, data driven instruction, and increased instructional time. The positive effects extend to college: charters nearly double the likelihood that English Language Learners enroll in four-year colleges and quadruple the likelihood that special education students graduate from two-year college. A multiple instrument strategy suggests that high quality general education practices drive the gains and finds no detrimental effect from lower classification rates.
New advances in neurobiology are revealing that brain development and the learning it enables are directly dependent on social-emotional experience. Growing bodies of research reveal the importance of socially-triggered epigenetic contributions to brain development and brain network configuration, with implications for social-emotional functioning, cognition, motivation and learning. Brain development is also impacted by health-related and physical developmental factors, such as sleep, toxin exposure, and puberty, which in turn influence social-emotional functioning and cognition. An appreciation of the dynamic interdependencies of social-emotional experience, health-related factors, brain development and learning underscores the importance of a “whole child” approach to education reform, and leads to important insights for research on Social-Emotional Learning (SEL). To facilitate these interdisciplinary conversations, here we conceptualize within a developmental framework current evidence on the fundamental and ubiquitous biological constraints and affordances undergirding SEL-related constructs and learning more broadly. Learning indeed depends on how nature is nurtured.
The sustaining environments hypothesis refers to the popular idea, stemming from theories in developmental, cognitive, and educational psychology, that the long-term success of early educational interventions is contingent on the quality of the subsequent learning environment. Several studies have investigated whether specific kindergarten classroom and other elementary school factors account for patterns of persistence and fadeout of early educational interventions. These analyses focus on the statistical interaction between an early educational intervention – usually whether the child attended preschool – and several measures of the quality of the subsequent educational environment. The key prediction of the sustaining environments hypothesis is a positive interaction between these two variables. To quantify the strength of the evidence for such effects, we meta-analyze existing studies that have attempted to estimate interactions between preschool and later educational quality in the United States. We then attempt to establish the consistency of the direction and a plausible range of estimates of the interaction between preschool attendance and subsequent educational quality by using a specification curve analysis in a large, nationally representative dataset that has been used in several recent studies of the sustaining environments hypothesis. Both analyses yield interaction estimates of approximately 0. We consider a range of plausible explanations, including 1) the null hypothesis is true, 2) we did not have statistical power to detect interactions of a realistic magnitude, 3) model misspecification because of theoretical ambiguity, and 4) heterogeneity of these interactions across treatments, contexts, and children. We offer a set of recommendations for future research on the sustaining environments hypothesis.
The vast majority of literature on school choice, and charter schools in particular, focus on attending an elementary or middle school and often focus on test scores or other proximal outcomes. Much less is known about the long-term effects of attending a charter high school. It is important to fill this information void for a few reasons. First, schools in general affect more than just students’ test scores. Second, high schools make up a larger share of the charter sector. Third, school choice depends on freely available information for parents and students to make informed decisions about where to attend, including potential long-term benefits. We add to the empirical research on charter school effects by using a doubly-robust regression-adjusted propensity score matching approach to evaluate the impacts of charter high school attendance on 9th grade behavioral outcomes and individuals propensity to commit crime and participate in elections as young adults in North Carolina, a state with a large and growing charter school sector.
Student loan borrowing for higher education has emerged as a top policy concern. Policy makers at the institutional, state, and federal levels have pursued a variety of strategies to inform students about loan origination processes and how much a student has cumulatively borrowed, and to provide students with greater access to loan counseling. We conducted an experiment to evaluate the impact of an outreach campaign that prompted loan applicants at a large community college to make informed and active borrowing decisions and that offered them access to remote, one-onone assistance from a loan counselor. The intervention led students to reduce their unsubsidized loan borrowing by 7 percent, resulted in worse academic performance, and increased the likelihood of loan default during the three years after the intervention occurred. Our results suggest policy makers and higher education leaders should carefully examine the potential unintended consequences of efforts to reduce student borrowing, particularly in light of growing evidence regarding the counter-intuitive positive relationship between reduced borrowing levels and worse student academic and financial outcomes.
This paper considers an unavoidable feature of the school environment – what are the long run effects of a student’s ordinal rank in elementary school? Using administrative data from all public school students in Texas, we show that students with a higher third grade academic rank, conditional on ability and classroom fixed effects, have higher subsequent test scores, are more likely to take AP classes, to graduate high school, and to enroll in college, and ultimately have higher earnings 19 years later. The paper concludes by exploring the tradeoff between higher quality schools and higher rank in the presence of these rank-based peer effects.