Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
Advanced course-taking in high school sends an important signal to college admissions officers, helps reduce the cost and time to complete a post-secondary degree, and increases educational attainment and future earnings. However, Black and Hispanic students in the U.S. are underrepresented in Advanced Placement coursework and dual enrollment (i.e. early college). In this paper, we systematically examine the social, demographic, economic, and policy factors that are predictive of racial gaps in AP enrollment and access to DE across the U.S. We find that many of the same factors that predict higher AP access overall also predict higher racial/ethnic gaps in AP, suggesting that policies aimed at increasing AP access need to specifically attend to the inequitable access, rather than simply focusing on increasing access overall. We also find evidence that that might indicate opportunity hoarding by White families contributes to AP gaps – but not DE gaps – suggesting that DE acts as a more equitable avenue for access to college coursework. Our most novel contribution to the literature is our analysis of policies aimed at reducing teacher shortages in high needs areas, in which we find no evidence that the disparities in access to advanced coursework were reduced following implementation of these policies.
In multisite experiments, we can quantify treatment effect variation with the cross-site treatment effect variance. However, there is no standard method for estimating cross-site treatment effect variance in multisite regression discontinuity designs (RDD). This research rectifies this gap in the literature by systematically exploring and evaluating methods for estimating the cross-site treatment effect variance in multisite RDDs. Specifically, we formalize a fixed intercepts/random coefficients (FIRC) RDD model and develop a random effects meta-analysis (Meta) RDD model for estimating cross-site treatment effect variance. We find that a restricted FIRC model works best when the running variables' relationship to the outcome is stable across sites but can be biased otherwise. In those instances, we recommend using either the unrestricted FIRC model or the meta-analysis model; with the unrestricted FIRC model generally performing better when the average number of in-bandwidth observations is less than 120 and the meta-analysis model performing better when the average number of in-bandwidth observations is above 120. We apply our models to a high school exit exam policy in Massachusetts that required students who passed the high school exit exam but were still determined to be nonproficient to complete an ``Education Proficiency Plan" (EPP). We find the EPP policy had a positive local average treatment effect on whether students completed a math course their senior year on average across sites, but that the impact varied enough such that a third of schools could have had a negative impact.
Federal law defines eligibility for English learner (EL) classification differently for Indigenous students compared to non-Indigenous students. Indigenous students, unlike non-Indigenous students, are not required to have a non-English home or primary language. A critical question, therefore, is how EL classification impacts Indigenous students’ educational outcomes. This study explores this question for Alaska Native students, drawing on data from five Alaska school districts. Using a regression discontinuity design, we find evidence that among students who score near the EL classification threshold in kindergarten, EL classification has a large negative impact on Alaska Native students’ academic outcomes, especially in the 3rd and 4th grades. Negative impacts are not found for non-Alaska Native students in the same districts.
A core motivation for the widespread teacher evaluation reforms of the last decade was the belief that these new systems would promote teacher development through high-quality feedback. We examine this theory by studying teachers’ perceptions of evaluation feedback in Boston Public Schools and evaluating the district’s efforts to improve feedback through an administrator training program. Teachers generally reported that evaluators were trustworthy, fair, and accurate, but that they struggled to provide high-quality feedback. We find little evidence the training program improved perceived feedback quality, classroom instruction, teacher self-efficacy, or student achievement. Our results illustrate the challenges of using evaluation systems as engines for professional growth when administrators lack the time and skill necessary to provide frequent, high-quality feedback.
From 2010 onwards, most US states have aligned their education standards by adopting the Common Core State Standards (CCSS) for math and English Language Arts. The CCSS did not target other subjects such as science and social studies. We estimate spillovers of the CCSS on student achievement in non-targeted subjects in models with state and year fixed effects. Using student achievement data from the NAEP, we show that the CCSS had a negative effect on student achievement in non-targeted subjects. This negative effect is largest for underprivileged students, exacerbating racial and socioeconomic student achievement gaps. Using teacher surveys, we show that the CCSS caused a reduction in instructional focus on nontargeted subjects.
Four-day school weeks have proliferated across the United States in recent years, reaching over 650 public school districts in 24 states as of 2019, but little is known about the effects of the four-day school week on high school students. This study uses district-level panel data from Oklahoma and a difference-in-differences research design to provide the first estimates of the causal effect of the four-day school week on high school students’ ACT scores, attendance, and disciplinary incidents during school. Results indicate that four-day school weeks decrease per-pupil bullying incidents by approximately 31% and per-pupil fighting incidents by approximately 27%, but have no detectable effect on other incident types, ACT scores, or attendance.
Poor program implementation constitutes one explanation for null results in trials of educational interventions. For this reason, researchers often collect data about implementation fidelity when conducting such trials. In this article, we document whether and how researchers report and measure program fidelity in recent cluster-randomized trials. We then create two measures—one describing the level of fidelity reported by authors and another describing whether the study reports null results—and examine the correspondence between the two. We also explore whether fidelity is influenced by study size, type of fidelity measured and reported, and features of the intervention. We find that as expected, fidelity level relates to student outcomes; we also find that the presence of new curriculum materials positively predicts fidelity level.
Free and reduced-price meal (FRM) eligibility is commonly used in education research and policy applications as an indicator of student poverty. However, using multiple data sources external to the school system, we show that FRM status is a poor proxy for poverty, with eligibility rates far exceeding what would be expected based on stated income thresholds for program participation. This is true even without accounting for community eligibility for free meals, although community eligibility has exacerbated the problem in recent years. Over the course of showing the limitations of using FRM data to measure poverty, we provide promising validity evidence for a new, publicly-available measure of school poverty based on local-area family incomes.
Using newly available data on all civil rights complaints submitted to the U.S. Department of Education’s Office for Civil Rights related to racial discrimination in discipline between 1999 and 2018, I provide the first systematic evidence on how modern federal civil rights enforcement is used to address racial discrimination in discipline. I find that less than 50 percent of complaints received each year result in a federal investigation. I also find that 70 to 80 percent of investigations are closed due to insufficient evidence of a civil rights violation. Results also suggest that districts with higher shares of minoritized students, higher levels of segregation, and districts with larger racial educational gaps are more likely to receive a civil rights complaint after controlling for other district factors.
Local school boards have primary authority for running educational systems in the U.S. but little is known empirically about the merits of this arrangement. State takeovers of struggling districts represent a rare alternative form of educational governance and have become an increasingly common response to low performance. However, limited research explores whether this effectively improves student outcomes. We track all takeovers nationwide from the late 1980s, when the first takeovers occurred, through 2016 and describe takeover districts. While these districts are low performing, we find academic performance plays less of a role in predicting takeover for districts serving larger concentrations of African American students. We then use a new data source allowing for cross-state comparisons of student outcomes to estimate the effect of takeovers that occurred between 2011 and 2016. On average, we find no evidence that takeover generates academic benefits. Takeover appears to be disruptive in the early years of takeover, particularly to English Language Arts achievement, although the longer-term effects are less clear. We also observe considerable heterogeneity of effects across districts. Takeovers were least effective in districts with higher baseline achievement and least harmful in majority Latinx communities. Leaders should be cautious about using takeover without considering local context and a better understanding of why some takeovers are more effective than others.