Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Robert P. Strauss.

This paper compares and contrasts two required building level school violence measures under NCLB, arrests and incidents of well-defined school misconduct acts, across 20 years of Pennsylvania’s approximately 3,000 public school buildings. Generally, both arrests for school violence and incidents of school violence are rare events. Over 20 years, the third quartile arrest rate was zero and, the third quartile incident rate was 3.3%. Relatively few, 4.1% overall, of Pennsylvania’s school buildings were persistently dangerous as defined and reported pursuant to Pennsylvania’s state plan to the US Department of Education; however, these buildings represented about 7.8% of the student population statewide. When we measure whether or not a school building is dangerous based on reported school violence incidents, that is without an arrest requirement, fully 36.9% of Pennsylvania’school buildings were dangerous, and they represented 46.7% of the students statewide. Both Philadelphia and Pittsburgh public school buildings were disproportionately unsafe and among the top 20 districts in the state which were unsafe over the 20 year study period.

Exploratory regression analysis of mean building scale scores for math and language arts explained about 58% of the variation in such learning outcome measures. As expected, household poverty, holding all else constant, has very strong, negative effects on learning outcomes. A school building composed entirely of low income students will score about 240 scale points lower, about 1.24 standard deviations lower, than a school building without any low income students. A school building at the 90th percentile in terms of student misconduct and poverty rates, would have lower student test scores by about 1 to 1.28 standard deviations. Were a school administrator to reduce student misconduct rates from the 90th percentile to the 50th percentile, our regression coefficients predict learning gains on the order of (100-43) = 2/3 of a standard deviation in mean scale scores.

More →


Sarah A. Cordes, Christopher Rick, Amy Ellen Schwartz.

School buses may be a critical education policy lever, breaking the link between schools and neighborhoods and facilitating access to school choice. Yet little is known about the commute for bus riders, including the average length of the bus ride or whether long commutes harm academic outcomes. We begin to fill this gap using data from New York City to explore the morning commutes of over 120,000 bus riders. We find that long bus rides are uncommon and that those with long bus rides are disproportionately Black and more likely to attend charter or district-choice schools. We find deleterious effects of long bus rides on attendance and chronic absenteeism of district-choice students.

More →


Zachary Bleemer, Aashish Mehta.

Underrepresented minority (URM) college students have been steadily earning degrees in relatively less-lucrative fields of study since the mid-1990s. A decomposition reveals that this widening gap is principally explained by rising stratification at public research universities, many of which increasingly enforce GPA restriction policies that prohibit students with poor introductory grades from declaring popular majors. We investigate these GPA restrictions by constructing a novel 50-year dataset covering four public research universities' student transcripts and employing a staggered difference-in-difference design around the implementation of 29 restrictions. Restricted majors’ average URM enrollment share falls by 20 percent, which matches observational patterns and can be explained by URM students’ poorer average pre-college academic preparation. Using first-term course enrollments to identify students who intend to earn restricted majors, we find that major restrictions disproportionately lead URM students from their intended major toward less-lucrative fields, driving within-institution ethnic stratification and likely exacerbating labor market disparities.

More →


M. Danish Shakeel, Paul E. Peterson.

Principals (policymakers) disagree as to whether U. S. student performance has changed over the past half century. To inform conversations, agents administered seven million psychometrically linked tests in math (m) and reading (rd) in 160 survey waves to national probability samples of cohorts born between 1954 and 2007. Estimated change in standard deviations (sd) per decade varies by agent (m: -0.10sd to 0.27sd, rd: -0.02sd to 0.12sd). Consistent with Flynn effects, median trends show larger gains in m (0.19sd) than rd (0.04sd), though rates of progress for cohorts born since 1990 have increased in rd but slowed in m. Greater progress is shown by students tested at younger ages (m: 0.31sd, rd: 0.08sd) than when tested in middle years of schooling (m: 0.17sd, rd: 0.03sd) or toward end of schooling (m: 0.06sd, rd: 0.02sd). Young white students progress more slowly (m: 0.28sd, rd: 0.09sd) than Asian (m: 46sd, rd: 0.28sd), black (m: 0.36sd, rd: 0.19sd) and Hispanic (m: 0.29sd, rd: 0.13sd) students. These ethnic differences generally attenuate as students age. Young students in the bottom quartile of the SES distribution show greater progress than those in the top quartile (difference in m: 0.08sd, in rd: 0.15sd), but the reverse is true for older students. Moderators likely include not only changes in families and schools but also improvements in nutrition, health care, and protection from contagious diseases and environmental risks. International data suggest that subject and age differentials may be due to moderators more general than just the United States.

More →


Robert C. Carr, Tyler Watts, Jade M. Jenkins, Yu Bai, Ellen S. Peisner-Feinberg, Clara G. Muschkin, Helen F. Ladd, Kenneth A. Dodge.

Prior research has found that financial investments in North Carolina’s early childhood education programs—Smart Start and NC Pre-K—generated positive effects on student achievement in reading and mathematics through eighth grade (Bai et al., 2020). The current study examined if these effects were moderated by two dimensions of educational opportunity in NC public school districts, as measured by (1) the average academic achievement level in third grade and (2) the rate of growth in academic achievement from third to eighth grade. The Smart Start effect on eighth grade reading achievement was larger in school districts with higher levels of average achievement. Also, the NC Pre-K effect on eighth grade reading achievement was smaller in school districts with higher rates of achievement growth.

More →


Reagan Mozer, Luke Miratrix, Jackie Eunjung Relyea, James S. Kim.

In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This process is both time and labor-intensive, which creates a persistent barrier for large-scale assessments of text. Furthermore, enriching ones understanding of a found impact on text outcomes via secondary analyses can be difficult without additional scoring efforts. Machine-based text analytic and data mining tools offer one potential avenue to help facilitate research in this domain. For instance, we could augment a traditional impact analysis that examines a single human-coded outcome with a suite of automatically generated secondary outcomes. By analyzing impacts across a wide array of text-based features, we can then explore what an overall change signifies, in terms of how the text has evolved due to treatment. In this paper, we propose several different methods for supplementary analysis in this spirit. We then present a case study of using these methods to enrich an evaluation of a classroom intervention on young children’s writing. We argue that our rich array of findings move us from “it worked” to “it worked because” by revealing how observed improvements in writing were likely due, in part, to the students having learned to marshal evidence and speak with more authority. Relying exclusively on human scoring, by contrast, is a lost opportunity.

More →


Biraj Bisht, Zachary LeClair, Susanna Loeb, Min Sun.

Paraeducators perform multiple roles in U.S. classrooms, including among others preparing classroom activities, working with students individually and in small groups, supporting individualized programming for students with disabilities, managing classroom behavior, and engaging with parents and communities. Yet, little research provides insights into this key group of educators. This study combines an analysis of national administrative data to describe the paraeducator labor market with a systematic review of collective bargaining agreements and other job-defining documents in ten case-study districts. We find a large and expanding labor market of paraeducators, far more diverse along ethnic and racial lines than certified teachers but with far lower wages, fewer performance incentives, less professional development, and fewer opportunities for advancement within the profession.

More →


Eric Bettinger, Benjamin L. Castleman, Alice Choe, Zachary Mabel.

Nearly half of students who enter college do not graduate. The majority of efforts to increase college completion have focused on supporting students before or soon after they enter college, yet many students drop out after making significant progress towards their degree. In this paper, we report results from a multi-year, large-scale experimental intervention conducted across five states and 20 broad-access, public colleges and universities to support students who are late in their college career but still at risk of not graduating. The intervention provided these “near-completer” students with personalized text messages that encouraged them to connect with campus-based academic and financial resources, reminded them of upcoming and important deadlines, and invited them to engage (via text) with campus-based advisors. We find little evidence that the message campaign affected academic performance or attainment in either the full sample or within individual higher education systems or student subgroups. The findings suggest low-cost nudge interventions may be insufficient for addressing barriers to completion among students who have made considerable academic progress.

More →


Dylan Lukes, Christopher Cleveland.
Between 1935-1940 the Home Owners' Loan Corporation (HOLC) assigned A (minimal risk) to D (hazardous) grades to neighborhoods that reflected their lending risk from previously issued loans and visualized these grades on color-coded maps, which arguably influenced banks and other mortgage lenders to provide or deny home loans within residential neighborhoods. In this study, we leverage a spatial analysis of 144 HOLC-graded core-based statistical areas (CBSAs) to understand how HOLC maps relate to current patterns of school and district funding, school racial diversity, and school performance. We find that schools and districts located today in historically redlined D neighborhoods have less district per-pupil total revenues, larger shares of Black and non-White student bodies, less diverse student populations, and worse average test scores relative to those located in A, B, and C neighborhoods. Conversely, at the school level, we find that per-pupil total expenditures are better for those schools operating in previously redlined D neighborhoods. Consequently, these schools also have the largest shares of low-income students. Our nationwide results are, on the whole, consistent by region and after controlling for CBSA. Finally, we document a persistence in these patterns across time, with overall positive time trends regardless of HOLC security rating but widening gaps between D vs. A, B, and C outcomes. These findings suggest that education policymakers need to consider the historical implications of redlining and past neighborhood inequality on neighborhoods today when designing modern interventions focused on improving the life outcomes of students of color and students from low-socioeconomic backgrounds.

More →


Lauren Sartain, Matthew P. Steinberg.

Personnel evaluation systems have historically failed to identify and remediate low-performing teachers. In 2012, Chicago Public Schools implemented an evaluation system that incorporated remediation and dismissal plans for low-rated teachers. Regression discontinuity estimates indicate that the evaluation reform increased the exit of low-rated tenured teachers by 50 percent. The teacher labor supply available to replace low-rated teachers was higher performing on multiple dimensions, and instrumental variables estimates indicate that policy-induced exit of low-rated teachers significantly improved teacher quality in subsequent years. Policy simulations show that the teacher labor supply in Chicago is sufficient to remove significantly more low-performing teachers.

More →