Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

Michael Dinerstein, Isaac M. Opper.

What happens when employers would like to screen their employees but only observe a subset of output? We specify a model in which heterogeneous employees respond by producing more of the observed output at the expense of the unobserved output. Though this substitution distorts output in the short-term, we derive three sufficient conditions under which the heterogenous response improves screening efficiency: 1) all employees place similar value on staying in their current role; 2) the employees' utility functions satisfy a variation of the traditional single-crossing condition; 3) employer and worker preferences over output are similar. We then assess these predictions empirically by studying a change to teacher tenure policy in New York City, which increased the role that a single measure -- test score value-added -- played in tenure decisions. We show that in response to the policy teachers increased test score value-added and decreased output that did not enter the tenure decision. The increase in test score value-added was largest for the teachers with more ability to improve students' untargeted outcomes, increasing their likelihood of getting tenure. We find that the endogenous response to the policy announcement reduced the screening efficiency gap -- defined as the reduction of screening efficiency stemming from the partial observability of output -- by 28%, effectively shifting some of the cost of partial observability from the post-tenure period to the pre-tenure period.

More →


Monnica Chan, Zachary Mabel, Preeya Pandya Mbekeani.

Performance-based funding models for higher education, which tie state support for institutions to performance on student outcomes, have proliferated in recent decades. Some states have designed these policies to also address educational attainment gaps by including bonus payments for traditionally low-performing groups. Using a Synthetic Control Method research design, we examine the impact of these funding regimes on race-based completion gaps in Tennessee and Ohio. We find no evidence that performance-based funding narrowed race-based completion gaps. In fact, contrary to their intended purpose, we find that performance-based funding widened existing gaps in certificate completion in Tennessee. Across both states, the estimated impacts on associate degree outcomes are also directionally consistent with performance-based funding exacerbating racial inequities in associate degree attainment.

More →


Kevin J. Mumford.

Financing college expenses through an income share agreement (ISA) is an arrangement where the student agrees to pay a fixed percentage of future earned income for a designated period of time in exchange for college funding. Using administrative and survey data for all eligible applicants to a university ISA program, I estimate the adverse selection into the ISA and provide preliminary estimates of the moral hazard for ISA participants. Identification of adverse selection comes from being able to observe the full set of eligible students who apply to the program. There is evidence of selection on the offered income share rate (which is determined by the student’s major) as well as on parent characteristics, though not parent income. Surprisingly, there is no evidence of adverse selection on student ability as measured by SAT score and college grades. I find no differential selection on other student characteristics including demographics and measures of debt aversion, risk aversion, and time preference. Controlling for observable factors, ISA participation increases the likelihood of college graduation by 3 percentage points and decreases starting salary by $5,000 on average.

More →


Kathleen Lynch, Lily An, Zid Mancenido.
We present results from a meta-analysis of 37 contemporary experimental and quasi-experimental studies of summer programs in mathematics for children in Grades pre-K-12, examining what resources and characteristics predict stronger student achievement. Children who participated in summer programs that included mathematics activities experienced significantly better mathematics achievement outcomes, compared to their control group counterparts. We find an average weighted impact estimate of +0.10 standard deviations on mathematics achievement outcomes. We find similar effects for programs conducted in higher- and lower-poverty settings. We undertook a secondary analysis exploring the effect of summer programs on non-cognitive outcomes and found positive mean impacts. The results indicate that summer programs are a promising tool to strengthen children’s mathematical proficiency outside of school time.

More →


Mark J. Chin.

In this paper I study how school desegregation by race following Brown v. Board of Education affected White individuals’ racial attitudes and politics in adulthood. I use geocoded nationwide data from the General Social Survey and differences-in-differences to identify causal impacts. Integration significantly reduced White individuals’ political conservatism as adults in the U.S. South but not elsewhere. I observe similar geographic impact heterogeneity for individuals’ attitudes towards Blacks and policies promoting racial equity, but positive effects emerge less consistently across specifications. Results suggest that this heterogeneity may depend on the effectiveness of integration policies. In the south, Black-White exposure was greater following desegregation, and White disenrollment was lower. My study provides the first causal evidence on how different theories concerning intergroup contact and racial attitudes (i.e., the contact and racial threat hypotheses) may have applied to school contexts following historic court mandates to desegregate.

More →


Sarah R. Cohodes, Helen Ho, Silvia C. Robles.

The federal government and many individual organizations have invested in programs to support diversity in the STEM pipeline, including STEM summer programs for high school students, but there is little rigorous evidence of their efficacy. We fielded a randomized controlled trial to study a suite of such programs targeted to underrepresented high school students at an elite, technical institution. The STEM summer programs differ in their length (one week, six weeks, or six months) and modality (on-site or online). Students offered seats in the STEM summer programs are more likely to enroll in, persist through, and graduate from college, with gains in institutional quality coming from both the host institution and other elite universities. The programs also increase the likelihood that students graduate with a degree in a STEM field, with the most intensive program increasing four-year graduation with a STEM degree attainment by 33 percent. The shift to STEM degrees increases potential earnings by 2 to 6 percent. Program-induced gains in college quality fully account for the gains in graduation, but gains in STEM degree attainment are larger than predicted based on institutional differences.

More →


Stephanie Owen.

Beliefs about relative academic performance may shape field specialization and explain gender gaps in STEM enrollment, but little causal evidence exists. To test whether these beliefs are malleable and salient enough to change behavior, I run a randomized controlled trial with 5,700 undergraduates across seven introductory STEM courses. Providing relative performance information shrinks gender gaps in biased beliefs substantially and closes ten percent of the gender gap in subsequent STEM course-taking. The gap closes due to men taking fewer STEM credits; women’s behavior is unchanged, implying that male overconfidence rather than female underconfidence contributes to gaps in specialization. Beliefs matter, but may not be a useful target for facilitating female STEM participation.

More →


Jackie Eunjung Relyea, James S. Kim, Patrick Rich.

The current study replicated and extended the previous findings of content-integrated literacy intervention focusing on its effectiveness on first- and second-grade English learners’ (N = 1,314) reading comprehension, writing, vocabulary knowledge, and oral proficiency. Statistically significant findings were replicated on science and social studies vocabulary knowledge (ES = .51 and .53, respectively) and argumentative writing (ES = .27 and .41, respectively). Furthermore, treatment group outperformed control group on reading (ES = .08) and listening comprehension (ES = .14). Vocabulary knowledge and oral proficiency mediated treatment effects on reading comprehension, whereas only oral proficiency mediated effects on writing. Findings replicate main effects on vocabulary knowledge and writing, while also extending previous research by highlighting mechanisms underlying improved reading comprehension and writing.

More →


Angel H. Harris, Darryl V. Hill, Matthew A. Lenard.

Disparities in gifted representation across demographic subgroups represents a large and persistent challenge in U.S. public schools. In this paper, we measure the impacts of a school-wide curricular intervention designed to address such disparities. We implemented Nurturing for a Bright Tomorrow (NBT) as a cluster randomized trial across elementary schools with the low gifted identification rates in one of the nation’s largest school systems. NBT did not boost formal gifted identification or math achievement in the early elementary grades. It did increase reading achievement in select cohorts and broadly improved performance on a gifted identification measure that assesses nonverbal abilities distinct from those captured by more commonly used screeners. These impacts were driven by Hispanic and female students. Results suggest that policymakers consider a more diverse battery of qualifying exams to narrow disparity gaps in gifted representation and carefully weigh tradeoffs between universal interventions like NBT and more targeted approaches.

More →


James S. Kim, Patrick Rich, Ethan Scherer.

To address the challenge of improving third grade reading comprehension, we developed and evaluated the long-term effects of a sustained content literacy intervention called the Model of Reading Engagement (MORE), which emphasizes building domain and topic knowledge schemas from Grade 1 to Grade 3. The MORE intervention emphasizes thematic lessons that provide an intellectual framework for helping students connect new learning to a general schema (e.g., how scientists study past events, how systems function properly). Over three years, the treatment group students participated in (a) spring Grade 1 thematic content literacy lessons in science and social studies, (b) fall to spring Grade 2 thematic content literacy lessons in science, (c) remote Grade 3 thematic content literacy lessons in science, and (d) wide reading of thematically related informational texts in the summer months following Grade 1 and Grade 2. During the third grade school year (SY 2020-21), the COVID-19 pandemic required remote schooling to be in place from fall to spring and the Grade 3 MORE was provided to both treatment and control students. Accordingly, we examine long-term effects on third graders’ outcomes comparing a treatment group that received the Grade 1, Grade 2, and Grade 3 MORE treatment to a control condition that received the Grade 3 MORE treatment. Intent-to-treat estimates show that the students randomly assigned to the treatment condition outperformed control students in reading comprehension (ES = 0.11) and mathematics (ES = 0.14) on third grade state standardized assessments. Subgroup analyses also revealed positive impacts for student living in low- to moderate-socioeconomic status neighborhoods on both reading comprehension (ES = .13) and mathematics (ES = .20). Findings indicate that a sustained content literacy intervention may be a scalable approach for accelerating and equalizing third-graders’ reading comprehension and math outcomes.

More →