Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Matthew A. Kraft, John A. List, Jeffrey A. Livingston, Sally Sadoff.

A substantial body of experimental evidence demonstrates that in-person tutoring programs can have large impacts on K-12 student achievement. However, such programs typically are costly and constrained by a limited local supply of tutors. In partnership with CovEducation (CovEd), we conduct a pilot program that has potential to ease both of these concerns. We conduct an experiment where volunteer tutors from all over the country meet 1-on-1 with middle school students online during the school day. We find that the program produces consistently positive (0.07σ for math and 0.04σ for reading) but statistically insignificant effects on student achievement. While these estimates are notably smaller than those found in many higher-dosage in-person tutoring programs, they are from a significantly lower-cost program that was delivered within the challenging context of the COVID-19 pandemic. We provide evidence that is consistent with a dosage model of tutoring where additional hours result in larger effects.

More →

Katharine Meyer, Lindsay C. Page, Eric Smith, B. Tyler Walsh, C. Lindsey Fifield, Michael Evans.

Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on a randomized controlled trial testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation. We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by eight percentage points. We find large and significant treatment effects for first-generation students, estimating the intervention increased their final course grades by about 11 points on a 100-point scale (and a 16 percentage point increase in earning a B or higher) as well as their completion of and performance on individual course deliverables (e.g., readings, activities, exams).

More →

Jing Liu, Monica Lee, Xiaoxue Zhou.

Student absenteeism is often conceptualized and quantified in a static, uniform manner, providing an incomplete understanding of this important phenomenon. Applying growth curve models to detailed class-attendance data, we document that secondary school students' unexcused absences grow steadily throughout a school year and over grades, while the growth of excused absences remain essentially unchanged. Importantly, students starting the school year with a high number of unexcused absences, Black and Hispanic students, and low-income students accumulate unexcused absences at a significantly faster rate than their counterparts. Lastly, students with higher growth rates in unexcused absences consistently report lower perceptions of all aspects of school culture than their peers. Interventions targeting unexcused absences and/or improving school culture can be crucial to mitigating disengagement.

More →

Michael F. Lovenheim, Jonathan Smith.

Early research on the returns to higher education treated the postsecondary system as a monolith.  In reality, postsecondary education in the United States and around the world is highly differentiated, with a variety of options that differ by credential (associates degree, bachelor’s degree, diploma, certificate, graduate degree), the control of the institution (public, private not-for-profit, private for-profit), the quality/resources of the institution, field of study, and exposure to remedial education. In this Chapter, we review the literature on the returns to these different types of higher education investments, which has received increasing attention in recent decades. We first provide an overview of the structure of higher education in the U.S. and around the world, followed by a model that helps clarify and articulate the assumptions employed by different estimators used in the literature. We then discuss the research on the return to institution type, focusing on the return to two-year, four-year, and for-profit institutions as well as the return to college quality within and across these institution types. We also present the research on the return to different educational programs, including vocational credentials, remedial education, field of study, and graduate school. The wide variation in the returns to different postsecondary investments that we document leads to the question of how students from different backgrounds sort into these different institutions and programs. We discuss the emerging research showing that lower-SES students, especially in the U.S., are more likely to sort into colleges and programs with lower returns as well as results from recent U.S.-based interventions and policies designed to support success among students from disadvantaged backgrounds. The Chapter concludes with some broad directions for future research.

More →

Douglas D. Ready, Jeanne L. Reid.

New York City’s Pre-K for All (PKA) is the Nation’s largest universal early childhood initiative, currently serving some 70,000 four-year-olds. Stemming from the program’s choice architecture as well as the City’s stark residential segregation, PKA programs are extremely segregated by child race/ethnicity. Our current study explores the complex forces that influence this segregation, including the interplay between family choices, seat availability, site-level enrollment priorities, and the PKA algorithm that weighs these and other considerations. We find that a majority of PKA segregation lies within rather than between local communities, suggesting that reducing segregation would not necessarily require families to choose programs far from home. On a more troubling note, areas with increased options and greater racial/ethnic diversity also exhibit the most extreme segregation.

More →

Christopher Doss, Hans Fricke, Susanna Loeb, Justin B. Doromal.

This study assesses the effects of two text messaging programs for parents that aim to support the development of math skills in prekindergarten students. One program focuses purely on math, while the other takes an identical approach but focuses on a combination of math, literacy, and social-emotional skills. We find no evidence that the math-only program benefits children’s math development. However, the combination program shows greater promise, particularly for girls. Quantile regressions indicate that the effects are concentrated in the lower half of the outcome distribution. We discuss and provide evidence for various hypotheses that could explain these differences.


  • We test the effects of two text messaging curricula that leverage behavioral economics principles to help parents support the math development of prekindergarteners in the home.
  • We find that a program that cycles through literacy, mathematics, and social-emotional skills increases math achievement for girls, while a program focusing solely on mathematics has no effects.
  • Benefits for girls are concentrated on those with weaker performance on mathematics assessments.
  • We posit potential mechanisms based on the literature.

More →

Dan Goldhaber, Cyrus Grout, Kris Holden, Josh B. McGee.

Over the last two decades, twenty-two states have moved away from traditional defined benefit (DB) pension systems and toward pension plan structures like the defined contribution (DC) plans now prevalent in the private sector. Others are considering such a reform as it is seen as a means of limiting future pension funding risk. It is important to understand the implications of such reforms for end-of-career exit patterns and workforce composition. Empirical evidence on the relationship between pension plan structure and retirement timing is currently limited, primarily because, most state pension reforms are so new that few employees enrolled in those alternative plans have reached retirement age. An exception, and the subject of our analysis, is the teacher retirement system in Washington State, which introduced a hybrid DB-DC plan in 1996 and allowed employees in its traditional DB plan to transfer into the new plan. Our analysis focuses on a years-of-service threshold, the crossing of which grants employees early retirement eligibility and, in many cases, a large upward shift in retirement wealth. The financial implications of crossing this threshold are far greater under the state’s traditional DB plan than under the hybrid plan. We find that employees are responsive to crossing the years-of-service threshold, but we fail to find significant evidence that the propensity to exit the workforce varies according to plan enrollment.

More →

Maxwell J. Cook, Cory Koedel, Michael Reda.

We estimate the education and earnings returns to enrolling in technical two-year degree programs at community colleges in Missouri. A unique feature of the Missouri context is the presence of a highly-regarded, nationally-ranked technical college: State Technical College of Missouri (State Tech). Compared to enrolling in a non-technical community college program, we find that enrolling in a technical program at State Tech greatly increases students’ likelihoods of graduation and earnings. In contrast, there is no evidence that technical education programs at other Missouri community colleges increase graduation rates, and our estimates of the earnings impacts of these other programs are much smaller than for State Tech. Our findings exemplify the importance of institutional differences in driving the efficacy of technical education and suggest great potential for high-quality programs to improve student outcomes.

More →

Leiah Groom-Thomas, Monica Lee, Cate Smith Todd, Kathleen Lynch, Susanna Loeb, Scott McConnell, Lydia Carlis.

Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing COVID-19 pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on adaptable practices of assessing young children during the pandemic. We detail practices used to assess learning in 336 Head Start children across four states during three different time periods in the 2020-21 school year, using adaptation of traditionally in-person assessments of early numeracy, early literacy, and executive functioning. In doing so, we distill early lessons for the field from the application of a novel, virtual assessment method with the early childhood population. The paper describes adaptations of assessment administration for virtual implementation and incorporation of feedback into continued virtual delivery of assessments. Applications and limitations in broader contexts are discussed.

More →

Kathleen Lynch, Monica Lee, Susanna Loeb.

The COVID-19 pandemic’s impact on preschool children’s school readiness skills remains understudied. This research investigates whether exposure to in-person (versus virtual) Head Start preschool predicted children’s early numeracy, literacy, and executive function outcomes during a pandemic-affected school year, using a novel virtual assessment methodology. Study children (N = 336; mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) experienced low in-person preschool exposure compared to national pre-pandemic norms. However, study children experienced gains during the pandemic-affected year of 0.08 SD in executive function, 0.34 SD in print knowledge, and 0.49-0.75 in early numeracy skills. For two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool.

More →