Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.


Taylor Odle, Lauren Russell.

Reverse transfer associate degrees are credentials retroactively awarded to current bachelor’s degree seekers that combine current four-year credits with credits previously earned at a community college. Providing students with an associate degree may not only increase motivation and persistence en route to completing a bachelor’s but may also provide important labor market benefits by way of increased marketability and earnings potential. Despite the proliferation of reverse transfer policies across at least 15 states to date, there is no causal evidence documenting their effect on students’ outcomes. Leveraging administrative data from Tennessee matched with records on its statewide reverse transfer program and a difference-in-differences design, we find reverse transfer degrees generally have little impact on students’ short- and intermediate-term academic and labor market outcomes. Our results point to suggestive yet small positive gains in GPA and short-term employment for recipients, but these estimates accompany no impacts on bachelor’s degree attainment and estimates that confidently reject any meaningful impacts on recipients’ earnings. Our findings contrast those of existing descriptive works on reverse transfer that reported large benefits for students, due in part to our methodological improvements and more robust data. These findings should guide policymakers considering the adoption, design, and ongoing operation of reverse transfer programs.

More →

Brian Heseung Kim, Katharine Meyer, Alice Choe.

Interactive, text message-based advising programs have become an increasingly common strategy to support college access and success for underrepresented student populations. Despite the proliferation of these programs, we know relatively little about how students engage in these text-based advising opportunities and whether that relates to stronger student outcomes – factors that could help explain why we’ve seen relatively mixed evidence about their efficacy to date. In this paper, we use data from a large-scale, two-way text advising experiment focused on improving college completion to explore variation in student engagement using nuanced interaction metrics and automated text analysis techniques (i.e., natural language processing). We then explore whether student engagement patterns are associated with key outcomes including persistence, GPA, credit accumulation, and degree completion. Our results reveal substantial variation in engagement measures across students, indicating the importance of analyzing engagement as a multi-dimensional construct. We moreover find that many of these nuanced engagement measures have strong correlations with student outcomes, even after controlling for student baseline characteristics and academic performance. Especially as virtual advising interventions proliferate across higher education institutions, we show the value of applying a more codified, comprehensive lens for examining student engagement in these programs and chart a path to potentially improving the efficacy of these programs in the future.

Open source code on GitHub.

More →

Matthew A. Kraft, John A. List, Jeffrey A. Livingston, Sally Sadoff.

A substantial body of experimental evidence demonstrates that in-person tutoring programs can have large impacts on K-12 student achievement. However, such programs typically are costly and constrained by a limited local supply of tutors. In partnership with CovEducation (CovEd), we conduct a pilot program that has potential to ease both of these concerns. We conduct an experiment where volunteer tutors from all over the country meet 1-on-1 with middle school students online during the school day. We find that the program produces consistently positive (0.07σ for math and 0.04σ for reading) but statistically insignificant effects on student achievement. While these estimates are notably smaller than those found in many higher-dosage in-person tutoring programs, they are from a significantly lower-cost program that was delivered within the challenging context of the COVID-19 pandemic. We provide evidence that is consistent with a dosage model of tutoring where additional hours result in larger effects.

More →

Matthew A. Kraft, Joshua Bleiberg.

Economic downturns can cause major funding shortfalls for U.S. public schools, often forcing districts to make difficult budget cuts including teacher layoffs. In this brief, we synthesize the empirical literature on the widespread teacher layoffs caused by the Great Recession. Studies find that teacher layoffs harmed student achievement and were inequitably distributed across schools, teachers, and students. Research suggests that specific elements of the layoff process can exacerbate these negative effects. Seniority-based policies disproportionately concentrate layoffs among teachers of color who are more likely to be early career teachers. These “last-in first-out” policies also disproportionately affect disadvantaged students because these students are more likely to be taught by early career teachers. The common practice of widely distributing pink slips warning about a potential job loss also appears to increase teacher churn and negatively impact teacher performance. Drawing on this evidence, we outline a set of policy recommendations to minimize the need for teacher layoffs during economic downturns and ensure that the burden of any unavoidable job cuts does not continue to be borne by students of color and students from low-income backgrounds.

More →

Taylor Odle.

Promoting equality in college enrollment and completion must start early in students’ college-going journeys, including with their expectations to first earn a college degree. With a nationally representative sample of high school students, I evaluate the ability of a recent collection of college access policies (place-based “promise” scholarships or “free” college programs) to increase students’ college expectations and test the heterogeneity of these impacts across students’ race and family income. Evidence from a difference-in-differences design and lagged-dependent-variable regressions suggest the introduction of promise programs increased the likelihood a student expected to attain an associate degree or higher by 8.5 to 15.0 percentage points by the end of high school, with larger effects for low-income and racially minoritized students. This study is the first to test the power of “free” college in shaping pre-college students’ educational plans, and, in doing so, not only addresses an existing gap in the literature but also identifies a key mechanism through which many of the positive college-going impacts observed across promise programs in the current literature may in fact originate. Given the rapid proliferation of promise programs across the nation, this study provides policymakers with a fuller view of the potential impacts of these programs, particularly concerning how they influence students’ outcomes along dimensions of race and income.

More →

Katharine Meyer, Lindsay C. Page, Eric Smith, B. Tyler Walsh, C. Lindsey Fifield, Michael Evans.

Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on a randomized controlled trial testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation. We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by eight percentage points. We find large and significant treatment effects for first-generation students, estimating the intervention increased their final course grades by about 11 points on a 100-point scale (and a 16 percentage point increase in earning a B or higher) as well as their completion of and performance on individual course deliverables (e.g., readings, activities, exams).

More →

Stephen B. Holt, Katie Vinopal, Heasun Choi, Lucy C. Sorensen.

While a growing body of literature has documented the negative impacts of exclusionary punishments, such as suspensions, on academic outcomes, less is known about how teachers vary in disciplinary behaviors and the attendant impacts on students. We use administrative data from North Carolina elementary schools to examine the extent to which teachers vary in their use of referrals and investigate the impact of more punitive teachers on student attendance and achievement. We also estimate the effect of teachers' racial bias in the use of referrals on student outcomes. We find more punitive teachers increase student absenteeism and reduce student achievement. Moreover, more punitive teachers negatively affect the achievement of students who do not receive disciplinary sanctions from the teacher. Similarly, while teachers with a racial bias in the use of referrals do not negatively affect academic outcomes for White students, they significantly increase absenteeism and reduce achievement for Black students. The results suggest punitive disciplinary measures do not aid teachers in productively managing classrooms; rather, teachers taking more punitive stances may undermine student engagement and learning. Moreover, bias in teachers' referral usage contributes to inequities in student outcomes.

More →

Jing Liu, Monica Lee, Xiaoxue Zhou.

Student absenteeism is often conceptualized and quantified in a static, uniform manner, providing an incomplete understanding of this important phenomenon. Applying growth curve models to detailed class-attendance data, we document that secondary school students' unexcused absences grow steadily throughout a school year and over grades, while the growth of excused absences remain essentially unchanged. Importantly, students starting the school year with a high number of unexcused absences, Black and Hispanic students, and low-income students accumulate unexcused absences at a significantly faster rate than their counterparts. Lastly, students with higher growth rates in unexcused absences consistently report lower perceptions of all aspects of school culture than their peers. Interventions targeting unexcused absences and/or improving school culture can be crucial to mitigating disengagement.

More →

Michael F. Lovenheim, Jonathan Smith.

Early research on the returns to higher education treated the postsecondary system as a monolith.  In reality, postsecondary education in the United States and around the world is highly differentiated, with a variety of options that differ by credential (associates degree, bachelor’s degree, diploma, certificate, graduate degree), the control of the institution (public, private not-for-profit, private for-profit), the quality/resources of the institution, field of study, and exposure to remedial education. In this Chapter, we review the literature on the returns to these different types of higher education investments, which has received increasing attention in recent decades. We first provide an overview of the structure of higher education in the U.S. and around the world, followed by a model that helps clarify and articulate the assumptions employed by different estimators used in the literature. We then discuss the research on the return to institution type, focusing on the return to two-year, four-year, and for-profit institutions as well as the return to college quality within and across these institution types. We also present the research on the return to different educational programs, including vocational credentials, remedial education, field of study, and graduate school. The wide variation in the returns to different postsecondary investments that we document leads to the question of how students from different backgrounds sort into these different institutions and programs. We discuss the emerging research showing that lower-SES students, especially in the U.S., are more likely to sort into colleges and programs with lower returns as well as results from recent U.S.-based interventions and policies designed to support success among students from disadvantaged backgrounds. The Chapter concludes with some broad directions for future research.

More →

Christine Mulhern, Isaac M. Opper.

There is an emerging consensus that teachers impact multiple student outcomes, but it remains unclear how to measure and summarize the multiple dimensions of teacher effectiveness into simple metrics for research or personnel decisions. We present a multidimensional empirical Bayes framework and illustrate how to use noisy estimates of teacher effectiveness to assess the dimensionality and predictive power of teachers' true effects. We find that it is possible to efficiently summarize many dimensions of effectiveness and most summary measures lead to similar teacher rankings; however, focusing on any one specific measure alone misses important dimensions of teacher quality.

More →