Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
Program and policy effects
Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on two randomized controlled trials testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation in introductory courses (political science and economics). We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by 5-6 percentage points and reduced the likelihood students dropped the course.
Decisions to invest in human capital depend on people’s time preferences. We show that differences in patience are closely related to substantial subnational differences in educational achievement, leading to new perspectives on longstanding within-country disparities. We use social-media data – Facebook interests – to construct novel regional measures of patience within Italy and the United States. Patience is strongly positively associated with student achievement in both countries, accounting for two-thirds of the achievement variation across Italian regions and one-third across U.S. states. Results also hold for six other countries with more limited regional achievement data.
Generally, need-based financial aid improves students’ academic outcomes. However, the largest source of need-based grant aid in the United States, the Federal Pell Grant Program (Pell), has a mixed evaluation record. We assess the minimum Pell Grant in a regression discontinuity framework, using Kentucky administrative data. We focus on whether and how year-to-year changes in aid eligibility and interactions with other sources of aid attenuate Pell’s estimated effects on post-secondary outcomes. This evaluation complements past work by assessing explanations for the null or muted impacts found in our analysis and other Pell evaluations. We also discuss the limitations of using regression discontinuity methods to evaluate Pell—or other interventions with dynamic eligibility criteria—with respect to generalizability and construct validity.
Students’ college choices can affect their chances of earning a degree, but many lack the support to navigate the opaque college application and admissions process. This paper evaluates whether guaranteeing four-year college admissions based on transparent academic standards affected community college students’ enrollment choices and graduation rates. Guaranteed admissions increased high-GPA graduates’ transfer rates to highly-selective colleges by 30 percent. Increased transfers to highly-selective colleges also accompanied higher graduation rates and lower student debt. Gains were largest for students with historically lower transfer rates. Transparent admissions standards can increase access to selective colleges at low to no cost.
This study contributes to the science of teaching reading by illustrating how a ubiquitous classroom practice – read alouds – can be enhanced by fostering teacher language practices that support students’ ability to read for understanding. This experimental study examines whether and to what extent providing structured teacher read aloud supplements in a social studies read aloud can allow students to leverage a familiar science schema and thereby positively impact reading comprehension outcomes. Treatment students received a single social studies read-aloud on the story of Apollo 11 with structured teacher read aloud supplements while control students received the same read-aloud story but without structured supplements. Effect sizes from hierarchical linear models indicated that students in the treatment condition significantly outperformed students in the control condition on four measures of domain-specific reading comprehension. Further exploratory analyses using structural equation modeling examined the extent that teacher language mediated the treatment effect. Results indicated that teachers going above and beyond the intervention script explained 67 percent of the treatment effect. Structured supplements for read alouds can help students see important connections between schemas, which ultimately aids in reading comprehension.
We develop a unifying conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then generate and test hypotheses about geographic and subject variation in teacher shortages using data on unfilled teaching positions in Tennessee during the fall of 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective.
This paper discusses the importance of incorporating personal assistance into interventions aimed at improving long-term education and labor market success. While existing research demonstrates the cost-effectiveness of low-touch behavioral nudges, this paper argues that the dynamic nature of human capital accumulation requires sustained habits over time. To foster better habits, social connections are critical for encouraging enduring effort and intrinsic motivation. The paper showcases examples from various stages of human capital accumulation, including early childhood, adolescence, and adulthood, in which interventions that incorporate personal assistance substantially out-perform less intensive nudges. We underscore the importance of interactive support, guidance, and motivation in facilitating significant progress and explore the challenges associated with implementing cost-effective policies to provide such assistance.
Targeted instruction is one of the most effective educational interventions in low- and middle-income countries, yet reported impacts vary by an order of magnitude. We study this variation by aggregating evidence from prior randomized trials across five contexts, and use the results to inform a new randomized trial. We find two factors explain most of the heterogeneity in effects across contexts: the degree of implementation (intention-to-treat or treatment-on-the-treated) and program delivery model (teachers or volunteers). Accounting for these implementation factors yields high generalizability, with similar effect sizes across studies. Thus, reporting treatment-on-the-treated effects, a practice which remains limited, can enhance external validity. We also introduce a new Bayesian framework to formally incorporate implementation metrics into evidence aggregation. Results show targeted instruction delivers average learning gains of 0.42 SD when taken up and 0.85 SD when implemented with high fidelity. To investigate how implementation can be improved in future settings, we run a new randomized trial of a targeted instruction program in Botswana. Results demonstrate that implementation can be improved in the context of a scaling program with large causal effects on learning. While research on implementation has been limited to date, our findings and framework reveal its importance for impact evaluation and generalizability.
The impact of test-optional college admissions policies depends on whether applicants act strategically in disclosing test scores. We analyze individual applicants’ standardized test scores and disclosure behavior to 50 major US colleges for entry in fall 2021, when Covid-19 prompted widespread adoption of test-optional policies. Applicants withheld low scores and disclosed high scores, including seeking admissions advantages by conditioning their disclosure choices on their other academic characteristics, colleges’ selectivity and testing policy statements, and the Covid-related test access challenges of the applicants’ local peers. We find only modest differences in test disclosure strategies by applicants’ race and socioeconomic characteristics.
Prior research has clearly established the substantial expected payoffs to investments in early childhood education. However, the ability to deliver early childhood programs differs across communities with access to high quality programing especially hard to establish in rural communities. We study one program, Early Steps to School Success, to understand whether the provision of home visiting and book exchange programs in rural Kentucky can influence kindergarten readiness. Linking program data with the state longitudinal data system in Kentucky we create multiple comparison groups by matching children on known program qualification indicators to estimate whether Early Steps program participation was related to school readiness. Our estimates suggest that program participation resulted in small improvements to children’s kindergarten readiness, as measured by the Brigance kindergarten readiness assessment overall score and sub-scores in language, cognitive, and physical development. Results are not sensitive to our choice of comparison group, though they appear driven by the experiences of children who participate from birth through age five or from ages three-to-five only. Our findings suggest that the Early Steps home visiting intervention may be a worthwhile intervention for improving kindergarten preparedness for children living in rural contexts.