Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Covid-19 Education Research for Recovery

Kelli A. Bird, Benjamin L. Castleman, Yifeng Song.

Predictive analytics are increasingly pervasive in higher education. However, algorithmic bias has the potential to reinforce racial inequities in postsecondary success. We provide a comprehensive and translational investigation of algorithmic bias in two separate prediction models -- one predicting course completion, the second predicting degree completion. Our results show that algorithmic bias in both models could result in at-risk Black students receiving fewer success resources than White students at comparatively lower-risk of failure. We also find the magnitude of algorithmic bias to vary within the distribution of predicted success. With the degree completion model, the amount of bias is nearly four times higher when we define at-risk using the bottom decile than when we focus on students in the bottom half of predicted scores. Between the two models, the magnitude and pattern of bias and the efficacy of basic bias mitigation strategies differ meaningfully, emphasizing the contextual nature of algorithmic bias and attempts to mitigate it. Our results moreover suggest that algorithmic bias is due in part to currently-available administrative data being less useful at predicting Black student success compared with White student success, particularly for new students; this suggests that additional data collection efforts have the potential to mitigate bias.

More →


Maria Marta Ferreyra, Camila Galindo, Sergio Urzua.

This paper estimates the heterogeneous labor market effects of enrolling in higher education short-cycle (SC) programs. Expanding access to these programs might affect the behavior of some students (compliers) in two margins: the expansion margin (students who would not have enrolled in higher education otherwise) and the diversion margin (students who would have enrolled in bachelor’s programs otherwise). To quantify these responses, we exploit local exogenous variation in the supply of higher education institutions (HEIs) facing Colombian high school graduates in an empirical multinomial choice model with several instruments. According to our findings, the presence of at least one HEI specialized in SC programs in the vicinity of the student’s high school municipality increases SC enrollment by 3.7-4.5 percentage points (40-50% of the SC enrollment rate). The diversion margin largely drives this effect. For female compliers, enrollment in SC programs increases formal employment relative to the next-best alternative. For male compliers, in contrast, it lowers formal employment and wages. These results should alert policymakers of the unexpected consequences of higher education expansionary policies.

More →


Suchitra Akmanchi, Kelli A. Bird, Benjamin L. Castleman.

Prediction algorithms are used across public policy domains to aid in the identification of at-risk individuals and guide service provision or resource allocation. While growing research has investigated concerns of algorithmic bias, much less research has compared algorithmically-driven targeting to the counterfactual: human prediction. We compare algorithmic and human predictions in the context of a national college advising program, focusing in particular on predicting high-achieving, lower-income students’ college enrollment quality. College advisors slightly outperform a prediction algorithm; however, greater advisor accuracy is concentrated among students with whom advisors had more interactions. The algorithm achieved similar accuracy among students lower in the distribution of interactions, despite advisors having substantially more information. We find no evidence that the advisors or algorithm exhibit bias against vulnerable populations. Our results suggest that, especially at scale, algorithms have the potential to provide efficient, accurate, and unbiased predictions to target scarce social services and resources.

More →


David M. Houston, Matthew P. Steinberg.

In spring 2020, nearly every U.S. public school closed at the onset of the Covid-19 pandemic. Existing evidence suggests that local political partisanship and teachers union strength were better predictors of fall 2020 school re-opening status than Covid case and death rates. We replicate and extend these analyses using data collected over the 2020-21 academic year. We demonstrate that Covid case and death rates were meaningfully associated with initial rates of in-person instruction. We also show that all three factors—Covid, partisanship, and teachers unions—became less predictive of in-person instruction as the school year continued. We then leverage data from two nationally representative surveys of Americans’ attitudes toward education and identify an as-yet undiscussed factor that predicts in-person instruction: public support for increasing teacher salaries. We speculate that education leaders were better able to manage the logistical and political complexities of school re-openings in communities with greater support for educators.

More →


Cassandra M. D. Hart, Michael Hill, Emily Alonso, Di Xu.

While the COVID-19 pandemic necessitated the short-term use of online courses, colleges’ experiences with COVID-era online course delivery may also affect the way that they offer and approach online courses going forward. We draw on interviews with 35 distance education leaders from the California Community Colleges system to provide insights into how the use of online education may change in the system going forward. Leaders predicted that post-pandemic, colleges would increase their online course offerings, and that many instructional innovations to online courses from the pandemic—such as the use of synchronous courses—would persist. They hoped that a more prominent position for online education within the system would be matched by more resources to provide supports for online learning.

More →


Matthew A. Kraft, Melissa Arnold Lyon.

We examine the state of the U.S. K-12 teaching profession over the last half century by compiling nationally representative time-series data on four interrelated constructs: professional prestige, interest among students, preparation for entry, and job satisfaction. We find a consistent and dynamic pattern across every measure: a rapid decline in the 1970s, a swift rise in the 1980s, relative stability for two decades, and a sustained drop beginning around 2010. The current state of the teaching profession is at or near its lowest levels in 50 years. We identify and explore a range of factors that might explain these historical patterns including education funding, teacher pay, outside opportunities, unionism, barriers to entry, working conditions, accountability, autonomy, and school shootings.  

More →


Danielle Sanderson Edwards, Matthew A. Kraft, Alvin Christian, Christopher A. Candelaria.

We develop a unifying conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then generate and test hypotheses about geographic, grade level, and subject variation in teacher shortages using data on teaching vacancies in Tennessee during the fall of 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective.

More →


Kathleen Lynch, Monica Lee, Susanna Loeb.
The COVID-19 pandemic’s impact on preschool children’s school readiness skills remains understudied. This research investigates Head Start preschool children’s early numeracy, literacy, and executive function outcomes during a pandemic-affected school year, using a novel virtual assessment methodology. Study children (N = 336; mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) in a network of Head Start centers in four states (Nevada, New Jersey, Pennsylvania, and Wisconsin) experienced low in-person preschool exposure compared to national pre-pandemic norms. However, study children experienced fall to spring score gains during the pandemic-affected year of 0.05 SD in executive function, 0.27 SD in print knowledge, and 0.45-0.71 SD in early numeracy skills, growth not outside the general range of that observed in pre-pandemic research studies. For two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool. We discuss implications for future research and policy.

More →


Alvin Christian, Brian Jacob, John D. Singleton.

The COVID-19 pandemic drew new attention to the role of school boards in the U.S. In this paper, we examine school districts' choices of learning modality -- whether and when to offer in-person, virtual, or hybrid instruction -- over the course of the 2020-21 pandemic school year. The analysis takes advantage of granular weekly data on learning mode and COVID-19 cases for Ohio school districts. We show that districts respond on the margin to health risks: all else equal, a marginal increase in new cases reduces the probability that a district offers in-person instruction the next week. Moreover, this negative response is magnified when the district was in-person the prior week and attenuates in magnitude over the school year. These findings are consistent with districts learning from experience about the effect of in-person learning on disease transmission in schools. We also find evidence that districts are influenced by the decisions of their peers.

More →


Sara White, Leiah Groom-Thomas, Susanna Loeb.

This study synthesizes existing research on the implementation of tutoring programs which we define as one-to-one or small-group instruction in which a human tutor supports students grades K-12 in an academic subject area. Tutoring has emerged as an especially promising strategy for supporting students’ academic success with strong causal evidence finding large, positive effects on students' math and reading test scores across grade levels. Prior studies have reviewed this causal evidence of effects, but none have summarized the evidence on implementation. We iteratively developed search and selection criteria to identify studies addressing key research questions and synthesized these 40 studies which employ a range of research methodologies to describe how tutoring is implemented and experienced. We find that existing research provides rich descriptions of tutoring implementation within specific programs of focus, with most studies describing after-school tutoring and small-scale programs run by university professors. While few elements of implementation are studied in depth across multiple studies, common patterns emerge. Tutoring program launch is often facilitated by strategic relationships between schools and external tutoring providers and strengthened by transparent assessments of program quality and effectiveness. Successful tutoring implementation often hinges on the support of key school leaders with the power to direct the use of school funding, space, and time. Tutoring setting and schedule, tutor recruitment and training, and curriculum identification influence whether students are able to access tutoring services and the quality of the instruction provided. Ultimately, the evidence points to strong tutoring being driven by positive student-tutor relationships through which tutors provide instruction strategically targeted for students’ strengths and needs driving towards a long-term academic goal.

More →