Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Carly D. Robinson, Biraj Bisht, Susanna Loeb.

Billions of dollars are invested in opt-in, educational resources to accelerate students’ learning. Although advertised to support struggling, marginalized students, there is no guarantee these students will opt in. We report results from a school system’s implementation of on-demand tutoring. The take up was low. At baseline, only 19% of students ever accessed the platform, and struggling students were far less likely to opt in than their more engaged and higher achieving peers. We conducted a randomized controlled trial (N=4,763) testing behaviorally-informed approaches to increase take-up. Communications to parents and students together increase the likelihood students access tutoring by 46%, which led to a four-percentage point decrease in course failures. Nonetheless, take-up remained low, showing concerns that opt-in resources can increase—instead of reduce—inequality are valid. Without targeted investments, opt-in educational resources are unlikely to reach many students who could benefit.

More →


Bobby W. Chung, Jian Zou.

States increasingly require prospective teachers to pass exams for program completion and initial licensure, including the recent controversial roll-out of the educative Teacher Performance Assessment (edTPA). We leverage the quasi-experimental setting of different adoption timing by states and analyze multiple data sources containing a national sample of prospective teachers and students of new teachers in the US. With extensive controls of concurrent policies, we find that the edTPA reduced prospective teachers in traditional route programs, less-selective and minority-concentrated universities. Contrary to the policy intention, we do not find evidence that edTPA increased student test scores.

More →


Sara White, Leiah Groom-Thomas, Susanna Loeb.

This study synthesizes existing research on the implementation of tutoring programs which we define as one-to-one or small-group instruction in which a human tutor supports students grades K-12 in an academic subject area. Tutoring has emerged as an especially promising strategy for supporting students’ academic success with strong causal evidence finding large, positive effects on students' math and reading test scores across grade levels. Prior studies have reviewed this causal evidence of effects, but none have summarized the evidence on implementation. We iteratively developed search and selection criteria to identify studies addressing key research questions and synthesized these 40 studies which employ a range of research methodologies to describe how tutoring is implemented and experienced. We find that existing research provides rich descriptions of tutoring implementation within specific programs of focus, with most studies describing after-school tutoring and small-scale programs run by university professors. While few elements of implementation are studied in depth across multiple studies, common patterns emerge. Tutoring program launch is often facilitated by strategic relationships between schools and external tutoring providers and strengthened by transparent assessments of program quality and effectiveness. Successful tutoring implementation often hinges on the support of key school leaders with the power to direct the use of school funding, space, and time. Tutoring setting and schedule, tutor recruitment and training, and curriculum identification influence whether students are able to access tutoring services and the quality of the instruction provided. Ultimately, the evidence points to strong tutoring being driven by positive student-tutor relationships through which tutors provide instruction strategically targeted for students’ strengths and needs driving towards a long-term academic goal.

More →


Kaitlyn O'Hagan, Leanna Stiefel, Amy Ellen Schwartz.

Students with disabilities (SWDs) educated in traditional public schools alongside general education students (GENs) typically move to middle school in sixth grade, rather than continuing in a K-8/12. The documented negative effects of this move on GEN academic outcomes suggests similar negative—and perhaps larger—effects on SWDs. Using an instrumental variables strategy and NYC data on nine cohorts of students, we find the middle school transition causes a 0.30 (0.16) standard deviation decline in SWD math (ELA) performance and increases grade retention. Low-income SWDs and SWDs with a specific learning disability or emotional disturbance fare worse. However, the move does not widen the SWD-GEN gap, suggesting the need to ease the middle school transition for all students.

More →


Carly D. Robinson.

Few question the value of teacher-student relationships (TSRs) for educational outcomes. TSRs are positively associated with students’ achievement and engagement, as well as teachers’ well-being. Building and maintaining these crucial classroom relationships, however, is not easy. Drawing on prominent motivation theories in educational psychology, I present the Motivating Teacher-Student Relationships framework for understanding what motivates teachers to build positive TSRs. In particular, I focus on how teachers’ motivational beliefs about TSRs energize, direct, and sustain their efforts to engage in relationship-building behaviors and, thus, lead to positive relationships with their students. To build positive TSRs, teachers must believe it is their role to build TSRs, value TSRs, and believe they can successfully build TSRs (i.e., have relational self-efficacy). These beliefs are shaped by teachers’ sociocultural contexts and can facilitate or undermine the development of these learning relationships. With a greater understanding of how motivational beliefs influence social relationships, the field of education can more effectively develop theoretically grounded interventions to improve TSRs and mitigate inequality.

More →


Matthew A. Kraft, Sarah Novicoff.

Policymakers have renewed calls for expanding instructional time in the wake of the COVID-19 pandemic. We establish a set of empirical facts about time in school, synthesize the literature on the causal effects of instructional time, and conduct a case study of time use in an urban district. On average, instructional time in U.S. public schools is comparable to most high-income countries, with longer days but shorter years. However, instructional time varies widely across U.S. public schools with a 90th-10th percentile difference of 190 total hours. Empirical literature confirms that additional time can increase student achievement, but how this time is structured matters. Our case study suggests schools might also recover substantial lost learning time within the existing school day.

More →


Christina Weiland, Meghan McCormick, Jennifer Duer, Allison Friedman-Kraus, Mirjana Pralica, Samantha Xia, Milagros Nores, Shira Mattera.

Nearly all states with public prekindergarten programs use mixed-delivery systems, with classrooms in both public schools and community-based settings.  However, experts have long raised concerns about systematic inequities by setting within these public systems.  We used data from five large-scale such systems that have taken steps to improve equity by setting  (Boston, New York City, Seattle, New Jersey, and West Virginia) to conduct the most comprehensive descriptive study of prekindergarten setting differences to date. Our public school sample included 2,395 children in 383 classrooms in 152 schools, while our community-based sample is comprised of 1,541 children in 201 classrooms in 103 community-based organizations (CBOs).  We examined how child and teacher demographic characteristics, structural and process quality features, and child gains differed by setting within each of these systems.  We found evidence of sorting of children and teachers by setting within each locality, including of children with higher baseline skills and more educated teachers into public schools. Where there were differences in quality and children’s gains, these tended to favor public schools.  The localities with fewer policy differences by setting – NJ and Seattle – showed fewer differences in quality and child gains.  Our findings suggest that inequities by setting are common, appear consequential, and deserve more research and policy attention.  

More →


Alex Eble, Maya Escueta.

How much does family demand matter for child learning in contexts of extreme poverty? In rural Gambia, families with high aspirations for their children's future education and career, measured before children start school, go on to invest substantially more than other families in their children’s education. Despite this, essentially no children are literate or numerate three years later. When villages receive a highly-impactful, teacher-focused supply-side intervention, however, children of these families are 25 percent more likely to achieve literacy and numeracy than other children in the same village. Furthermore, improved supply enables these children to acquire other higher-level skills necessary for later learning and child development. In such settings, greater demand can map onto developmentally meaningful learning differences, but only with adequate complementary inputs.

More →


Kai Hong, Syeda Sana Fatima, Sherry Glied, Leanna Stiefel, Amy Ellen Schwartz.

There is increasing concern about risky behaviors and poor mental health among school-aged youth. A critical factor in youth well-being is school attendance. This study evaluates how school organization and structure affect health outcomes by examining the impacts of a popular urban high school reform -- “small schools” -- on youth risky behaviors and mental health, using data from New York City. To estimate a causal estimate of attending small versus large high schools, we use a two-sample-instrumental-variable approach with the distance between student residence and school as the instrument for school enrollment. We consider two types of small schools – “old small schools,” which opened prior to a system-wide 2003 reform aimed at increasing educational achievement and “new small schools,” which opened in the wake of that reform. We find that girls enrolled in older small schools are less likely to become pregnant, and boys are less likely to be diagnosed with mental health disorders than their counterparts in large schools. Both girls and boys enrolled in more recently opened small schools, however, are more likely to be diagnosed with violence-associated injuries and (for girls only) with mental health disorders. These disparate results suggest that improving a school’s organization and inputs together is likely more effective in addressing youth risky behaviors than simply reducing school size.

More →


Ann Mantil, John Papay, Preeya Pandya Mbekeani, Richard J. Murnane.

Preparing K-12 students for careers in science, technology, engineering and mathematics (STEM) fields is an ongoing challenge confronting state policymakers. We examine the implementation of a science graduation testing requirement for high-school students in Massachusetts, beginning with the graduating class of 2010. We find that the design of the new requirement was quite complicated, reflecting the state’s previous experiences with test-based accountability, a broad consensus on policy goals among key stakeholders, and the desire to afford flexibility to local schools and districts. The consequences for both students and schools, while largely consistent with the goals of increasing students’ skills and interest in STEM fields, were in many cases unexpected. We find large differences by demographic subgroup in the probabilities of passing the first science exam and of succeeding on retest, even when conditioning on previous test-score performance. Our results also show impacts of science exit-exam performance for students scoring near the passing threshold, particularly on the high-school graduation rates of females and on college outcomes for higher-income students. These findings demonstrate the importance of equity considerations in designing and evaluating ambitious new policy initiatives.

More →