Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
We explore the potential for mobile technology to facilitate more frequent and higher-quality teacher-parent communication among a sample of 132 New York City public schools. We provide participating schools with free access to a mobile communication app and randomize schools to receive intensive training and guidance for maximizing the efficacy of the app. User supports led to substantially higher levels of communication within the app in the treatment year, but had few subsequent effects on perceptions of communication quality or student outcomes. Treatment teachers used the app less frequently the following year when they no longer received communication tips and reminders. We analyze internal user data to suggest organizational policies schools might adopt to increase the take-up and impacts of mobile communication technology.
Despite substantial evidence that resources and outcomes are transmitted across generations, there has been limited inquiry into the extent to which anti-poverty programs actually disrupt the cycle of bad outcomes. We explore how the effects of the United States’ largest early childhood program, Head Start, transfer across generations. We leverage the rollout of this federally funded, means-tested preschool program to estimate the effect of early childhood exposure among mothers on their children’s long-term outcomes. We find evidence of intergenerational transmission of effects in the form of increased educational attainment, reduced teen pregnancy, and reduced criminal engagement in the second generation.
This paper reports improvements in teacher job performance, as measured by student test scores, resulting from a program of (zero-) low-stakes peer evaluation. Teachers working at the same school observed and scored each other’s teaching. Students in randomly-assigned treatment schools scored 0.07σ higher on math and English exams (0.09σ lower-bound on TOT). Within each treatment school, teachers were further randomly assigned to roles: observer and observee. Teachers in both roles improved, perhaps slightly more for observers. The typical treatment school completed 2-3 observations per observee teacher. Variation in observations was generated partly by randomly assigning a low and high (2*low) dose of suggested number of observations. Benefits were quite similar across dose conditions.
Students’ level of academic skills at school entry are a strong predictor of later academic success, and focusing on improving these skills during the preschool years has been a priority during the past ten years. Evidence from two prior nationally representative studies indicated that incoming kindergarteners’ math and literacy skills were higher in 2010 than 1998, but no national studies have examined trends since 2010. This study examines academic skills at kindergarten entry from 2010 and 2017 using data from over 2 million kindergarten students. Results indicated kindergarteners in 2017 have slightly lower math and reading skills than in 2010, but that inequalities at school entry by race/ethnicity and school poverty level have decreased during this period.
The sustaining environments hypothesis refers to the popular idea, stemming from theories in developmental, cognitive, and educational psychology, that the long-term success of early educational interventions is contingent on the quality of the subsequent learning environment. Several studies have investigated whether specific kindergarten classroom and other elementary school factors account for patterns of persistence and fadeout of early educational interventions. These analyses focus on the statistical interaction between an early educational intervention – usually whether the child attended preschool – and several measures of the quality of the subsequent educational environment. The key prediction of the sustaining environments hypothesis is a positive interaction between these two variables. To quantify the strength of the evidence for such effects, we meta-analyze existing studies that have attempted to estimate interactions between preschool and later educational quality in the United States. We then attempt to establish the consistency of the direction and a plausible range of estimates of the interaction between preschool attendance and subsequent educational quality by using a specification curve analysis in a large, nationally representative dataset that has been used in several recent studies of the sustaining environments hypothesis. The meta-analysis yields small positive interaction estimates ranging from approximately .00 to .04, depending on the specification. The specification curve analyses yield interaction estimates of approximately 0. Results suggest that the current mix of methods used to test the sustaining environments hypothesis cannot reliably detect realistically sized effects. Our recommendations are to combine large sample sizes with strong causal identification strategies, and to study combinations of interventions that have a strong probability of showing large main effects.
Recent attempts to measure schools’ influence on students' SEL show differences across schools, but whether these differences measure the true effect of schools is unclear. We examine the stability of school-by-grade effects on students' SEL across two years using a large-scale survey. Correlations among effects in the same grades across different years are positive but lower than those for math and English. Schools in the top or bottom of the effect distribution have more persistent impacts across years than those in the middle. Overall, the results suggest that these school effects measure real contributions to students' SEL. However, their low stability draws into question whether including school value-added measures of self-reported SEL in school performance systems would be beneficial.
We examine whether virtual advising – college counseling using technology to communicate remotely – increases postsecondary enrollment in selective colleges. We test this approach using a sample of approximately 16,000 high-achieving, low- and middle-income students identified by the College Board and randomly assigned to receive virtual advising from the College Advising Corps. The offer of virtual advising had no impact on overall college enrollment, but increased enrollment in high graduation rate colleges by 2.7 percentage points (5%), with instrumental variable impacts on treated students of 6.1 percentage points. We also find that non-white students who were randomly assigned to a nonwhite adviser exhibited stronger treatment effects.
In recognition of the complexity of the college and financial aid application process, and in response to insufficient access to family or school-based counseling among economically-disadvantaged populations, investments at the local, state, and federal level have expanded students’ access to college and financial aid advising. Experimental and quasi-experimental studies of these programs demonstrate that they can generate substantial improvements in the rate at which low-income students enroll and persist in college. While these programs are successful at the level of individual communities, the individualized, in-person college advising model faces numerous barriers to scale. In this paper, we report early results from an RCT of CollegePoint, an innovative, national college advising initiative that pursues a technology-enabled approach to provide students with sustained, intensive advising. Students assigned to CollegePoint are modestly more likely (1.5 percentage points, or 7.5 percent relative to the control) to enroll at the most selective colleges and universities (Barron’s 1 institutions), though we find no difference in enrollment patterns on other measures of college quality. We find suggestive evidence of variation in the impact of CollegePoint based on when students enrolled in the program. Students who enrolled in the spring of their junior year were 5.6 percentage points (22 percent relative to the control) more likely to enroll at one of the most selective colleges and universities in the country than students in the control group who also signed up in the spring of junior year but who were not assigned to the program.
Holzer and Baum’s recent book, ‘Making College Work: Pathways to Success for Disadvantaged Students,’ provides an excellent up-to-date review of higher education. My review first summarizes its key themes: 1) who gains from college and why? 2) mismatch and the need for more structure; 3) problems with remediation; 4) financial barriers and 5) the promise of comprehensive support. I then critique the book’s proposed solutions using some of my own qualitative and quantitative data. Some recommendations are worth considering, while others are too expensive or unlikely to make a meaningful difference without addressing the underlying lack of preparedness and motivation of college students. I argue that making mandatory some existing services, such as application assistance and advice, proactive tutoring and advising, and greater career transition support, has the most immediate potential.
Child care subsidies play an important role in stabilizing parental employment and helping low- income families access care. With limited federal requirements under CCDBG, states developed divergent subsidy program policies. Our study examines how variations in six state policy levers that capture CCDF administrative burdens and generosity relate to stability in children’s care in the CCDF program, known as subsidy “spells.”: (1) length of eligibility redetermination; (2) reporting requirements for income changes; (3) grace period for care before termination; (4) provider reimbursement rates; (5) parent copay amounts; and (6) difference between initial and continuing eligibility income thresholds. We exploit states’ changes in these policies during a 10- year period (2004-2013) using state fixed effects analyses to identify their impact on spell length. We find that administrative burdens robustly affect child care spell length; increasing states’ redetermination period length by one month increased state median subsidy spell length by 1.4 weeks, but requiring all changes in family income to be reported while enrolled in CCDF decreased spell length by 2.3 weeks. Switching to a 12-month redetermination period increased median spell length by 30%. CCDF policy generosity was not related to spell length. Results are discussed in the context of the 2014 CCDBG reauthorization.