- Susanna Loeb
Search for EdWorkingPapers here by author, title, or keywords.
School Improvement Grants (SIG) represent one type of governments’ capacity-building investment to spur sustainable changes in America’s persistently under-performing public schools. This study examines both short- and long-run effects of the first two cohorts of SIG schools from two states and two urban districts across the country. Using dynamic event analyses, we observe that SIG showed larger effects in the second and third years of the intervention than the first year on 3-8th grade student test scores—a pattern of gradually increase over the course the intervention. These positive effects are largely sustained three or four years after the funding ended. In high schools, the SIG effects on 4-year graduation rates were steadily increasing throughout the period of six or seven years after the initial start of the intervention. These patterns of SIG effects mostly apply to each of the four locations, but the magnitude of effects varies across locations, suggesting differential implementations. Moreover, SIG effects on students of color or low-socioeconomic students are similar to, and sometimes a bit larger than, the overall SIG effects. We also conduct a variety of sensitivity and robustness checks. Lastly, we discuss the policy implications of our findings on states’ continuing efforts of transforming public organizations and building their long-term capacity for better performance.
While the importance of social-emotional learning for student success is well established, educators and researchers have less knowledge and agreement about which social-emotional skills are most important for students and how these skills distribute across student subgroups. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students in California districts, this paper describes growth mindset gaps across student groups, and confirms, at a large scale, the predictive power of growth mindset for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills. Average annual growth in English language arts and math corresponding to differences between students with fixed and growth mindset in a same school and grade level is 0.07 and 0.05 standard deviations respectively, after adjusting for students’ characteristics and previous achievement. This estimate is equivalent to 48 and 35 additional days of learning.
Existing research on self-management skills shows that measures of self-management predict student success. However, these conclusions are based on small samples or narrowly defined self-management measures. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students, this paper describes self-management gaps across student groups, and confirms, at a large scale, the predictive power of self-management for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills. Self-management is a better predictor of student learning than are other measures of socioemotional skills. Average growth in English language arts due to changing from a low to a high level of self-management is between 0.091 and 0.112 standard deviations, equivalent to almost 80 days of learning.
Recent attempts to measure schools’ influence on students' SEL show differences across schools, but whether these differences measure the true effect of schools is unclear. We examine the stability of school-by-grade effects on students' SEL across two years using a large-scale survey. Correlations among effects in the same grades across different years are positive but lower than those for math and English. Schools in the top or bottom of the effect distribution have more persistent impacts across years than those in the middle. Overall, the results suggest that these school effects measure real contributions to students' SEL. However, their low stability draws into question whether including school value-added measures of self-reported SEL in school performance systems would be beneficial.
The time children spend with their parents affects their development. Parenting programs can help parents use that time more effectively. Text-messaged-based parenting curricula have proven an effective means of supporting positive parenting practices by providing easy and fun activities that reduce informational and behavioral barriers. These programs may be more effective if delivered during times when parents are particularly in need of support, such as after work, or, alternatively when parents have more time to interact with their child, such as on a day off of work. This study compares the effects of an early childhood text-messaging program sent during the weekend to the same program sent on weekdays. We find that sending the text messages on the weekend is, on average, more beneficial to children’s literacy and math development. This effect is particularly strong for initially lower achieving children, while the weekday texts show some benefits for higher achieving children on higher order skills. These results are consistent with the hypothesis that the parents of lower achieving students, on average, face such high barriers during weekdays that supports are not enough to overcome these barriers, while for parents of higher achieving students, weekday texts are more effective because weekdays are more challenging, but not so difficult as to be untenable for positive parenting. In sum, the findings suggest that parenting support works best when parents have time, attention, and need.
Despite growing concern over teachers’ ability to live comfortably where they work, we know little about the systematic impacts of affordability on teachers’ well-being, particularly in high-cost urban areas. We use novel survey data from San Francisco Unified School District to identify the patterns and prevalence of economic anxiety among teachers and assess how this anxiety relates to teachers’ attitudes, behaviors, and turnover. We find that San Francisco teachers have far higher levels of economic anxiety on average than a national sample of employed adults, and that younger teachers are particularly financially anxious. Furthermore, such anxiety relates to job performance and teacher retention— economically anxious teachers tend to have more negative attitudes about their jobs, have worse attendance, and are 50 percent more likely to depart the district within two years after the survey.
Ten years ago, the reform of teacher evaluation was touted as a mechanism to improve teacher effectiveness. In response, virtually every state redesigned its teacher evaluation system. Recently, a growing narrative suggests these reforms failed and should be abandoned. This response may be overly simplistic. We explore the variability of New York City principals’ implementation of policies intended to promote teaching effectiveness. Drawing on survey, interview, and administrative data, we analyze whether principals believe they can use teacher evaluation and tenure policies to improve teaching effectiveness, and how such perceptions influence policy implementation. We find that principals with greater perceived agency are more likely to strategically employ tenure and evaluation policies. Results have important implications for principal training and policy implementation.
Teachers’ impact on student long-run success is only partially explained by their contributions to students’ short-run academic performance. For this study, we explore a second dimension of teacher effectiveness by creating measures of teachers’ contributions to student class-attendance. We find systematic variation in teacher effectiveness at reducing unexcused class absences at the middle and high school level. These differences across teachers are as stable as those for student achievement, but teacher effectiveness on attendance only weakly correlates with their effects on achievement. We link these measures of teacher effectiveness to students’ long-run outcomes. A high value-added to attendance teacher has a stronger impact on students’ likelihood of finishing high school than does a high value-added to achievement teacher. Moreover, high value-added to attendance teachers can motivate students to pursue higher academic goals as measured by Advanced Placement course taking. These positive effects are particularly salient for low-achieving and low-attendance students.
Text-message based parenting programs have proven successful in improving parental engagement and preschoolers’ literacy development. The tested programs have provided a combination of (a) general information about important literacy skills, (b) actionable advice (i.e., specific examples of such activities), and (c) encouragement. The regularity of the texts – each week throughout the school year – also provided nudges to focus parents’ attention on their children. This study seeks to identify mechanisms of the overall effect of such programs. It investigates whether the actionable advice alone drives previous study’s results and whether additional texts of actionable advice improve program effectiveness. The findings provide evidence that text messaging programs can supply too little or too much information. A single text per week is not as effective at improving parenting practices as a set of three texts that also include information and encouragement, but a set of five texts with additional actionable advice is also not as effective as the three-text approach. The results on children’s literacy development depend strongly on the child’s pre-intervention literacy skills. For children in the lowest quarter of the pre-treatment literacy assessments, only providing one example of an activity decreases literacy scores by 0.15 standard deviations relative to the original intervention. Literacy scores of children in higher quarters are marginally higher with only one tip per week. We find no positive effects of increasing to five texts per week.