Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
Program and policy effects
Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers, but conventional classroom observations are costly, prone to rater bias, and hard to implement at scale. Using nearly 1,000 word-to-word transcriptions of 4th- and 5th-grade English language arts classes, we apply novel text-as-data methods to develop automated, objective measures of teaching to complement classroom observations. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers’ average classroom observation scores. The interactive instruction factor predicts positive value-added scores.
States and districts are increasingly incorporating measures of achievement growth into their school accountability systems, but there is little research on how these changes affect the public’s perceptions of school quality. We conduct a nationally representative online survey experiment to identify the effects of providing participants with information about their local school districts’ average achievement status and/or average achievement growth. In the control group, participants who live in higher status districts tend to grade their local schools more favorably. The provision of status information does not fundamentally alter this relationship. The provision of growth information, however, reshapes Americans’ views about educational performance. Once informed, participants’ evaluations of their local public schools better reflect the variation in district growth.
We provide the first estimated economic impacts of students’ access to an entire sector of public higher education in the U.S. Approximately half of Georgia high school graduates who enroll in college do so in the state’s public four-year sector, which requires minimum SAT scores for admission. Regression discontinuity estimates show enrollment in public four-year institutions boosts students’ household income around age 30 by 20 percent, and has even larger impacts for those from low income high schools. Access to this sector has little clear impact on student loan balances or other measures of financial health. For the marginal student, enrollment in such institutions has large private returns even in the short run and positive returns to state budgets in the long run.
English Learners (ELs) lag behind their peers in postsecondary attainment. As the EL population in the U.S. continues to grow, so does concern over their underrepresentation in higher education. Research shows that Early College High Schools have a significant impact on high school and college outcomes for students from low income and racial/ethnic minority backgrounds, but how similar opportunities might extend to ELs remains unknown. We report findings from the first three years of an intervention that offers Early College opportunities in high schools serving large EL populations. Leveraging an exogenous policy change and rich administrative records, we examine the outcomes of pre- and post-program cohorts of ELs (N=15,090) in treated and untreated high schools. We find a large, significant impact on the number of college credits earned in 12th grade but no effect on immediate college attendance after high school. The probability of attending a four-year college significantly decreased.
While teacher evaluation policies have been central to efforts to enhance teaching quality over the past decade, little is known about how teachers change their instructional practices in response to such policies. To address this question, this paper drew on classroom observation and survey data to examine how early career teachers’ (ECTs’) perceptions of pressure associated with teacher evaluation policies seemed to affect their enactment of ambitious mathematics instruction. As part of our analysis, we also considered the role that mathematical knowledge for teaching (MKT) and school norms regarding teaching mathematics shape the potential influence of teacher evaluation policies on ECTs’ instructional practices. Understanding how the confluence of these factors is associated with teachers’ instruction provides important insights into how to improve teaching quality, which is one of the most important inputs for student learning.
Family and social networks are widely believed to influence important life decisions but identifying their causal effects is notoriously difficult. Using admissions thresholds that directly affect older but not younger siblings’ college options, we present evidence from the United States, Chile, Sweden and Croatia that older siblings’ college and major choices can significantly influence their younger siblings’ college and major choices. On the extensive margin, an older sibling’s enrollment in a better college increases a younger sibling’s probability of enrolling in college at all, especially for families with low predicted probabilities of enrollment. On the intensive margin, an older sibling’s choice of college or major increases the probability that a younger sibling applies to and enrolls in that same college or major. Spillovers in major choice are stronger when older siblings enroll and succeed in more selective and higher-earning majors. The observed spillovers are not well-explained by price, income, proximity or legacy effects, but are most consistent with older siblings transmitting otherwise unavailable information about the college experience and its potential returns. The importance of such personally salient information may partly explain persistent differences in college-going rates by geography, income, and other determinants of social networks.
Researchers commonly interpret effect sizes by applying benchmarks proposed by Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this paper, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, cost, and scalability into the process of interpreting the policy importance of effect sizes.
Using detailed administrative data for public schools, we document racial and ethnic segregation at the classroom level in North Carolina, a state that has experienced a sharp increase in Hispanic enrollment. We decompose classroom-level segregation in counties into within-school and between-school components. We find that the within-school component accounted for a sizable share of total segregation in middle schools and high schools. Recognizing its importance could temper the praise for school assignment policies that reduce racial disparities between schools but allow large disparities within them. More generally, we observe between the two components a complementary relationship, with one component tending to be large when the other one is small. Comparing the degree of segregation for the state’s two largest racial/ethnic minority groups, we find that White/Hispanic segregation was more severe than White/Black segregation, particularly within schools. Analyzed as separate administrative units, schools with large shares of Black students tended to have more White/Black segregation across classrooms than schools with smaller shares. Finally, we examine enrollment patterns by course and show that school segregation brings with it differences by race and ethnicity in the courses that students take, with White students more likely to be enrolled in advanced classes.
Castañeda v. Pickard (1981) mandated that educational programs for emergent bilinguals be tested for program efficacy. Since English language development (ELD) curricular materials are one part of an instructional program, we assess this mandate by examining the effectiveness of ELD materials in Texas. Using local linear matching, we find that schools that do not purchase any ELD curricula have significantly lower English language proficiency scores relative to schools that purchase state-adopted ELD materials. These results are robust across various matching models—inverse probability weights with regression adjustment, kernel matching, and nearest neighbor matching--and a comparative interrupted time series design. There is no significant difference between schools that adopt the two most popular ELD curricula—Rigby On Our Way to English and National Geographic Reach. This study suggests that emergent bilinguals (EBs) who attend schools that have instructional materials that explicitly foreground English language proficiency standards outperform those in schools that do not have such materials.
Although qualitative research suggests that school choice and other interventions are more beneficial for moderately disadvantaged than severely deprived students, the subject has barely been explored by quantitative studies with either observational or experimental designs. We estimate experimentally the impact of a voucher offer on college attainment of poor minority students by household income and parental education. Estimates are obtained from a 1997 private, lottery-based voucher intervention in New York City. National Student Clearinghouse provided 2017 postsecondary outcomes. Positive impacts on moderately disadvantaged students do not extend to the severely deprived.