Search EdWorkingPapers by author, title, or keywords.
Researchers use test outcomes to evaluate the effectiveness of education interventions across numerous randomized controlled trials (RCTs). Aggregate test data—for example, simple measures like the sum of correct responses—are compared across treatment and control groups to determine whether an intervention has had a positive impact on student achievement. We show that item-level data and psychometric analyses can provide information about treatment heterogeneity and improve design of future experiments. We apply techniques typically used in the study of Differential Item Functioning (DIF) to examine variation in the degree to which items show treatment effects. That is, are observed treatment effects due to generalized gains on the aggregate achievement measures or are they due to targeted gains on specific items? Based on our analysis of 7,244,566 item responses (265,732 students responding to 2,119 items) taken from 15 RCTs in low-and-middle-income countries, we find clear evidence for variation in gains across items. DIF analyses identify items that are highly sensitive to the interventions—in one extreme case, a single item drives nearly 40% of the observed treatment effect—as well as items that are insensitive. We also show that the variation of item-level sensitivity can have implications for the precision of effect estimates. Of the RCTs that have significant effect estimates, 41% have patterns of item-level sensitivity to treatment that allow for the possibility of a null effect when this source of uncertainty is considered. Our findings demonstrate how researchers can gain more insight regarding the effects of interventions via additional analysis of item-level test data.
This paper conceptualizes segregation as a phenomenon that emerges from the intersection of public policy and individual decision-making. Contemporary scholarship on complex decision-making describes a two-step process—1) Editing and 2) Selection— and has emphasized the individual decision-maker’s agency in both steps. We build on this work by exploring, both theoretically and empirically, how policy can structure the choices individuals face at each step. We conduct this exploration within the empirical context of enrollment decisions among families in the Wake County Public School System (WCPSS), which used a controlled school choice system to help achieve diversity aims. We first investigate the schooling choice sets that WCPSS constructed for families and then examine families’ schooling selections. We find that families were offered choice sets containing schools varying racial compositions, but that the racial makeup of schools in families’ choice set systematically varied by schooling type and student race/ethnicity. We further show that a majority of families enrolled in their district-assigned default school, with Black and Hispanic families more likely than white or Asian families to attend this option. Finally, we demonstrate that white or Asian families enroll in their default school at lower rates as the share of Black students increases.
How progressive is school spending when spending is measured at the school-level, instead of the district-level? We use the first dataset on school-level spending across schools throughout the United States to ask to what extent progressivity patterns previously examined across districts are amplified, nullified, or reversed, upon disaggregation to schools. We find that progressivity is systematically greater when we conduct a school-level analysis, rather than district-level analysis. This may be surprising, given the traditional view in public economics that local governments cannot effectively redistribute. We thus probe the data for explanations for this pattern, uncovering evidence that federal policies play an important role in driving within-district progressive allocations. In particular, we can explain about 83% of the within-district contribution to progressivity by the federal component of spending plus allocations that are empirically attributable to special education and English language learning programs. Our findings are thus consistent with the traditional view of redistribution being primarily the purview of central governments, operationalized in this context through mandates.
Challenging the conventional wisdom that the spread of democracy was a leading driver of the expansion of primary schooling, recent studies show that democratization in fact did not lead to an average increase in primary school enrollment rates. One reason for this null effect is that there was already considerable provision of primary education before democratization. Still, it is possible that the spread of democracy did impact other aspects of education systems, such as the content of education and the extent to which teaching jobs are politicized. Studying this possibility cross-nationally has been infeasible due to data limitations. To address this gap, we take advantage of an original dataset covering 160 countries from 1945 to 2021 that contains information about these aspects of education. We document that transitions to democracy tend to be preceded by a decline in the politicization of both education content and teaching jobs. However, soon after democratization occurs, this decline usually halts. Counterfactual estimates suggest that democratization roughly halves the degree to which teacher hiring and firing decisions are politicized, but has a smaller impact on the content of education. The empirical patterns that we uncover have important implications for future research.
Inequality related to standardized tests in college admissions has long been a subject of discussion; less is known about inequality in non-standardized components of the college application. We analyzed extracurricular activity descriptions in 5,967,920 applications submitted through the Common Application platform. Using human-crafted keyword dictionaries combined with text-as-data (natural language processing) methods, we found that White, Asian American, high-SES, and private school students reported substantially more activities, more activities with top-level leadership roles, and more activities with distinctive accomplishments (e.g., honors, awards). Disparities decrease when accounting for other applicant demographics, school fixed effects, and standardized test scores. Still, salient differences remain, especially those related to first-generation applicants. Implications and recommendations for college admissions policy and practice are discussed.
This is one of the first studies of the mismatch between students’ test scores and teachers’ estimations of those scores in low- and middle-income countries. Prior studies in high-income countries have found strong correlations between these metrics. We leverage data on actual and estimated scores in math and language from India and Bangladesh and find that teachers misestimate their students’ scores and that their estimations reveal their misconceptions about students in most need of support and variability within their class. This pattern is partly explained by teachers’ propensity to overestimate the scores of low-achieving students and to overweight the importance of intelligence. Teachers seem unaware of their errors, expressing confidence in estimations and surprise about their students’ performance once revealed.
Discussion of the rising price of higher education and associated student debt in America has been a key feature of political discourse in recent memory, with renewed interest sparked by the announcement of the student loan forgiveness plan. Federal student debt has increased by 756% since 1995, and total student debt tripled from 2007 to 2022. Concurrently, state support for public universities fell by 18% from 2000 to 2015. This phenomenon has drawn interest in the literature, with works by Jaquette and Curs (2015), Bound et al. (2016), Deming and Walters (2017), Webber (2017), and Mathias (2022) examining the effect of state disinvestment on higher education pricing and enrollment. This paper uses data from IPEDS to examine Colorado's College Opportunity Fund, which eliminated state appropriations to Colorado universities in 2006. I advance the literature by being the first to employ quasi-experimental methods, using a synthetic control identification strategy to measure the impact of this funding shock on enrollment and tuition revenue recuperation by Colorado universities. I find that Hispanic enrollment increased by 3 percentage points relative to the synthetic counterfactual, and that tuition revenue increased by 42% as a result of the policy. These results are robust to threats to identification, and placebo tests conrm the validity of the design. These findings provide robust evidence of the pitfalls of state disinvestment in higher education, and the consequences for students who are left to foot the bill.
A controversial, equity-focused mathematics reform in the San Francisco Unified School District (SFUSD) featured delaying Algebra I until ninth grade for all students. This descriptive study examines student-level longitudinal data on mathematics course-taking across successive cohorts of SFUSD students who spanned the reform’s implementation. We observe large changes in ninth and tenth grades (e.g., delaying Algebra I and Geometry). Participation in Advanced Placement (AP) math initially fell 15% (6 pp.) driven by declines in AP Calculus and among Asian/Pacific-Islander students. However, growing participation in acceleration options attenuated these reductions. Large ethnoracial gaps in advanced math course-taking remained.
The effects of the COVID-19 pandemic on students’ experiences in school were widespread. Early research show reductions in test scores across grade levels and student groups. This study extends research evidence to additional student outcomes – absences, course grades, and grad retention – and to examine how pandemic effects are distributed across students. Using a combination of descriptive and regression analyses, we find negative average impacts on all outcomes. These effects are largest at the high end of the absence distribution and the low end of the grade distribution. Effects are also largest in middle school for most outcomes and are typically larger among historically marginalized groups of students. These findings reflect widening achievement gaps and the need for targeted supports.
College attendance has increased significantly over the last few decades, but dropout rates remain high, with fewer than half of all adults ultimately obtaining a postsecondary credential. This project investigates whether one-on-one college coaching improves college attendance and completion outcomes for former low- and middle-income income state aid recipients who attended college but left prior to earning a degree. We conducted a randomized control trial with approximately 8,000 former students in their early- to mid-20s. Half of participants assigned to the treatment group were offered the opportunity to receive coaching services from InsideTrack, with all communication done remotely via phone or video. Intent-to-treat analyses based on assignment to coaching shows no impacts on college enrollment and we can rule out effects larger than a two-percentage point (5%) increase in subsequent Fall enrollment.