Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Paul T. von Hippel, Laura Bellows.

At least sixteen US states have taken steps toward holding teacher preparation programs (TPPs) accountable for teacher value-added to student test scores. Yet it is unclear whether teacher quality differences between TPPs are large enough to make an accountability system worthwhile. Several statistical practices can make differences between TPPs appear larger and more significant than they are. We reanalyze TPP evaluations from 6 states—New York, Louisiana, Missouri, Washington, Texas, and Florida—using appropriate methods implemented by our new caterpillar command for Stata. Our results show that teacher quality differences between most TPPs are negligible—.01-.03 standard deviations in student test scores—even in states where larger differences were reported previously. While ranking all a state’s TPPs may not be possible or desirable, in some states and subjects we can find a single TPP whose teachers stand out as significantly above or below average. Such exceptional TPPs may reward further study.

More →


Celeste K. Carruthers, Christopher Jepsen.

Vocational education is formal education about work, and vocational programs of study typically target a narrow subset of middle-income occupations. In this chapter, we trace vocational education from competing 20th century education philosophies to its varied structures throughout the 21st century world. We then review the body of economic research on labor market returns to vocational education. Three themes from this rapidly expanding literature are that (1) workers with a vocational education tend to have a flatter age-employment profile than workers with an academic education, (2) individuals who seek and gain access to more secondary vocational education tend to have better attainment and early-career outcomes, whereas the effects of large-scale changes to tracking in secondary grades are more ambiguous; and (3) vocational postsecondary education is associated with improved labor market outcomes relative to no or incomplete postsecondary education, particularly for multi-year programs. We close by highlighting areas where more empirical research is needed, which include a deeper understanding of the long-term and inter-generational effects of vocational education on stability and growth in earnings, and the effects of vocational education in the developing world.

More →


Jon Valant, Lindsay Weixler.

In order for school choice reforms to fulfill their potential, school choosers must be informed about their options. We conducted a randomized controlled trial during the school choice application period in New Orleans to assess the effects of providing information to parents. Families with children entering pre-K, kindergarten, or ninth grade were assigned to one of two treatment groups or a control group. A “performance” group received lists of the highest-performing schools or programs available (via U.S. mail, email, and text message). A “neighborhood” group received lists of the schools or programs in their home geographic zone. We find that the performance treatment made applicants significantly more likely to request high-performing schools, though the effects were concentrated among high school choosers. The performance treatment had especially strong effects among families of students with disabilities. The neighborhood treatment had only modest effects. We consider these findings in the context of questions about the role of information in school choice markets, as well as which families may be in particular need of support.

More →


Michel Grosz, Michal Kurlaender, Ann Stevens.

This article asks whether small changes to community college courses and programs can help improve student outcomes. We use administrative data from the California Community College system, including millions of student records and detailed course-level information for most career-technical education programs in the state. We construct a summary measure of each program’s flexibility, incorporating many components of the availability and scheduling of its courses. We show considerable variation in this flexibility measure across programs and over time. An increase in a program’s flexibility is associated with increases in enrollment and completions, but not with changes in its completion rate.

More →


Rebecca Unterman, Christina Weiland.

While there is a consensus that attending preschool better prepares children for kindergarten, evidence on the factors that sustain the preschool boost into the early elementary years is still emerging.  To add to this literature, we use lottery data from applicants to oversubscribed schools in Boston Public Schools (BPS) prekindergarten program to estimate variation in the effects of the program across school sites through the end of third grade.  Student outcomes include children’s kindergarten-through-second-grade retention, kindergarten-through-third-grade special education placement, and third-grade state English Language Arts and math test scores.  We find statistically significant variation in effects in all student outcomes and we predict this variation with multiple proxies for early elementary school quality.  We find that the academic proficiency of third-graders within the schools for which prekindergarten children competed is most strongly associated with prekindergarten program effects. Prekindergarten gains persisted if students applied to and won a seat in a higher-quality elementary school. Our findings appear to be driven by the schools themselves and not by student selection in higher-scoring schools, nor by the counterfactual.  These findings imply that policymakers and practitioners interested in sustained gains may need to also invest in improving the quality of children’s K-3 experience.

More →

Appendices266.36 KB

Todd R. Jones, Daniel Kreisman, Ross Rubenstein, Cynthia Searcy, Rachana Bhatt.

For years Georgia's HOPE Scholarship program provided full tuition scholarships to high achieving students. State budgetary shortfalls reduced its generosity in 2011. Under the new rules, only students meeting more rigorous merit-based criteria would retain the original scholarship covering full tuition, now called Zell Miller, with other students seeing aid reductions of approximately 15 percent. We exploit the fact that two of the criteria were high school GPA and SAT/ACT score, which students could not manipulate when the change took place. We compare already-enrolled students just above and below these cutoffs, making use of advances in multi-dimensional regression discontinuity, to estimate effects of partial aid loss. We show that, after the changes, aid flowed disproportionately to wealthier students, and find no evidence that the financial aid reduction affected persistence or graduation for these students. The results suggest that high-achieving students, particularly those already in college, may be less price sensitive than their peers.

More →


Mark J. Chin.

In this paper I study the impact of court-mandated school desegregation, which began in the late 1950s, on White individuals’ racial attitudes and politics in adulthood. Using geocoded nationwide data from the General Social Survey, I compare outcomes between respondents living in the same county who were differentially exposed to desegregated schools, based on respondent age and the year of court-mandated integration. With this differences-in-differences approach, I find that exposure to desegregated schools increased White individuals’ conservatism and negatively impacted their racial attitudes and support for policies promoting racial equity, such as affirmative action. Heterogeneity analyses indicate that effects are particularly pronounced in counties where opposition to integration was strongest: Southern counties desegregating after the passage of the Civil Rights Act of 1964 and counties where support for the Democratic presidential candidate between the 1960 and 1968 elections substantially decreased. My study provides causal evidence for key tenets of the contact hypothesis, which theorizes that Black-White contact in integrated schools can improve outgroup racial attitudes only under certain conditions, including when this intergroup contact has institutional support.

More →


Katharine Meyer, Kelli A. Bird, Benjamin L. Castleman.
With rapid technological transformations to the labor market along with COVID-19 related economic disruptions, many working adults return to college to obtain additional training or credentials. Using a comparative individual fixed effects strategy and an administrative panel dataset of enrollment and employment in Virginia, we provide the first causal estimates of credential “stacking” among working adults. We find stacking increases employment by four percentage points and quarterly wages by $570 (seven percent). Returns are larger for individuals whose first credential is a short-term certificate and in Health and Business.

More →


Heather C. Hill, Erica Litke, Kathleen Lynch.

Background:
For nearly three decades, policy-makers and researchers in the United States have promoted more intellectually rigorous standards for mathematics teaching and learning. Yet, to date, we have limited descriptive evidence on the extent to which reform-oriented instruction has been enacted at scale.

Purpose:
The purpose of the study is to examine the prevalence of reform-aligned mathematics instructional practices in five U.S. school districts. We also seek to describe the range of instruction students experience by presenting case studies of teachers at high, medium and low levels of reform alignment.

Participants:
We draw on 1,735 video-recorded lessons from 329 elementary teachers in these five U.S. urban districts.

Research Design:
We present descriptive analyses of lesson scores on a mathematics-focused classroom observation instrument. We also draw upon interviews with district personnel, rater-written lesson summaries, and lesson video in order to develop case studies of instructional practice.

Findings:
We find that teachers in our sample do use reform-aligned instructional practices, but that they do so within the confines of traditional lesson formats. We also find that the implementation of these instructional practices varies in quality. Furthermore, the prevalence and strength of these practices corresponds to the coherence of district efforts at instructional reform.

Conclusions:
Our findings suggest that unlike other studies in which reform-oriented instruction rarely occurred (e.g. Kane & Staiger, 2012), reform practices do appear to some degree in study classrooms. In addition, our analyses suggest that implementation of these reform practices corresponds to the strength and coherence of district efforts to change instruction.

More →


Silvia Robles, Max Gross, Robert W. Fairlie.

One frequently cited yet understudied channel through which money matters for college students is course availability- colleges may respond to budgetary pressure by reducing course offerings. Open admissions policies, binding class size constraints, and heavy reliance on state funding may make this channel especially salient at community colleges, which enroll 47% of U.S. undergraduates in public colleges and 55% of underrepresented minority students. We use administrative course registration data from a large community college in California to test this mechanism. By exploiting discontinuities in course admissions created by waitlists, we find that students stuck on a waitlist and shut out of a course section were 25% more likely to take zero courses that term relative to a baseline of 10%. Shutouts also increased transfer rates to nearby, but potentially lower quality, two-year colleges. These results document that course availability- even through a relatively small friction- can interrupt and distort community college students’ educational trajectories.

More →