Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Methodology, measurement and data

Eric A. Hanushek, Paul E. Peterson, Laura M. Talpey, Ludger Woessmann.
Concerns about the breadth of the U.S. income distribution and limited intergenerational mobility have led to a focus on educational achievement gaps by socio-economic status (SES).  Uintertemporally linked assessments from NAEP, TIMSS, and PISA, we trace the achievement of U.S. student cohorts born between 1954 and 2001.  Achievement gaps between the top and bottom deciles and the top and bottom quartiles of the SES distribution have been large and remarkably constant for a near half century.  These unwavering gaps have not been offset by overall improvements in achievement levels, which have risen at age 14 but remained unchanged at age 17 for the most recent quarter century.  The long-term failure of major educational policies to alter SES gaps suggests a need to reconsider standard approaches to mitigating disparities.

More →

Christina Ciocca Eller.

The rise of accountability standards has pressed higher education organizations to oversee the production and publication of data on student outcomes more closely than in the past. However, the most common measure of student outcomes, average bachelor's degree completion rates, potentially provides little information about the direct impacts of colleges and universities on student success. Extending scholarship in the new institutionalist tradition, I hypothesize that higher education organizations today exist as, “superficially coupled systems,” where colleges closely oversee their technical outputs but where those technical outputs provide limited insight into the direct role of colleges and universities in producing them. I test this hypothesis using administrative data from the largest, public, urban university system in the United States together with fixed effects regression and entropy balancing techniques, allowing me to isolate organizational effects. My results provide evidence for superficial coupling, suggesting that inequality in college effectiveness exists both between colleges and within colleges, given students' racial background and family income. They also indicate that institutionalized norms surrounding accountability have backfired, enabling higher education organizations, and other bureaucratic organizations like them, to maintain legitimacy without identifying and addressing inequality.

More →

Robert Garlick, Joshua Hyman.

We use a natural experiment to evaluate sample selection correction methods' performance. In 2007, Michigan began requiring that all students take a college entrance exam, increasing the exam-taking rate from 64 to 99%. We apply different selection correction methods, using different sets of predictors, to the pre-policy exam score data. We then compare the corrected data to the complete post-policy exam score data as a benchmark. We find that performance is sensitive to the choice of predictors, but not the choice of selection correction method. Using stronger predictors such as lagged test scores yields more accurate results, but simple parametric methods and less restrictive semiparametric methods yield similar results for any set of predictors. We conclude that gains in this setting from less restrictive econometric methods are small relative to gains from richer data. This suggests that empirical researchers using selection correction methods should focus more on the predictive power of covariates than robustness across modeling choices.

More →