Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Benjamin L. Castleman

Brian Heseung Kim, Kelli A. Bird, Benjamin L. Castleman.

Despite decades and hundreds of billions of dollars of federal and state investment in policies to promote postsecondary educational attainment as a key lever for increasing the economic mobility of lower-income populations, research continues to show large and meaningful differences in the mid-career earnings of students from families in the bottom and top income quintiles. Prior research has not disentangled whether these disparities are due to differential sorting into colleges and majors, or due to barriers lower socioeconomic status (SES) graduates encounter during the college-to-career transition. Using linked individual-level higher education and Unemployment Insurance (UI) records for nearly a decade of students from the Virginia Community College System (VCCS), we compare the labor market outcomes of higher- and lower-SES community college graduates within the same college, program, and academic performance level. Our analyses show that, conditional on employment, lower-SES graduates earn nearly $500/quarter less than their higher-SES peers one year after graduation, relative to higher-SES graduate average of $10,846/quarter. The magnitude of this disparity persists through at least three years after graduation. Disparities are concentrated among non-Nursing programs, in which gaps persist seven years from graduation. Our results highlight the importance of greater focus on the college-to-career transition.

More →

Andrew C. Barr, Kelli A. Bird, Benjamin L. Castleman, William L. Skimmyhorn.

Non-traditional students disproportionately enroll in institutions with weaker graduation and earnings outcomes. One hypothesis is that these students would have made different choices had they been provided with better information or supports during the decision-making process. We conducted a large-scale, multi-arm field experiment with the U.S. Army to investigate whether personalized information and the offer of advising assistance affect postsecondary choices and attainment among non-traditional adult populations. We provided U.S. Army service members transitioning out of the military with a package of research-based information and prompts, including quality and cost information on a personalized set of matched colleges, messages targeted at addressing veteran-specific concerns or needs, and reminders about key stages in the college and financial aid application process. For a randomly selected subset of the experimental sample, we also provided service members with opportunities to connect with a college advisor. We find no overall impact of the intervention on whether service members enroll in college, on the quality of their college enrollment, or on their persistence in college. We find suggestive evidence of a modest increase in degree completion within the period of observation, with these impacts mainly driven by increased attainment at for-profit institutions. Our results suggest that influencing non-traditional populations’ educational decisions and outcomes will require substantially more intensive programs and significant resources.

More →

Kelli A. Bird, Benjamin L. Castleman, Gabrielle Lohner.

The COVID-19 pandemic led to an abrupt shift from in-person to virtual instruction in Spring 2020. We use two complementary difference-in differences frameworks, one that leverages within-instructor-by-course variation on whether students started their Spring 2020 courses in person or online and another that incorporates student fixed effects. We estimate the impact of this shift on the academic performance of Virginia’s community college students. With both approaches, we find modest negative impacts (three to six percent) on course completion. Our results suggest that faculty experience teaching a given course online does not mitigate the negative effects. In an exploratory analysis, we find minimal long-term impacts of the switch to online instruction.

More →

Kelli A. Bird, Benjamin L. Castleman, Brett Fischer, Benjamin T. Skinner.

Recent state policy efforts have focused on increasing attainment among adults with some college but no degree (SCND). Yet little is actually known about the SCND population. Using data from the Virginia Community College System (VCCS), we provide the first detailed profile on the academic, employment, and earnings trajectories of the SCND population, and how these compare to VCCS graduates. We show that the share of SCND students who are academically ready to reenroll and would benefit from doing so may be substantially lower than policy makers anticipate. Specifically, we estimate that few SCND students (approximately three percent) could fairly easily re-enroll in fields of study from which they could reasonably expect a sizable earnings premium from completing their degree.

More →

Katharine Meyer, Kelli A. Bird, Benjamin L. Castleman.
With rapid technological transformations to the labor market, many working adults return to college after graduation to obtain additional training or credentials. Using a comparative individual fixed effects strategy and an administrative panel dataset of enrollment and employment in Virginia, we provide the first causal estimates of credential “stacking” – earning two or more community college certificates or degrees – among working adults. We find stacking increases employment by four percentage points and quarterly wages by $375 (four percent). Returns are larger for individuals studying in Health and who return to college after first completing a short-term certificate.

More →

Eric Bettinger, Benjamin L. Castleman, Alice Choe, Zachary Mabel.

Nearly half of students who enter college do not graduate. The majority of efforts to increase college completion have focused on supporting students before or soon after they enter college, yet many students drop out after making significant progress towards their degree. In this paper, we report results from a multi-year, large-scale experimental intervention conducted across five states and 20 broad-access, public colleges and universities to support students who are late in their college career but still at risk of not graduating. The intervention provided these “near-completer” students with personalized text messages that encouraged them to connect with campus-based academic and financial resources, reminded them of upcoming and important deadlines, and invited them to engage (via text) with campus-based advisors. We find little evidence that the message campaign affected academic performance or attainment in either the full sample or within individual higher education systems or student subgroups. The findings suggest low-cost nudge interventions may be insufficient for addressing barriers to completion among students who have made considerable academic progress.

More →

Andrew C. Barr, Benjamin L. Castleman.

We combine a large multi-site randomized control trial with administrative and survey data to demonstrate that intensive advising during high school and college leads to large increases in bachelor's degree attainment. Novel causal forest methods suggest that these increases are driven primarily by improvements in the quality of initial enrollment. Program effects are consistent across sites, cohorts, advisors, and student characteristics, suggesting the model is scalable. While current and proposed investments in postsecondary education focus on cutting costs, our result suggest that investment in advising is likely to be a more efficient route to promote bachelor's degree attainment.

More →

Zach Sullivan, Benjamin L. Castleman, Gabrielle Lohner, Eric Bettinger.

In-person college advising programs generate large improvements in college persistence and success for low-income students but face numerous barriers to scale. Remote advising models offer a promising strategy to address informational and assistance barriers facing the substantial majority of low-income students who do not have access to community-based advising, yet the existing evidence base on the efficacy of remote advising is limited. We present a comprehensive, multi-cohort experimental evaluation of CollegePoint, a national remote college advising program for high-achieving low- and moderate-income students. Students assigned to CollegePoint are modestly more likely (1.3 percentage points) to attend higher-quality institutions. Results from mechanism experiments we conducted within CollegePoint indicate that moderate changes to the program model, such as a longer duration of advising and modest expansions of the pool of students academically eligible to participate, do not lead to larger program effects. We also capitalize on across-cohort variation in whether students were affected by COVID-19 to investigate whether social distancing required by the pandemic increased the value of remote advising. CollegePoint increased attendance at higher-quality institutions by 3.2 percentage points for the COVID-19-affected cohort. Acknowledgements.

More →

Kelli A. Bird, Benjamin L. Castleman, Zachary Mabel, Yifeng Song.

Colleges have increasingly turned to predictive analytics to target at-risk students for additional support. Most of the predictive analytic applications in higher education are proprietary, with private companies offering little transparency about their underlying models. We address this lack of transparency by systematically comparing two important dimensions: (1) different approaches to sample and variable construction and how these affect model accuracy; and (2) how the selection of predictive modeling approaches, ranging from methods many institutional researchers would be familiar with to more complex machine learning methods, impacts model performance and the stability of predicted scores. The relative ranking of students’ predicted probability of completing college varies substantially across modeling approaches. While we observe substantial gains in performance from models trained on a sample structured to represent the typical enrollment spells of students and with a robust set of predictors, we observe similar performance between the simplest and most complex models.

More →

Benjamin L. Castleman, Denise Deutschlander, Gabrielle Lohner.

Growing experimental evidence demonstrates that low-touch informational, nudge, and virtual advising interventions are ineffective at improving postsecondary educational outcomes for economically-disadvantaged students at scale. Intensive in-person college advising programs are a considerably higher-touch and more resource intensive strategy; some programs provide students with dozen of hours of individualized assistance starting in high school and continuing through college, and can cost thousands of dollars per student served. Despite the magnitude of this investment, causal evidence on these programs' impact is quite limited, particularly for programs that serve Hispanic students, the fastest growing segment of U.S. college enrollees. We contribute new evidence on the impact of intensive college advising programs through a multi-cohort RCT of College Forward, which provides individualized advising from junior year of high school through college for a majority Hispanic student population in Texas. College Forward leads to a 7.5 percentage point increase in enrollment in college, driven entirely by increased enrollment at four-year universities. Students who receive College Forward advising are nearly 12 percentage points more likely to persist to their third year of college. While more costly and harder to scale than low-touch interventions, back of the envelope calculations suggest that the benefit from increased college graduation likely induced by the program outweighs operating costs in less than two years following college completion.

More →