Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
Post-secondary education
Recent interest to promote and support replication efforts assume that there is well-established methodological guidance for designing and implementing these studies. However, no such consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies. Our general approach is derived from the Causal Replication Framework (CRF), which formalizes the assumptions under which replication success can be expected. The assumptions may be understood broadly as replication design requirements and individual study design requirements. Replication failure occurs when one or more CRF assumptions are violated. In design-based approaches to replication, CRF assumptions are systematically tested to evaluate the replicability of effects, as well as to identify sources of effect variation when replication failure is observed. In direct replication designs, replication failure is evidence of bias or incorrect reporting in individual study estimates, while in conceptual replication designs, replication failure occurs because of effect variation due to differences in treatments, outcomes, settings, and participant characteristics. The paper demonstrates how multiple research designs may be combined in systematic replication studies, as well as how diagnostic measures may be used to assess the extent to which CRF assumptions are met in field settings.
Developmental education, in which college students deemed unprepared for college-level coursework enroll in non-credit bearing courses, is widespread in American higher education. The current study evaluates the effect of mobile app courseware on the college outcomes of developmental education students, using a research design which randomly assigned course sections to receive access to the apps or not. The results show that access to the apps significantly improved student performance in developmental education outcomes, marginally improved medium-term college persistence and performance, but did not effect credential attainment in the study timeframe. Despite a number of barriers to implementation, the results suggest the intervention has the potential to improve the short-term outcomes of developmental education students in addition to being low-cost and scalable.
We draw on administrative data from the country of Colombia to assess differences in student learning in online and traditional on-campus college programs. The Colombian context is uniquely suited to study this topic, as students take an exit examination at the end of their studies. We can therefore directly compare performance on the exit exam for students in online and on-campus programs both across and within institutions, degrees, and majors. Using inverse probability weighting methods based on a rich set of background characteristics coupled with institution-degree-major fixed effects, our results suggest that bachelor’s degree students in online programs perform worse on nearly all test score measures (including math, reading, writing, and English) relative to their counterparts in on-campus programs. Results for shorter technical certificates are more mixed. While online students perform significantly worse than on-campus students on exit exams in private institutions, they perform better in SENA—the main public vocational institution in the country.
The transfer between two-year and four-year colleges is a critical path to baccalaureate attainment. Yet, students face a number of barriers in transfer pathways, including a lack of coherent coordination and articulation between their community colleges and four-year institutions, resulting in excess units and increased time to degree. In this paper we evaluate the impact of California’s Student Transfer Achievement Reform Act, which aimed to create a more seamless pathway between the Community Colleges and the California State University. We investigate whether the reform effort met its intended goal of improving baccalaureate receipt, and greater efficiency in earning these degrees, among community college transfer students. We tease out plausibly causal effects of the policy by leveraging the exogenous variation in the timing of the implementation of the reform in different campuses and fields of study. We find that this reform effort has led to significant reductions in time to baccalaureate receipt among community college transfers and reduced total unit accumulation. These positive effects were shared across all student subgroups.
I study how postsecondary admission policies affect the composition and subsequent academic outcomes of new cohorts. I leverage the staggered replacement of lotteries and waitlists at California's community college nursing programs with admissions that rely on grades, work experience, and other measures. The change in admissions increased the average prior academic performance of incoming cohorts, but did not improve academic outcomes such as completion rates or pass rates on the national licensing exam. I find suggestive evidence that the change in admissions decreased the share of new students who were not White, but by small amounts. The change in admissions also substantially reduced the time students spent waiting between taking prerequisite coursework and entering the nursing programs.
The Top 10% Plan admissions policy has now been in place in Texas for over two decades. We analyze 18 years of post-Top 10% Plan data to look for evidence of increased access to the selective Texas flagship campuses among all Texas high schools. We provide a detailed description of changes in enrollment patterns at the flagship campuses from Texas high schools after the implementation of the Top 10% Plan, focusing on whether the policy resulted in new sending patterns from high schools that did not have a history of sending students to the flagship campuses. Our analysis reveals an increase in the likelihood that high schools in non-suburban areas sent students to the flagship campuses, but ultimately little to no equity-producing effects of the Top 10% Plan over this 18-year period. In fact, the representation of traditional, always-sending, feeder high schools on the flagship campuses continued to dwarf the population of students from other high schools. Thus, the purported high school representation benefits of the policy appear to be overstated and may not go as far as advocates might have hoped in terms of generating equity of access to the flagship campuses in the state.
Multiple studies have documented the positive effect of school choice on college attendance. We focus instead on the quality of colleges, which is linked to higher graduation rates and later-in-life wages, especially for Black and Hispanic students. We examine the effect of the New Orleans school reforms, a district-wide reform creating an almost all-charter school district, on the quality of colleges that students attended. Using difference-in-differences analysis of statewide microdata, we find that the reforms led students to attend four-year colleges, and higher-quality ones, at higher rates. The reforms also increased the share of college-goers who were well matched to their colleges and this had little effect on transfer or persistence rates. Overall, these results reinforce that the reforms led students to attend higher-quality colleges that will improve long-term life outcomes.
Using data with detailed instructor employment information from a state college system, this study examines disciplinary variations in the characteristics and effects of non-tenure-track faculty hired through temporary and long-term employment. We identify substantial differences in the demographic and employment characteristics between the two types of non-tenure-line faculty, where the differences are most pronounced in STEM fields at four-year colleges. Using an instrumental variables strategy to address student sorting, our analyses indicate that taking introductory courses with temporary adjuncts reduces subsequent interest, and the effects are particularly large in STEM fields at four-year colleges. Long-term non-tenure faculty are generally comparable to tenure-track faculty in student subsequent interest, but tenure-track faculty are associated with better subsequent performance in a handful of fields.
We examine U.S. children whose parents won the lottery to trace out the effect of financial resources on college attendance. The analysis leverages federal tax and financial aid records and substantial variation in win size and timing. While per-dollar effects are modest, the relationship is weakly concave, with a high upper bound for amounts greatly exceeding college costs. Effects are smaller among low-SES households, not sensitive to how early in adolescence the shock occurs, and not moderated by financial aid crowd-out. The results imply that households derive consumption value from college and household financial constraints alone do not inhibit attendance.
This article asks whether small changes to community college courses and programs can help improve student outcomes. We use administrative data from the California Community College system, including millions of student records and detailed course-level information for most career-technical education programs in the state. We construct a summary measure of each program’s flexibility, incorporating many components of the availability and scheduling of its courses. We show considerable variation in this flexibility measure across programs and over time. An increase in a program’s flexibility is associated with increases in enrollment and completions, but not with changes in its completion rate.