Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
EdWorkingPapers
Over the last twenty years, education researchers have increasingly conducted randomised experiments with the goal of informing the decisions of educators and policymakers. Such experiments have generally employed broad, consequential, standardised outcome measures in the hope that this would allow decisionmakers to compare effectiveness of different approaches. However, a combination of small effect sizes, wide confidence intervals, and treatment effect heterogeneity means that researchers have largely failed to achieve this goal. We argue that quasiexperimental methods and multi-site trials will often be superior for informing educators’ decisions on the grounds that they can achieve greater precision and better address heterogeneity. Experimental research remains valuable in applied education research. However, it should primarily be used to test theoretical models, which can in turn inform educators’ mental models, rather than attempting to directly inform decision making. Since comparable effect size estimates are not of interest when testing educational theory, researchers can and should improve the power of theory-informing experiments by using more closely aligned (i.e., valid) outcome measures. We argue that this approach would reduce wasteful research spending and make the research that does go ahead more statistically informative, thus improving the return on investment in educational research.
Not all students who could benefit from college apply. With novel data on over 1.2 million high schoolers, we show that nearly 25% start but never complete a college application. We use descriptive techniques, data visualizations, and fixed effects models to explore this population of college-interested “non-submitters” to observe application behaviors; document differences across individual, school, and community contexts; and identify factors most predictive of non-submission. We find large gaps by race/ethnicity, socioeconomic status, and education-career plans, as well as by school type and community features. We also find that early application tasks and engagement strongly predict non-submission. This study breaks ground for future research into this unexplored group and informs strategies to support those at risk of non-submission.
Tutoring has emerged as an especially promising strategy for supporting students academically. This study synthesizes 33 articles on the implementation of tutoring, defined as one-to-one or small-group instruction in which a human tutor supports students grades K-12 in an academic subject, to better understand the facilitators and barriers to program success. We find that policies influenced tutoring implementation through the allocation of federal funding and stipulation of program design. Tutoring program launch has often been facilitated by strategic relationships between schools and external tutoring providers and strengthened by transparent assessments of program quality and effectiveness. Successful implementation hinged on the support of school leaders with the power to direct school funding, space, and time. Tutoring setting and schedule, recruitment and training, and curriculum influenced whether students are able to access quality tutoring and instruction. Ultimately, evidence suggests that tutoring was most meaningful when tutors fostered positive student-tutor relationships which they drew upon to target instruction toward students’ strengths and needs.
Middle school transitions are increasingly required, despite documented negative effects on general education students (GENs). We explore if and how the move to middle school differentially affects students with disabilities (SWDs), a large and low-performing group of students. Using an instrumental variables strategy and NYC data on nine cohorts of students, we find the middle school transition causes a 0.29 standard deviation decline in SWD math performance, a 0.16 standard deviation decline in ELA performance, and a one percentage point increase in grade retention. However, after accounting for potential mediators (e.g. peer cohort stability) effects are similar for SWDs and GENs, suggesting the need to ease the middle school transition for all students.
The number of English learners enrolled in public schools has grown substantially in the United States over the past two decades. The growth is especially large in states in the South and Midwest that have not been traditional destinations for recent immigrants. In this study, we examine the effects of an influx of new English learners on students in receiving schools in Delaware, which is one of the so-called “new destination” states. We find significant positive spillover effects in the short term of new English learners on the test scores of the other students in the receiving schools. The positive effects are mainly concentrated among current and former English learners.
This mixed methods research study explores superintendents’ beliefs about and engagement in state education policymaking processes. Through interviews with 58 superintendents and a national survey of superintendents, I find that many superintendents feel that their voices have value in state policymaking spaces; however, superintendents’ actual engagement in state policymaking processes is relatively low. Three factors shape superintendents’ state policy engagement: (1) personal capacity, (2) proximity to state capitol, and (3) district size. This work can guide the development of formal educational opportunities and experiences to better prepare superintendents to engage in policy spaces. Colleges of education, researchers, and policymakers can also draw on this work as they consider solutions to reduce inequalities in superintendents’ access to shape state education policy.
School climate is critical to school effectiveness, but there is limited large-scale data available to examine the magnitude and nature of the relationship between school climate and school improvement. Drawing on statewide administrative data linked with unique teacher survey data in Michigan, we examine whether school climate appeared to play a role in the effects of a state-level school turnaround intervention. Using comparative interrupted time series models and descriptive mediation analysis, we find that students in schools with more positive school climate appeared to fare better than their peers in schools with less positive climate. Certain elements of climate—relational trust and school leadership—also mediated the effect of turnaround on student achievement. Our findings have implications for school improvement planning, for the design of evaluations of school turnaround initiatives, and for data collection by states aiming to improve their lowest performing schools.
The Core Knowledge curriculum is a K-8 curriculum focused on building students General Knowledge about the world they live in that is hypothesized to increase reading comprehension and Reading/English-LA achievement. This study utilizes an experimental design to evaluate the long term effects of attending Charter schools teaching the Core Knowledge curriculum. Fourteen oversubscribed kindergarten lotteries for enrollment in nine Core Knowledge Charter schools using the curriculum had 2310 students applying from parents in predominately middle/high income school districts. State achievement data was collected at 3rd- 6th grade in Reading/English-LA and Mathematics and at 5th Grade in Science. A new methodology addresses two previously undiscovered sources of bias inherent in kindergarten lotteries that include middle/high income families. The unbiased confirmatory Reading-English-LA results show statistically significant ITT (0.241***) and TOT (0.473***) effects for 3rd-6th grade achievement with statistically significant ITT and TOT effects at each grade. Exploratory analyses also showed significant ITT (0.15*) and TOT (0.300*) unbiased effects at 5th grade in Science. A CK-Charter school in a low income school district also had statistically significant, moderate to large unbiased ITT and TOT effects in English Language Arts (ITT= 0.944**; TOT = 1.299**), Mathematics (ITT= 0.735*; TOT = 0.997*) and positive, but insignificant Science effects (ITT= 0.468; TOT = 0.622) that eliminated achievement gaps in all subjects.
In this study, we examine an at-scale effort to encourage the formation of career pathways in California, with the goal of estimating the initiative’s causal effects on community college enrollment. We leverage a discontinuous assignment rule used to award grant funds to obtain credibly causal estimates of an ambitious $500 million effort to expand and establish career and technical education pathways between K-12 and community colleges. The competitive grant application process used a standardized rubric, and those receiving a score above a predetermined threshold were awarded funding (i.e., treatment group) while those just below received no funding (i.e., control group), allowing for a regression discontinuity (RD) design. We found that successful grantees did not experience overall enrollment increases in postsecondary partnerships; however, there were enrollment increases of 13.5 percent to 14.8 percent in CTE health sector courses, the program targeted the most for expansion. Manufacturing, and information communication technology, the other programs with the most expanded offerings, experienced no increases in postsecondary enrollment. The enrollment increases for the health sector were concentrated amongst female students in line with earlier findings by Bonilla (2020) documenting reductions in high school dropout rates for female students. These findings suggest that partnerships between K-12 schools and postsecondary institutions may be a viable avenue for increasing alignment between enrollment and high- growth sectors.