Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
Despite calls for more evaluative research in teacher education, formal assessments of the effectiveness of novel teacher education practices remain rare. One reason is that we lack designs and measurement approaches that appropriately meet the challenges of causal inference in the field. In this article, we seek to fill this gap. We first outline the difficulties of doing evaluative work in teacher education. We then describe a set of replicable practices for developing measures of key teaching outcomes, and propose evaluative research designs that can be adapted to suit the needs of the field. Finally, we identify community-wide initiatives that are necessary to advance useful evaluative research.
The COVID-19 pandemic has put virtual schooling at the forefront of policy concerns, as millions of children worldwide shift to virtual schooling with hopes of “slowing the spread”. Given the emergency shift to online education coupled with the large increase in demand for virtual education over the last decade it is imperative to explore the impacts of virtual education on student outcomes. This paper estimates the causal effect of full-time virtual school attendance on student outcomes with important implications for school choice, online education, and education policy. Despite the increasing demand for K-12 virtual schools over the past decade little is known about the impact of full-time virtual schools on students’ cognitive and behavioral outcomes. The existing evidence on the impact of online education on students’ outcomes is mixed. I use a longitudinal data set composed of individual-level information on all public-school students and teachers throughout Georgia from 2007 to 2016 to investigate how attending virtual schools influences student outcomes. I implement a variety of econometric specifications to account for the issue of potential self-selection into full-time virtual schools. I find that attending a virtual school leads to a reduction of 0.1 to 0.4 standard deviations in English Language Arts, Mathematics, Science, and Social Studies achievement test scores for students in elementary and middle school. I also find that ever attending a virtual school is associated with a 10-percentage point reduction in the probability of ever graduating from high school. This is early evidence that full-time virtual schools as a type of school choice could be harmful to students’ learning and future economic opportunities, as well as a sub-optimal use of taxpayer money.
Enrollment in higher education has risen dramatically in Latin America, especially in Chile. Yet graduation and persistence rates remain low. One way to improve graduation and persistence is to use data and analytics to identify students at risk of dropout, target interventions, and evaluate interventions’ effectiveness at improving student success. We illustrate the potential of this approach using data from eight Chilean universities. Results show that data available at matriculation are only weakly predictive of persistence, while prediction improves dramatically once data on university grades become available. Some predictors of persistence are under policy control. Financial aid predicts higher persistence, and being denied a first-choice major predicts lower persistence. Student success programs are ineffective at some universities; they are more effective at others, but when effective they often fail to target the highest risk students. Universities should use data regularly and systematically to identify high-risk students, target them with interventions, and evaluate those interventions’ effectiveness.
India took a decisive step toward universal basic education by proclaiming a constitutionally-guaranteed Right to Education (RTE) Act in 2009 that called for full access of children aged 6-14 to free schooling. This paper considers the offsetting effects to RTE from induced expansion of private tutoring in the educationally competitive districts of India. We develop a unique database of registrations of new private educational institutions offering tutorial services by local district between 2001-2015. We estimate the causal impact of RTE on private supplemental education by comparing the growth of these private tutorial institutions in districts identified a priori as having very competitive educational markets to those that had less competitive educational markets. We find a strong impact of RTE on the private tutoring market and show that this holds across alternative definitions of highly competitive districts and a variety of robustness checks, sensitivity analyses, and controls. Finally, we provide descriptive evidence that these private tutoring schools do increase the achievement (and competitiveness) of students able to afford them.
This study used college dormitory room and social group assignment data to investigate the peer effect on the probability of college students switching their major fields of study. The results revealed strong evidence of peer effects on students’ decisions to switch majors. In particular, the number of a student’s peers who have the same major significantly reduces the student’s likelihood of switching majors; however, when a same-major peer switches majors, it significantly increases a student’s probability of switching majors. This study also found that peers’ majors affected students’ choice of destination majors. Students in the same peer group are more likely to choose the same destination majors, compared to non-peers. Finally, we found that in general peer effects at the dormitory room level, both in choice and persistence of major, were stronger than were peer effects at the social group level.
Narrative accounts of classroom instruction suggest that external interruptions, such as intercom announcements and visits from staff, are a regular occurrence in U.S. public schools. We study the frequency, nature, and duration of external interruptions in the Providence Public School District (PPSD) using original data from a district-wide survey and classroom observations. We estimate that a typical classroom in PPSD is interrupted over 2,000 times per year, and that these interruptions and the disruptions they cause result in the loss of between 10 to 20 days of instructional time. Administrators appear to systematically underestimate the frequency and negative consequences of these interruptions. We propose several organizational approaches schools might adopt to reduce external interruptions to classroom instruction.
Success in postsecondary education requires students to engage with their institution both academically and administratively. As with the transition to college, administrative requirements students face once enrolled can be substantial. Missteps with required processes can threaten students’ ability to persist. During the 2018-19 academic year, Georgia State University implemented an artificially intelligent text-based chatbot to provide proactive outreach and support to help undergraduates navigate administrative processes and take advantage of campus resources. A team of centralized university administrators orchestrated outreach “campaigns” to support students across three broad domains: (1) academic supports; (2) social and career supports; and (3) administrative processes. We investigate GSU’s implementation of this persistence-focused chatbot through an experimental study. Of the three message domains, outreach was most effective when focused on administrative processes, many of which were time-sensitive and for which outreach could be targeted specifically to students for whom it was relevant based on administrative data. In contrast, outreach to encourage take up of other supports had little effect on student behavior. By the end of the academic year, rates of FAFSA filing and registration for the subsequent fall semester were approximately three percentage points higher, suggesting positive effects on year-to-year college persistence. The positive effects on fall enrollment persisted into summer 2019, at which time the GSU administration judged that the study results were compelling enough to conclude the experiment and roll the chatbot system out to all students. We situate our findings in the literature on nudge-type efforts to support college access and success to draw lessons regarding their effective use.
Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers, but conventional classroom observations are costly, prone to rater bias, and hard to implement at scale. Using nearly 1,000 word-to-word transcriptions of 4th- and 5th-grade English language arts classes, we apply novel text-as-data methods to develop automated, objective measures of teaching to complement classroom observations. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers’ average classroom observation scores. The interactive instruction factor predicts positive value-added scores.
States and districts are increasingly incorporating measures of achievement growth into their school accountability systems, but there is little research on how these changes affect the public’s perceptions of school quality. We conduct a nationally representative online survey experiment to identify the effects of providing participants with information about their local school districts’ average achievement status and/or average achievement growth. In the control group, participants who live in higher status districts tend to grade their local schools more favorably. The provision of status information does not fundamentally alter this relationship. The provision of growth information, however, reshapes Americans’ views about educational performance. Once informed, participants’ evaluations of their local public schools better reflect the variation in district growth.
We provide the first estimated economic impacts of students’ access to an entire sector of public higher education in the U.S. Approximately half of Georgia high school graduates who enroll in college do so in the state’s public four-year sector, which requires minimum SAT scores for admission. Regression discontinuity estimates show enrollment in public four-year institutions boosts students’ household income around age 30 by 20 percent, and has even larger impacts for those from low income high schools. Access to this sector has little clear impact on student loan balances or other measures of financial health. For the marginal student, enrollment in such institutions has large private returns even in the short run and positive returns to state budgets in the long run.