Exploring the Impacts of an Early Childhood Educational Intervention on Later School Selection
In the current article, we examine the long-run school selection patterns of children randomly assigned to the Chicago School Readiness Project, an early childhood educational (ECE) intervention that aimed to improve the quality of Head Start classrooms serving low-income communities. Analyses suggest that adolescents who participated in the program were more likely to opt out of their assigned neighborhood school and attend schools with better indicators of academic performance. Further analyses suggested that these selection patterns began in elementary school, although elementary school quality explained only a small portion of the effect on high school selection. Results suggest that intensive ECE interventions could have lasting effects on children’s patterns of selection into later educational environments.
The current article reexamines the correlation between achievement test scores and earnings by providing new evidence on the association between academic skills and measures of adult earnings assessed when participants were in their 30s, 40s, and 50s. Results suggest that math and reading scores are strong predictors of economic attainment throughout participants’ careers, but these associations may also be sensitive to controls for other characteristics—including measures of the early family environment, general cognitive functioning, and socioemotional skills. Although these associations demonstrate the likely importance of achievement skills in determining labor market productivity, the variability in the achievement-to-earnings correlation suggests that researchers should apply caution when using the correlation to project the long-run effects of educational interventions.
Longitudinal studies of development often rely on correlational methods to examine linkages between early-life constructs and later-life outcomes. As highlighted by responses to our article, “Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Delay of Gratification and Later Outcomes,” interpretations of these linkages can be difficult. In this commentary, we address criticisms that our approach “over-controlled” for key factors related to a child’s ability to delay gratification, allay concerns over multicollinearity, and discuss how multivariate regression techniques can help clarify the interpretation of observed predictive relations.
Aiming Further: Addressing the Need for High Quality Longitudinal Research in Education
In educational research, the importance of longer-run follow-ups has been continually identified as a key priority for the field, with policy reports (Martin et al., 2018; McCormick, Hsueh, Weiland, & Bangser, 2017; Phillips et al., 2018), conference keynote addresses [see SREE invited lectures by Duncan (2015) and Singer (2019)], and “future directions” sections of research manuscripts noting the need to conduct evaluations with longitudinal follow-up. In recent years, the field has experienced substantial growth in the use of randomized control trials (RCTs) for the evaluation of educational programs, and at the same time, the wide availability of secondary administrative data sources has made longitudinal follow-up for these RCTs more possible than ever before (Penner & Dodge, 2019). However, despite these important innovations, educational interventions reporting long-run follow-up are still scarce, leaving a critical gap in the evaluation literature. In this commentary, we argue that this gap hampers the field’s progress, stifling our ability to empirically test fundamental theories regarding long-run development, and incentivizing research practices that are counter-productive to our widely-held goals. Below, we offer several options that researchers and funders could pursue to substantially strengthen our understanding of how educational programs influence long-term student outcomes.
The current paper reports long-term treatment impact estimates for a randomized evaluation of an early childhood intervention designed to promote children’s developmental outcomes and improve the quality of Head Start centers serving high-violence and high-crime areas in inner-city Chicago. Initial evaluations of end-of-preschool data reported that the program led to reductions in child behavioral problems and gains in measures of executive function and academic achievement. For this report, we analyzed adolescent follow-up data taken 10 to 11 years after program completion. We found evidence that the program had positive long-term effects on students’ executive function and grades, though effects were somewhat imprecise and dependent on the inclusion of baseline covariates. Results also indicated that treated children had heightened sensitivity to emotional stimuli, and we found no evidence of long-run effects on measures of behavioral problems. These findings raise the possibility that developing programs that improve on the Head Start model could carry long-run benefits for affected children.
We replicated and extended Shoda, Mischel, and Peake’s (1990) famous “marshmallow” study, which showed strong bivariate correlations between a child’s ability to delay gratification just before entering school and both adolescent achievement and socioemotional behaviors. Concentrating on children whose mothers had not completed college, we found that an additional minute waited at age 4 predicted a gain of approximately 1/10th of a SD in age-15 achievement. But this bivariate correlation was only half the size of those reported in the original studies, and was reduced by two-thirds in the presence of controls for family background, early cognitive ability, and the home environment. Most of the variation in adolescent achievement came from being able to wait at least 20 seconds. Associations between delay time and age-15 measures of behavioral outcomes were much smaller and rarely statistically significant.
Risky Business: Correlation and Causation in Developmental Studies- American Psychologist, 2018
Developmental theories often posit that changes in children’s early psychological characteristics will affect much later psychological, social, and economic outcomes. However, tests of these theories frequently yield results that are consistent with plausible alternative theories that posit a much smaller causal role for earlier levels of these psychological characteristics. Our paper explores this issue with empirical tests of skill building theories, which predict that early boosts to simpler skills (e.g., numeracy or literacy) or behaviors (e.g, anti-social behavior or executive functions) support the long-term development of more sophisticated skills or behaviors. Substantial longitudinal associations between academic or socioemotional skills measured early and then later in childhood or adolescence are often taken as support of these skill-building processes. Using the example of skill-building in mathematics, we argue that longitudinal correlations, even if adjusted for an extensive set of baseline covariates, constitute an insufficiently risky test of skill-building theories. We first show that experimental manipulation of early math skills generates much smaller effects on later math achievement than the non-experimental literature has suggested. We then conduct falsification tests that show puzzlingly high cross-domain associations between early math and later literacy achievement. Finally, we show that a skill-building model positing a combination of unmeasured stable factors and skill-building processes is able to reproduce the pattern of experimental impacts on children’s mathematics achievement. Implications for developmental theories, methods, and practice are discussed.
Causal Impact of Early Math Learning- Child Development, 2017
The current study estimated the causal links between preschool mathematics learning and late elementary school mathematics achievement, using variation in treatment assignment to an early mathematics intervention as an instrument for preschool mathematics change. Estimates indicate (n= 410) that a standard-deviation of intervention-produced change at age 4 is associated with a 0.24 standard deviation gain in achievement in late elementary school. This impact is approximately half the size of the association produced by correlational models relating later achievement to preschool math change, and is approximately 35% smaller than the effect reported by highly-controlled OLS regression models (Claessens et al., 2009; Watts et al., 2014) using national datasets. Implications for developmental theory and practice are discussed.
Early Math Intervention and Stable Learning Processes- JREE, 2017
Early educational intervention effects typically fade in the years following treatment, and few studies have investigated why achievement impacts diminish over time. The current study tested the effects of a preschool mathematics intervention on two aspects of children’s mathematical development. We tested for separate effects of the intervention on “state” (occasion-specific) and “trait” (relatively stable) variability in mathematics achievement. Results indicated that, although the treatment had a large impact on state mathematics, the treatment had no effect on trait mathematics, or the aspect of mathematics achievement that influences stable individual differences in mathematics achievement over time. Results did suggest, however, that the intervention could affect the underlying processes in children’s mathematical development by inducing more transfer of knowledge immediately following the intervention for students in the treated group.
State and Trait Models of Math Achievement- Psychological Science, 2014
Substantial longitudinal relations between children’s early mathematics achievement and their much later mathematics achievement are firmly established. These findings are seemingly at odds with studies showing that early educational interventions have diminishing effects on children’s mathematics achievement across time. We hypothesized that individual differences in children’s later mathematical knowledge are more an indicator of stable, underlying characteristics related to mathematics learning throughout development than of direct effects of early mathematical competency on later mathematical competency. We tested this hypothesis in two longitudinal data sets, by simultaneously modeling effects of latent traits (stable characteristics that influence learning across time) and states (e.g., prior knowledge) on children’s mathematics achievement over time. Latent trait effects on children’s mathematical development were substantially larger than state effects. Approximately 60% of the variance in trait mathematics achievement was accounted for by commonly used control variables, such as working memory, but residual trait effects remained larger than state effects. Implications for research and practice are discussed.
Early Growth in Math Learning and Later Achievement- Educational Researcher, 2014
Although previous research has established the association between early-grade mathematics knowledge and later mathematics achievement, few studies have measured mathematical skills prior to school entry, and few have investigated the predictive power of early gains in mathematics ability. The current paper relates mathematical skills measured at 54 months to adolescent mathematics achievement using multi-site longitudinal data. We find that preschool mathematics ability predicts mathematics achievement through age 15, even after accounting for early reading, cognitive skills, and family and child characteristics. Moreover, we find that growth in mathematical ability between age 54 months and first grade is an even stronger predictor of adolescent mathematics achievement. These results demonstrate the importance of prekindergarten mathematics knowledge and early math learning for later achievement.
Tyler W. Watts 