(Abstracts and links provided below)
Understanding heterogeneity in the impact of public preschool programs
We examine the North Carolina Pre-K (NC Pre-K) program to test the hypothesis that observed variation in effects resulting from exposure to the program can be attributed to interactions with other environmental factors that occur before, during, or after the pre-k year. We examine student outcomes in 5th grade and test interaction effects between NC’s level of investment in public pre-k and moderating factors. Our main sample includes the population of children born in North Carolina between 1987 and 2005 who later attended a public school in that state, had valid achievement data in 5th grade, and could be matched by administrative record review (n = 1,207,576; 58% White non-Hispanic, 29% Black non-Hispanic, 7% Hispanic, 6% multiracial and Other race/ethnicity). Analyses were based on a natural experiment leveraging variation in county-level funding for NC Pre-K across NC counties during each of the years the state scaled up the program. Exposure to NC Pre-K funding was defined as the per-4-year-old-child state allocation of funds to a county in a year. Regression models included child-level and county-level covariates and county and year fixed effects.
Estimates indicate that a child’s exposure to higher NC Pre-K funding was positively associated with that child’s academic achievement 6 years later. We found no effect on special education placement or grade retention. NC Pre-K funding effects on achievement were positive for all subgroups tested, and statistically significant for most. However, they were larger for children exposed to more disadvantaged environments either before or after the pre-k experience, consistent with a compensatory model where pre-k provides a buffer against the adverse effects of prior negative environmental experiences and protection against the effects of future adverse experiences. In addition, the effect of NC Pre-K funding on achievement remained positive across most environments, supporting an additive effects model. In contrast, few findings supported a dynamic complementarity model. Instrumental variables analyses incorporating a child’s NC Pre-K enrollment status indicate that program attendance increased average 5th grade achievement by approximately 20% of a standard deviation, and impacts were largest for children who were Hispanic or whose mothers had less than a high school education. Implications for the future of pre-k scale-up and developmental theory are discussed.
This study examines whether changes in classroom quality predict within-child changes in achievement and behavioral problems in elementary school (ages spanning approximately 6–11 years old). Drawing on data from a longitudinal study of children in predominantly low-income, nonurban communities (n = 1,078), we relied on child fixed effects modeling, which controlled for stable factors that could bias the effects of classroom quality. In general, we found that changes in classroom quality had small and statistically nonsignificant effects on achievement and behavior. However, we found that moving into a high-quality classroom, particularly those rated as high in Classroom Organization, had positive effects on achievement and behavior for children with significant exposure to poverty in early life.
Exploring the Impacts of an Early Childhood Educational Intervention on Later School Selection
In the current article, we examine the long-run school selection patterns of children randomly assigned to the Chicago School Readiness Project, an early childhood educational (ECE) intervention that aimed to improve the quality of Head Start classrooms serving low-income communities. Analyses suggest that adolescents who participated in the program were more likely to opt out of their assigned neighborhood school and attend schools with better indicators of academic performance. Further analyses suggested that these selection patterns began in elementary school, although elementary school quality explained only a small portion of the effect on high school selection. Results suggest that intensive ECE interventions could have lasting effects on children’s patterns of selection into later educational environments.
The current article reexamines the correlation between achievement test scores and earnings by providing new evidence on the association between academic skills and measures of adult earnings assessed when participants were in their 30s, 40s, and 50s. Results suggest that math and reading scores are strong predictors of economic attainment throughout participants’ careers, but these associations may also be sensitive to controls for other characteristics—including measures of the early family environment, general cognitive functioning, and socioemotional skills. Although these associations demonstrate the likely importance of achievement skills in determining labor market productivity, the variability in the achievement-to-earnings correlation suggests that researchers should apply caution when using the correlation to project the long-run effects of educational interventions.
Longitudinal studies of development often rely on correlational methods to examine linkages between early-life constructs and later-life outcomes. As highlighted by responses to our article, “Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Delay of Gratification and Later Outcomes,” interpretations of these linkages can be difficult. In this commentary, we address criticisms that our approach “over-controlled” for key factors related to a child’s ability to delay gratification, allay concerns over multicollinearity, and discuss how multivariate regression techniques can help clarify the interpretation of observed predictive relations.
Aiming Further: Addressing the Need for High Quality Longitudinal Research in Education
In educational research, the importance of longer-run follow-ups has been continually identified as a key priority for the field, with policy reports (Martin et al., 2018; McCormick, Hsueh, Weiland, & Bangser, 2017; Phillips et al., 2018), conference keynote addresses [see SREE invited lectures by Duncan (2015) and Singer (2019)], and “future directions” sections of research manuscripts noting the need to conduct evaluations with longitudinal follow-up. In recent years, the field has experienced substantial growth in the use of randomized control trials (RCTs) for the evaluation of educational programs, and at the same time, the wide availability of secondary administrative data sources has made longitudinal follow-up for these RCTs more possible than ever before (Penner & Dodge, 2019). However, despite these important innovations, educational interventions reporting long-run follow-up are still scarce, leaving a critical gap in the evaluation literature. In this commentary, we argue that this gap hampers the field’s progress, stifling our ability to empirically test fundamental theories regarding long-run development, and incentivizing research practices that are counter-productive to our widely-held goals. Below, we offer several options that researchers and funders could pursue to substantially strengthen our understanding of how educational programs influence long-term student outcomes.
The current paper reports long-term treatment impact estimates for a randomized evaluation of an early childhood intervention designed to promote children’s developmental outcomes and improve the quality of Head Start centers serving high-violence and high-crime areas in inner-city Chicago. Initial evaluations of end-of-preschool data reported that the program led to reductions in child behavioral problems and gains in measures of executive function and academic achievement. For this report, we analyzed adolescent follow-up data taken 10 to 11 years after program completion. We found evidence that the program had positive long-term effects on students’ executive function and grades, though effects were somewhat imprecise and dependent on the inclusion of baseline covariates. Results also indicated that treated children had heightened sensitivity to emotional stimuli, and we found no evidence of long-run effects on measures of behavioral problems. These findings raise the possibility that developing programs that improve on the Head Start model could carry long-run benefits for affected children.
We replicated and extended Shoda, Mischel, and Peake’s (1990) famous “marshmallow” study, which showed strong bivariate correlations between a child’s ability to delay gratification just before entering school and both adolescent achievement and socioemotional behaviors. Concentrating on children whose mothers had not completed college, we found that an additional minute waited at age 4 predicted a gain of approximately 1/10th of a SD in age-15 achievement. But this bivariate correlation was only half the size of those reported in the original studies, and was reduced by two-thirds in the presence of controls for family background, early cognitive ability, and the home environment. Most of the variation in adolescent achievement came from being able to wait at least 20 seconds. Associations between delay time and age-15 measures of behavioral outcomes were much smaller and rarely statistically significant.
Risky Business: Correlation and Causation in Developmental Studies- American Psychologist, 2018
Developmental theories often posit that changes in children’s early psychological characteristics will affect much later psychological, social, and economic outcomes. However, tests of these theories frequently yield results that are consistent with plausible alternative theories that posit a much smaller causal role for earlier levels of these psychological characteristics. Our paper explores this issue with empirical tests of skill building theories, which predict that early boosts to simpler skills (e.g., numeracy or literacy) or behaviors (e.g, anti-social behavior or executive functions) support the long-term development of more sophisticated skills or behaviors. Substantial longitudinal associations between academic or socioemotional skills measured early and then later in childhood or adolescence are often taken as support of these skill-building processes. Using the example of skill-building in mathematics, we argue that longitudinal correlations, even if adjusted for an extensive set of baseline covariates, constitute an insufficiently risky test of skill-building theories. We first show that experimental manipulation of early math skills generates much smaller effects on later math achievement than the non-experimental literature has suggested. We then conduct falsification tests that show puzzlingly high cross-domain associations between early math and later literacy achievement. Finally, we show that a skill-building model positing a combination of unmeasured stable factors and skill-building processes is able to reproduce the pattern of experimental impacts on children’s mathematics achievement. Implications for developmental theories, methods, and practice are discussed.
Causal Impact of Early Math Learning- Child Development, 2017
The current study estimated the causal links between preschool mathematics learning and late elementary school mathematics achievement, using variation in treatment assignment to an early mathematics intervention as an instrument for preschool mathematics change. Estimates indicate (n= 410) that a standard-deviation of intervention-produced change at age 4 is associated with a 0.24 standard deviation gain in achievement in late elementary school. This impact is approximately half the size of the association produced by correlational models relating later achievement to preschool math change, and is approximately 35% smaller than the effect reported by highly-controlled OLS regression models (Claessens et al., 2009; Watts et al., 2014) using national datasets. Implications for developmental theory and practice are discussed.
Early Math Intervention and Stable Learning Processes- JREE, 2017
Early educational intervention effects typically fade in the years following treatment, and few studies have investigated why achievement impacts diminish over time. The current study tested the effects of a preschool mathematics intervention on two aspects of children’s mathematical development. We tested for separate effects of the intervention on “state” (occasion-specific) and “trait” (relatively stable) variability in mathematics achievement. Results indicated that, although the treatment had a large impact on state mathematics, the treatment had no effect on trait mathematics, or the aspect of mathematics achievement that influences stable individual differences in mathematics achievement over time. Results did suggest, however, that the intervention could affect the underlying processes in children’s mathematical development by inducing more transfer of knowledge immediately following the intervention for students in the treated group.
State and Trait Models of Math Achievement- Psychological Science, 2014
Substantial longitudinal relations between children’s early mathematics achievement and their much later mathematics achievement are firmly established. These findings are seemingly at odds with studies showing that early educational interventions have diminishing effects on children’s mathematics achievement across time. We hypothesized that individual differences in children’s later mathematical knowledge are more an indicator of stable, underlying characteristics related to mathematics learning throughout development than of direct effects of early mathematical competency on later mathematical competency. We tested this hypothesis in two longitudinal data sets, by simultaneously modeling effects of latent traits (stable characteristics that influence learning across time) and states (e.g., prior knowledge) on children’s mathematics achievement over time. Latent trait effects on children’s mathematical development were substantially larger than state effects. Approximately 60% of the variance in trait mathematics achievement was accounted for by commonly used control variables, such as working memory, but residual trait effects remained larger than state effects. Implications for research and practice are discussed.
Early Growth in Math Learning and Later Achievement- Educational Researcher, 2014
Although previous research has established the association between early-grade mathematics knowledge and later mathematics achievement, few studies have measured mathematical skills prior to school entry, and few have investigated the predictive power of early gains in mathematics ability. The current paper relates mathematical skills measured at 54 months to adolescent mathematics achievement using multi-site longitudinal data. We find that preschool mathematics ability predicts mathematics achievement through age 15, even after accounting for early reading, cognitive skills, and family and child characteristics. Moreover, we find that growth in mathematical ability between age 54 months and first grade is an even stronger predictor of adolescent mathematics achievement. These results demonstrate the importance of prekindergarten mathematics knowledge and early math learning for later achievement.
Tyler Watts 