A new study set out to replicate a controversial claim in the SMART relational skills training literature: that training children in derived relational responding fluency can meaningfully raise IQ scores. Researchers assigned two groups of 10–11 year olds (based on school enrollment rather than individual random assignment) to treatment or control conditions, measuring attentional skills at baseline and IQ (via the WASI-II) at both baseline and follow-up. On the surface, the results looked like a win for SMART: the treatment group showed a significant IQ increase from baseline to follow-up, while the control group did not. But once the researchers controlled for baseline IQ, or for any of three attentional sub-indices, the effect vanished entirely.
The authors argue this points to a more cautionary story than a confirmatory one. Because the treatment group started with a notably lower average IQ than the control group, much of the apparent gain looks attributable to regression to the mean rather than to the training itself. Combined with the non-random, school-based group assignment and a modest sample size, the study ends up illustrating how easily uncontrolled pre/post designs can generate the appearance of an intervention effect. The authors frame their own study as something of a cautionary case: even a seemingly clean “significant effect” can dissolve under basic statistical controls, and they call for future SMART research to use proper randomization, larger a priori power analyses, and baseline IQ as a standard covariate.
Link to full article: https://doi.org/10.1016/j.jcbs.2023.03.012