Is Trying Harder Enough? Causal Analysis of the Effort-IQ Relationship Suggests Not

IQ Research
1

A major article by Timothy Bates was just published in the ICA Journal showing that incentives make people more motivated when taking tests. But the higher motivation does NOT cause IQ to increase. And the finding was replicated (n=500 in 1st study; n = 1,237 in the replication).

Article 1
Article 1975×814 83.3 KB

In both studies, self-reported effort was correlated with test performance, but only when the effort was reported after taking the test. Pre-test effort (e.g., “I will give my best effort on this test.”) is NOT correlated with test performance. Therefore, the post-test effort reports are distorted by people’s beliefs about how well they did on the test.

Half of participants in both studies were randomly selected to receive an extra incentive in which they would be paid more if they did better on a second test. In both studies, the incentive was shown to impact pre-test effort. But this did NOT lead to higher test score in either study. This is seen in the value of “0” in the path leading from pre-test effort to cognitive test score in the figure below.

Article 2
Article 2482×460 28.2 KB

Here is the same finding in the replication, which had more statistical to detect any effect that might have been present:

Article 3
Article 3484×403 26.3 KB

The author stated, “. . . these findings support the hypothesis that effort does not causally raise cognitive score. Both studies, then, showed that, while incentives reliably and substantially manipulated effort, increased effort did not manifest in any statistically or theoretically significant causal effect on cognitive scores” (p. 101).

These results don’t mean that we shouldn’t try on tests. Instead, they mean that claims that IQ scores are susceptible to changes in effort is incorrect. In other words, intelligence tests (including the online tests used in this article) are measuring cognitive ability–not test-taking effort.

Another implication of this research is that motivating people to try harder won’t change their underlying ability. Telling students to “try harder” on school tests is not a very effective strategy to raise scores (assuming that they were already putting some effort into their performance in the first place).

Link to original post: https://x.com/RiotIQ/status/1952369432149545429?s=20

Link to full article: https:/​/​doi.org/​10.65550/​001c.142071

3

This is huge for understanding what IQ tests actually measure. The fact that incentives increased effort but didn’t improve scores proves that IQ tests are measuring ability, not motivation. The pre-test vs post-test effort finding is really clever, it shows people retrospectively judge their effort based on how well they think they did, not the other way around. This also kills the common criticism that group IQ differences are just motivation differences. If financial incentives can’t raise scores, then differences in “test-taking motivation” probably aren’t explaining much variance either.

4

The replication with over 1,200 people makes this really solid. People say “IQ tests just measure who tries hard,” but this directly disproves that. You can make people try harder with money, but it doesn’t change their scores. The correlation between effort and performance exists because people who do well retrospectively report trying harder, not because trying harder makes you do better. This has practical implications too. Telling struggling students to “just try harder” won’t work if they’re already putting in effort. You need to address actual skill gaps or use different teaching methods, not just demand more motivation.