RIOT IQ Test Queries

IQ Scores
1

Hi,

I completed the full RIOT IQ test back in September and I have a few queries relating to it:

  1. Do you plan to provide a General Ability Index (GAI)? My T-scores were - verbal 68 / fluid 74 / spatial 70 / memory 65 / speed 59 / reaction time 57, which gave FSIQ of 134. Based on typical g-loading of these indices I would estimate GAI of 136-137?
  2. My vocabulary subtest score was strangely low, at 59 scaled score, which is much lower than I would have expected as in verbal reasoning tests I would typically always score in top 1-2%. My other verbal scores were Information = 70 / Analogies =68. I did retake the vocabulary subtest approximately 4 weeks later (using custom test option) and I got 67 scaled score, which is more valid? Would you expect almost 1 SD jump due to practice effect or familiarity?
  3. Likewise I got a very low score on the Exposure Memory subtest (42 scaled score) - after retaking this test 4 weeks later (again using custom test option) I got 58 scaled score. Which is more valid? Similarly to question 2, would you expect > 1.5 SD jump due to practice effect or familiarity?
  4. How would you say RIOT IQ compares with other popular online tests such as CORE on cognitivemetrics.com? Having taken both I would say that while RIOT IQ has the benefit of a large professional norming sample (which the makers of CORE say they plan to do but haven’t yet) I would have concerns about its ability to differentiate at high ranges as surely only 15-20 or so individuals of your norming sample achieved scores in the top 1%? Surely statistically that makes it less reliable in these ranges of > 135 say? CORE has been normed on quite a high IQ population so would it potentially have better give more reliable results in these higher ranges of > 135? While potentially also being deflated in the 90-110 range. I would say that after sitting both the RIOT full test and the CORE full battery (which is 3 hours long), the CORE has more subtests (particularly in verbal reasoning, fluid reasoning, quantitative reasoning areas) and has more challenging questions in each subtest, with each subtest being significantly longer also. On RIOT the subtests are very short so I would have some concerns on reliability, as evidenced by my questions (2-3) above. While the makers of CORE have yet to publish a validity study they have estimated g-loading at 0.94. How does RIOT compare in this regard?

Thanks,

Peter.

2

Any thoughts on these questions from any of the RIOT team?

3

Hi, Peter!

  1. There are currently no plans for a GAI on the RIOT. GAIs and IQs are highly correlated with one another, and there’s little unique information that GAI provides that RIOT doesn’t. On the WAIS-IV, GAI and FSIQ correlate .90 or higher. When we have calculated a few GAIs to compare with the global IQs on the RIOT, we almost always get scores within about 3 points.

2 and 3. Generally, the first testing should be more accurate. We are conducting a study now to see how stable scores are after retesting. If you retook the test shortly after the initial examination, then a large jump of ~1 SD is not out of the question.

  1. There are other IQ tests online, but only the RIOT is designed to meet all of the technical and ethical standards established by the field. There are some tests designed by some of my colleagues that are useful for research purposes, but only the RIOT is suitable for low- and medium-stakes decisions (e.g., as a screening tool during the hiring process). There is no online test that is suitable for high-stakes decisions, such as diagnosing. For these purposes, always take a one-on-one test administered by a licensed professional (often administered in person by a psychologist).

The people creating the CORE are well-meaning, but they are amateurs who do not know what they are doing. On the CORE web site, I’ve seen basic errors in understanding of factor analysis and item analysis. Given these errors, their test will almost certiainly produce distorted scores across the entire range of IQs.

Creating a professional-quality psychological test takes advanced training and a lot of experience in research and test development. There is no evidence whatsoever that the CORE’s creators have that experience, knowledge, or training. A psychological test is a sophisticated scientific instrument on par with a medical test… You wouldn’t trust a blood test created by amateurs, and so why should you trust a psychological test created by amateurs? The fact that they’re anonymous is a major red flag. Anonymity prevents accountability.

1 Like
4

Hi Dr Warne, many thanks for the replies, I appreciate it.

In terms of my interest in GAI, I know that for people with FSIQ over 130 there can often be a disparity between their highest and lowest indices. In my case for instance I have 1.5-1.7 SD disparity between fluid reasoning and processing speed/reaction time, which are both above average but not on same level as my fluid reasoning. I think in WAIS guidelines they recommend when you have 1.5 SD disparity or greater that GAI may better reflect intelligence than FSIQ.

Regarding CORE, I agree that they wouldn’t have the expertise that you guys have as they aren’t psychometric professionals and wouldn’t claim it has diagnostic/clinical relevance but it does seem to have relatively good correlation with WAIS-IV/V performance for high IQ ranges at least (based on experience of others in CognitiveTesting subreddit community) and they are quite open about their methods, i.e., preliminary validity report published, and very responsive to any support queries I’ve made to them.