In recent years there has been a significant increase in demand from outside of clinics and research laboratories for structured psychological measurement. Measures are now being used by human resources departments, coaches, teachers, product researchers and by individuals themselves. This increasing use raises a number of questions. What is driving this demand? How do different measures fare when assessed critically? How should results obtained from such measures be dealt with by the practitioner?
What is Actually Driving the Surge in Demand
Recently, the cost of distribution has decreased drastically for online assessments. This means that once supplied online, psychometric tests can be completed instantly by users of browsers on mobile or fixed devices. In addition, organizations now use behavior information as legitimate inputs into decisions around team design, training programs, and customer segmentation.
Thirdly, test-takers arrive at the assessment of their psychological strength with preconceptions about what such a test is likely to say about them, usually somewhat better than they really are. This strengthens the self-report, as they expect to read a description of themselves rather than receive a verdict on them.
Why Measurement Beats Intuition in Applied Settings
Judging someone in an unstructured way is how most organizations are used to making decisions regarding the behavior of individuals and groups. This unstructured form of judgment has consistently been proven to be incorrect in many contexts. A structured psychological measure of individual and group behavior, on the other hand, does not guarantee a correct assessment but does allow for the possibility of error to become apparent. This in itself is of huge value.
Which Psychometric Properties Separate Credible Tools from Noise
Quality differs significantly between measurement tools. Never rely solely on marketing promises. Check for evidence specific to the chosen test.
- Internal consistency, usually reported as alpha or omega, showing that the items on a scale cohere.
- Test retest stability over an interval long enough to distinguish trait from mood.
- Convergent and discriminant validity, showing that the scale correlates with related constructs and stays separate from unrelated ones.
- Criterion validity against an outcome that matters, not just against another questionnaire.
- Norms drawn from a population resembling the people you intend to assess.
- Measurement invariance across the language groups, age bands, and cultures you plan to compare.
Heck, a very basic instrument could still give good results for individuals but awful results for group comparison. We tend to overlook this important distinction.
Format choices and the trade-offs they carry
| Assessment type | Best used for | Main limitation |
| Trait self-report inventory | Broad personality description, stable dispositions | Vulnerable to impression management and self-insight limits |
| Forced-choice or ipsative format | Reducing socially desirable responding | Scores are relative within the person, complicating comparison |
| Cognitive ability and reasoning tasks | Predicting performance on novel, complex work | Requires controlled conditions, sensitive to practice effects |
| Situational judgment items | Context-bound decision tendencies | Scoring keys reflect a specific organizational culture |
| Behavioral or reaction-time tasks | Implicit associations, processing style | Lower reliability, results often over-interpreted |
How to read individual differences without overclaiming
The score itself is just a location in a distribution which has error attached. Therefore, two individuals scoring 5-8 points apart on the percentiles are more or less indistinguishable from each other. Thus reporting a confidence interval (CI) along with a score is critical to how scores are perceived by the individuals taking the test and others to whom results are reported. Anyone wanting to build that intuition should spend time with well documented psychology tests that publish their norms and error margins openly.
Common interpretive errors worth naming
- Treating continuous traits as categorical types, which discards information and invents boundaries the data do not support.
- Reading a profile as a fixed explanation of behavior rather than a probabilistic description of tendencies.
- Assuming a trait score predicts behavior in a specific situation, when aggregation across situations is where trait measures perform best.
- Ignoring the assessment context, since a respondent completing a questionnaire for selection answers differently than one completing it for self-knowledge.
Where online administration helps and where it introduces risk
Screening for straight-lining as well as for implausibly fast responses can be implemented at little cost. In addition to improving data quality, digital delivery of tests also allows for assessment of response times (e.g., how quickly a person completes a set of questions), for the use of adaptive item selection, and for the opportunity to assess data for evidence of test-taking strategies (e.g., duplicate items that are answered inconsistently).
Unproctored online testing poses a number of risks including test taker misrepresentation, facilitation, identity theft, use of support in ability tests, incompatibility with test taker’s devices, violation of test taker’s privacy, and failure to obtain informed consent in employment settings.
A Practical Standard for Applied Use
Using measures within other inputs and documenting the reasons for using particular measures are as important as ensuring one has the ability to explain a score to the individual who completed the measure. It is also important to remember that rigor in the application of measures is far more important than the actual sophistication of the measures themselves. In particular, decisions based on psychological assessments of individuals in employment, education or healthcare contexts should be discussed with and interpreted by qualified experts rather than relying solely on automated reports.
