You’ve posted a job, and applications are pouring in. But where does the candidate’s effort stop and AI take over?
Every HR professional is asking the same question. A quick LinkedIn scroll shows recruiters bemoaning the fact that AI is now writing cover letters, editing CVs, completing interview tasks—and claiming that candidates are even trying to fake psychometric tests. It’s changing the hiring game. It’s difficult to dispute. And it’s making recruitment a lot more arduous.
But it got us wondering. Can AI really fake personality psychometrics? And does it help candidates? We put it to the test.
First of all – let’s be clear. At Thomas we are firmly in favour of AI. It can speed up CV screening, streamline interviews, and enhances workplace communication (as seen with our own AI coaching tool, Thom).
But while there are thousands of great use cases for AI, using it to fake or cheat within personality psychometrics is not one of them. In fact, it doesn’t even give potential candidates an advantage.
Research1 has shown that when there is a clear ‘right’ answer (for example, on an untimed IQ test), AI can perform very well. But on the timed assessments often used in recruitment (such as Thomas’ General Intelligence Assessment), AI can’t currently come close to comparing to human performance, so would be of no use in faking a result.
What is less understood is whether AI can predict the ‘correct’ or ‘optimal’ responses within a personality assessment. Does it have what it takes to accurately predict a candidate’s fit for a job role?
Can AI fake the personality you need to succeed?
We tested whether AI, or even a human, could complete our workplace personality assessment (HPTI) and fake the traits that determine success in certain job roles.
The Thomas personality assessment measures six traits: Adjustment, Ambiguity Acceptance, Conscientiousness, Risk Approach, Curiosity and Competitiveness. It is made up of 78 questions and takes around 7-8 minutes to complete.
Thomas’ proprietary job profiling system allows hiring teams to create a job profile and then compares applicants’ assessment results with this ideal profile based on our extensively tested algorithm. Applicants scoring closer to a 5-star rating overall, would be better aligned to the demands of that role, which helps you to hire the person best suited to your needs.
The experiment:
- We asked five different AI platforms to generate the ‘ideal’ responses to each of the personality (HPTI) questions for each of the job roles. Our prompts specifically asked AI to respond in a favourable way, as if taking the assessment as part of a recruitment process.
- We also asked a sample of 18 individuals to answer the personality (HPTI) assessment in a socially desirable way for three of the job roles. We wanted them to deliberately try to fake or cheat in the assessment.
- To ground our findings even further, we drew upon our databases to look at data from job incumbents, allowing us to understand the average star ratings for people that were already within each of these roles.
The result:
Across all 10 job roles, real employees always outperformed AI and human fakers on personality assessments. Job incumbents always had a higher overall personality star rating in comparison to the average AI-generated score and human faking responses.
Interestingly, this demonstrates that both AI and humans struggle to fake personality assessments to appear more suitable. And it seems that job incumbents embody traits that align to their role, as we would expect.
Generally, it appeared that humans and AI did tend to show similar star ratings which were not very closely aligned to the 5-star rating. This shows that neither AI nor humans know exactly ‘what good looks like’ and may in fact be basing their answers on general stereotypes. For example, for a salesperson, AI and humans were generally quite accurate in portraying high ‘competitiveness’ but this did not seem to be the case with some other traits. As AI is trained on human data, it isn’t especially surprising to see human and AI faking is so similar.
Interestingly, there also appeared to be inconsistencies and wide variation within the different AI systems and human faking responses when we looked at the overall star rating. So not only do AI and humans not know what actually drives excellent performance within a role, but they can be inconsistent and find it difficult to hide their lack of understanding about what is expected. AI’s responses varied widely across platforms, proving it struggles with consistency.
Conclusion: Faking psychometrics hurts more than it helps
Happily for HR, AI is just as bad at faking psychology psychometrics as people are. Although we should note that our research-backed job profiling system ensures success is based on real predictors—not stereotypes. This may have been why AI struggled to fake, as job profiles are not based on stereotyped personalities in given job roles, but on what we know makes incumbents successful. It’s also important to note that this research was based on our own personality (HPTI) assessment. AI faking may look different in other personality inventories.
Footnote:
Further reading:
When Machines Make Hiring Decisions: Examining the Risks and Limitations of AI-Based Recruitment Tools – (Taylor O’Brien, 2024)
How Candidates Use AI to Cheat in Assessments—and What Employers Can Do About It | Onrec
