One of the questions we get asked a lot is “what’s the difference between psych testing and predictive analytics”. So today we’re going to unpack this a little bit and look at how the two differ, and where they are similar
Psych Testing
Psych testing has been around for decades. It’s an old-school form of predictive analytics. You look at a big group of people who are in the same role and figure out what’s common with their profiles, define a set of questions to test for the common attributes for that role and then apply that as a broad-based test for anyone who is applying for that same role.
What makes psych testing compelling?
It’s been around for a while, so people are familiar with the practice.
Read more: The Changing Role of the Organisational Psychologist
What makes psych tests limiting?
It’s generally expensive, cumbersome to interpret, and based on a very big assumption that if you fit the profile you will be successful in the role.
Psychological assessments have long been used to identify ‘hidden talent’ or ‘potential’ in people with limited work experience. Whilst these traditional assessments have reduced the hiring and promotional error rates, they take time to analyse, are costly, and are built off competencies inherently imbued with bias. It gives a suggestion that you are a fit, but we know that this rarely correlates to actually being successful in the role.
Psych tests are testing your ability to do a test. That’s it. Traditional psychological assessments do not link to actual performance in the role, nor do they have any self-learning functionality. There is no performance data that feeds into psychological assessments and therefore they have limited predictive power.
Predictive Analytics
In the context of Sapia, we use actual performance data to predict a candidate’s likelihood of success in the role they have applied for. The applicant completes an online questionnaire, but in-between the questions asked and the applicant’s responses is a data model. This statistical model draws on many different objective data points to predicts a candidate’s success in the role.
This also enables an efficient and immediate feedback loop about the actual performance of the hired candidate, improving the accuracy of the predictive model over time. Very quickly the predictive model that you use to select high performers becomes completely customised for your business. You build your own bespoke Intellectual Property, which becomes even more valuable with use.
Where does the prediction come from?
We all try to find patterns to help us make decisions, whether it’s ‘this restaurant looks busy so it must be good’, to ‘this person went to the same university as me, so they must know what they’re talking about’. We are often blinded by our innate cognitive biases, such as our tendency to overweight the relevance of our own experience. We end up in a tourist trap eating overcooked steak because that’s what everyone else was doing.
Our predictions are based on analysing objective data – someone’s responses to a set of questions, compared to the objective performance metrics for that same person in the role. This is a much more reliable and fairer way to make the decision. The democracy of numbers can help organisations eliminate unconscious preferences and biases, which can surface even when those responsible have the best of intentions.
Alchemy of high performers
We work closely with the recruiter or hiring manager to drill down into the qualities of a high performer, and then structure a bespoke application process to search for this. This could be a high level of empathy for customer service or the drive and resilience needed in sales.
Like all AI, our system improves with data. It learns what kind of hires drive results for your business, and then automatically begins to look for this with future applications. Ultimately, the more applicants that apply, the better it gets in identifying which people best match your requirements. And the longer you leverage our system, the more effective it gets.
Hopefully, that’s given you a good overview of where we differ, and what some of the advantages of implementing this into your recruitment process. Still, looking for more?
You can try out Sapia’s Chat Interview right now, or leave us your details to book a demo
Related
Blog
How can we make hiring more inclusive for people with disabilities?
The Americans with Disabilities Act (ADA) passed in 1990. This year, Australia’s Disability Discrimination Act turned 30. Even after all that time, bias and discrimination against candidates and employees with disabilities continues to be an important topic.
The unemployment rate for those with a disability (10.1%) in 2021 was about twice as high as the rate for those without a disability (5.1%) (U.S. Bureau of Labor Statistics, 2022). Coupled with increased laws and regulations regarding the protection of disabled job applicants and employees (e.g., U.S. EEOC, 2022), it is no surprise that academics, employers, and selection vendors are keen to understand where potential disability bias exists so it can be reduced or, ideally, eliminated.
Traditional face-to-face interviews are a large entry barrier for people with disabilities
Traditional face-to-face or video interviews in particular create potential barriers for individuals with disabilities, due to the well-documented stigma and prejudice against those with disabilities (Scior, 2011; Thompson et al., 2011). One study found that fake accountant job applicants that had disclosed a disability were 26% less likely to receive employment interest from the employer than those with no disability. Worse, experienced candidates with disabilities were 34% less likely to receive interest, despite presenting equally high levels of qualifications (Ameri et al., 2015). In addition to the bias held by hiring managers or recruiters, another concern is that certain selection methods create a very poor candidate experience for individuals with disabilities, causing them stress or anxiety and therefore stopping them from putting their best foot forward. For individuals with Autism Spectrum Disorder (ASD) in particular, in-person or video interviews can be very stressful, with less than 10% believing they are given the opportunity to demonstrate their skills and abilities in this process (Cooper & Kennady, 2021).
Stuttering is another form of disability where traditional in-person and video interviews where the candidate has to speak may lead to stress and anxiety (Manning and Beck, 2013). One study found that people who stutter find their stuttering to be a “major handicap” in their working lives and over 70% thought that they had a decreased opportunity to be hired and promoted (Klein & Hood, 2004). Other disabilities, such as dyslexia and other learning and language disabilities may cause candidates to struggle with timed online selection assessments, so it is important to identify and remove these barriers (Hyland & Rutigliano, 2013).
How do we better accommodate people with disabilities or neurodiversity in the way we interview and hire?
Cooper and Mujtaba (2022) recommend alternative approaches that allow candidates with ASD to showcase their skills without having to verbally communicate them or properly interpret nonverbal cues.
The use of an online, untimed, chat-based interview – that is, our Ai Smart Interviewer – can not only help reduce discrimination against those with disabilities but also create a more positive candidate experience for them.
This format is particularly helpful for individuals with disabilities where traditional in-person interviews, video interviews, or timed assessments may cause stress or discomfort, therefore not allowing the candidate to express themselves freely and adequately demonstrate their skills.
The power of a Smart Interviewer, supported by research
Our Sapia Labs data science team has submitted a paper on reducing bias for people with disabilities to SIOP for 2023.
In the study, the data comes directly from our Smart Interviewer, which, as we said above, is an online untimed chat-based interview platform.
Candidates can give feedback after the interview process, and some candidates include self-report disability conditions in their feedback. While a number of different disabilities were mentioned, we had sufficient sample sizes to examine candidates with autism, dyslexia, and stutter. We compared their machine learning-generated final interview scores and yes/maybe/no hiring recommendations to a randomly sampled, demographically similar group of candidates that did not disclose a disability.
Effect sizes, 4/5ths ratios, and Z-tests revealed no adverse impact against candidates with autism, stutter or dyslexia. Additionally, feedback from these groups tended to indicate the experience was positive and allowed them the opportunity to do their best.
- “It was an unusual experience but as an autistic person, I appreciated being able to interview via text rather than phone. It gave me the chance to really consider my responses in my own time.”
- “I must admit this is much more relieving than a face-to-face interview as I fear that I would stutter and accidentally say something stupid.”
- “Being dyslexic, this interview gives me a fantastic opportunity to think and re-read my responses before delivery.”
True diversity and inclusion starts with the way you hire. Our Ai Smart Interviewer allows people with disabilities and neurodiversity – real people, with real ambitions – to represent themselves fairly.
Blog
How our fully-automated Video Interview solution delivers unprecedented hiring velocity
Our smart interviewer aims to make life easier for hiring and talent acquisition professionals, while making job-getting a fairer and more pleasant experience for candidates. This requires a product suite that takes both stakeholders from one end of the journey to the other – application to offer – as seamlessly and personably as possible.
To that end, we’ve introduced a new product to supplement our candidate-favourite Chat Interview tool. Called Video Interview, this new tool revolutionises the current market approach to video-based interviewing and assessment. Now, Sapia’s user journey looks like this:
- Candidates start with a blind, unbiased chat interview with our market-leading Ai, Smart Interviewer
- Then, if suitable, they progress automatically to a short and sharp video-based interview
Read also: Our Talent Acquisition Transformation Guide, a free playbook to help TA teams win more talent
Thanks to the combination of Chat Interview and Video Interview, hiring managers can conduct hundreds of interviews without having to schedule calls, resulting in amazing hiring velocity. As a result, the average time-to-decision for Sapia’s customers is now under 24 hours, giving our customers the flexibility they need to hire fast in a highly competitive candidate market. Better still, all candidates also receive personalized feedback once they complete the process, whether they are hired or not.
What’s the difference between Sapia’s Video Interview, and other video interviewing solutions?
In contrast to our ethical smart interviewer, Video Interview does not use an Ai or algorithm to screen candidate responses. We recognise that, unlike with text, bias is almost impossible to eliminate once video and voice are introduced; to suggest that an Ai can remove bias from the assessment of a video interview is fundamentally dishonest.
The early success with our chat and video-based interviewing approach speaks for itself
Video Interview is currently delivering a Candidate Happiness Score of 9/10, with a completion rate of nearly 76%. 86% of candidates say that the end-to-end experience has made them more likely to recommend the hiring company as an employer of choice.
The new product innovation was created as a direct response to a need by Australia’s biggest private employer, Woolworths Group, who urgently needed to hire better and faster.
Our CEO, Barb Hyman, said that this need has been reflected by many talent acquisition specialists and hiring teams across the globe.
“In a world where there is a massive shortage of talent, even for recruiters themselves, businesses need to find a less taxing way to assess talent,” Hyman said.
“We believe that the introduction of Video Interview makes us the only truly automated hiring solution that addresses fairness, fit and speed, while genuinely engaging candidates,” she said.
Woolworths Group (ASX: WOW) handles more than 1 million candidates, applying for 40,000 roles a year. It estimates that it has saved about 5,000 recruitment hours in the first week of using Sapia alone.
“We love the tool, and we knew candidates would love it because it’s mobile and it’s interactive,” Keri Foti, Head of Advisory & Talent Acquisition Services at Woolworths, said. “We also love the fact that we can measure advocacy for Woolworths at the end of it.”
Blog
How Victoria defeated COVID with individual action and data
Action and data
COVID has taught us that on reflection the focus on individual action with a community benefit as a goal is really a focus that leads to the greater good. In our home state of Victoria, Australia now 7 straight days with ZERO new cases. It has been an effort founded on facts and science over misinformation. In Victoria, many sacrificed a lot for their well-being for ALL. If anything, there is now proof, thanks to Victorians, that when we see facts, listen to science and let data show you how to lead that change, you can make it happen.
We’re using this approach to build a new vision for inclusive hiring.
AI, especially predictive machine learning models, are an outcome of a scientific process, it’s no different to any other scientific theory, where a hypothesis is being tested using data.
The beauty of the scientific method is that every scientific theory needs to be falsifiable, a condition first brought to light by the philosopher of science Karl Popper. In other words, a theory has to have the capacity to be contradicted with evidence.
It is how science is able to progress without bias.
There are three decisions that are made by a human in building that scientific experiment.
- Forming a meaningful hypothesis
- Data collection methodology (experiment set up)
- The data you rely on to test the hypothesis
One can argue 2 and 3 are the same as if the methodology is not sound the data collection wouldn’t be either. That’s why there is so much challenge and curiosity as there should be about the data that goes into an algorithm.
Think of an analogy in a different field of science: the science of climate change.
A scientist comes up with a hypothesis that certain factors drive an increase in objective measures of climate warming, eg CO2 emissions, cars on the road, etc. That’s a hypothesis and then she tests it using statistical analysis to prove or disprove that her hypothesis holds beyond random chance.
The best way to make sure you are following a sound scientific approach is to share your findings with the broader scientific community. In other words, publish in peer-reviewed mediums such as journals or conferences so that you are open to scrutiny and arguments against your findings.
Or to put it another way, be open for your hypothesis to be falsified.
In AI especially, it is also important to keep testing whether your hypothesis holds over time as new data may show patterns that lead to disproving your initial hypothesis. This can be due to limitations in your initial dataset or assumptions made that are no longer valid. For example, assuming the only information in a resume related to gender are name and explicit mention of gender or a certain predictive pattern such as detecting facial expressions are consistent across race or gender groups. Both of these have been proven wrong*.
The only way to improve our ability to predict, be it climate change or employee performance, is to start applying the scientific method and be open to adjusting your models to better explain new evidence.
Therefore the idea that a human can encode their own biases in the AI — well it’s just not true if the right science is followed.
* Amazon scraps secret AI recruiting tool that showed bias against women (https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G)
* Researchers find evidence of bias in facial expression data sets (https://venturebeat.com/2020/07/24/researchers-find-evidence-of-bias-in-facial-expression-data-sets/)
To keep up to date on all things “Hiring with Ai” subscribe to our blog!
You can try out Sapia’s Chat Interview right now, or leave us your details to get a personalised demo