Sapia.ai, the world’s only smart hiring automation platform powered by deep-learning AI, has released a new function which will detect and flag responses sourced by generative AI, such as ChatGPT, in real time.
A world first, the new function draws from Sapia.ai’s growing proprietary dataset of over one billion words — collated from over 12 million responses from 2.5 million candidates that have used its platform.
Brands including Woolworths Group, Holland & Barrett, and WOLT trust Sapia.ai to accelerate and enhance their recruitment and promotion processes. A conversational, Natural Language Processing (NLP) based chat AI interviews, assesses and screens for the best talent at scale via an easy to use messaging platform.
In addition to improving diversity outcomes by eliminating unconscious bias, it also allows companies to re-allocate thousands of hours spent screening talent towards higher value tasks.
This new feature prevents candidates from using generative AI tools to respond to prompts from Sapia.ai’s platform. Candidates will be alerted in real-time as they respond when their answers are likely to be AI generated content (AGC), giving them an opportunity to change it ahead of the final submission. Failing this, it will then flag to the decision-maker the likely inclusion of AGC in the candidate’s response for further review.
Barb Hyman, Sapia.ai CEO and founder, said: “This is something our competitors can’t do. It’s our competitive moat. While it is possible to detect use of generative ChatGPT through analysis, we’re conducting it in real-time. Our data set also gives us the ability to readily adapt to new iterations of generative AI.”
“It builds on the plagiarism flag which we released in 2019. It identified candidates who sourced their answers from the internet or other candidates. Adding a new flag for AI generated content maintains the integrity of the structured interview as a fair, and accurate way to assess talent.“
“What is equally important to our product integrity is that we are transparent with the individual – transparent right from the outset that their interview responses are being assessed for AGC, and giving them the option real time to change their answers. We have always built our product with the human experience front of mind. We aim to put people at ease, having the interview blind and untimed and letting them know how their data is being used so they trust the experience.
“We believe every candidate matters. Today’s applicant is tomorrow’s customer. This is why we strive for exceptional candidate satisfaction metrics, with up to 94% completion rate for our long-standing customer Qantas Group, and an average completion rate of 91% across all our customers.
Sapia.ai’s Chief Data Scientist Dr. Buddhi Jayatilleke says the accuracy of the Sapia.ai’s AGC detector comes from the unique data asset the company can leverage to build and test the feature.
“We tested our AGC flag with thousands of generated answers from GPT-2, GPT-3 and ChatGPT on various prompts related to multiple role families. We were able to achieve a ROC-AUC of over 95%, which is a strong indicator of the accuracy of a classifier. This is due to our ability to distinguish the differences between human-written text and the formulaic nature of content coming from generative AI models, leveraging our large human-written response data set,” he said.
Related
Blog
Recruitment metrics: How and why to track your candidate abandonment rate
In the candidate short market we’re in, it’s absolutely critical to keep talent engaged throughout the entire application process. You simply cannot afford to lose the talent that you’ve spent time and money attracting.
This sounds obvious, of course, but abandonment is a key problem – and few companies know where, when, and why it is happening.
Let’s start with the metric, and then talk about how we apply it to your wider talent acquisition journey.
Overall candidate abandonment rate = number of candidates still in the process at shortlist stage, minus the total number of candidates who landed on your careers page, divided by that total number again. Or:
At the very minimum, this is the metric you need to start tracking, because it is a generalized diagnostic for the health of your recruitment process.
If you know that you had 100 visitors to your careers (or job ad) page, but your shortlist has only 10 candidates in it, you’ve lost 90% of your possible talent pool at one stage or another.
Simple math, yes, but in our experience, many recruiters and talent acquisition managers don’t look at what their starting pool of candidate interest was – and therefore, what their theoretical talent pool might have been – and look only at actual applicants.
This poses another, related question: How do I know what my abandonment rate is at each stage of the application process?
Let’s say, like the example above, that you had 100 visitors to your careers (or job ad) page, and 20 of them completed the first-step application form on that page. You’ve lost 80% of your possible pool right there.
Not great, but at least you know – now you can examine that page to uncover possible issues preventing conversion.
- Is the page too long?
- Does it have too much text?
- Is the ‘apply’ button clearly shown?
- Is the form too long, requiring too much information to fill out?
- Are your perks/EVP attributes clearly displayed?
Without examining stage progression in isolation, you might never know why people aren’t sticking around.
To reiterate: As well as an overall abandonment rate, you need to measure the drop out rates at each of the stages of your talent acquisition journey. The next section can help show you what to focus on.
Where, when and why do candidates drop out of the application process?
Conventional wisdom tells us that the longer your application and interview process goes on, the higher your dropout rate will be.
But that’s a generalized issue – it tells you nothing about how to fix the problem, beyond simply making it shorter. You need specific, localized data to diagnose and fix your leakage spots.
Data from a 2022 Aptitude Research report on key interviewing trends found that candidates tend to drop out at the following stages, in the following proportions:
- 22% of candidates drop out at the application stage
- 24% at the screening stage
- 25% at the interview stage
- 18% at the assessment stage
- 9% at the offer stage
Good to know, right? If you audit your own journey, looking at these stages and using these numbers as benchmarks, you can quickly identify your weak areas.
For example: You might be proud of your four-step culture-building interview process, in which candidates have a coffee meet-and-greet with the team they’re hoping to join.
But if it’s cumbersome for the applicant and relies on several stakeholders to orchestrate, it may be dragging your process out unnecessarily, and doing more harm than good.
The most common job application abandonment stage: The interview
25% of candidates drop out here. Shouldn’t really be a surprise, should it? Job interviews are long, numerous, and in many cases, ineffective. According to Aptitude Research, 33% of companies aren’t confident in how they interview; 50% believe they’ve lost talent due to poor interviewing.
When asked about their top interviewing challenges, surveyed HR and TA leaders responded:
- Our interview process is too long (52%)
- We make candidates go through too many interviews (39%)
- Out interview process is inconsistent (38%)
- We don’t use objective data to drive decisions (32%)
- We have bias in the interview process (21%)
Let’s focus on that second-last challenge: lack of objective data. Almost a third of companies are approaching their interview and application process with assumptions and gut feelings; and half of them believe their interview process is too long.
Despite this, 68% of companies say they have not made any improvements surrounding candidate experience this year. How many, then, are looking seriously at their entire talent acquisition journey to see where it’s failing?
This is why we’re focusing on candidate abandonment rate in this post: It is a simple metric to show the health of your application process, easier to measure than many of the other recruitment metrics for which you’re responsible (the ever-nebulous quality-of-hire being a prime example). As the saying goes, what gets measured, gets managed.
Start here today, and see what you learn.
(P.S. Sapia’s Ai Smart Chat Interviewer combines the first four stages of your process – application, screening, interviewing, and assessment – together, resulting in an application process that can secure top talent in as little as 24 hours.
Because it’s a chat-based interview with a smart little AI, your team doesn’t need to do anything – everyone who applies gets an interview, immediately. That maximizes your talent pool right from the get-go.
What’s more, our candidate dropout rate is just 15%, on average. That means that 85% of your starting talent pool will stick around.
Why do our candidates stick around? More than 90% of them love the experience. See how we can help you here, today.)
Blog
Sapia is an AIconics Awards Finalist: “AI for Good”
Sapia is recognised as an Alconics Awards finalist in the ‘AI for Good’ category.
It’s AI that gives every candidate a fair chance of landing their dream job and gives candidates something of value back.
The AIconics Awards recognize the outstanding achievements of individuals, projects, teams and their organizations that are responsible for breakthrough innovations in the Artificial Intelligence for Business space.
These prestigious global awards create the ultimate showcase for the best and brightest people, projects and transformational innovations. The AIconics acknowledges the advances in technologies and disciplines being made, as we explore and push at the very definition of what can be accomplished by AI.
The AIconic Awards are being announced on 9th December 2020.
AWARD: AI FOR GOOD
Artificial Intelligence has the potential to help overcome humanity’s biggest challenges, there are a huge number of applications where AI will not only deliver value for businesses but also improve the world itself. This award applauds companies for utilising AI as a positive force for change; the innovations in research and product development that work to create a more sustainable and accessible world; and the AI pioneers that hold the values of leveraging AI for good at their core.
Why is Sapia a finalist in the “AI for Good” category?
Sapia’s Chat Interview gives every candidate a fair chance of landing their dream job.
Have you ever met a recruiter that is totally free from bias and discrimination, and truly embraces diversity and inclusion, and is fair and equitable for everyone? You have now!
The interviews are a true blind assessment. This is the first step in the hiring process for organisations championing a positive culture change to realise their goals of embracing inclusion and celebrating diversity.
Ai is used to hire with heart. It’s making recruitment ‘human’ again.
Applying for a job is notoriously a heart-wrenching and time-consuming experience. Right now, during C-19 times, the hardest job in the world is applying for a job.
Organisations partner with Sapia to help them hire with heart. From the job advert candidates access a link to access their interview.
This means EVERY candidate gets an interview. The basic right of fairness and equality is available to every candidate, every time.
It takes around 15-20 minutes to answer 5-7 questions. After that, candidates receive their personalised coaching tips. This means EVERY candidate also gets something of value back, something that motivates them and teaches something about themselves they didn’t know. A candidate experience that helps them get this job or the next job or just makes them feel good.
For these reasons, candidate satisfaction is 99%.
Hiring with heart is good for business.
65% of candidates with a positive experience would be a customer again, even if they were not hired and 81% will share their positive experience with family, friends and peers.
As consumers, we buy products while sitting on our computer or scrolling through our phone. Texting, messaging – it’s what we all do every day. To be candidate-centric means connecting with candidates the way they connect every day. The long-term payback to customers and employer brand is substantial and enduring.
In the short-term AI can assess 100,000 people in 6 hours compared to what it would take a team of recruiters 476 days to do. It’s 600 times faster and 3 x cheaper.
No recruiter or team of recruiters can ever come close to the kind of efficiency of a smart AI system.
Using AI to augment decision making is fast becoming mainstream.
It certainly helps to reduce the impact of unconscious bias in hiring decisions. Testing for bias and removing it from algorithms is possible. Whereas for humans, it’s not.
Bias can be removed with the right data. Algorithms and Ai learn according to the profile of the data we feed it. If the data it learns from is taken from a CV, it’s only going to amplify our existing biases. Only clean data, like the answers to specific job-related questions, can give us a true bias-free outcome.
We continuously test the data so that if ever the slightest bias is found, it can be corrected. These include all assumed biases that can be added to a suite of tests. Examples of tests include: Proportional Parity Test, Score Distribution Test and Fairness Test
Candidates say:
- I don’t know if it is a human or a robot answering me, but if it is a robot then technology is quite amazing.
- I love the personal feeling … It creates a safe environment to write freely and honestly.
- I don’t know how you gathered such accurate information about my personality from my responses.
- I feel that I’ve had a supportive counselling session. You know me so well!!
- This is amazingly accurate that I can’t even begin to express, I’m in tears!!
- It is fascinating how much our choice of words, our expressions, reveal about us.
CANDIDATE REVIEWS HERE >
Have you seen the Inclusive e-Book?
It offers a pathway to fairer hiring in 2021.
Get diversity and inclusion right whilst hiring on time and on budget.
In this Inclusivity e-Book, you’ll learn:
- How to design an inclusive recruitment path. From discovery to offer and validation of the process.
- The hidden inclusion challenges that are holding your organisation back.
- How to tell if Ai technology is ethical.
Download Inclusivity Hiring e-Book Here >
Blog
Hiring automation: It’s Time To Use Your Words
This Recruiter Says How You Write and What You Write Can Tell an Employer a Great Deal About Your Fit.
Barbara Hyman believes the most important skill for people looking for a job in the post-COVID world will be the ability to write.
“People who think clearly, write clearly,’’ says the chief executive of the artificial intelligence-powered recruiting firm Sapia, which judges its candidates on the most basic of skills.
The firm, which has big-name backers including Myer family member Rupert Myer, former Aconex founder turned venture capitalist Leigh Jasper, fund manager Dion Hershan and former JB Were partner Sam Brougham, gives every job candidate a first interview by asking them five text-based behavioural questions on their phone that take around 20 minutes to answer.
Then the company’s predictive models assign a “suitability” score to each candidate using over 80 features extracted from their responses and the system specifically precludes the use of names, gender and age to determine the recommended shortlist, removing unconscious bias from the recruitment process.
The company’s technology has already been used by 400,000 candidates looking for roles in retail, healthcare, customer service, hospitality, contact centres and graduate and trainee roles across 34 countries.
Its clients include Qantas, Bunnings, the Iceland Group grocery chain in the UK and Spanish telecoms giant Telefonica.
But Hyman says her biggest target client in the post-COVID world is government.
She believes the economy can only be sustainably reactivated through large-scale job security and that requires redeploying existing skillsets to meet in-demand industries.
“This requires a sophisticated and scaleable solution to find jobs for those whose industries have been decimated by the pandemic and have no jobs to return to. Our solution can immediately activate these job seekers into the new economy, steering them to the jobs they will be good at, she says.
She claims if the government activated this sort of technology for a range of growth industries the economic and social impact would be unprecedented.
“In a healthy economy, the cost benefit in Australia alone is $1bn net benefit (cost) for every 100,000 workers that get back to work one month earlier through reduced welfare payments and increased consumer spending. That is significantly higher when accounting for government subsidies as a result of COVID,” she says.
“A big part of getting back to work is the confidence and the mindset. We are exploring different avenues to allow people to use our chat bot to find their true role in the new economy. This is the vision we are trying to sell to government – you have your own personalised career coach that helps you find the ideal role.”
Hyman said one of the company’s big-name backers Rupert Myer, the chair of the Australia Council for the Arts and an emeritus trustee of The National Gallery of Victoria, had given her “amazing introductions” into the government and university sectors.
“When I came into the business in February 2018 it was running out of money. I had to get a bunch of the existing investors to support me,’’ says Hyman, a former chief human resources officer at REA Group and a human resources and marketing director at Boston Consulting.
Her data science leader at Sapia is Sri Lankan-born Buddhi Jayatilleke, who has a diverse background in machine learning, software engineering and academic research.
The firm has raised $4m in the past 2 years, including bringing in Australian global recruitment and talent management firm Hudson as a strategic investor last year.
“That gave us credibility because the number two recruitment firm in the market believes in what we are doing,’’ Hyman says.
“Whether you like it or not, there is enormous amount we can learn about you in 200 words. Just the very fact we don’t use any secret or behavioural data, you have to build trust from the beginning with your candidate. The completion rates are 95 per cent, the engagement rates are 99 per cent. But the key point is when we give you back your feedback. It is effectively a public service we are performing with this feedback.”
One of the firm’s initial backers was Rampersand, the venture capital firm which has a focus on early growth stage tech businesses.
Rampersand co-founder Paul Naphtali says the firm invested in Sapia for its ability to put data at the centre of a company’s people strategy.
“It’s a massive challenge for a start-up to aggregate the data and build the algorithms that can identify an individual’s suitability to a role quickly and accurately. It was a bold and ambitious plan from the beginning, and Sapia is now well on its way to becoming that data-centric engine,’’ he says.
“The company started with working to turbocharge the recruitment process by quickly identifying the right talent for the right roles.
“It’s taken time to build the tech and the data sets, but it’s paying off as a number of Australia’s leading companies now have Sapia as a default part of the process.”
He says the firm is now entering a new phase “where it also powers internal people management as well as for job seekers, which is obviously very relevant in the current environment”.
Recently in London Sapia was awarded the TIARA Talent Tech Star which honours the businesses globally in the talent acquisition industry.
Source: DAMON KITNEY, The Australian, October 30, 2020
To keep up to date on all things “Hiring with Ai” subscribe to our blog!
You can try out Sapia’s Chat Interview right now, or leave us your details to get a personalised demo