Sapia.ai Partners with iCIMS to Revolutionize the Hiring Process with AI Chat Interview Platform

The Royal Commission has brought about a lot of scrutiny on the banks, and for good reason. But we have to give them credit where it’s due.

Compared to HR teams across the country, banks know a thing or two when it comes to managing risk. Which is funny, as I’d argue that hiring a staff member is a much riskier proposition for a business than a bank having one of its customers default on a loan.
Imagine if your bank lent you money with the same process that your average recruiter used to hire for a role.
They would ask you to load your all of your personal financial information into an exhaustive application form. Your salary, your weekly spend, your financial commitments. All of it.

The same form would include a lot of probing questions, such as: Will you pay this money back on time? When have you borrowed in the past and paid back on time? Describe a time that you struggled to repay a loan and what you did about it?

Then, assuming your form piqued their interest, they would bring you in for one on one meeting with the bank manager. That manager would grill you with a stern look, asking the same questions. This time though, they will be closely watching your eye movement to see if you were lying when you answered.

In each part of the process you get a score, and then if that number is above a certain threshold, you get the loan.
It’s almost laughable, right?

Banks wouldn’t have any customers if they used that approach. Only people who desperately need money would put themselves through that process. And they’re likely not the best loan candidates.

Banks work hard to attain incredibly high accuracy levels in assessing loan risk.

Meanwhile in HR, if you use turnover as a measure of hiring accuracy its as low as 30–50% per cent in some sectors. If you combine both turnover and performance data (how many people who get hired really raise a company’s performance), it might be even lower than that.

Banks wouldn’t exist if their risk accuracy was anywhere close to those numbers.

Well, that’s how most recruitment currently works — just usually involving more people.

There’s more parallels here than you think.

Just like a bank manager, every recruiter wants to get it right and make the best decisions for the sake of their employer. As everyone in HR knows, hiring is one of the greatest risks a business can take on.

But they are making the highest risk decision for an organisation based on a set of hypotheses, assumptions and lots of imperfect data.

So, let’s flip the thought experiment.

What if a bank’s risk management department was running recruitment? What would the risk assessment look like?

Well, the process wouldn’t involve scanning CVs, a 10 minute phone call, a face to face interview and then a decision.
That would be way too expensive given exponentially more people apply for jobs than apply for loans each year. Not to mention the process itself is too subjective.

I suspect they would want objective proof points on what traits make a candidate successful in a role, data that matches the candidate against those proof points and finally, further cross validation with other external sources.

They wouldn’t really care if you were white, Asian, gay female. How could you possibly generalise about someone’s gender, sexuality or ethnicity and use it as a lead indicator of hiring risk. (Yet, in HR this is still how we do it.)

Finally, they’d apply a layer of technology to the process. They would make it a positive customer experience for the candidates and with mobile-first design. Much like a loan, you’ll lose your best customers if the funnel is long and exhaustive.

I’m not saying that banks are a beacon of business. The Royal Commission definitely showed otherwise. But for the most part, they have gotten with the times and upgraded their processes to better manage their risk. It’s time HR do the same.

In June 2022, we announced that, thanks to our partnership with AWS, we now have introduced regional data hosting. This means that customers and their candidates will have increased speed when they use the Sapia platform, and means companies using the platform can have confidence that candidate data is treated in line with data sovereignty legislation, such as the EU’s General Data Protection Regulation (GDPR).

Here is the full list of improvements to data security and sovereignty for Sapia customers.

World-leading protections
Sapia’s platform is built on AWS, and is protected by anti-virus, anti-malware, intrusion detection, intrusion protection, and anti-DDoS protocols. We comply with most major cybersecurity requirements, including ISO 27001, Soc 2 Type 1 (Type 2 in progress), and GDPR.

Scalablility
We use AWS’ serverless solution, which can automatically support billions of requests per day. Our sophisticated tech stack includes React.js, GraphQL, MongoDB, Node.js and Terraform.

Regional data hosting
Sapia offers regional data hosting via AWS. All data is processed within highly secure and fault-tolerant data centres, located in the same geography as the one in which the data is stored. All data is stored in and served from AWS data centres using industry standard encryption; both at rest and in while transit.

Availability and reliability
Sapia uses a purpose-built, distributed, fault-tolerant, self-healing storage system that replicates data six ways across three AWS Availability Zones (AZs), making it highly durable. Our storage system is automatic, features continuous data backup, and allows for point-in-time restore (PITR).

Voluntary employee turnover can have a direct financial impact on organisations. And, at the time of this pandemic outbreak where the majority of the organisations are looking to cut down their employee costs, voluntary employee turnover can create a big concern for companies. And thus, the ability to predict this turnover rate of employees can not only help in making informed hiring decisions but can also help in saving a substantial financial crisis in this uncertain time.

What Drives Job-Hopping?

Acknowledging that, researchers and data scientists from Sapia, a AI recruiting startup, built a language model that can analyse the open-ended interview questions of the candidate to infer the likelihood of a candidate’s job-hopping. The study — led by Madhura Jayaratne, Buddhi Jayatilleke — was done on the responses of 45,000 job applicants, who used a chatbot to give an interview and also self-rated themselves on their possibility of hopping jobs.

The researchers evaluated five different methods of text representations — short for term frequency-inverse document frequency (TF-IDF), LDS, GloVe Vectors for word representations, Doc2Vec document embeddings, and Linguistic Inquiry and Word Count (LIWC). However, the GloVe embeddings provided the best results highlighting the positive correlation between sequences of words and the likelihood of employees leaving the job.

Researchers have also further noted that there is also a positive correlation of employee job-hopping with their “openness to experience.” With companies able to predict the same for freshers and the ones changing their career can provide significant financial benefits for the company.

Regression Model To Infer Job Hopping

Apart from a financial impact of on-boarding new employees, or outsourcing the work, increased employee turnover rate can also decrease productivity as well as can dampen employee morale. In fact, the trend of leaving jobs in order to search for a better one has gained massive traction amid this competitive landscape. And thus, it has become critical for companies to assess the likelihood of the candidate to hop jobs prior to selections.

Traditionally this assessment was done by surfing through candidates’ resume; however, the manual intervention makes the process tiring as well as inaccurate. Plus, this method only was eligible for professionals with work experience but was not fruitful for freshers and amateurs. And thus, researchers decided to leverage the interview answers to analyse the candidates’ personality traits as well as their chances of voluntary turnover.

To test the correlation of the interview answers and likelihood of hopping jobs, the researchers built a regression model that uses the textual answers given by the candidate to infer the result. The chosen candidates used the chatbot — Chat Interview by Sapia for responding to 5-7 open-ended interview questions on past experience, situational judgement and values, rated themselves on a 5-point scale on their motives of changing jobs. Further, the length of the textual response along with the distribution of job-hopping likelihood score among all participants formed the ground truth for building the predictive model.

To initiate the process, the researchers leveraged the LDA-based topic modelling to understand the correlation between the words and phrases used by the candidate and the chances of them leaving the company. Post that, the researchers evaluated four open-vocabulary approaches that analyse all words for understanding the textual information.

Open vocabulary approaches are always going to be preferred over closed ones like LIWC, as it doesn’t rely on category judgement of words. These approaches are further used to build the regression model with the Random Forest algorithm using the scores of the participants. Researchers used 80% of the data to train the model, and the rest of the 20% was used to validate the accuracy of the model.

Additionally, researchers also experiment with various text response lengths, especially with the shorter ones, which becomes challenging as there is not much textual context to predict. However, they found a balance between the short text responses along with the data available and trained the model predicts for even those.

To test the accuracy, the models are evaluated based on the actual likelihood of the turnover with relation to the score produced by the model. To which, the GloVe word embedding approach with the minimum text of 150 words achieved the highest correlation. This result demonstrated that the language used in responding to typical open-ended interview questions could predict the chances of candidates’ turnover rate.

Wrapping Up

Leveraging data from over 45,000 individuals researchers built a regression model in order to infer the likelihood of the candidates leaving the job. It will not only remove the dependency of companies on candidate resumes and job histories but also enhances the process of hiring into a multi-measure assessment process that can be conducted digitally for recruiting.

By Sejuti Das, Analytics India Magazine, 02/08/2020

Join the movement

To keep up to date on all things “Hiring with Ai” subscribe to our blog!

You can try out Sapia’s Chat Interview right now, or leave us your details here to get a personalised demo.