The language candidates use in conversation can reliably indicate their propensity to ‘job hop’, new research shows.

Sapia, which uses text-based communication to interview candidates, has uncovered a correlation between candidate language and job churn that is “stronger than what you would find normally in traditional psychometric testing of job-hopping”, says CEO Barbara Hyman.

HEXACO Personality Model & Job Hopping

Similar to its recent study measuring candidate personality traits, researchers used data from 46,000 job applicants who completed an online chat interview and used the six-factor HEXACO personality model to analyse responses.

The HEXACO traits are honesty-humility, emotionality, extraversion, agreeableness (versus anger), conscientiousness, and openness to experience.

The ‘openness to experience’ trait has long been considered in organisational psychology circles as an indicator of job-hopping, and this has been reinforced by Sapia’s research, says Hyman

“Low agreeableness also correlates with people who may move and look for better opportunities,” she adds.

Analysing candidates’ responses to determine their job-hopping likelihood is especially useful for many entry-level roles, where people do not have prior experience on their CV.

“We know ‘flight risk’ or staff churn is a really big problem for our customers, particularly those who hire at volume into low-skilled roles. For them to be able to identify this upfront and avoid or minimise it was really valuable,” Hyman says.

And, from the candidate’s point of view, “we’re seeing a real craving and an appetite for understanding yourself and understanding where your strengths are best placed”, she adds.

The researchers also note further work is required to assess the true predictive validity of the outcome – that is, establishing the correlation between inferred job-hopping likelihood and actual job-hopping behaviour.

Addressing bias

Sapia has also incorporated the job-hopping measurement into its algorithms to provide this additional information to recruiters, says Hyman.

Importantly, however, “we don’t automatically discount someone who has a high job-hopping likelihood; it’s just another data point you get to look at”.

For some employers and roles, the ‘openness to experience’ trait is generally desirable, Hyman says.

“In investment banking, you want people who are comfortable with looking outside of the box and being really curious and questioning,” she says by way of example.

She stresses the intention is to allow recruitment decision-makers to use the technology as a “co-pilot, not an autopilot”.

Read more here: When used properly, data amplifies inclusive hiring.

Barbara Hyman, Shortlist, Thursday 27 August 2020 2:20 pm

Join the movement

To also keep up to date on all things “Hiring with Ai” subscribe to our blog!

Finally, you can try out Sapia’s Chat Interview right now, or leave us your details here to get a personalised demo.

Addressing valid concerns in the HR industry about AI, Sapia has released an ethical framework to encourage transparency in the sector

MELBOURNE, Jan 18, 2021 – Sapia (https://sapia.ai/), an Australian technology company that has pioneered transparent AI-assisted hiring solutions, today announced the global release of its Fair Ai for Recruitment (FAIR™) framework to educate HR executives in assessing Ai technology for use in their organisations, as well as act as spark conversations  for Ai developers in the space:  https://sapia.ai/fair-ai-recruitment-framework/

The framework has been released to begin conversations around transparency in HR technology against an explosion of Ai solutions in the sector, with many using algorithms that operate in a ‘black box’. The absence of any form of accreditation of vendors, and the fact that regulation is light years behind tech innovation, has meant a lack of collaboration among vendors to champion Ai ethics in the sector, something Sapia hopes to help change.

The Fair AI for Recruitment (FAIR™) framework :

– Focuses on establishing a data-driven approach to fairness that provides an objective pathway for evaluating, challenging and enhancing fairness considerations.

– Includes a set of measures and guidelines to implement and maintain fairness in AI based candidate selection tools.

-For hiring managers and organisations, it provides an assurance as well as a template to query fairness related metrics of Ai recruitment tools.
-For candidates, FAIR™ ensures that they are using a system built with fairness as a key performance metric.

In launching the framework, Sapia CEO Barb Hyman said: “We have created a framework that we hope can be used as inspiration to ensure that Ai is being used to build inclusive teams – something humans are not capable of doing on their own because we cannot subvert our biases.”

“Our mission is to help HR leaders make bias interruption more than rhetoric, which is why we also published this guide to Making inclusion an HR priority, not a PR one”.

About Sapia

Sapia has become one of the most trusted mobile-first Ai recruitment platforms, used by companies across Australia, India, South Africa, UK and the US, with a candidate every two minutes engaging with their unique Ai chat bot Smart Interviewer.

What makes their approach unique it it’s disruption of three paradigms in recruitment -candidates being ghosted, biased hiring and the false notion that automation diminishes the human experience.

The end result for companies – bias is interrupted at the top of the funnel, your hiring managers make more objective decisions empowered by Smart Interviewer their co-pilot, inclusivity is enhanced, and your hired profile starts to look more like your applicant profile.

Media contacts

Barb Hyman, CEO
barb@sapia.ai

Voluntary employee turnover can have a direct financial impact on organisations. And, at the time of this pandemic outbreak where the majority of the organisations are looking to cut down their employee costs, voluntary employee turnover can create a big concern for companies. And thus, the ability to predict this turnover rate of employees can not only help in making informed hiring decisions but can also help in saving a substantial financial crisis in this uncertain time.

What Drives Job-Hopping?

Acknowledging that, researchers and data scientists from Sapia, a AI recruiting startup, built a language model that can analyse the open-ended interview questions of the candidate to infer the likelihood of a candidate’s job-hopping. The study — led by Madhura Jayaratne, Buddhi Jayatilleke — was done on the responses of 45,000 job applicants, who used a chatbot to give an interview and also self-rated themselves on their possibility of hopping jobs.

The researchers evaluated five different methods of text representations — short for term frequency-inverse document frequency (TF-IDF), LDS, GloVe Vectors for word representations, Doc2Vec document embeddings, and Linguistic Inquiry and Word Count (LIWC). However, the GloVe embeddings provided the best results highlighting the positive correlation between sequences of words and the likelihood of employees leaving the job.

Researchers have also further noted that there is also a positive correlation of employee job-hopping with their “openness to experience.” With companies able to predict the same for freshers and the ones changing their career can provide significant financial benefits for the company.

Regression Model To Infer Job Hopping

Apart from a financial impact of on-boarding new employees, or outsourcing the work, increased employee turnover rate can also decrease productivity as well as can dampen employee morale. In fact, the trend of leaving jobs in order to search for a better one has gained massive traction amid this competitive landscape. And thus, it has become critical for companies to assess the likelihood of the candidate to hop jobs prior to selections.

Traditionally this assessment was done by surfing through candidates’ resume; however, the manual intervention makes the process tiring as well as inaccurate. Plus, this method only was eligible for professionals with work experience but was not fruitful for freshers and amateurs. And thus, researchers decided to leverage the interview answers to analyse the candidates’ personality traits as well as their chances of voluntary turnover.

To test the correlation of the interview answers and likelihood of hopping jobs, the researchers built a regression model that uses the textual answers given by the candidate to infer the result. The chosen candidates used the chatbot — Chat Interview by Sapia for responding to 5-7 open-ended interview questions on past experience, situational judgement and values, rated themselves on a 5-point scale on their motives of changing jobs. Further, the length of the textual response along with the distribution of job-hopping likelihood score among all participants formed the ground truth for building the predictive model.

To initiate the process, the researchers leveraged the LDA-based topic modelling to understand the correlation between the words and phrases used by the candidate and the chances of them leaving the company. Post that, the researchers evaluated four open-vocabulary approaches that analyse all words for understanding the textual information.

Open vocabulary approaches are always going to be preferred over closed ones like LIWC, as it doesn’t rely on category judgement of words. These approaches are further used to build the regression model with the Random Forest algorithm using the scores of the participants. Researchers used 80% of the data to train the model, and the rest of the 20% was used to validate the accuracy of the model.

Additionally, researchers also experiment with various text response lengths, especially with the shorter ones, which becomes challenging as there is not much textual context to predict. However, they found a balance between the short text responses along with the data available and trained the model predicts for even those.

To test the accuracy, the models are evaluated based on the actual likelihood of the turnover with relation to the score produced by the model. To which, the GloVe word embedding approach with the minimum text of 150 words achieved the highest correlation. This result demonstrated that the language used in responding to typical open-ended interview questions could predict the chances of candidates’ turnover rate.

Wrapping Up

Leveraging data from over 45,000 individuals researchers built a regression model in order to infer the likelihood of the candidates leaving the job. It will not only remove the dependency of companies on candidate resumes and job histories but also enhances the process of hiring into a multi-measure assessment process that can be conducted digitally for recruiting.

By Sejuti Das, Analytics India Magazine, 02/08/2020

Join the movement

To keep up to date on all things “Hiring with Ai” subscribe to our blog!

You can try out Sapia’s Chat Interview right now, or leave us your details here to get a personalised demo.