A good candidate experience doesn’t cut it in the current recruitment climate. It’s the most basic thing that hiring managers need to fulfil, but if that’s all you are delivering you are going to miss out on talent. 

You need to hire fast (interview-to-offer in 24 hours) and your process needs to be frictionless – ie. you can’t be asking candidates to jump through hoops to prove themselves to you. No games, no CVs, no third interviews, no asking them to make time for an interview when you are free. 

Those days are over. The global talent shortage is being felt across every industry and recruitment needs to be re-imagined.

And, there’s only one solution: automation. 

Woolworths Hiring Process: Automation is the key

Yes, it can be hard to cut through a lot of hype around automation, but it is possible that leaders can develop a  clear-eyed way to think about how these technologies will improve their organizations.

This is not about replacing jobs of HR managers, but giving them the tools to help them grapple with the challenges in the current hiring environment.  It’s about empowering HR teams to be able to do the seemingly impossible – and be good at it. Or to put it another way, help them create a human-centric organization with super-human intelligence.

Introducing Video Interview

This was the challenge that Woolworths Group brought to us.  Even before the pandemic, the Group was realising that they needed to invest in more efficient processes to keep up with the recruitment demands of the company. But remaining fair about who they hired and treating candidates with respect was not up for compromise. 

They had just recovered from surge hiring needs brought on by COVID and wanted to make sure they never had to go through that as a team again. The task had been slow and manual,  took too long to hire and the tech they had used was not reliable. 

They were looking to redefine their whole approach and realised they could not deliver a positive experience to candidates without the help of technology. 

Sapia’s current chat-based candidate assessment was chosen as a front-runner after an extensive search for tools, given the fact that it meant that bias was removed from reviewing candidates and because it also meant they could give every candidate constructive feedback – even if they didn’t get the job.

However, there was still an opportunity to solve for the sheer volume of video interviews that had to occur in the next step.

This made Woolworths a perfect candidate to roll out Sapia’s Video Interview product, a video interview delivered via conversational chat that candidates can do in their own time, and doesn’t require any scheduling input from hiring managers.

Woolworths Interview Process: 24-hour time to decision

This means anyone can now run a fully automated hiring process that is both fair, candidate-friendly and insanely fast. Woolworths was the first customer to go live with Video Interview and as the largest private employer in Australia it was a true test of the effectiveness of the product.

Within one week of going live Smart Interviewer, our text chatbot, had interviewed more than 10k candidates, all without bias. The introduction of an end-to-end fully automated chat based assessment process where every candidate is interviewed and every candidate receives personalised feedback transformed their recruitment for candidates.

But, what was transformational for hiring managers was that the top candidates were then able to do Video Interviews through the platform by video recording answers to a set of questions on their phone. No-one had to schedule an interview and hiring managers could quickly assess the best candidates for the role by simply watching a video – also, in their own time. 

Time-to-decision is as little as 24 hours in some cases with the Group achieving an NPS score of 8.8. 

Automation the answer for the current climate 

The issues that Woolworths faced are felt by most large companies hiring at scale.

With the introduction of Video Interview, Sapia can now create and deliver a solution that streamlines the Woolworths recruitment process and improves the efficiency of large-scale recruitment for other companies as well.

• Giving everyone a fair go at getting the job 
• Giving every candidate feedback 
• Keeping it frictionless for candidates
• Removing the need to schedule any follow up interviews asynchronously 
• Empowering hiring managers with objective intelligence about every candidate 
• Enabling hiring managers to have the final say by watching a simple video in just a few minutes.

If you’d like to know more about Video Interview and how we have integrated video into our product suite without compromising fairness, please get in touch.

You can also download our Woolworths case study.

This research paper is part of our accepted submission to SIOP, and will be presented at the 2023 SIOP Conference in Boston.

Discrimination based on race and ethnicity in personnel selection is a well known and pervasive issue highlighted in numerous studies (Bertrand & Mullainathan, 2004; Kline et al., 2021; Pager et al., 2009). Most of these studies report name-based inference of race and ethnicity by human reviewers leading to differential outcomes in the recruitment process. Linguistic racism is a form of discrimination that occurs based on one’s use of language, especially English (De Costa, 2020), and is highly associated with race and ethnicity.

As machine learning models are adopted to automate tasks like interview scoring, race or ethnicity encoded signals in language can lead to biased outcomes, if not mitigated. Hence understanding the level of ethnicity encoded signals in language is important when building natural language-based machine learning models in order to avoid biased outcomes, for example by using feature scores rather than raw text to score responses (Jayaratne, Jayatilleke, Dai, 2022).

In this work, we sought to quantify and compare the amount of ethnicity encoded information in over 300,000 candidates’ raw text interview responses to language-derived feature scores, including personality, behavioral competencies, and communication skills.

First, we trained machine learning models to predict candidate ethnicity from raw-text chat interview responses. Specifically, we trained an Attention-Based Bidirectional Long Short-Term Memory (Attn-BiLSTM) (Zhou et al., 2016) model for predicting ethnicity from textual responses.

Secondly, we tested the same for the language-derived features used in the automated scoring of the interview responses. We trained multiple models using a variety of machine learning algorithms (a linear model, tree models with bagging and boosting, and a neural network model with a single hidden layer) suitable for tabular data for predicting ethnicity from the 21 derived features.

Each model was then used to predict ethnicity for the 10% of the sample left out of the training dataset. The results from the classification tasks show a clear distinction between the ability to infer ethnicity based on natural language and inferred features. As hypothesized, we found that features derived according to a clearly defined rubric contain significantly less ethnicity information compared to raw candidate responses. That is, the models based on derived features recorded consistently weaker accuracy, precision, recall, and F1 values across all models compared to the model for the raw text candidate responses.

This research demonstrates the benefit of using algorithmically derived feature values in mitigating ethnicity related biases when scoring structured interview responses. Specifically, our results show that natural language responses to interview questions carry higher amounts of ethnicity information compared to features derived according to a clearly defined rubric for assessing interview responses. This further strengthens the case for using structured interviews that have been shown to reduce bias over unstructured interviews (Levashina et al., 2014) with much stronger criterion validity (Sackett et al., 2021).

References:

Bertrand, M., & Mullainathan, S. (2004). Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination. American Economic Review, 94(4), 991–1013.

De Costa, P. I. (2020). Linguistic racism: Its negative effects and why we need to contest it. International Journal of Bilingual Education and Bilingualism, 23(7), 833–837.

Jayaratne, M., Jayatilleke, B., & Yimeng Dai (2022). Identifying and mitigating gender bias in structured interview responses [Paper presentation]. 2022 Society for Industrial Organizational Psychology Conference. Seattle, Washington, United States.

Kline, P. M., Rose, E. K., & Walters, C. R. (2021). Systemic Discrimination Among Large U.S. Employers (NBER Working Papers No. 29053). National Bureau of Economic Research, Inc.

Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature. Personnel Psychology, 67(1), 241–293.

Pager, D., Western, B., & Bonikowski, B. (2009). Discrimination in a Low-Wage Labor Market: A Field Experiment. American Sociological Review, 74(5), 777–799.

Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2021). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology.

Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., & Xu, B. (2016). Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 207–212.

Right now video screening is the solution of choice for many, given the challenges of recruiting during the pandemic. Every day I’m asked about video solutions, and every week there seems to be a new video solution for hiring.

This isn’t people simply switching to Zoom, but rather embracing AI  video platforms where you are judged by algorithms. Often algorithms crawl these videos to identify top candidates. This is not great. In fact, it’s horrifying. Not all video interviews are bad, given the pandemic it’s often become a necessity as a default for face-to-face interviews in the final stages of a recruitment process. But when it comes to top-of-the-funnel screening with first interviews, video interviews lead to biased outcomes.

Put simply, image and video recognition is built to favour white faces. In the documentary Coded Bias an M.I.T. Media Lab researcher Joy Buolamwini found that the algorithm couldn’t detect her face–until she put on a white mask. There are hundreds of validated research findings which confirm this.

Video Screening

Video invites judgement. It adds stress to the candidate with added pressure around hair and makeup, picking the right fake backdrop (yes, there are hundreds of advice columns on this), and practising and rehearsing your answers until you nail the recording. It turns a simple interview into a small theatre production.

Not everyone is comfortable on video, most especially introverts, people with autism, and people who feel marginalised. These factors do not influence or speak to a person’s ability to do a job, but by using video as part of the interview process they are put at a deep disadvantage. What percentage of people are you excluding just by using video?

Chat is a better option. It solves the challenges of remote interviews while being inclusive.

Try it for yourself, we’ll send you real results.