Blog

Back

Written by Nathan Hewitt

Three diversity goals you can actually track and achieve in 2022

More money is flowing into Environmental, Social and Governance (ESG) than ever. In 2021, investors poured $649 billion into ESG-focused funds worldwide, up 90% from the $542 billion invested in 2020. In the UK, over 21% of investors plan to back funds and companies with comprehensive ESG strategies by 2025. And in Australia, more than 55% of super funds are using responsible investment approaches to inform strategic asset allocation.

All this investment has prompted a sharper focus on social issues across major companies – the S in Environmental, Social, and Governance. The great news is that investment in the big S, in turn, means more money and attention toward progress in Diversity, Equity and Inclusion (DEI).

Executives that care about diversity know that an effective strategy must start at the top – take Australian superannuation fund HESTA and its 40:40 vision as an example. But, to be truly successful, we need DEI goals at all levels, and we need to track, accurately, the degree to which we meet them.

Both boards and shareholders want measurable change in DEI, and fast. According to a Harvard Business Review study of S&P 500 earnings calls, the frequency with which CEOs talk about issues of equity, fairness and inclusion has increased by 658% since 2018. You can bet that this will only increase further in the coming years.

Diversity goals need to be measurable, today

According to another HBR article, 40% of US companies discussed DEI in their Q2 2020 earnings calls, which is a huge step up from the 4% of companies that did the year before. And with 1,600 CEOs pledging to take action on DEI, setting goals and tracking progress remain top priorities.

DEI and ESG are big challenges, and we might take myriad possible approaches in trying to solve them. Some companies may start at the executive level (HESTA, as an example), while others may invest in partnerships and outreach programs. The spectrum of options can easily become overwhelming.

“Interestingly, I’m just looking at our workforce profile and have been discussing the changes in diversity since we updated our recruitment approach last March. Not only have we hired three times more ethnic minorities and 1.5 times more women, but we now have twice as many LGBTQI+ colleagues in our business than we did three years ago! Other initiatives have played a part, but I’d imagine the game changer has been Sapia as we’ve had some direct feedback from a transgender colleague that they felt more confident with our recruitment process than they did in other applications! 

David Nally, HR Manager, Woodie’s UK

So why not start with the people you bring into your company, at all levels? Why not begin with the way you attract, assess, and select talent?

With a Smart conversational Ai, you can set realistic DEI targets and measure them, at scale, with little extra effort – ensuring you access the best talent from the widest possible pool. A Smart Interviewer is different to the simple chatbots used to automate routine tasks according to a fixed set of rules. For example, our conversational Ai is able to analyse interview responses to gain deeper insights about each candidate’s personality and competencies, in a fair and objective way. 

Our Smart Interviewer helps you track and meet these three key diversity goals.

  1. Gender bias

Our proprietary interview response database is made up of more than 500,000,000 words, enabling us to conduct the most sophisticated response analysis in the recruitment industry. We can do this on a macro scale (e.g. across countries, cultures, industries, and role types); or for individual companies.

Take these findings, combining data from a range of our customers, globally:

Diversity and inclusion analytics

Figure 1: Gender stats across applicants, Ai recommendations and hired

Thanks to our machine-learning capabilities, and the size of our database, we can provide the hiring team with real-time analytics on the following parameters:

  1. Number of observed female applicants vs number of expected female applicants (and the same for male applicants)
  2. Overall hiring rate of females vs males
  3. Overall rate of female vs male recommendations made by our Ai

By employing a smart interviewing Ai at the first stage of recruitment, we can prove progress with regards to inclusivity and bias reduction. These aggregate company data show that while the expected number of female applicants exceeded the number of those that actually applied, the number of recommendations made by our Smart Interviewer also beat expectations (effectively compensating for the top-of-funnel bias). We can also see that the rate of observed female hires far exceeded the expected number. 

What does this show? With just three metrics, you can see the progress being made in your recruitment process – and if performance is below expectation, you can see the stage at which targets are not being hit.

It is important to note that the recommendations of our Ai are based solely on its analysis of candidate responses in the chat-based interview. Its suitability criteria is based, among other factors, on HEXACO personality modeling and accurate assessments of various job related competencies such as team work, critical thinking and communication skills.

Our data also keep biases in check at each stage of the recruitment process, depending on the role type. As you can see, for all three roles, this company’s hiring outcomes were within regulatory limits (as stipulated by the US Equal Employment Opportunity Commission (EEOC)) across the three stages of their funnel: Applications received, recommendations made, and the hiring decisions ultimately made by the hiring team. The final step, it is important to note, happens independently of our Ai: It is a human decision. Despite this, the outcome data is recorded, so that the company can compare its outcomes against inputs and recommendations to see if late-funnel biases are occurring.

Solving gender bias with data

Figure 2: Role-type-based gender bias. Mid line indicates 0 bias. Shaded areas indicate the tolerance level. Right of line favours females and left favours males.

The feedback from candidates is extremely positive: Company A’s strivings for fairness and equality in its processes has resulted in a candidate satisfaction score of 98.7% for females, and 98.1% for males. Better still, the interview dropout rate across the board is less than 10%.

  1. Ethnicity bias

As with gender, our ethnicity analytics help hiring managers to easily set and accurately track goals for ethnicity representation in recruitment. Company A (whose data were shown in Figure 2) is, again, leading the way in this regard: Its BAME (Black, Asian and Ethnic Minorities) recommendation rate is at 46.5%, exceeding expectations – meanwhile, its non-BAME recommendation rate sits at 37.1%.

Our data has also helped Company A to increase its hiring commitments for First Nations people: The rate currently sits at 4.5%, from 4,000 candidates, above the national average of 1.8% (2018-19). This number is expected to increase over the coming year.

  1. Personality biases

The data we collect helps us, as well as our customers, understand the extent to which personality determines role suitability and general workplace success. It also helps us to eliminate long-standing biases that negatively impact certain candidates, despite the fact that said candidates may be highly suitable to the roles for which they are applying. 

For example, people high in trait agreeableness (compassionate, polite, not likely to dissent or proffer controversial viewpoints) tend to underperform in the traditional face-to-face interviews. Hiring managers may assume, based on this, that they are unable to lead, or are not a ‘culture fit’. However, a face-value assessment of agreeableness is not a reliable predictor of candidate potential. Only scientific analysis of HEXACO traits can make this call with accuracy.

Take these two visualizations, showing how different personality traits affect the recommendations made by our Ai. Females (red dot) and males (blue dot) are slightly different in agreeableness, but there is virtually no difference in their conscientiousness, a strong predictor of job performance. As a result of being able to measure conscientiousness accurately, our system can effectively allow for higher levels of agreeableness – or cancel out the negative face-value judgements typically made in face-to-face interviews. Despite these personality differences, as shown in Figure 1, Sapia Ai recommendations for both male and female groups remain similar (~40%). This results in a fairer chance for all, and a wider pool of candidates. In this case, this is to the benefit of females.

diversity by personality type

Figure 3: Male (blue) and Female (red) personality trait differences

Bringing it all together

The world is changing, and we can no longer continue to take a “We’ll see what happens” approach to the ‘S’ in ESG. Many investors are pushing companies for better diversity and inclusion outcomes. At Sapia, our data show that fair, scientifically valid, and explainable Ai can produce better outcomes for peoples of all genders and ethnicities. The companies that have adopted our Ai approach are seeing strong improvement in their own DEI practices and results.

Over and above assisting our clients, our commitment to DEI is embodied in a guiding vision of our own: Our FAIR Framework. This embeds an approach that ensures our systems and processes are ethical and transparent. Many similar Ai systems operate in a ‘black box’, providing little knowledge about how their algorithms help make important decisions or create issues like amplifying biases. We are committed to a fairer world, free of bias – and, with every candidate interviewed, our data is bringing us closer.


Blog

Introducing InterviewBERT: A world-first algorithm for better interviews

Sapia labs, our R&D department, has developed a world-first innovation that will help us deepen our understanding of the contextual meaning of words in written job interviews. Called InterviewBERT, this algorithm combines Google’s model for Natural Language Processing (NLP) with our proprietary dataset of more than 330 million words. BERT, meet Smart Interviewer. Together, they’ll usher in a new generation of pre-employment assessment tools and recruitment software solutions.

Put simply, InterviewBERT makes Smart Interviewer, the most sophisticated conversational Ai in the world. Ours is no simple chat-bot – already, Smart Interviewer is capable of discovering personality traits and communication skills, accurately and reliably, using a candidate’s written responses. With InterviewBERT, Smart Interviewer will learn more about candidates than ever before, faster than ever before. With this speed and accuracy also comes reductions to the unfairnesses and biases that plague the hiring process.

Why, and how, are we the first to transform pre-employment assessment technology with BERT?

Through sound Ai infrastructure, we have been able to accumulate a vast and accurate dataset. This dataset grows by the minute – we interview a new candidate every 30 seconds – and, coupled with the expertise of our Sapia labs team, we can assess candidate suitability for a role in milliseconds.

“The smartest companies know that the fairest and most accurate way to assess someone’s suitability for a role is through a structured interview,” our CEO, Barb Hyman, said. “Text increases accuracy and speed of assessing candidates, while removing biases that come through voice or video interviews.

InterviewBERT and Google NLP | PredictiveHire recruitment software

Dr Buddhi Jayatilleke, Chief Data Scientist and head of Sapia labs, said the team is excited at the finding that InterviewBERT had such a profound impact on trait accuracy.

“Written language encodes personality signals predictive of ‘fit’,” Dr Jayatilleke said. “The ability to understand people through language has limitless applications, and we are excited to keep inventing more ways to use language data for our customers.”

Dr Jayatilleke said decades of research had confirmed that language has long been seen as a source of truth for personality. 

“What our R&D team has proven is just how powerful language data is when you combine it with enormous data volumes and scientific rigour,” he said. “This capability can be used for assessment and for offering personalised career coaching – a game changer for job seekers, universities, and employers.”

Sapia labs will present its findings from a new research paper, Identifying and Mitigating Gender Bias in Structured Interview Responses, at a SIOP symposium in April. 

Read Online
Blog

What can Ai help us discover?

We are asked often in customer meetings – What does the data reveal?

What can AI help us discover? How can we make better people and business decisions by looking at the data?

By using SOM maps  https://en.wikipedia.org/wiki/Self-organizing_map to map personality for more than 85,000 applicants using their HEXACO scores, 47/53% male and female, candidates spread across 2 regions – the UK and Australia, we identified 400 unique personality profiles. 

It turns out that personality is somewhat more complex than the 16 types long promoted by Myers Briggs.

Following SOM’s show the percentage density of male, female and sales candidates across the 400 different HEXACO profile groups. The size of each bubble represents the total count of individuals mapped to each profile. Darker shades represent higher % of each category.

The ‘so what’

  • People are different. There is a whole spectrum of personality. Even 400 groupings of personality don’t convey the variance in personality in the population
  • Gender is less meaningful as a lens into personality than what role family you sit in. The genders are not that different when it comes to personality profiling
  • Role family differences are noticeable.  Gender personality differences at the role family level are negligible. “Sales” candidates come from specific personality profiles (i.e. HEXACO combinations) and it is clear that certain personality profiles are drawn to sales more than others. See the SOM on Sales that clearly shows some profiles are more likely to be sales candidates than others (the darker bubbles).

Why NLP is a better approach to discovering personality traits

Personality is widely accepted as an indicator of job performance. Until now, the only way to accurately measure personality was through long and repetitive 100+ item personality tests, where the candidate experience is proven to be weak. The Sapia team breaks new ground disrupting decades of assessment practice. They do this by showing that answers to standard interview questions related to past behaviour and situational judgement can be used to reliably infer personality traits. Thus by leveraging NLP, machine learning and personality theory, we validate that text is a reliable indicator of hidden personality traits. Additionally, this approach to candidate interviews is blind to gender, race and any characteristics that are not directly relevant in job selection. Instead, every applicant is given a fair opportunity to express themselves and be evaluated equally.


Join the movement

To keep up to date on all things “Hiring with Ai” subscribe to our blog!

Finally, you can try out Sapia’s SmartInterview right now, or leave us your details here to get a personalised demo.

Read Online
Blog

How Ai can build equity in hiring for the future

Traditional psychological assessment has reduced the hiring and promotional error rate in modern businesses successfully for decades. They have also been used extensively to identify ‘hidden talent’ or ‘potential’ in people with limited work experience such as graduates, and also applied as a means for identifying future leaders at different levels of seniority, as well as in succession planning.

Psych testing is essentially an old-school form of predictive analytics, but they are limited in insight, providing a test of your ability to do a test. That’s it. Traditional psychological assessments do not link to actual performance in the role, nor do they have any self-learning functionality. There is no performance data that feeds into psychological assessments and therefore they have limited predictive power and no learning capability.

The worst aspect of psych tests is that you need multiple tests to test for multiple attributes. This is because they are just not that smart. This is where innovation necessarily disrupts an old formula. The difference lies in the data – volume and variance. A psych test is usually multi-choice questions repeated in different ways to achieve validity. You and I might pick the same option for each question and the only way to distinguish between you and me is to ask us a lot of questions and hope we pick some that are different to recognise our differences.

Data that comes from free-text answers to open-ended questions is by definition going to be hugely varied. A question like ‘what’s a favourite experience of working in a team’ asks us to each delve into our own personal experience, a behavioural interview question which means our answers will naturally be different. 

This formula of using data that is uniquely personalised delivers variance that psych tests just can’t deliver. Ever. When it comes to developing an Ai based assessment the questions that a candidate is asked, and the answers to the questions are suitably diverse, psychologically robust and designed with the same rigour in standardised Psychological assessments.

With the processing power and advances in Natural Language Processing (natural language being the origin of all psych tests) instead of having to force a candidate through multiple tests you can distil many attributes from one test. That test is usually 20 minutes, asks 5 questions, with up to 80 features able to be discovered about that candidate including their critical thinking, their drive, self-awareness, accountability and team orientation, their propensity to stay in a role or not, their HEAXCO traits and their communication skills.

The ability to better understand individuals based on their answers to questions means we can provide accurate and insightful feedback to everyone within a couple of hours. Feedback allows everyone the opportunity to be heard, understood and cared for. This is equity.

____________________

We cover this and so much more in our report: Hiring for Equality. Download the report here.

Read Online