Workday with Sapia: Faster Hiring Results

There are some steps we can take to eliminate bias in recruitment and it begins with not relying on CVs as a method of evaluating candidates.

CVs are full of information that is irrelevant to assessing a person’s suitability to do a job. They instead highlight things that we often use to confirm our biases, and draw our attention from other key attributes or aptitudes that might make someone especially suitable for a job.

For example, if a CV mentions a certain university it might pique our attention (a form of pedigree bias). This is problematic, as there may be socio- economic reasons why someone attended a certain university (or did not attend another) and CVs do little to reveal this. Situations like this confirm the bias that lead to it in the first place, compounding bias for these long-term systemic issues.

Additionally, CV data reduces a candidate pool in a way that is not optimising for better fits for the role, by relying on the wrong input data and criteria to find a candidate. Amazon discovered this when it abandoned its machine learning based recruiting engine that used CV data when it was discovered the engine did not like women.

Automation has been key to Amazon’s dominance, so the company created an experimental hiring tool that used artificial intelligence to give job candidates scores ranging from one to five stars.

The issue was not the use of Ai, but rather its application. Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. As a result of being fed predominantly male resumes, Amazon’s system taught itself that male candidates were preferable. It penalised resumes that included the word ‘women’ as in “women’s chess club captain.” It also downgraded graduates of all-women’s colleges.

Studies have shown systemic unintended bias occurs when reviewing resumes that are identical apart from names that signify a racial background or gender, or a signifier of LGBTQIA+ status. The solution for this has been to remove names or any identifiable data from an interview or CV screening, but these have still experienced bias issues like those discussed earlier.

In order to be truly blind, any input data needs to be clean and objective. This means that it gives no insight into someone’s age, gender, ethnicity, socio-economic standing, education, or even past professional experience.

To truly disrupt bias, recruiters and hiring managers should utilise a new wave of HR tech tools such as Sapia, stepping away from using CV data as a way to determine job suitability.

____________________

We cover this and so much more in our report: Hiring for Equality. Download the report here.

This year, we have helped more than 60,000 people get a job. From retail assistants, graduates entering the workforce for the first time, sales professionals, team leaders and cabin crew, to name just a few. We have supported recruiters across various industries and roles to find the right people to join their organisations.

Yet, for every successful candidate we helped, there were 21 unsuccessful ones. Recruiters do a lot more rejecting than hiring, which begs the question of how good the typical recruitment function is at dealing with unsuccessful candidates. And what the consequences are of not getting it right?

Anecdotally, there are plenty of stories of candidates getting ghosted, dragged through onerous processes, subjected to poorly conducted interviews, or participating in questionable assessment practices. Many of us have experienced lousy recruitment firsthand, or someone close to us has, but when we look at the data it gives credence to these anecdotal experiences.

Starred, a candidate experience benchmarking firm, use Candidate Net Promoter Score (cNPS)* to measure the quality of the candidate experience across several touch points during the recruitment process. Not surprisingly, the cNPS for rejected candidates is -15, meaning that rejected candidates have formed a significant negative view of the company and will likely  detract friends and family from applying with the organisation.

Social media amplifies the consequences of providing a negative experience. A separate study by the Talent Board across 240 companies and 183,000 candidates found that disgruntled candidates will broadcast their negative experience 34% of the time.

Disgruntled candidates impact the brand reputation and the bottom line. Virgin Media recently quantified the cost of poor candidate experience, estimating that across 123,000 rejected candidates every year, if 6% cancel their monthly subscription, then about US$5.4 million in revenue is lost. 

When a recruitment provides a poor experience, they are creating customers for competitors. But it doesn’t have to be that way. Candidates with a positive experience can be one of the most potent tools in building a positive brand and presence in the marketplace.  According to Talent Board, 64% of candidates with a positive experience will expand their relationship with the brand, even if they didn’t get the job. 

What are the hallmarks of a process that consistently delivers a great candidate experience? 

Communication is right up there. Timely and accurate communication with candidates every step of the way. When we look at the feedback candidates provide us, a major theme around the importance of communication emerges. Not surprisingly, candidates appreciate NOT being ghosted. And they also appreciate being given honest and constructive feedback immediately following their interview.

Sapia and it’s blind smart interview

Also important is convenience. Sapia’s chat-based interview format allows candidates to conduct the interview at a time and place where they feel they can be at their best. It is fast, easy and safe.

And finally, candidates need to trust that the process is giving them a fair go. In a recent experiment conducted by Monash University, using Sapia’s chat-based interview platform, female candidates were more likely to continue with the application process when told that the interview will be with AI rather than a human. Using an AI process, such as what Sapia provides, offers a unique opportunity to put the candidate at the centre. And it not only results in better experiences for candidates. Recruiters also benefit by spending less time on administrative tasks and more time building relationships with candidates.

So while we helped more than 60,000 people get a job and are proud of contributing in that way, we are equally as proud of the fact that for the 1.2 million candidates that didn’t get the job, we were able to provide a positive experience for more than 60% of them. This translates into a cNPS of 58, well above the -15 cNPS reported by Starred for unsuccessful candidates.

And we did that by giving everybody an equal opportunity to be at their best during the recruitment process, providing everybody with useful insights straight after their interview, and freeing up recruiters to spend more time with candidates.

*cNPS is the % of advocates less the % of detractors. A positive value indicates more advocates than detractors.

With so many candidates in the market, it’s more important than ever to create an engaging and human candidate experience. But you need to balance that with finding the best talent for your role.

Skill testing can give recruiters a competitive advantage in today’s job market. Candidates who are hired on merit, rather than background, tend to stay longer and perform better over the long term. Here’s how to use skills assessments to fill your open positions, no matter how many applicants you are dealing with.

Your Guide to Skill Testing

• What is a skill test?
• How skill testing works
• How to run a skill test
• Using skill tests during hiring
• Skill test examples and templates
• Do new hires like doing skill tests?
• What are the benefits of a skill test?
• Do skill tests work?
• Are skill tests valid?
• Skill tests vs. interviewing
• Additional resources

What is a Skill Test?

A skills test is an assessment used to provide an unbiased, validated evaluation of a candidate’s ability to perform the duties listed in the job description.

Typically, a skills test asks a variety of questions in different formats to see how candidates perform on-the-job tasks. A good skills test includes questions that are capable of being answered by someone already doing the job and can accurately measure key performance metrics. Questions should also be specifically tailored to relate to the responsibilities of an open position. Many skills tests include immersive experiences, like coding challenges or job simulations, to mimic how a candidate performs when faced with a real-life scenario.

Other types of job-readiness evaluations deploy validated psychometric assessments to identify those in-demand soft skills: things like motivation, conscientiousness, resilience, and emotional intelligence. A personality assessment varies from a skills test in that it predicts how a person will behave in a specific scenario, rather than their ability to complete a task.

Related: Should You Use Psychometric Tests for Hiring?

While skills test cover task-related abilities, like coding, copywriting, or sales, some pre-employment assessments integrate the less tangible capabilities – things like teamwork and leadership. These qualities are sought after by executives at more than 900 companies, according to a Wall Street Journal survey of executives.

Yet, 89% of those surveyed said they have a “very or somewhat difficult time finding people with the requisite attributes.” Where traditional hiring methods fall short, a skills test can easily clarify a candidate’s true talent.

“Many service companies, including retailers, call centers, and security firms, can reduce costs and make better hires by using short, web-based tests as the first screening step. Such tests efficiently weed out the least-suitable applicants, leaving a smaller, better-qualified pool to undergo the more costly personalized aspects of the process.”

Research by John Bateson, Jochen Wirtz, Eugene Burke and Carly Vaughan via Harvard Business Review

Overall, skills tests can play a critical role in predicting on-the-job success. More so than resumes or job interviews, a skills test can assess the true potential of a new hire to go the distance with the company. Here’s how skill testing works, and why more companies than ever are starting to integrate skill testing into the recruitment and hiring process.

How Skill Testing Works

Skill testing works best when the questions being asked are specifically crafted to the role and needs of the team hiring the new candidate. In designing a skills test, combine different types of questions to get a 360-degree view of how a candidate will perform in different scenarios.

There are a variety of ways to set up a skills test – and we’ll get into the mechanics of how to actually run the assessment in the next section. But, designing a thoughtful aptitude test takes some initial foresight on behalf of the hiring manager and team.

Research by Deloitte suggests this sample process for selecting and implementing skill testing questions:

1. Define the “human elements” needed to perform the job
2. Compile questions that will measure and predict these human elements
3. Use the data gathered by the skills assessments to empower the next round of the screening process
4. Post-hiring, evaluate the efficacy of the hiring assessment to ensure the questions delivered the best result.

Ultimately, the best use for a skills assessment is to help recruiters move away from the resume and allow candidates to prove they are the real deal. Crafting the right series of questions should be a collaborative process between the recruiting team and the team hiring the new employee. Here’s how these teams can set up and run a skills test.

How to Set Up and Run a Skill Test

In designing a skills test or pre-employment assessment, there are a few specific steps to take in order to thoughtfully structure your questions.

Related: 5 Steps to Creating an Engaging Skill Assessment

Based on our work with over 8,000 customers, we recommend following these best practices in setting up and running your skills test. These tips can help with candidate engagement and lead to high rates of completion.

• Your skills test should include a minimum of six questions; somewhere in the eight to ten range is best.
• At least a few questions should require text answers; start with a text-based response in the first question, rather than a video or immersive question.
• Include an “immersive” style question, in which the candidate edits a document, spreadsheet, or presentation.
• To retain a candidate over the entire experience, start with easier questions and build up to more difficult ones later in the assessment.
• Try to minimize the use of timers to account for technical difficulties and give the candidate the best chance of success.

We also suggest that video responses not be timed; there are too many technical issues that can result from a candidate trying to film a one-way video interview. If you do wish to set a time limit, make sure it’s at a minimum of five minutes.

Running a skills test through Vervoe, or any other platform, is relatively straightforward. Vervoe’s skills assessments let you select questions from a library of assessment tools, or design your own questions based on the specific needs of your company. The Expert Assessment Library offers questions and trials created by experts in their fields, meaning they have at least 3+ years of experience in their specific area of expertise. You can preview questions from any of the assessments and add them seamlessly through the Vervoe platform.

Now that you know how to set up an assessment, when should you deploy this tool during the hiring process?

Using Skill Tests During Hiring

Timing is everything when it comes to adding a skill assessment to your hiring process.

Research by Harvard Business Review revealed that skills tests should come early in the hiring process. According to their study, “Many service companies, including retailers, call centers, and security firms, can reduce costs and make better hires by using short, web-based tests as the first screening step. Such tests efficiently weed out the least-suitable applicants, leaving a smaller, better-qualified pool to undergo the more costly personalized aspects of the process.”

Skill tests should be used to screen candidates in, not out. The issue many recruiters face is that the volume of candidates makes it impossible to carefully consider each person’s ability. Smart algorithms and AI tools can turbo-charge candidate assessments by scoring results quickly and removing human bias from the equation.

Vervoe’s algorithm scores candidates using a multi-layered approach. Candidates are ranked based on how well they performed, rather than filtered out if they didn’t achieve a certain benchmark. The top candidates easily rise to the top; but no one misses out on being considered for the next round. When used early in the hiring process, skill tests can select a more diverse pool of applicants to continue onto the next phase.

Skill Test Examples and Templates

There are many ways to set up a skills test, depending on the position for which you are hiring. Pre-employment skills tests can cover a range of positions: administrative assistant, finance and accounting, and call center reps are just a few roles that companies hire for using skills assessments.

Excel skill tests, coding skill tests, typing skill tests, and other computer skill tests are the most common forms of pre-employment assessments. Some companies focus on questions that are task-related, e.g. “Create a Powerpoint Slide that has a video embedded in the presentation.” Questions can get hyper-specific to test a niche skill, like a coding language, or be posed more broadly to test the general requirements for success at a certain level.

Some companies choose to focus on verifying the skills that will help a candidate succeed beyond the immediate position. This approach skews closer to a pre-employment assessment, with questions designed to reveal if a candidate can climb the corporate ladder, adapt in a challenging work environment, or respond under pressure.

For example, one call center rep test included questions such as, “You have an elderly customer on the phone who is having trouble understanding your instructions. A colleague is also trying to transfer a call from a customer you served before, and you have a scheduled follow-up call happening in 5 minutes. How would you handle and prioritize in this situation?”

Multiple choice, open-ended questions, and pre-recorded video responses are all great ways to see if a candidate has what it takes to do the job well. But, do candidates enjoy answering these types of questions?

Do new hires like doing skill tests?

By most accounts, candidates appreciate the opportunity to showcase what makes them great at their job. Orica, the world’s largest provider of commercial explosives, integrated skill-testing into their interview process to the delight of their job candidates. In revamping the interview process for graduate students looking to join the Orica team, recruiters consolidated their online evaluation components into one platform, Vervoe. The skill assessment combined questions focusing on skills, logic, and values.

An average of 86% of candidates completed the online process, and the reviews were mostly positive. Here’s what the candidates had to say about the skills test:

“The tests required total engagement and thought, and were a clear demonstration of what makes Orica different from any other company.”

“I think the questions were very diverse and it allowed me to showcase myself, my skills and abilities in different ways.”

“It gave me an opportunity to showcase who I am as well as challenge my skills”

This is just one example of how a skill test can change the entire interview process for a potential new hire. In a job market where people spend an average of 11 hours a week looking for a new job, it’s easy to get burned out, fast. Every job description starts to look the same; every interview begins to feel stale.

When given the opportunity to showcase their talent through real-world tasks, job candidates will jump at the chance to be engaged with the job description, rise above their resume, and challenge themselves. Companies that use Vervoe’s assessments experience a 97% candidate completion rate, which is among the highest engagement rates in the industry. Candidates love the opportunity to stand out from the crowd. Even if they aren’t hired, skills testing offers a break from the repetition of the stale interview experience.

What are the benefits of a skill test?

The benefits of a skills test aren’t limited to the candidate experience.

Recruiters looking to hire diverse, high-performing teams with better efficiency and consistency can use pre-employment tests to their advantage. Skills tests are a better predictor of performance than resume screenings or traditional interviews alone. Resume screenings are bad for three reasons. First, studies suggest that it’s common for candidates to lie on their CV. The person you think you’re hiring may not actually possess the qualifications you think they do.

“We just wouldn’t be able to interview 2000 people in two weeks. But what we could do is utilize Vervoe to more accurately and in quite an unbiased way, assess everybody’s application during that period.

Rather than just assess the first 200 [applicants] and maybe hire 150 of them, Vervoe allowed us to actually assess all 3000 applicants in a two week period and still be able to select the best 150.”

Jeremy Crawford, Head of Talent Acquisition at Medibank

Second, resumes only provide a high-level view of a candidate’s credentials and work experience. These items don’t offer qualitative insight into actual on-the-job performance. Coupled with recruiting biases that are built into the process, the third threat is that recruiters are privileging candidates based on background and demographics, rather than talent. Perhaps this is why new hires crash out as often as they do. According to one study, 46% of new hires “fail” within the first 18 months of being hired.

Skill tests can help take some of the bias out of the interview process, give recruiters a new evaluation metric to consider, and lead to happier, long-term hires. There’s ample evidence to suggest they really do work better than many of the other traditional hiring methods recruiters have relied on in the past.

Related: How to Avoid the 12 Kinds of Hiring Bias

Do skill tests work?

In our experience, skill testing works better than traditional hiring methods – with some caveats.

Without a doubt, aptitude tests can be used to replace resume screening. This style of sorting through candidates increases the chance that the best candidates will be unfairly eliminated. Good people get screened out, rather than screened in. So-called “pedigree proxies” – resumes and cover letters – are not indicative of job performance, yet they are often the quickest way a recruiter or algorithm can think of to cut down on their stack of candidate resumes.

Skills tests improve time to hire while allowing the hiring manager to see how someone will do the job, before they get the offer. This reduces turnover costs, which add up quickly: the cost of making the wrong hire can be up to 2.5x salary, easily over $100,000. Working with Vervoe’s skills assessments, on the other hand, can help a recruiter identify the best people at under $100 per hire.

The best skills tests, however, need the right formula to help the candidates succeed. Some recruiters focus narrowly on the skills that will help a new hire succeed in the immediate position for which they are hiring. Yet, many CEOs emphasize the importance of soft skills – things like leadership and teamwork.

Related: 5 Ways To Turn Rejected Candidates Into Allies

New hires may end up being disappointed and leaving because they lacked the soft skills needed to adapt to their new team, not necessarily the skills to perform the job. Recruiters must integrate questions into their skill assessment that focus on critical soft skills that predict long-term success. These validated psychometric assessments are key to assessing “culture fit” without defaulting to recruiter bias.

Is skill testing valid?

With any kind of assessment, there’s a common concern that’s quite commonly raised: is this assessment valid?

In summary:

• Any test that directly mimics what a person will do on the job can be considered “validated.”
• Tests of personality and soft skills are a riskier prospect even when they are “validated,” because they often lack the proper validation required to be EEOC compliant.
• Positive candidate experience and perceived fairness are two of the primary reasons why skill testing is an effective and expedient hiring practice.

There are many types of validity, and it’s rare that a test will satisfy every type. Looking specifically at tests for finding job fit, there are a few different types of validity that are particularly relevant, not just to ensure that the hire is a good one, but to ensure compliance with EEOC regulations.

Types of Validity

• Face validity
  The most basic form of validity, and sometimes the only one that can be obtained when a test is first created. Face validity essentially asks whether the test looks like it’s assessing what it claims to measure.One example is testing someone’s arithmetic skills. A set of math problems would have more face validity in this instance than, say, a word problem because a word problem is assessing both arithmetic skills and comprehension.
• Content validity
  For skill tests used in recruitment, the question of validity should be most focussed on this kind of validity. Content validity asks whether the test covers the full range of the construct that it’s supposed to measure.This means that in any assessment, the group of questions being asked needs to cover a wide enough range of skills, so that the person evaluating can be sure that the results show the candidate is capable of doing the tasks required on the job.
• Construct Validity
  When people ask if a test is “validated” or has “psychometric validity,” this is the kind of validity that they’re usually talking about. Construct validity asks whether the test actually measures the theory-based construct that it claims to measure.So, if you’re testing for general cognitive ability or personality, construct validity is absolutely essential, because they are indirectly related to whether someone can perform the job.But when it comes to testing skills that used directly on the job, face and construct validity are far more important.
• Predictive/External Validity
  This kind of validity is about whether or not the assessment predicts performance in other situations. So, if someone scores highly on the test, does that mean they’ll perform well on the job?There’s a big difference between tasks that are assessed without context, and tests reflect the day-to-day skills and tasks someone would need to have to perform the role.

Related: Skill Testing Validity

In all cases where assessments are used, and in every step of the recruitment process, it’s essential that employers track and remain aware of differences in performance that are biased toward particular demographic factors. At Vervoe, we constantly monitor assessments to make sure candidates take tests that are fair, and based solely on skills that reflect how they would perform on the job.

Skill tests vs. interviewing

In conclusion, we’ll leave you with few thoughts on skill tests compared to interviews.

First, interviews, in general, need a total overhaul. Recruiters have been asking the same, outdated interview questions for decades. Many candidates get overwhelmed by the performance anxiety inherent in the interview and may make (forgivable) mistakes. Nevertheless, many recruiters like the security of meeting someone before making an offer.

Many recruiters seek the same insight from a group interview or case study that they would get from an individual skill test. Unfortunately, using these methods can’t give you the same valuable information as a straightforward aptitude assessment. Case studies can be too conceptual; rather than seeing how a candidate will approach the work listed in the job description, case studies ask abstract questions. The goal of asking “how many tennis balls can fit on a Boeing 757” is not to see if the candidate can guess the right answer, but to see how they approach the question and reason through their response.

But this knowledge doesn’t always serve a recruiter with the best predictor of on-the-job success.

Group interviews provide more insight – into a candidate’s teamwork, leadership, and communication, for example. Yet, in a group scenario, extroverts tend to dominate. It can be difficult to see how each candidate performs as an individual while trying to consider the group at once.

In summary, skill testing is all about understanding whether a candidate can do something or knows something. It’s about verifying their ability to go the distance with your company. Pre-employment assessments differ slightly in that they focus on predicting how a candidate will behave in certain scenarios, not what they can do. By combining questions from skills testing and pre-employment assessments, recruiters can get a more accurate picture of the candidate’s ability.

Additional Resources

For more reading, check out some of these great resources.

• https://www.mindtools.com
• https://www.techgig.com/skilltest
• https://www.3smartcubes.com
• https://www.criteriacorp.com
• https://www.smartrecruiters.com
• https://www.revelian.com