Measuring Potential in the Era of Digital, Data-based Hiring: a case study with re:work
“How do you determine who you give opportunities to?”
This simple question, posed by Shelton Banks, lies at the heart of his job as the CEO of re:work. re:work is a Chicago-based non-profit that provides individuals coming from non-traditional backgrounds training and opportunities to secure jobs in technology sales. Its training program, which gives stipends to each participant, takes place over 8 weekends and helps candidates create professional resumes, gain the skills required for the jobs, and prepare for interviews, among other services.
To select candidates, re:work uses an interview custom-designed to screen candidates for an inside sales representative position through HireVue, a digital interviewing software with artificial intelligence capabilities that allows people to record themselves responding to prompts.
“We spent a lot of time trying to figure out who to accept into our program,” said Banks. “Every candidate that actually applies to the program takes the HireVue assessment because we are able to be fair with the questions we ask and get a healthy chunk of data from the candidates that apply.”
Kevin Parker, the CEO of HireVue, explained that the structured interview questions are designed by industrial organization psychologists to discover each person’s potential in a job’s relevant skills.
“[The] challenge is that we’re not looking for an archetype of a particular person. Every job is unique, and we analyze each job to figure out what that particular job is looking for.”
For re:work’s purposes, the key skills are communication, drive for results, negotiation and persuasion, problem solving, cognitive ability, and personal stability. Based on results in those categories, HireVue’s artificial intelligence algorithm assigns a score to each candidate from 1 - 99. re:work currently accepts candidates with scores above 33.
re:work previously used volunteer in-person interviews to assess candidates, but received inconsistent data. When they initially switched to HireVue, they didn’t trust the HireVue AI’s assessment and accepted everyone who took the assessment.
“We did the entire program, and we were able to compare the data,” said Banks. “We found that no one that scored in the bottom 33 percentiles got employed. Individuals who scored above that more frequently got a job, so we were able to take that data and say we were pretty comfortable with HireVue’s scores.”
HireVue extensively tests every model and assessment to avoid creating adverse impact, said Parker.
“We [test our models] on very diverse sets of people—in some cases, up to 40,000 people—in terms of gender, race, ethnicities, etc. So in the course of the on-demand interview, I can ask you three or four questions about the job, three or four cognitive games, and get a pretty good sense of who you are as a candidate, as neutral as possible from a gender or race perspective.”
re:work doesn’t solely rely on the HireVue assessment to select its candidates. For example, when a candidate scores low overall, but has high competencies in specific skills, re:work will sometimes still accept them, depending on what their weaknesses are.
The expectations the HireVue assessments create aren’t always accurate. Banks recalled a candidate who scored above 90 in the initial assessment, exhibiting poise and confidence. The candidate didn’t receive a job offer, and when looking back on their video interview, Banks explained that despite sounding polished, the candidate never answered the questions asked.
“HireVue isn’t a fortune telling machine that’s going to guarantee anything. Using it, we are able to get a snapshot of what we’re dealing with in a person when they start our program.”
re:work creates its curriculum with the input and assistance of the employers it partners with, including LinkedIn, Groupon, and Glassdoor, making sure it trains its candidates in the top 10 - 15 things the employers are looking for. This approach also creates buy-in from the companies, who participate in the training.
re:work does not guarantee its candidates job offer, only phone screens. Candidates often are applying to jobs, both within and outside of re:work’s network, by week 3 of the program.
“The [traditional hiring] system will identify words that are or aren’t in a candidate resume and will send that canned, automated email saying that this person isn’t a good fit,” said Banks. “If the employer doesn’t select the candidate, we ask that they give us feedback on why that person wasn’t a fit. They can’t use the blanketed ‘wasn’t a good culture fit.’”
Banks emphasized that companies could be missing out on the diversity of people. He added that the companies considered the skill set required to do a job as the most important thing, which looks different for someone from a non-traditional background.
“Clearly, traditional ways of using software have failed because we have a diversity in tech problem,” he said.
Even with artificial intelligence streamlining re:work’s process of accurately selecting candidates with the most potential, Banks knows they can’t help everyone.
“If we had unlimited capital, we’d love to help everybody, and we want to. But from a business standpoint, [re:work] helps us figure out how to best use our resources.”