The Role Of Assessments In A World Of AI: Learning From SHL - E180

August 28, 2024 00:22:32
The Role Of Assessments In A World Of AI: Learning From SHL - E180
The Josh Bersin Company
The Role Of Assessments In A World Of AI: Learning From SHL - E180

Aug 28 2024 | 00:22:32

/

Show Notes

In this WhatWorks podcast I interview Andy Bradshaw, the CEO of SHLSHL is one of the world’s largest psychometric assessment vendors, bringing together psychology, skills assessment, and other forms of measurement to identify the capabilities, and potential of an individual at work.

These highly scientific assessments are widely used for pre-hire screening, job-fit analysis, development planning, leadership development, and succession planning. But in a world of AI, where large HR platforms are “inferring” skills and capabilities, where do they fit?

Andy, who has led SHL for many years, explains how psychometric assessment fits into the new world of AI and Talent Intelligence. He also shares how SHL itself has amassed a large data set on skills and roles that makes SHL a big data vendor as well.

Keywords

SHL, talent intelligence, assessments, AI, talent acquisition, talent management, leadership assessment, benchmarking, data-driven decisions

Episode Chapters

00:00 Introduction 00:57 SHL: A Talent Intelligence Business 02:21 Psychometric Assessments and Tailored Solutions 03:47 Customization and Defensibility of Assessments 06:37 Consolidation in the Assessment Market 07:07 The Complementary Role of AI and Assessments 08:56 Assessments vs. AI: The Value of Insights 12:49 Assessing Leaders: Experience, Ability, and Potential 16:09 Benchmarking and Data-driven Decisions 19:03 Talent Intelligence: Aggregating Data for Better Decisions 21:17 Conclusion

Additional Information

Introducing the HR Career Navigator

The Talent Intelligence Primer

When Will The Trillions Invested In AI Pay Off? Sooner Than You Think.

View Full Transcript

Episode Transcript

[00:00:04] Speaker A: The AI tools that are used predict based on patterns of data, but they can't predict some of the underlying context in which you're trying to understand the individual. It might predict that I'm good at communication. What it doesn't really tell me is my potential. And that's really where we come in with some of the assessment design and measurement that we provide. Customers bring us in to measure. They want that decision to be robust and fair and defensible. [00:00:33] Speaker B: Hi, everyone. Welcome to a new episode of the what Works podcast series. You just heard from today's very special guest, Andy Bradshaw, CEO of SHL, a London based talent intelligence and assessment company. I'm sure many of you recognize SHL as a leader in this space. In this episode, Andy and I dive deep into the evolution of Shljdehdeh, the role of psychometric assessments, and how AI is shaping the future of talent management. Let's get started. Andy, thank you for joining me. [00:01:05] Speaker A: Thanks, Josh. [00:01:06] Speaker B: So let's start with just a little bit of basics. Tell everybody about who SHL is and a little bit of the history of the company, because it's actually a very interesting company. It's been around a long time and has a pretty big presence in the whole HR domain. [00:01:19] Speaker A: Yeah, absolutely. I think several people might recognize SHL as a leading assessment business and probably a leading psychometric assessment business. As you say, we've been operating for almost 50 years now, but we made a decision a few years back to move away from just providing purely assessments to packaging our solutions based on customer needs. And so we've built out a portfolio of not only products, but also functionality in our software, which means we can do workflow for customers, branding, provide digital feedback. And so we've evolved really to provide tailored solutions for companies and individuals actually within their customers through the SaaS platform. So the way I describe the business today, rather than where we've come from, is a talent intelligence business that some of the largest organizations around the world use, where we provide package solutions for both talent acquisition and talent management from entry level all the way through to the c suite. [00:02:21] Speaker B: So just to go back to assessments in general, for people listening to the podcast that don't understand the assessment industry, give us a one or two minute background on what a psychometric assessment is and how they were developed. [00:02:35] Speaker A: Yeah, I mean, psychometric assessments have been around for a long time, and as I said, SHL was out very early as one of the, the leaders. Essentially, psychometrics assessments are designed to understand more about an individual, to predict performance in a job. Now, that could be used in a hiring sense. So is the individual capable of doing the job that we want to do, or it could be used in a talent management sense. You know, how can we develop this individual going forward? And over the years, assessments have really broadened, and that's one of the things that we've invested in. You know, not just personality assessments, but skills assessments, which is a really big topic at the moment. Moving into language assessments, coding assessments, but also newer tools that are moving more towards the same assessment methodology, such as video interviewing and other tools like that. So it's really broadened out recently. But the way I describe assessments to customers is anything that provides individual to the person, either for hiring purposes or for some sort of talent management purposes, that is defensible going forward. [00:03:41] Speaker B: So when a customer buys a solution from SHL or another vendor, they're getting off the shelf assessments, you're customizing them. How does it get tailored to that organization? [00:03:53] Speaker A: Yeah, we provide off the shelf, but we do tailoring rather than customization. There are organizations that just do purely custom design. Part of our business was a company called PDRI that does that for the us government specifically. We like to take off the shelf assessments or components as we describe them and put them together into a solution so they can still feel like it's tailored. You know, a lot of the work that we do with some of the largest organizations on the planet, you know, they want their brand to come across. They want their workflow to come across. You know, they may want to give feedback to the candidate as they go through, whereas another organization or a different solution might be a lot quicker. [00:04:31] Speaker B: Okay, so you get the benefit of the years of research on what this job role is all about and the role skills and so forth, but you can customize it so it looks and feels the way your company wants it to be. [00:04:42] Speaker A: Correct? Correct. Because over those years, you build really strong what's called validation. So predictability, essentially, of what the tool is supposed to do. And that's important, especially for the customers in markets where there's quite a lot of legislation, is being able to stand behind the assessment and the decision off the back of the assessment to make sure that that is not having undue adverse impact. [00:05:03] Speaker B: As an example, now there used to be hundreds of assessment firms, small companies that had built different kinds of assessments. It seems to me a lot of that's collapsed. What is your sense of your fit into different size of companies or industries versus other assessment providers, or has the market really collapsed a lot? [00:05:24] Speaker A: Yeah, there's been a lot of consolidation in the market. There are a few larger vendors, but there's still a lot of local vendors. And the way to think about it is either in country or specific customer segment. So there are those vendors who just do, let's say, volume hiring in the us market, which is a huge market in itself. And equally there are organizations such as Hackerrag, Hackeroth who just do coding. And what's happened over the years is there were a lot of assessment providers and there were lots of industrial occupational psychologists within the customer base who were looking at those different assessments. I think over time there are more generalists now and HR generalists making the decision. So once you've kind of got the check in the box for science, those individuals are making different buying decisions now, such as what's the candidate experience like? How much insight can you give to me outside of just purely the test itself? How can I make sure my branding comes across? So that's why we in particular have moved more into this solution zone, whereas a lot of the traditional ones have stayed more just test provision. [00:06:39] Speaker B: Okay, so you guys have been around a long time. You went through the acquisition by CEb and then the spin back out. So you've been through kind of a reinvention in a way. And along comes AI and AI platforms that have tried to do some form of skills inference or skills assessment through general AI models. What is your reaction and position sort of relative to the AI platforms out there that are trying to do this? [00:07:10] Speaker A: Yeah, no, I think on the whole it's been really actually quite helpful for us in a funny sort of way. I mean, as you know, the amount of platforms using AI, there's quite a few. The challenge is that the AI tools that are used, they are trying to predict based on patterns of data, but they can't predict some of the underlying traits or behaviors or skills or even the context in which you're trying to understand the individual. And that's really where we come in with some of the assessment, design and measurement that we provide. AI is only as good as the data it's learning from. So let's say I've been in a call center, and let's say I've been in a call center for five years into my first position. It might predict that I'm good at communication or problem solving or I'm able to follow a process because you would expect that from somebody in the call center, assuming that I've done a good job. But what it doesn't really tell me is my potential and what else within my character, my skills, my behaviors that I might be able to do? I might be high on emotional resilience, I might be high on adaptability, but that wouldn't necessarily come out from inferred data. You'd need to actually measure. So what we're seeing is happening within customers is the AI platforms do play a useful role in trying to look across an organization very quickly and infer the skills that are there and that are available. What then happens is customers bring us in to then measure them because they want to measure them and they want to make a decision, and they want that decision to be robust and fair and defensible. So the combination of two, I actually think, is quite powerful. [00:08:55] Speaker B: Okay, so there's no sort of competition between an AI based inference tool and an assessment. You really see them as complementary. I would agree with that. But can you think of an example of a company that's doing it that way? [00:09:07] Speaker A: We have a number of customers who do that. And again, the AI tools are quite different, as you know. So if you look at somebody like Novartis, they use gloat, slightly different use of AI, but it's a marketplace model and trying to connect people. So Andy thinks he's good in the call center I put forward. He tries to connect me with other roles. But what Marcus and the team wanted then, is to be able to measure that and actually put us alongside that individual and say, well, here's actually what we think you're really good at. So when you go into making that decision, there's some context, and I think that's helpful for the candidate, not only the company as well, because then the candidate is less likely to be disappointed when they go for that role. That isn't a good fit because we're actually giving them some guidance. So that's an example of where we're sitting alongside in that example, an AI driven marketplace. [00:09:56] Speaker B: And as I understand the way the world works today, if a company has an AI platform, like gloat or Eightfold or one of the other ones, and they use SHL, they can take the SHL data and put it into that platform. So they get the benefit of both. [00:10:11] Speaker A: Yeah, they get the benefit of both. And we live in an ever increasingly integrated world. And I think that's really a big step forward that we'll start to see over the next few years, which is how can these pockets of data be better integrated to get better outcomes for organizations and individuals? Unfortunately, they are still relatively siloed. Going back to the example we just used my data sits in that marketplace. My assessment data sits somewhere else. I think the power of WikiLeaks, high intelligence that could be unlocked, is where those two things can come together. [00:10:48] Speaker B: What do you say to companies that are maybe not familiar with what you guys do, but they've bought a seek out or eatfold or one of the other AI based systems and said, oh, this is great, we're going to use this for sourcing. I don't think we need any assessment. How do you compare the value of the assessment based solution versus the AI solution? [00:11:09] Speaker A: We would always say that assessments provide more insights for hiring decisions. So why wouldn't you? Especially in high stakes roles, but it actually also happens in low stakes roles as well. That individual that's opening a store, are they safe? Can they follow a process, those types of things? Why wouldn't you? Because the wall for talent is greater than it's ever been, you know, since certainly I've been working and getting it right, whether it's customer service, whether it's leadership that's important, and the kind of cost return for assessments is relatively low in that sense. So I'd say yes. And I would also say where you have something in place, think about the measurement. So it's not just the predictability and the inference is what, what are you doing to actually measure such that when you give that individual the job or that promotion or that succession opportunity, you've got something you can go, yeah, measured it and it's a good fit. [00:12:07] Speaker B: Right. I mean, as much as I'm a fan of AI, I have to agree with you completely. I think a lot of the inference that comes out of those tools is very vague, and I'm not sure how accurate it is all the time. Yeah. [00:12:18] Speaker A: And I think customers are genuinely concerned by making decisions off the back of it, in particular, like the us market's an example, it's got to be legally defensible, right? [00:12:29] Speaker B: So let's talk about leadership. One of the missing links in the AI world is how do you assess leaders? Now, Heydrich is working on an initiative with what's called the Heidrick Navigator, which you may be familiar with, that is using their assessment methodologies in concert with Eightfold in their AI system. What would you tell people about how to assess leaders? [00:12:56] Speaker A: I mean, I think the first thing to say is we believe that leadership is more critical than ever before. I mean, the speed of change across organizations is ever increasing. The need for agility, resilience, emotional intelligence is greater than ever. And the leadership decisions are, they're high stakes decisions, right? I mean, you know, look, when you get leadership wrong, you know, maybe a bad example, but Boeing, you know, to me, is it kind of like getting the leadership wrong? It's kind of like has not helped them in the position they're at. And equally, we can see where people, when they do get it right, they can really fly. So, you know, I think when, whenever we think about assessing people for positions, we're really looking at three aspects. We're looking at their experience, we're looking at their ability, and we're looking at their future potential. And obviously, as you go through your careers, you build experience. And that often for leaders, drives them into a certain way of operating and a certain way of being successful. We did a very large study on leaders, and we found one of the most important factors is context. So you could have the same leader in a different contextual position and get a very different outcome. And if you give you an example, an individual who is tasked with transforming a business might not be the best person to drive high margin, in a low margin environment. The pure skills maybe doesn't mean they're a bad leader. They're still a good leader. But getting the individual into the right role at the right time, I think, is really important. And then obviously, if you can get that right and you can understand the individual, you can think about those development and learning interventions that helps that individual grow. Either get stronger of what they're doing or build out some extra skills. Because fundamentally, I think most organizations want their leadership teams to thrive, and that's what they're trying to do when they're assessing leaders. And the more insight and the more data points you can get around that, then the better you are. And certainly for leaders, when we assess them, it's just not one assessment. You're looking at multiple assessments. You're looking at things like 360 feedback. You're trying to take those different inputs. [00:15:19] Speaker B: So you do senior leadership assessment as well as prehir and hiring? [00:15:24] Speaker A: We do all the way through, yeah. I mean, one of the uniques about the business is the fact that we can do entry level through to c suite, and it's based on the same framework underneath. So you can kind of compare outputs at different levels as well. So when it comes to full workforce planning, which we're starting to see organizations do so not just tiers of managers or leaders or not just segments of functions. That starts to become a really important part when you're looking across the data across the whole of an organization. [00:15:57] Speaker B: Now, one of the things you were doing prior to CEB that I talked to you about, I remember was you were building a big benchmark database. [00:16:04] Speaker A: Yeah. [00:16:05] Speaker B: At the time. Did you manage to keep that? [00:16:09] Speaker A: We've had a lot of investment, and in particular around the software and the technology and the data. We've got 50 billion data points on people in the workforce. [00:16:21] Speaker B: So let's just talk about what that means. So I could go through, I could hire you guys to do assessments of our leaders or first line managers or whatever, and you could probably benchmark lie people against other companies of my industry, of my size. Correct? [00:16:36] Speaker A: Correct. Yeah. We can benchmark against people in your industry, your size. We can benchmark against your aspirational industry. If you go through some sort of transformation, I might like my leadership to look more like this. We have the ability to do all of that, and what we're starting to do as well is we talked about AI just a moment ago, but is use AI to really drive better outcomes across that data. So one of our customers in the technology world, we've got data now on 25,000 of their leaders, and it's not one data point, they're multiple data points. So we're working with them to be able to ask the question of who could be my head of America's in a year's time? What are the skills that my leaders overall are lacking, and how do I go about doing that? So being able to use AI, because it's quite hard if you've got that much data and you're trying to cut and paste it through Excel and everything else, but AI enables you now to ask more of those free language questions to get the information back. I think it's quite exciting. [00:17:43] Speaker B: What's really contemporary right now that you think is driving success and performance in companies from your data in which you're seeing in all the assessments you're doing? [00:17:53] Speaker A: Yeah, I think context is really important. As I said earlier, I think understanding the context you're putting leaders in is one thing. I think we are seeing the rise of leaders needing strong emotional intelligence and being authentic across organizations that is increasing and this need for agility, being able to switch between different contexts, being able to react. I mean, you know, the world is changing all the time, right? You know, whether it's driven by wars, pandemics, you know, you just think about the last three or four years. The one constant is change. And having leaders who are capable of spotting adapting to those change is becoming more and more important as well. The other big one actually is also digitization. Having the digital skills needed, especially in terms of embracing AI technologies and new technologies around automation, is also a big part of leadership change. [00:18:55] Speaker B: What does talent intelligence mean to you? [00:18:57] Speaker A: Well, I was going to say, because as you rightly said, we kind of came up with the talent intelligence term about ten years ago, but it was in a slightly different context. It was very much in the context of I'm trying to understand this individual. So it's really at an individual level. But now talent intelligence to us is how can you aggregate data about the workforce that gives you some sort of insight that enables you to come to make decisions and get better outcomes. That's what we're really trying to do. And obviously we come at it from an assessment perspective. So how do we understand the individual more and more? But you need to complement that with other data sources as well, from the HCM system, the HRIs system. And then I think once you do that, you can really start to make really insightful decisions across the whole of the workforce. That's all real time. So we've kind of moved on from there. You know, I want to understand this individual. I'm going to make a single decision. I want to understand this individual, and I want to add to them other data points as they go through their career, such that I can continue to make decisions about them throughout their career. And also I can give them the insight so they can also make decisions about their career. So I'd agree with a comment that you said where you said the talent intelligence platform is the backbone of integrated talent management, matching people to opportunities. I think that is really what it's about. And we are very much, I think, within touching distance of being to do that. So in the same way as CRM systems are very informed about all the actions that a customer takes and the buying decisions and the touch points, I think we'll end up with something very similar in the workforce. That's not the static system of record that it's traditionally been in the HCM system, but actually this ongoing data access data insight that senior leaders and individuals can use to make better decisions and get better outcomes, I love that, Andy. [00:21:03] Speaker B: I mean, that's a really good way of framing it. We're going to spend more time with you, for sure, because I really think with all of the focus on AI platforms, people have sort of forgotten a little bit about how important assessment is because it plays such a major role. Anyway, congratulations on all that you've been through. I mean, going through an acquisition and spinning a company back out, I'm sure is exciting. I've been through a little bit of that myself. I will continue to talk, and thank you so much for your time today. [00:21:31] Speaker A: Thanks, Josh. [00:21:32] Speaker B: Okay, thanks a lot. Thank you for tuning in to today's episode of the what Works podcast series. I hope you found our conversation with Andy insightful and thought provoking. We covered a lot of ground discussing the evolution of SHL from a psychometric assessment company to a talent intelligence company, the complementary role of AI in assessments, and, of course, the importance of robust, fair, and deficit sensible decision making in all of our talent management. If you enjoyed the episode, don't forget to subscribe to our podcast. We've had more than 3 million downloads. Leave a review, share it with your colleagues, and please stay tuned for more discussions with industry leaders, experts, and practitioners who are shaping the future of work. Thanks, everybody. See you next time.

Other Episodes

Episode

February 24, 2024 00:18:10
Episode Cover

The Wakeup Call To Colleges: A Growing "Worthlessness" Of Degrees

As companies shift heavily towards “skills-based hiring” to remove bias, we now see the unintended consequence. Half of all US college graduates are now...

Listen

Episode 0

March 14, 2022 00:18:01
Episode Cover

The Secret To Growth In Today's Job Market Is... Organization Design

In this podcast, I discuss one of the most important topics in business: Organization Design. Now that it’s almost impossible to hire people and...

Listen

Episode 0

September 23, 2023 00:20:52
Episode Cover

What's Up With The UAW Strike? And What Are The Real Costs of AI?

In this podcast I discuss the ongoing UAW strike and why it feels strangely out of sync with the economy, EV revolution, growth of...

Listen