The Leadership Growth Podcast

AI's Impact on Leadership Assessment

Daniel & Peter Stewart Season 1 Episode 47

Imagine being “paid to judge people”!

That’s how today’s guest jokingly describes his role. Tom Verboven is a Senior Director of Client Success at Mercer Talent Enterprise, where he advises clients on how to drive value through effective human capital strategies.

In this conversation with Daniel and Peter, Tom offers his insights and advice for navigating a leadership assessment landscape that now includes artificial intelligence (AI) tools.

Tune in to learn:

  • Three shifts in leadership assessment over the last decade
  • Three steps leaders should take as they use assessments in the future
  • One thing leaders should do to establish an accurate standard of success

We need to embrace both the human and the technological, Tom says. “In the end, a leader is a human and needs to do good with other humans,” he concludes.

Questions, comments, or topic ideas? Drop us an e-mail at podcast@stewartleadership.com.

In this episode:

:48 – Introduction: Tom Verboven

3:07 – Topic: AI’s Impact on Leadership Assessment

8:30 – The Changing World of Assessment

16:00 – The AI Avatar Experience

22:44 – Can AI Be Too Accurate?

25:03 – Steps for Leaders to Understand the Assessments of the Future

28:09 – Caution Around AI for Assessments

31:42 – The One Thing for a Leader to Establish an Accurate Standard of Success

Resources:

Tom Verboven LinkedIn

Mercer Talent Enterprise

Kahneman, D., Rosenfield, A., et al (2016, October). Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making. Harvard Business Review.

What Are the Big 5 Personality Traits?, Very Well Mind

Behind the Headline: Unpacking the Reported Drop in Conscientiousness

Stewart Leadership Insights and Resources:

7 Principles for Character Driven Leadership

The Advantages of an Online 360° Assessment

Six Ways to Leverage the Power of 360° Assessments for Your Leaders

When Should I Use a Lead Now! Assessment?

The Leadership Self-Assessment Question You Need

Stewart Leadership Assessment Services

Stewy: Your AI Leadership Coach





If you liked this episode, please share it with a friend or colleague, or, better yet, leave a review to help other listeners find our show, and remember to subscribe so you never miss an episode.

For more great content or to learn about how Stewart Leadership can help you grow your ability to lead effectively, please visit stewartleadership.com and follow us on LinkedIn, Instagram, and YouTube.

Hello, everyone, and welcome to another episode of the Leadership Growth Podcast. I'm your host, Daniel Stewart, along with my wonderful brother, Peter Stewart, as we will be hosting today, and we are very excited to be able to have a fantastic guest with us. Tom Verboven has joined us, a deep expert. Well, I'll get into his expertise, but we're going to be talking all about leadership, executive assessment, and AI. What is the impact that is having as we are wanting to assess and develop executives and leaders in all levels? So before we do that, let me share a little bit about Tom's background. With over 20 years of leadership assessment experience at Talogy, PwC, and currently at Mercer, Tom has lived in six countries and assessed leaders across the globe. A Belgian citizen who is now based in Dubai, yes, it's rather late for him, so thank you, he starts his day with a double espresso while watching NBA games. I love this. Bringing reflection and a global perspective to every conversation. I can just picture you, Tom, with a small little espresso, enjoying seeing some fantastic NBA action, and I've got to, well, first of all, welcome, Tom, to the program. Thank you very much, Daniel and Peter, for hosting me as well. Very much looking forward. There's a little correction I have to make. It's a double espresso with a pinch of milk, like a little pinch of milk. Just a little bit, just to add that extra there. Just a little bit, yeah. Well, that's great. And so, Tom, here's the pressing question, then. If you are an avid NBA fan, what's your favorite team? I'm actually, to be honest, more like an individual player, so I'm going to answer boring, but I'm a James LeBron fan, definitely. So, yeah, so it's Lakers now, but it can change. I'm not very loyal, to be honest. I just, yeah, the team that plays the nicest basketball, I'm a fan. I just want to see good basketball. Good basketball, exactly, yeah. You're talking to two guys who grew up loving the Boston Celtics.-Okay.-And so...-Oops.-Who likes the Lakers? There is some definite conflict. I think we'll allow it to happen. He did qualify it, saying he's a basketball fan, like he enjoys a good game, and who can't respect the king? I mean, LeBron, he's amazing. With that, we can dive into some real relevant leadership topics. I love it. And so, Tom, let's get us started here. Give us a sense, what led you into this field of leadership assessment, and kind of define what that looks like, because we're not just talking about sitting in a meeting and kind of looking at somebody and saying, oh, yeah, you're good at this. There you go. No, we're talking far more in-depth analysis. Give us a sense of what this looks like. Yeah, definitely. Now, I do jokingly sometimes say to my friends or my mom who doesn't have a clue what I'm doing, like, I'm paid to judge people. And then, oh, and then she thinks I have done really well in life because I can have a chat. I have my double espresso, having a chat and judging people. But it is, of course, much more to it. The judgment is scientifically based, data driven, whatever. But it comes down to first defining what good leadership looks like. And that depends on context, company, sectors, countries, et cetera. And that's, of course, a very important piece of work that we tend to forget. But that's the starting point, always. And once that is clear, and that should, of course, link to strategy, mission, vision, et cetera, and the culture we want to achieve, then you start assessing leaders with all kind of tools, exercise, psychometrics, AI. We'll come back to that. To understand and then select against that success profile what goods look like. And then, ideally, also there is a development dimension to it. So how do you get leaders to that next level of leadership as it's both individual as a leadership team level, because that's becoming more and more important. So that's what it is. To come back to your question, how did I end up in this field? It was definitely not part of a grand master career plan. Because I didn't have a clue what I wanted to do. And there was one thing I definitely had to be part of my job that was traveling. So my first was becoming a professional basketball player, but unfortunately, that failed. I wasn't that bad, but just not good enough to be professional active. Second option was becoming a diplomat, there I failed miserably. And then I had a period like trial and error. I tried everything, starting academics, banking, technology, at a certain point in time was an executive recruiter. And it was during a prospect, like a client conversation. And the recruitment manager, he was just talking and talking about his ex employer and that he had the opportunity to travel to China and to Kuala Lumpur and he was so excited and he was such a cool company, but you know, the traveling was too much. And I was listening to this guy and I said, I have to call this company. So I called up the company, the country manager at the time, no clue. I didn't even look up what they were doing exactly because I was so excited. And the guy asked me, so what are you doing now? I am an executive recruiter. Okay, that's an interesting background. Let's have a discussion. And then I started to work there. The guy who recruited me is still a mentor. So I followed him also to PwC. And in the second week, there was a French colleague who called me like, Tom, are you willing to travel? I said, that's why I'm here. Do you speak French? I say, yeah, I do speak French. Do you know about 360? I said, I don't have a clue about 360. Don't worry, we will guide you. And off I was on a plane to Kigali, Rwanda, giving a training for Heineken, the beer brewer on 360. And that's how it started. Spent 10 years there, absolutely loved it. And then I moved to PwC and now I'm at Mercer. But also in reflection, I think, or I try to rationalize it too much, but I think I'm in the place where I'm meant to be in the sense that I'm coming back to basketball. I was always fascinated, what's the role of a coach? Because sometimes you have a team that you think they're not going to make it. And then they make it because of a good coach, because of the team dynamics, because of the whole environment, supportive environment. So I think it makes a lot of sense that I ended up in the space where I am now. It's a long-winded answer, actually, but I don't want to skip your picture. Yeah, it's great to hear the journey and the story that you've gone on. This is one of... And I know, Daniel, we've talked about this before. It's one of our favorite parts of doing this podcast is hearing people's journeys. And as we're coaching leaders and we're getting to know, rarely is it just this straight line of oh, when I was five years old, I wanted to be this. And that's exactly what happened. It's this more meandering course of circumstances and people you meet and opportunities you try and sometimes we fail. And so then we pivot and shift. And I think that's just such an exciting part about the path that we take. And as we kind of jump back to what you started defining here, which I think is just fascinating, is you're laying out really the fundamentals of assessment. First, it's the identification of the standard by which we're trying to assess. You know, you're saying, OK, what does good leadership look like? And then it's through some measurement, through some method, we gather data, which then compares to that standard to then identify the gap or the delta. Exactly. And then we take steps to mitigate that. And what are we going to do from here? And I love that just basic fundamental assessment process. So in your view, then, how have you seen this process or change or what observances have you had over these last 10 plus years as you've been diving into this? That's an excellent question. And it's fascinating. I would say there, I do see three shifts since I've been involved. So in the beginning of the 2000s. So when I started this company, it's Cubix, it was acquired by Talogy. Anyway, it was a small UK based company with offices all over the world. But in the basement of our offices was like a stack of prints of personality questionnaires. So it was all pen and paper. Assessors had to kind of rate manually. Can you imagine? So I think that first, the first part is really the digitalization and automation of reports. I think that's the first wave shift, whatever you want to call it. And then there is a long period, probably 10, 15 years where not much really happened. I think the focus was more on the candidate experience, like the whole gamification. So we tried to make, so we did it at the time for L'Oreal, where you had to hunt for a treasure. And by hunting, you had a complete personality questionnaire and you had to do some reasoning tests. But in a way that the candidate didn't feel that they were assessed. So there was a whole gamification. People tried to be fancy with marketing, working together. So the candidate experience was definitely one. On the content, I think we evolved a lot around like adverse impact on gender, ethnicity, neurodiversity. All that stuff also improved quite a lot. A lot of discussion around integration with ATSs, HR systems, all that kind of discussion. That was also important. And then also, but that's also, that's recently actually the assessment platforms. You have one platform where everything is happening. So you send out invitational personality questionnaire through the platform. You do business exercises, you store all the data in there, you do reporting, dashboarding. So that's actually pretty new in the last five years, let's say. There were some interesting innovations during that period. On the AI side, you know the story about HireVue. I think they were a bit too early there and they didn't do their work that thoroughly. So they had a kind of reputational issues. There was very interesting bits and pieces. I had one company in the Netherlands and they were measuring your sales personality based on your saliva. So you had to spit in, like you do your DNA, you had to spit. Then they analyzed the saliva and then they could tell you, Daniel, you're a hunter and Peter, you're more like a relationship guy. So based on, that was interesting. I think they still exist, by the way. I don't know how successful they are, but definitely not scalable because you don't hear about it. But you had this kind of interesting innovation going around. So that's the second shift wave. And then the third is, of course, AI. And I think we're going to go more in depth, but basically, you don't need assessors anymore. So my job becomes obsolete. Luckily, I'm almost retiring. But if you want, in theory, you can do an assessment without any human intervention. So that's, I think, something that it's going at a pace that I've never seen. But secondly, even one step further is this invisible psychometrics. So you don't know, because now you know, like, Daniel, Peter, I'm going to send you a questionnaire, so you know that you're assessed for a reason. But this invisible psychometric is more like analyzing your digital interactions during chat, during Teams, emails, etc. And that's analyzed. And based on that, they can give you a profiling, in theory, also individual. Now it's used for cultural analysis, and people have to agree on the privacy, etc. But you can imagine one step further that's going to be used as well for individual profiling. Yeah.-The breadth of the ability to gather data has just exponentially increased, beyond just, here's the survey, oh, now we're going to take your entire email or chat history and load it in as metadata to then analyze. I mean, it is amazing.-Yeah. That's amazing. And so let's step back for a moment. The purpose for all of this assessment, so we do a fair amount of this assessment work as well. So it's really two things. And please, I want to get your thoughts on how you would add to any of this here. It's really candidate selection. So it's understanding and helping inform decision-making for a hire. To be able to help the organization, the hiring manager, look at candidates and run them through a battery of simulations, assessment, and here's a report to be able to assess. And then the other is after they are hired, so that we can assess, all right, where are they currently in terms of their leadership capabilities, their potential for succession? Maybe the board is interested or senior management or HR. And so here again is the report. Does that kind of sum up or what other circumstances, purposes have you experienced? How would you build off of that? It's mainly that. Yeah, it's for hiring selection. Once they're in, it's for succession. Also high potential programs out of these 100 people who are your high potential, we need to invest in. So those are the use cases. Development, purely development. You do an assessment as a starting point for a development program. But that are the key reasons, the key used cases, yeah. Gotcha. Yeah, so then let's just double click and dive deeper into the whole AI interaction with this process.-Yeah.-Like, what are the impacts now and in the near future that AI will have on assessment? Yeah, that's a good question. I give you my point of view because I cannot see in the future. But as for now, in theory, we can use AI in every single assessment tool, simulation, exercise. So that's why I said you don't need, in theory, you don't need any human interaction, any assessor-led intervention anymore. So you have, of course, you have your personality questionnaire that's automated. There's no human intervention. There is reasoning test, there is 360, whatever tool you want to use. But typically what we do is all these business simulations. So you have a business case presentation, you have role plays. All that can be done by an AI avatar. So like now what we typically do, Peter, we want to test your managerial skills, let's do a role play. I'm your direct report and you have, I'm underperforming, I'm watching too much NBA, I'm drinking too much coffee. I'm talking too much, I'm not working anymore. You have a scenario now and now you have, you can prepare and I have 20 minutes interaction with me. And I, of course, I adapt as an assessor based on what you're saying. All that can be done by an AI avatar. And the AI avatar is going to give you a scoring as well, like Peter, you were too tough on him. He didn't understand his background, just divorced, et cetera. But you were clear on expectations. And so it's going to give you a whole feedback directly on the spot. And it even can rate you on a scale to one to five. And that exists. Yeah. And this is not just typing prompts. This is, it's video interface. Video interface. Yeah. Yeah. Depending on what you're saying to me that day. Do you find... Do you find, Tom, that it's accurate? Or the other question is, is it helpful? Those are two angles as you're kind of seeing this play out and personally experiencing this with a vested interest as you're kind of working through this. How do you find the AI feedback and the AI integration as relating to accuracy and helpfulness? So I tried it myself a couple of times and I was impressed by the feedback I got from the AI avatar. Of course, this is based on one, two times trying, so it's hard to say. So there's still some work on, is it valid and reliable? Because that are the key criteria for us to use tools, to use personality questions, et cetera. So we still do need to do work. So if we use it, we probably need to use it as an AI and then a human will look at it to kind of validate the scoring. So I don't have the confidence yet to fully say, yes, it's accurate. That said, it eliminates noise. So I'm not sure if you're familiar with Canama's work on noise. So basically, even as an assessor, I'm biased. You're biased, we're all biased. That's one. But you also have the aspect of noise. Meaning if I get up here in Dubai, it's 40 degrees, it's way too hot and my airco doesn't work and I have to assess Peter on his managerial skills. I'm not in the mood. It has an impact. Even if I think it doesn't have an impact, it has an impact. Or I don't know, something, I mean, the Boston Celtics lost their game. I don't know which impact that depends who's assessing who, but it does have an impact. So the whole noise aspect just disappears. So it's more about is it reliable and valid or accurate? As you said, that's the key question. And I cannot give you an answer yet. We don't, there's still a lot of research to do. Before answering that question. But I think if there's still a human validation aspect or definitely in the beginning. And if you see, OK, that makes sense. And ideally different assessors rating to make sure that it's accurate. So, but it's still, yeah, that's the work that we still need to do. And then I think on the helpful. Yeah, not for me because I'm losing my job. For the client, it's it's cheaper. But yeah, so help, it depends which angle you're looking looking at it.-Yeah.-But yeah, it's helpful. What will be interesting as well is as you know, this methodology is just utilized more and we have more data that that whole helpfulness question can be answered. But it's also there's just the level of interactions. Do people act the same way when they're interacting with a simulation, with an A.I. as opposed to a human, you know, and just weighing out that variable? Because it does, as you're stating, it gets us closer and closer to more of that laboratory type setting where you're you're helping to control so many of the environmental factors and variables that could, you know, take a portion of the variance of response and account for that. Yeah, yeah, it's an excellent question. Again, not a straight answer, but also candidates are using A.I. So I think that's one of my concerns as well, that we kind of lose this authenticity, the vulnerability of people. You see that in LinkedIn as well, people posting and you think another A.I. driven post and you just lose all authenticity. And of course, candidates are also going to use A.I. to answer questions, role plays, whatever. So interesting, interesting to see. So let me let me ask you kind of a provocative question. So can A.I. be too accurate? And and I ask you that because I read a while back for like self-driving cars. One of the challenges with self-driving cars is they are too good. And and as they drive with human drivers, human drivers are like accurate, like 70 percent of the time, so like 30 percent of the time they're going to make mistakes. And we actually, as humans, expect some degree of mistakes and we're comfortable. It's a weird way of thinking. But like if the self-driving car was accurate 100 percent of the time, all the time and never varied, the human almost doesn't know how to interact and will actually get in accidents. It's weird to imagine. So I ask, might if we've veered too much toward using it, could it be too accurate or is that not so much a concern as we think of leadership? It's a very interesting question, Daniel. I'm not sure if it's a concern. I never thought about this one. This can't be too accurate. I'm just thinking it's got too accurate. It's going to be boring as well. No, maybe that's the part. And then there's no emotions involved like self-driving. It's so perfect. Like and you want to step in your car and be angry about. So, yeah, loss of emotions because too accurate. Right. I don't know. I really don't know. It could be these are the types of conversations that in many ways, I think just kind of blow the lid off of a jar, you know, and I think for many listeners that are hearing and thinking about the future of assessment and the role that even, you know, leaders or humans are playing in that process, it's like, wow, where this is going. And I love it as we talk about that because it helps prepare us for what's coming, because we can become so insulated to just what we're seeing every day in our interactions. And so broadening that perspective, looking at it more and more. So what would you say are, for leaders today that are looking to to leverage, to learn more, to understand the impact of what assessment really is going to be like in the future? What are steps they can take now? From the point of performing during an assessment or implementing assessments in I'd say, I'd say more of the latter for the utilization of assessments, as opposed to advice for the assessor. Yeah. Yeah. The guinea pig. Or the utilization as like, I'm a leader and I want to, I want to see my team who can, who can be my successor. Right. Or yeah. Of course you call Mercer or you call me to have a discussion. This will be the first step. No, I think there are a couple of best practices in the, in the sense and still best practices. And when we talked about that, it's use different tools. That's one. Don't depend on one single tool. That's one thing. Try to, to like, like we said, what, what is, if you're a leader and I'm looking for my successor, what does that, your success profile, no? Like what, what goods looks like? Let's, let's do that exercise, put effort in it. So that's, that's the first thing I put really time in, look at your strategy, vision, look at the future. So get that... Otherwise you're going to assess on the wrong things and that doesn't make sense. So that's, that's definitely step one. Step two is then, then if you assess, use different tools. That's one thing. Measure every single leadership trait multiple times. So if you look at needs to be strategic, try first to understand what does that mean, strategic, because you have 100,000 definition and nail it down. And how can we then measure strategic thinking? And you can do it with a reasoning test, abstract thinking or whatever. You can do it in a business case. You can, you can do it with the personality question. So, so you have to measure it in different data points to cover different data points. And also make always, always there needs to be for me, development dimension to it. So if you're very transparent around your team, like, no, I know a better feel of where you stand, the reasons should always be development, help them in their journey as a leader to become a better version of themselves. That's, that's, I think that's even if you don't get the job, that's, that should be the goal now to, to, to have better leaders in companies, in society. That's the thing that most important. That's, that's well said. And I want to get your thoughts on a caution, a caution around AI, and then Peter can kind of wrap us up here. So the caution that I keep seeing is this temptation to believe that AI can provide great analysis, which it can. It can look at so much data, it can compile, it can suggest, it can really do that work fast and much cheaper. The, the trick is sometimes we as humans might conclude we don't have to be as involved or we, we don't need to understand or take as much time to then understand things. It's almost like, well, it's been done. Fine. We'll just quickly do it. Versus taking the time to have a conversation, to really process ourselves and understand and integrate as well as understanding what is the success profile. It can be an easy temptation to say, oh, generally the AI is providing these five or 10 things I'll just pick here. And, and it's an easy way of making it quicker, which might be helpful, but it might shortchange the level of time that we actually need to spend. So talk about where we as humans still need to spend time and even utilizing what AI can provide. Yeah. No, that's an excellent question. There's an interesting study recently done, like in, in psychometry, we stick, we typically look at the, what we call the big five, which is extraversion, introversion, conscientiousness. And I think that one is very important. It's conscientiousness and that went down. So people are much less, as you said, much less in the depth of things. They don't, they don't think anymore what's behind. The whole reflection is moving away because like you said, okay, what's the success profile in the banking sector in, I don't know where, and you type it in, okay, hop, let's just copy paste it. And the whole thought process is, it's big. You become intellectually lazy a bit. Yeah. And, and that's, that's, that's a risk. And that's worrying. That's really, that's really worrying. Yeah. Yeah. That's really well stated. And I think a good way for us to kind of wrap things up today on this conversation. We've, we've talked about so many different avenues. I'm still thinking about this. Can it be too accurate? Now I'm, uh, my brain is, now I cannot sleep tonight because of that. It's a really interesting, it triggered me, Daniel. Right. Especially when you combine it with the caution you just said that we might become intellectually a little lazy. Uh huh. And just assume it's all too accurate and it's all right, but we still have to do work. We still have to think through it. We still need a process for ourselves. And that's our responsibility as humans. Instead of just—-We need to fail, no? We need to fail to learn as well. I don't think that's, that's. Yeah, I have to fail. And are we over trusting of whatever the output is from our algorithms? You know, is it, does that take away, make us lazy thinkers? Because we're like, oh, well, the AI is doing it for us. Yeah. And I think it's— It's also concerning, the reinforcing of biases. And now we just accept what comes out of the AI. So there's quite a lot of concern. As we think about this, we, we keep going back to this identification of the standard of success. Yeah. And, and I think that it's fundamental to all assessment. It's fundamental to this conversation. Uh, Tom, what's, what is the one thing you would advise a leader to establish an accurate standard of success? What advice would you give them? That's almost philosophical, no? It's because that, that's almost a question of what is good leadership. And then if I ask a hundred people, we'll get a hundred different... But for me, a leadership... leaders need to be, do good. Right. So for me, it's not only about profit and you should be profitable. That's why you're in business. But you should also take into account what are the consequences for, for humanity, for people, for planet. Uh, so I think that you need to do both good. Um, that's, that's my belief and that's, that's a bit philosophical world view, but I do think good leaders do good for the world, uh, and are high on integrity. I think that's a, that's a base for good leadership. Yeah. I love it. Cause you're highlighting the need for, yes, there are technical competencies, but in the end, a leader is a human and needs to do good with other humans. And there are those emotional, uh, warm, human, pure human types of characteristics that need to be present, need to be focused on. Absolutely. That's, that's well said. And frankly, a great way to kind of wrap up our conversation as we've been talking so much about this AI integration with executive assessment. Yes. AI can help do so much analysis. And in the end, how do we make sure that the human side is still present and in good balance? Tom Verboven, thank you so much for joining us on another episode of the Leadership Growth Podcast. We need to embrace both ends, right? And technology and human. Sorry, I was interrupting you. I think we have to end the story. Yeah. Great point. Tom, thank you for joining. It's been fantastic. All of our listeners, thank you for joining as well. Yes, thank you. It's been so good. Please reach out, link, like us and subscribe for future podcast episodes coming out to you and please take the things you've learned and elevate and continue to strengthen your own leadership journey. All the best to everybody. Take care. If you liked this episode, please share it with a friend or colleague or better yet, leave a review to help other listeners find our show. And remember to subscribe so you never miss an episode. For more great content or to learn more about how Stewart Leadership can help you grow your ability to lead effectively, please visit stewartleadership.com.

People on this episode