"Signal" has become one of the most used words in talent acquisition, and one of the least defined. Every stage of your process is meant to produce one: a piece of evidence that tells you whether a person will do well in the role. Get it right and good hiring decisions follow. Get it wrong and the cost turns up months later, once someone's in the seat and the work doesn't match the paperwork. Building better hiring signals starts with being precise about what a signal even is.
AI has made that much harder. When a candidate can generate a polished application, a tailored cover letter, or a fluent interview answer in seconds, the evidence teams have always relied on tells you less than it used to. So for anyone hiring at volume, the question has sharpened: is the signal you're collecting actually worth trusting?
That was the focus of a session at our first TA Disruptors Summit, led by two of the people who think about this for a living. Fiadhna McEvoy is a Chartered Occupational Psychologist with nearly 20 years designing selection processes for organisations including the Met Police and Lloyds Banking Group. Luke Montuori is a senior psychometrician and Associate Fellow of the British Psychological Society, with a background spanning behavioural neuroscience and assessment design. Together they set out a framework for what a trustworthy signal looks like once AI is in the mix, then showed how to build it into a real hiring funnel. Here's what TA leaders hiring at scale can take from it.
Fiadhna opened by slowing the conversation down, because a word this common is easy to use and hard to pin. She reached for an analogy from home. Her husband, Kevin, is a professor in brain imaging who puts research participants in MRI scanners and studies the blood flow in their brains. The quality of his findings rests entirely on one thing: the strength of the signal the scanner picks up against the surrounding noise.
"At every stage in your process, you are looking for a signal. Something that's going to tell you that this person is actually going to thrive in the role," she explained. The trap is assuming the presence of a signal is the same as a useful one. "A CV is a signal, but is that going to tell us something valuable that you can use to make really strong, confident hiring decisions?"
The point she landed on: every stage produces some signal. What matters is whether it means anything you can act on.
The honest answer is that a lot of the evidence hiring has leaned on for decades was already shaky, and AI has widened the cracks.
Take the CV. In our State of Volume Hiring 2026 report, 87% of candidates say their CV accurately reflects their skills and experience. Only 51% of hiring managers agree. That gap was there before generative AI arrived. Now that at least 46% of candidates use AI somewhere in their application, a polished document tells you even less about the person behind it.
The distrust runs deeper than the CV. Asked how much they trust different parts of the process to predict job performance, both hiring managers and candidates rank AI-driven CV screening dead last, at 28% each. And the cost of a weak signal shows up after the offer: 29% of hiring managers told us they've seen candidates who did well on text-based assessments but couldn't deliver once hired. That is the quiet expense underneath a lot of volume hiring, and it's exactly what a better signal is meant to prevent.
Fiadhna and Luke set out three components. A signal you can trust holds up on all three.
Luke started here, because measuring the wrong thing accurately still leaves you with the wrong answer. Arctic Shores spent years researching which qualities predict performance in a fast-changing world, work that became the skill enabler model: three domains covering how someone thinks, how they manage themselves, and how they interact with others.
The logic is simple. Most technical skills can be taught. What's difficult to teach, and difficult to fake, is how a person thinks, collaborates, and holds up under pressure. The World Economic Forum's Future of Jobs research backs this up, putting analytical thinking, resilience, and adaptability at the top of the skills that matter most. With 39% of today's skill sets likely to be transformed or outdated by 2030, Luke argued the question every team should be asking has moved on:
"We no longer need to be asking what can this person do in the job of today. We now need to be asking: does this person have the capacity to grow into what the role requires tomorrow?"
The second component is about how well you measure, which Luke framed as a question of evidence. Secondary evidence is a candidate telling you about themselves: what they'd do in a situation, what their strengths are, how they'd handle a challenge. It has its uses, but it sits one step removed from the thing you care about, and it's now trivially easy to manufacture.
"Large language models are able to produce compelling examples of secondary evidence. Basically, whatever you want to hear, you can hear it," he said. "If your process is relying on secondary evidence, what you are primarily measuring is the capability to produce that information, not necessarily the capability to perform."
Primary evidence is different. It's behaviour observed directly: watching how someone makes decisions and adapts in real time. It's harder to fake, though Luke was candid that some primary methods can be gamed too, and pointed to Arctic Shores research showing traditional psychometrics can be beaten with an LLM. His test is a practical one. If you haven't checked whether a stage of your process can be completed by AI, you may be collecting noise instead of signal.
Here Fiadhna returned to the scanner. Even a perfectly calibrated machine reads noise if the patient moves their head. Hiring is the same: you can measure the right thing, with the right method, and still get a distorted reading if the candidate can't bring their real self to the process.
"You can be measuring the right capabilities, you can be using the right methods and the right tools, but if the conditions aren't right to enable a candidate to bring their genuine best self, then you are not going to be capturing the optimal signal."
That puts the design and communication of the process at the centre. She pointed to a few levers: getting candidates into a flow state so they respond in the moment rather than second-guessing; making reasonable adjustments the default so nobody's performance is suppressed; getting the value exchange right by giving candidates something honest before asking them to be honest back; and being transparent at every stage about what's being assessed, why, and what's expected around AI use.
The framework is only useful if it survives contact with a real process. Fiadhna and Luke layered the three components over a typical funnel.
At the top, learning agility. Luke made the case for measuring adaptability over knowledge. "Hiring people based on what they know today is something of a fool's errand," he said, given how fast roles and required skills are shifting. A learning agility task measures something sturdier. "We're not asking them what their perception of their adaptability is. We're observing their adaptation as it happens." Future-focused, built on primary evidence, and hard to fake.
At the motivational screen, a redesign. A standard motivational sift, an asynchronous or live video interview, tends to ask open, generic questions, which are exactly the kind AI handles well. Fiadhna described situations customers and hiring managers had shared with her, where a candidate repeated the interviewer's question back loudly, paused, then read out an AI-generated answer. Her fix is to change the conditions. Lead with a genuine realistic job preview, an honest look at a day in the role, the challenges, the team, then build the motivational questions around what the candidate has just seen. "You are giving the candidate first something honest and genuine before you're asking them to be honest and genuine back to you." It lifts authenticity, and it's much harder to fake a response to material consumed live.
At the assessment centre, AI as context. Fiadhna gets asked constantly how to measure AI skills at an assessment centre, and she pushes back on the framing. Rather than testing technical AI proficiency, she suggests building AI into the environment of an exercise as a tool candidates use, while you measure the human behaviour around it. She reached for an exercise she once designed for a security services engineering programme, built around a Raspberry Pi that had just launched. Candidates arrived with ideas for using the device at a careers fair, and the real assessment was the debate: how they presented, collaborated, and reached a recommendation. "The Raspberry Pi was the context, but the human signal was what we were actually measuring. AI is the context, and the human signal is what we're measuring." The payoff, she added, is that "you're not inferring capability from a CV, you're actually seeing it live in front of you."
Fiadhna closed with a test you can run on every stage of your process. Hold each one up to three questions:
Is it future-ready? Does it capture what good performance looks like as the work evolves, rather than only what someone can do today?
Is it valid and reliable? Are you capturing primary evidence you can observe, or self-report a candidate (or their AI) can generate?
Are the conditions right? Do they give candidates the best opportunity to bring their full selves, so the signal you collect is clean?
Answer yes to all three and you've got a signal you can trust. Where a stage gives you pause, that's usually the one quietly costing you good hires.
Define the signal before you chase it. A signal exists at every stage. The job is working out whether it means anything you can act on.
Measure for adaptability, not just current skills. With 39% of skill sets set to change or date by 2030, how someone thinks, learns, and adapts outlasts any single skill on a CV.
Favour observed behaviour over self-report. AI can produce convincing secondary evidence on demand. Check whether any stage of your process could be completed by a machine, and if it could, treat the output with caution.
Design the conditions, not just the measure. Flow, reasonable adjustments as default, a fair value exchange, and transparent communication all shape the quality of what you collect.
Use AI as context, and measure the human signal. At the assessment centre, build AI into the task and assess how people think, collaborate, and decide around it.
Run the three-question test across your funnel. Future-ready, valid and reliable, right conditions. It's a fast way to find the stage that's letting you down.
This article covers the headlines. The full session from Fiadhna and Luke goes deeper on each component and the funnel design, and includes audience questions on cultural fit, reasonable adjustments, and mitigating bias. Watch it below. 👇
If you're rethinking how your organisation approaches high-volume hiring, explore how Arctic Shores can help.
Fiadhna McEvoy: Hi, thank you very much, Hung, for that introduction. My name is Fianna McEvoy and I'm a Chartered Occupational Psychologist and have been for near on 20 years now. And over that time, I've had the privilege of working with a number of different customers, real variety like Met Police, Fire Services, one of the security services, Lloyds Banking Group, Energy Services. And I think the fundamental way I've been partnering with these organizations is through end-to-end selection process design.
As well as designing assessment exercises from the ground up. Now, over the past five years, my focus has been at Arctic Shores, where we've been looking at real psychometric innovation. Essentially, how do you build an assessment that is scientifically rigorous and holds up in an AI enabled world, in a real world with real candidates and actually delivers good results off the back of it?
Increasingly we're having to grapple with the impact that AI is having on the signal that we are capturing through the hiring process, as has already been mentioned this morning. And today the session that we'll be talking through this morning, now really we'll be focusing on like what does that signal actually look like? Now I'm here with a really key team member, Luke, who can introduce himself.
Luke Montuori: Thank you
Hi everyone, I'm Luke Montouri. I'm a senior psychometrician at Arctic Shores and an associate fellow of the British Psychological Society. Across my career spanning behavioral neuroscience, psychometric design and assessment development, I've been interested in a relatively simple question. How do we measure the things that matter? Or more specifically, how do we create the conditions that allow us to observe the behaviors, the skills, the capabilities that we're most interested in trying to understand?
Now, at Arctic Shores, that's led me to work on our cognitive ability tasks, our learning assessments, and our broader skill enabler framework. And these days, I find myself interested in what happens when AI starts changing the signal that we have typically relied on. If people can think, if they can learn, and they can work with increasingly capable systems, what is it that we should be measuring, and what evidence can we still trust?
Fiadhna McEvoy: Thanks Luke. So what we're going to be focusing on this morning is which kind of signals really do matter in an AI enabled world. So what we're going to do is we're going to talk through three components of a framework that highlight what we think the key kind of signal quality components are. And then we're going to bring that to life practically and look at a funnel essentially that is going to, sorry, think my slides have gone back. So we will, I feel sorry.
So this morning we're going to go through a three component framework which is going to essentially bring to light what the kind of quality components of a signal actually look like and then we're going to lay those components alongside a hiring funnel. But before we get into that it's really important that we take a step back and like really understand, be precise about what we mean by signal. So signal is a word I don't know how many times it's been mentioned already this morning.
It's a word that was in our industry so much more prevenantly over the past kind of year or so. But what does it actually mean? It's talking about like, you know, is the signal strong enough? Are we losing signal? What's the impact the signal is having on our hiring process? But we really do need to define what it means. Now, I'd like to give a little bit of a personal analogy from my home life. My husband, Kevin, he has a slightly unusual job. He is a professor in brain imaging. So essentially what he does is he puts like research participants in MRI scanners and looks at the blood flow in their brain. And his research is looking at, you know, how healthy the blood vessels are in the brain to understand, what does that actually mean for things like dementia, for aging, and menopause? But what's really, really critical about like the validity of what he's looking at is how, what is the level of quality of that signal? Hull mentioned the phrase signal to noise ratio. I hear that phrase all the time.
And we've had many an animated pub conversation about him trying to explain the minutiae of the detail of what that actually means. But is that signal that the scanner picking up actually strong enough? Is the distortion in there? Is what he is looking at able to meaningfully tell him something about what he can see, what's going on in the brain? And that is directly relevant and absolutely key to the concepts that we are kind of grappling with now in the world of hiring. At every stage in your process, you are looking for a signal. Something that's gonna tell you that this person is actually really going to thrive in the role that you're recruiting for. And it's not, is there a signal to measure? It's about, is that signal actually meaningful? CV is a signal, but is that gonna tell us something valuable that you can use to make really strong, confident hiring decisions off the back of. So that's what we're gonna be talking about this morning. And obviously like we've got AI which has made this challenge 100 times harder to do this, not just at the start of the funnel, which a lot of us are navigating now, but the whole way through the funnel. How do we make sure that quality signal is coming through and we're actually able to pick that up? So we are going to be, we are, let me, I think I'm sorry.
Now it's not going. Would you like to move it along for me, Here we go. Thank you, technology. Here are our three components. have first one, future ready. Is the signal, are you measuring the right signal in the first place?
Is it something that is actually picking up on the capabilities that are actually going to predict whether someone is going to perform effectively in the future workforce? The second component is, is it the signal valid and reliable? Are you measuring it well? Are the methods you're using, are they measuring what they are actually claiming to measure, particularly in a scenario when we've got AI in the mix?
And then the third component is about authentic performance. So are you creating the best conditions for your candidates to be genuinely enabled to turn up to the process and give their true selves so you are then picking up the best quality signal from your candidate to the process? So this morning we're gonna give you definition of what these components mean and we will then talk you through how, bring that to life through the recruitment funnel. So Luke, think you're gonna start us off.
Luke Montuori: Thank you, Fina. Thank you, Simon. thank you, Fina. So this is somewhere that our own journey becomes particularly relevant. So we spent quite a few years trying to answer the question, what does future focus signal look like? And you'll likely recognize the results. It's our skill enabler model. Three domains, thinking style, self-management, and interaction with others.
All the characteristics that we believe underpin someone's ability to develop skills and perform, even if we can't predict what skills are going to be needed in the future. And the logic is relatively straightforward. Most technical skills can be taught.
what's particularly difficult to teach, to develop, and indeed to assess for, are things like how does someone think, how do they collaborate with others, and how do they manage themselves under pressure. And all of these things are characteristics that determine whether someone is able to adapt, to grow, and to ultimately succeed in a world that is changing around them. Now when we first started the research that ultimately ended up with the model that we have as the skill enablers,
We weren't thinking of the current AI wave. It was well beyond that. It was five, six years ago. It wasn't our focus. We were interested in identifying what skills would matter in a world that was characterized by increasing volatility, uncertainty, and rapid change. We looked at a variety of different organizations, and all of them framed the challenges somewhat differently, but ultimately ended up in the same place. And that place was, these are the three kinds of skills or characteristics that are going to be useful in the future, in an uncertain future. And we've now arrived in that uncertain future, right? And so this kind of model and the capabilities that underpin it are, as far as we're concerned, more relevant than the day that we launched the model.
And it wasn't just historical, this continues to be borne out by the data. In the World Economic Forum's recent Future of Jobs report, it's been made abundantly clear. The skills and capabilities that matter the most are human, or increasingly human. At the top of the list, we have analytical thinking, but that's closely followed by resilience and agility, leadership and social influence, self motivation and self awareness. And this exactly underpins our skill enabler model. These are the underlying capabilities that enable people to perform and develop skills even when the skills that they're going to need are evolving.
And with 39 % of skills likely to be transformed or even outdated by 2030, it is now more than ever that the question has shifted. We no longer need to be asking what can this person do in the job of today, and we now need to be asking does this person have the capacities to grow into what the role requires tomorrow?
And so that WEF data reinforces something that we identified a while ago. The capabilities that are increasing in importance are not technical ones, they're human, they're social, and they're increasingly difficult to automate. Now we asked some customers and we went out to the research literature again, looked at the thought leadership again, and tried to get a grasp on what social interaction skills and capabilities were likely to be more important in a future characterized by the kinds of volatility, uncertainty that we've been talking about. And the results of this were strikingly relevant. We have communication and sense-making, and so this is an ability to make sense and meaning out of ambiguous information. Collaboration, leadership, and social influence, and so this is someone's ability to effectively work through and with other people. Resilience and emotional regulation, self-explanatory, but your ability to remain unflappable as situations change around you. And your judgment and ethical awareness, ability to navigate complex human situations. Now, these capabilities are what's shaping our current roadmap. So today, we're happy to announce that we're, well, we've been in development, working on assessment that measures the behavioral and cognitive underpinnings of socially adaptive behavior, that examines how people perceive, interpret, and reason about the beliefs, intentions, and actions of the people around them. And we believe that these skills and characteristics or skill enablers are going to be more and more important in a world that is characterized by more collaboration, more ambiguity, and more human AI interaction.
So that was the domain of future focused signal. It's one component of high quality signal that we're looking for. But of course another component is the validity and reliability of the signal that we're picking up. Now there are lots of ways that we can cut this particular cake, but one way that we can think about or assess the validity and reliability of the data that we're collecting is by assessing the source of that data. So on the one hand, we have secondary evidence. Secondary evidence is our candidates telling you about their experiences, reflecting on what they think they could do or what they would do in a situation, and telling you what their personality is like. Now, don't get me wrong, this can be useful in some contexts, but it's one step removed from what really matters.
And that is observable information. And what we all know now is that increasingly large language models and other integrated AI tools are able to produce compelling examples of secondary evidence that mimic tailored responses, polished applications, basically whatever you want to hear, you can hear it.
And so if your process is relying on secondary evidence, what you are primarily measuring is the capability to produce that information, not necessarily an individual's capability to perform or behave in that way.
And on the other hand, we have primary or observable behavior. So these are behaviors that we observe directly in controlled situations. We watch decisions, how they adapt and how people respond in real time. This has always been a psychometric case for collecting behavioral observations over secondary or self-report data. But now it's more important than ever. And actually,
This question goes further than just primary versus secondary evidence because there are some situations where primary evidence can also be mimicked by large language models. So we've produced already some research showing how traditional psychometrics can be gamed using large language models. And this just drives the point home, really, that if you're not assessing whether a piece of your assessment process is able to be completed by artificial intelligence or some other kind of system, then you're potentially collecting noise rather than the signal that you're looking for.
Fiadhna McEvoy: Thank you, Luke.
So back to our MRI scanner analogy, because it really is relevant here. If we think about, you know, we've got the most powerful scanner, it's perfectly calibrated, but noise can still come into the picture. If a candidate moves their head, that distorts the concept and what has been read through that signal. So it's about getting the conditions right for the information that's coming through to make sure that we're able to be really robust. And that's the very same situation in the hiring situation.
So you can be measuring the right capabilities, you can be using the right methods, the right tools, they can be AI-resistant, resilient tools, but if the conditions aren't right to enable a candidate to bring their true self, their genuine best self to the hiring process, then you are not going to be capturing the optimal signal in that process.
So think about like, well, what are the key components that will allow these optimal conditions to be derived? And that's around the design and the communication of the process. So what are you doing to like remove the barriers that will prevent a candidate from bringing their true self to the process?
What are we creating the best conditions that a candidate can bring to themselves? Do they have, what about the candidate communications, the design of the assessment experience, what reasonable adjustments are available for the candidates? Are they clear about what your expectations are around AI usage? So all these kind of components really do allow a candidate, the clarity around those allow a candidate to be really settled and understand what they can expect to see when they actually get into the process itself. So there are a number of things that we would focus on. So the first being flow state, getting candidates into that kind of space. The research is really consistent that if somebody is really absorbed in a task, their performance is going to be better. Now if you think about our learning agility task or pretty much all of our tasks, those of you that have completed them, you'll remember you get into a of a rhythm of response in the process and you're in flow and there's no cognitive capacity to be second guess in your approach, you're in the moment. So that flow state is really key.
Then make reasonable adjustments the default. Removing friction for candidates who need reasonable adjustments is really important and help them to bring their best selves to the fore in the hiring process. Now shortly we're going to be releasing a new optional feature which is around self declaration of reasonable adjustments. So that should help candidates who may not be as confident requesting reasonable adjustments do that in a way and make sure that their full self has been brought to the process and they're not being suppressed and the signal then as hiring managers we're getting is as full as possible. Then the next couple of areas we've got get the value exchange right with your candidates. So candidates who you know bring the most authentic performance to the process are likely to feel that their time has been respected in the process. They've had an honest exchange through the process itself. And the realistic job preview concept is really useful in this capacity because it could be used to like give the candidate something genuine before you're asking for them to be genuine with you. So that underlying kind of psychological contract between the candidate and yourselves as a hiring organisation starts from more of a place of trust. And then mutual honesty rather than the kind of sense of one-sided extraction, trying to get information from them. And then the fourth area is around purposeful, transparent communications.
At every stage of the process, the candidate really should know exactly what they're doing, why they're doing it, again, what expectations are around AI usage and what reasonable adjustments there are available to them. And a candidate who has been given this information upfront, they're going to arrive at the process in the most settled state they can be. So they can then fully just embrace what they've been asked to do and providing that signal that you can then make a judgment as to whether they're fit for your role or not.
So we have covered and we've walked through the three components. So now we'd like to kind of take a typical recruitment funnel and layer over the components on the stage of the process. So Luke, I you're going to kick us off with first two
Luke Montuori: Excellent, thank you. So I think that learning agility tasks or the skill enable learning agility is probably one of the best examples of this research and product design philosophy in practice. We know that hiring people based on what they know today is something of a fool's errand. The roles are changing, the skills that are required continue to change and entire categories of work are appearing and disappearing overnight.
And so this is why we think that the important question is to focus on whether someone can continue to learn, to adapt, change as the environment around them changes. And so this is what we're measuring with learning agility. We don't measure what somebody knows. What we measure is how that person adapts in real time to new information, integrates that information, adapts to rule changes, and how they perform over time. We're not asking them what their perception of their adaptability is. We're observing their adaptation as it happens.
I think this is why it's one of the best examples because it's so clear that this is a huge difference in the way that you measure individual learning ability.
And it's for this reason that I think this characterises some of the principles that we've talked about. agility is future-focused, it's built for an uncertain world, it's primary evidence. We are collecting this data across trials in real time in a way that is difficult, not impossible, to fake at the moment. And because of this, it's an excellent representation of what we stand for.
Fiadhna McEvoy: So thinking about motivational screen and thinking about the concept of authentic performance within this stage, we've mentioned really realistic job previews this morning. I'm sure a number of you use them at the front end of your process. It's a really effective way of giving a candidate an insight into the role they've applied for and the organisation that they've applied for and get the sense of the values of the organisation. But the question is like, does it have to like remain at that and only be used at that front end of the process? Why can't we integrate it into more fundamentally into other stages in the process. For example, the motivational screen. If you kind of think about what a motivational screen typically looks like, it's an asynchronous video interview or a live video interview or maybe a telephone sift, it's kind of asking more like open generic questions that are in their nature a little bit more susceptible to AI interference as a result of that open generic component.
Speaking to customers and a few like hiring manager friends, there's been situations where in a live video interview they've had candidates, the hiring manager would ask the question, the candidate repeats the question back very loudly. There's a little pause.
And then the candidate starts responding pretty much blatantly reading the LLM generated response. that's not an ideal situation to be in. As a hiring manager, the signal you're collecting there isn't authentic. As a candidate, you're not doing yourself any justice in terms of being able to really project and authentically perform in that situation. So could we change the conditions, for example, introduce more of an RJP component within this motivational sift? So rather than starting with generic questions, could we give the candidate something genuine first? Could we give them a real honest insight into what does a day in the life of this role look like? What are the challenges they'll be expected to face? What are the team dynamics like? What are the challenges around demonstrating success? And then use that core concept to then build the questions and the motivational questions around those.
And I the sequencing really is important in this situation because you are giving the candidate first something honest and genuine before you're asking them to be honest and genuine back to you. And candidates who have that experience are more likely to be...
more authentic in the way they're interacting with the information you're giving. You'll see a sense of energy that they're kind of projecting within the way they're responding and that signal then can be used to help really get a steer for like is this candidate motivated for the role. So thinking a bit more creatively around that.
And one of the other components as well around integrating it into this situation is it's actually a lot more difficult to produce convincing LLM responses if the content is based on something that the candidate has just consumed live in that context. So it's kind of acting as a bit of an additional barrier there. So then the third area is around like assessment center and validity and reliability now.
We are a lot more often, as you can imagine, customers come to us and saying, how do I measure AI skills at the assessment centre? Now, the instinct in this question is 100 % on the money. But I would like challenge the framing of that, because if you kind of take a step back and think in the roles that we have that we're recruiting for at the moment, how many genuinely really need that really hard technical proficiency measurement? Do we need that for all roles? Or if you end up designing an assessment exercise based around that technical proficiency, are you actually going to get what you really want to understand from that candidate? In my view, don't measure AI skills, instead integrate AI into the environment of the exercises in a way that you're really reflecting how you as an organisation are adopting AI, reflecting the level of AI usage required within that particular role. So the AI acts as the context, but what you're actually measuring is human signal. And it kind of reminds me of an exercise I designed earlier on in my career for one of the security services.
Engineering program and they really wanted to kind of bring to life the fact that you know graduates get to play with really cool tools and Raspberry Pi had just recently been launched and it's like a really tiny, small, low-cost computer. What we did is we integrated that into a group exercise. Candidates were briefed, come to the assessment centre with a really creative idea of how you could use a Raspberry Pi at a careers fair to entice candidates. So a little bit meta on one level. So in the room though, the exercise was about the debate. It was about how candidates presented the information, their ideas, how they debated the ideas, how they collaborated, and then came to a recommended suggestion for what comes to the career fair. So the Raspberry Pi was the context, but the human signal was what we were actually measuring, and that's what we're suggesting in this situation. AI is the context, and the human signal is what we're actually measuring. And coming back to what Luke was talking a little bit earlier, being able to measure that primary evidence and see that in a context of a job-relevant task is incredibly valuable. You're not inferring any capability from a CV, you're actually seeing it live in front of you. So we have covered a lot this morning. So we would like to leave you with a bit of a summary and leave you with like three questions really. You can use these three questions to hold them up against each stage of your funnel.
So asking yourself, is this stage, is it future ready? Am I measuring something here that actually is going to really capture what good performance looks like as my workplace evolves? Is it valid and reliable?
Am I focusing more on primary evidence capture or is it more secondary? And really kind of questioning yourself around that. And then finally, do the conditions that I'm creating in this stage of the process, are they really giving my candidates the optimal opportunity to bring their full best selves to the hiring process so then I can capture that really good quality signal?
So if you find yourself answering yes to all of those areas, that sounds really solid, like you've got something that's really trust, signal there that you're able to trust and make really decent decisions off the back of. If any of the questions do give you pause.
That's what we're, come and find us, that's what we're here to talk through. It's exactly the type of conversation that we enjoy working through. So we'll leave it there. I hope, thank you so much for your attention and bearing with us with the clicker instance. We will not talk about it again. And so thank you again and I hope you provided some food for thought.
Hune Lee: Hang on, Fina and Luke. Stay on stage here. I think there's loads of questions that we might want to answer to ask here. So yeah, come back on stage, guys. So folks, there should be a Slido here. So scan this. And we're going to use this all day, by the way. So you might as well do it now. Get your phones out. Just give that a quick scan. And then put the event code in.
We're going to use that for any questions you might have for any of the sessions coming through. And yes, it depends on how fast you are typing. Do you have any questions for Fina and Luke? And whilst you're thinking about that, I've got a few myself. So firstly, very, very pleased to see the of sense of reciprocity that was required when we're demanding or asking candidates to do something. Oftentimes I think that's missing when we're trying to assess for candidates. We ask them to do tons of stuff, we're not giving anything back. The question is like how much do we give and how scalable is that give because you know is it possible for a stall giver, we'd love to get the hiring manager obviously to then give a pitch to the candidates on a per candidate basis before hiring for a thousand people or processing a thousand applications, we can't do that. So do you have a sense as to what would be like a scalable way to reciprocate?
Fiadhna McEvoy: I think it really needs to be based on you as an organisation and what is bespoke and required for and what would fit with what you're looking to measure and what is reasonable and manageable and practicable. Because as we all know, actually, we run a large scale assessment centres, it is a bit of a mind melt in terms of logistics involved in that. So it is about understanding and pinpointing elements of your process where you can see that there's an opportunity to give that real genuine sense of connection with the candidate. Because I think that's a lot of the time, that connection piece, I think, not just within the hiring process, in life more generally now, is missing. So if you were able to establish that really strong connection there, it helps kind of boost that authentic performance piece. But doing it in a way that's practical. And I don't think there's one answer if it's all. So sorry if that was, you're looking for one particular element, but I think it is that kind of understanding the process and being able to bespoke it.
Hung Lee: No, perfect. By the way, folks, that's a little bit of a homework exercise for all of us here, because I think probably in the room, we've got some really good ideas individually as to how we do that. And we'd be wonderful if we could share a few of those at the end of the day. Can you pass me the iPad? I forgot about this. It's very useful device, because this gives me access to the old Slido here.
Okay, let's see whether we've got any questions that people want to answer here. Okay, it's a very interesting one. Let's go with this. Luke, I wonder whether you'd give this a blast. How much weighting should be given to cultural fit?
How much weighting should be given to cultural fit? How are you assessing it, I think is the question.
Luke Montuori: I mean, cultural fit can be assessed in a huge number of ways. Some people just use personality assessments, in which case that's an example of that kind of secondary evidence that can be cheated now using LLMs where you take all of this information about a company, you put it in to whatever your provider is and say, answer it the way that you think that they would like me to respond. And all of a sudden, that's noise, that's unreliable.
The broader question might be about culture generally. I think that's more complex, but really it's about how you collect the data. And until you are sure about the data you're collecting, then it might actually be not worth less, but it's not particularly useful to be umming and ahhing about how to weight things. Zero times zero is, zero times one is still zero, right? Yeah. And it's a tricky, it's a tricky question on cultural fit. know that over the last decade or more, we've kind of moved that into the margins in terms of almost taboo speak. But at the same time, candidates are actually interested in the question. When they're looking at who the hiring manager is, who the team is and all that, they're assessing also for things that they would call cultural fit. So we also need to kind of rehabilitate that term, think, and discuss it, even though it feels taboo. Okay, another question here. Reasonable adjustments can be made and varied, but what constitutes a broad enough list to offer?
So for instance, we want to be inclusive of course, there's going to be lots of people with different requirements, but we've also got practical constraints. So where do we actually draw the line where we can say, know what, we've done as much as we can, but we can't optimise it for every particular case.
Fiadhna McEvoy: I think it's with the reasonable adjustment question again it's coming back to like well what is reasonable for the role itself so if this candidate was to be successful in the role what level of adjustment would they need in the role is that pragmatic is it practical that they're still able to perform the role to the level that you need them to be able to perform and so that that kind of level of adjustment is then replicated through the assessment process
So there's a kind of like an equation or an equality there between what's actually needed in the role, what's feasible in the role and then what's feasible within the reason. So again, I don't think this is one answer. think you need to be given, know, certain conditions are more prevalent in the population. So you need to be making sure that there is, you're covering the main ones, but then you're also giving all candidates the opportunity to discuss their specific individual needs that they require. Because again, somebody might have dyslexia but that might look different for one person compared to another so it's having that open conversation.
Hung Lee: Just being able to provide an option for the candidate, think this is sort of probably we need to elevate that to minimum, but I think that does a great deal from the candidate point of view to sort of on that reciprocative piece. Okay, final question. A great presentation, exclamation mark, which is true. Fantastic stuff, guys. How do you mitigate bias, the risk of bias coming through in the assessments? I think this is a classic question, and I'd love to hear your responses.
Luke Montuori: I'll take that. bias can manifest itself in a huge number of ways. Linguistically, is, think, questions. So the kinds of questions we've been talking about, where it's more about self-report, are particularly notorious for it because of the level of interpretation that's required. it comes to mitigating against bias in our behavioural assessments, our aptitude assessments, this is something that I think is baked in from the validity and like the task development perspective. And then from there on, essentially monitoring things, making sure that the level of bias doesn't exceed standard expectations, where it does improving tasks so that they don't.
It's another one of those things where it's not a single answer. It's a cultural thing, constantly monitoring and making sure that you meet the standards that are expected of society. Quick follow question for me. We know it's a big challenge to mitigate bias. How do we communicate that publicly to the wider community that it's something that we think is important, it's an ongoing sort of thing we're looking at? Is there a best practice to the way we describe how we mitigate bias in our system.
Fiadhna McEvoy: Is there a best practice? That's a really good question. Is there a best practice in the way that we actually describe that? I think as organisations we need to be, I think it comes back to one of the points you made during the talk was around open and transparent and purposeful communications. So organisations may have different levels of focus on bias within their process and monitoring bias, but it's about being clear with candidates about how you go about doing that and I'd also say that in terms of the space of bias
It has been part of hiring conversation from like since I, know, forever essentially. And one of the components is the actual, the human bias in the process and the constant reminder of all the individuals who are involved in all of the different stages in your process, reminding them in different ways and creative ways that so it doesn't become stale of the importance of the individual, their behaviour in relationship to a candidate, how critical that is to managing, monitoring bias. So yeah, I don't know if that's the way it goes towards answering that question, but yeah.
Hung Lee:
No, it's a really interesting one to talk about. How do we communicate it wider? We all need to get better at that. So fantastic stuff. There's actually 13 questions. It's escalating through. It feels like an application for sort of a job application here. We don't have time to go through it all. We'll get to them at some point. So Luke, Fina, thank you so much for your time. Round of applause, everybody.