Most employers now recognise that banning or detecting AI usage is more likely to damage diversity than help your process. But how exactly do you embrace AI? And how can you clearly communicate this with applicants?
HMRC recently published, ‘Understanding Artificial Intelligence in your job application’ for candidates, which provides clear guidelines on exactly how to use AI within the application process while also demonstrating how important it is for applicants to stand out, because ‘using a generic AI-generated response isn’t going to separate you from the crowd”.
Siobhan Stericker, Talent Insights and AI Lead at HM Revenue and Customs is the brains behind these new guidelines that are challenging the way that organisations view candidate usage of AI. Siobhan is responsible for leading an innovative team at HMRC who alongside these innovative guidelines are exploring how they can leverage GenAI within the whole of the talent acquisition process, while also upskilling the wider teams in their use of GenAI to improve efficiencies.
She’s joining us to talk through why and how she created this new guidance that’s helping candidates understand what exactly responsible usage of AI is, and more importantly, the benefit for them of leaving generic responses behind.
Join host Robert Newry and Siobhan as they discuss:
😱 How receiving 800 applications should be a thing of the past: Hear how the volume of applications has become unsustainable for hiring managers, and why Siobhan thinks this new guidance will improve the quality of candidate applications
⭐ Helping candidates to stand out: Learn how HMRC created and implemented its new candidate guidance which helps candidates understand why a generic AI response won’t help them advance in the application process
🤖 AI Skills are the future skills: Gain insight into how HMRC view AI Skills, and why the decision to embrace AI in the application phase will enable them to have a competitive edge with their future talent
💡Why the detection of AI is a risk: Learn why HMRC chose not to use AI detection for application screening, and how they’re striving to create a fair and equitable application process
💥 AI is a seismic shift in technology: Why we should be helping everyone through this technology transition, and how you can start to encourage your team to explore AI tools through gamification
This episode is packed with insights and actionable learnings for talent acquisition leaders navigating the era of the AI-enabled candidate in early careers recruitment.
Listen now 👇
Transcript:
Robert: Welcome to the TA Disruptors podcast. I'm Robert Neary, co-founder of Arctic Shores, the task-based psychometric company that helps organisations uncover potential and see more in people.
And in this third series we are focusing on Gen AI and the impact it is having on candidates in how they apply, but also to explore about how for recruiters the impact is coming to really change the way that not just how they assess, but what they assess for.
And today I’m excited to have guest Siobhan Stericker from HMRC. Siobhan is the Talent Insights and AI Lead at HM Revenue and Customs, which I think is a very interesting job title and we'll talk a little bit about that, but it just shows the way that the world is going, that these totally new types of jobs are being created as a result of AI. And in your LinkedIn profile, you describe this as responsible for data-driven insights on the labour market to inform talent acquisition strategies. I also lead an innovative team exploring how we can leverage Gen.ai within the whole of the talent acquisition process and upskilling the wider team in their use of Gen.ai.
A fascinating and quite broad job spectrum that you've got there, and even more interesting that this is within a public sector body too, not known traditionally for being at the leading edge of innovation. So certainly something that we'll explore. But the other point that I'd just like to bring out as well is that many people will assume that you have a technical background. And what I find exciting about your career journey is that you started life as a trainer. Spending time first with the Army and I know you continue to work with the Army Reserves and then the Home Office training new Border Force security officers.
So a very, very different kind of background coming into AI than most people would assume and as many people know, I'm absolutely passionate about potential and transferable skills and alternative pathways into some of the exciting jobs that are being created as a result of digitisation. And so I love to see this type of career journey, which I think will be an inspiration actually to many people. So welcome Siobhan to the podcast.
Siobhan: Thank you, Robert. I'm really happy to be here.
Robert: And can we start with how you have gone from training soldiers to being an AI lead, and what is the connection and the correlation between those two?
Siobhan: Of course, yeah, I appreciate it's not the most linear career path. So yeah, I've been in for the Army Reserve for 15 years now. I started as pretty much quite quickly a trainer within that environment, and it was something that I really enjoyed. I've gravitated towards helping people and sort of belonging to that wider purpose and I suppose hence why I'm also in the civil service.
So for me, I really enjoyed the trainer role and I started to come into the human resources side of thing and getting to understand the different areas within HR itself. And so I actually applied for the Civil Service Fast Stream, which is our graduate leadership development programme within the Civil Service.
Robert: And there was no prerequisite for that. I mean, that's the, I suppose, the exciting thing about applying for that type of role. You didn't have to have had any previous experience in it.
Siobhan: That's right, and there's a number of different streams. And of course, I was quite set on the HR stream at that point. At the time then, I was a operational trainer for border force. And I actually got that role from my education and training qualification that I received from the army. So had I not had that qualification, plus the operational experience as well, particularly in a law enforcement environment, I'm Royal Military Police by cat badge, I wouldn't have got that role. So it was a really great in into the civil service for me to begin with.
I spent about 18 months in that role and then moved on to my first role on the HR Faststream. And my first placement was in a talent acquisition function. I'll be honest, I'd never even come across talent acquisition before. And so I wasn't really quite sure what I'd let myself in for.
Robert: Except that it's people-based and I can see how that, as a trainer, you have to kind of understand people and you have to think about how you support them and develop them. But talent acquisition is slightly different actually, it's not a development piece. So this was one that just came along for you or was this something that you said, oh, this sounds interesting, so I'm gonna apply for talent acquisition?
Siobhan: No, not at all. I was, as I say, just dropped into the role really. I had some great supporting managers and peers that introduced me to the world of TA. And I...I started to get a feel for the data side of life. I definitely recognized that there was more that we could do with it. And I think because I was quite curious and I was in that mindset of, well, I'm new and actually, I don't know anything about talent acquisition. So I'm gonna ask loads of questions and I'm gonna challenge on why we do this stuff and what's the reason. And I felt quite comfortable challenging to be able to learn for a start, but then also to start to suggest better ways on how we can do things.
Robert: And is that a, because I think that challenging piece is such an interesting one, particularly within TA, because there's always so many things going on and it's hard enough just to get the day job done without then having to challenge the way things get done. And also I think in a public sector body, you can come across as a bit of. Well, you're the new kid on the block. We've been here for ages. We don't need somebody like you coming in and challenging us with, you know, your background from the military. So how did you cope with that? Was that welcomed in there or did you actually find you had to sort of knock some heads together to make change?
Siobhan: I think I came from it more as a position of curiosity. And I think because I was looking at… and I was willing to learn and I was willing to listen to what other people's views were, which of course I wanted to, cause I wanted to learn. I think that's the key thing really is, I was coming in a position of, I'm caring about what we're doing. And I'm curious and I just want to learn. And so the challenge came just from, I suppose my understanding really and building my understanding of what talent acquisition was all about.
I think there's a piece here as well on, I think from my military background and those transferable skills that I've brought in, I'm not afraid to hold my hands up and say, hey, actually, these are my limitations. These are where I'm my strengths, but actually I'm not very good at this. Could you help me develop in this? And so therefore, go in, trying something, trying a new project.
If it fails, then learning from that and moving forwards. And I think that's something key that I've not only do myself, but I pass it on to my teams as well. I sort of create that environment for people to experiment, of course, within certain limitations, but to sort of provide that top cover really of, hey, do you know what? We tried it, it didn't work. Let's try something else, but let's learn from it as well.
Robert: I love that. And that's one of the, I think, such an important transferable skill that often isn't highlighted or celebrated in any way, which is this willingness to try something and knowing that there's a high chance that you may fail on it if you're gonna do something completely new. But that's okay as long as you learn from it. And I think particularly in talent acquisition, we don't have enough of a culture around that, which is why...
I think somebody coming in from a completely different background where trying something new and learning from it is an accepted part of doing things. I suppose is that then what got you when Gen. AI came out? You were sort of, oh, this is something interesting and then jumped onto that or yeah, how did that then come into your life?
Siobhan: Yeah, absolutely. So. Originally, I started off with the contract management side of life and getting to understand our suppliers and who we work with and how we work with them. And I recognized that actually there was more that we could do around our value for money and the metric side. So this is where the data started to really come in to be able to inform these decisions and back those decisions up. That then moved into talent intelligence, which I found absolutely fascinating. And I think…
Again, this transferable skill theme is coming through here because I've come from a policing background in the military and having that investigative mindset. I really enjoyed the market research, the competitor analysis and understanding the labor market. And I would spend hours researching and going down rabbit holes. And I thought that was a day well spent and I really enjoyed it and I still do.
And then of course, November 22, we get the public release of Chat GPT. And I remember my director say, Shavon, have you heard of this Chat GPT? And I went, no, what's that? He explained, I was like, oh, that evening, I was straight on having a play around and that was it. I was completely hooked. And for a while, there was a small group of us that really were exploring and just having a play around and it really started to to dawn on us that, do you know what, there's this potential here, this is gonna be big and there's so much that we can do with this. And from there, you know, it was, this was a, my role was born really in terms of then how we can look to leverage Gen.AI within our talent acquisition process.
Robert: Well, I really like the way that you have a sort of optimistic, it's part of your sort of curiosity, but there's an optimism about U2 where some people I've come across with chat GPT when they first saw it thought, oh, well, I've put some stuff in there and I've got rubbish out of it and I don't see the potential of this and everybody's talking about that, you know, it may take away my job. And so it's very easy to go down a dark angle with AI as opposed to, well, actually, if you use it in the right way, it can be incredibly powerful. And...
And I wonder whether that again is part of that sort of transferable skill bit in there that you were looking at it from a, oh, how could I use this to help? But I'm also pleased, very pleased that you're, you know, you weren't shut down in any way because you could easily see that and probably say, look, this is all too new. We can't be involved in this. But there was an innovation unit that you set up or just came about because of your research?
Siobhan: I suppose it was, I was afforded the opportunity by my line managers and my boss to explore, of course, within the parameters of what we can do. And I suppose we just started off really simply in terms of day-to-day work, summarization, for example, and even just having fun with it as well. I think that was a real sell to begin with in terms of just getting people comfortable with using these tools. So especially when image generation came out and it was just, you know, who can create the funniest image of recruiters in the wild? And we had all sorts of suited and booted recruiters in jungles with leopards behind them and a LinkedIn logo on their laptop. And it was just a laugh and it was just getting people as I said, comfortable using it and no pressure to go, I need to use this for my role. But then by doing so, people then started to understand how to prompt and how to ask those questions and what the limitations of the tools were, but equally where the strengths were and actually then how that can translate into their roles.
So it was very much just starting off as a bit of gamification really. And I think… you know, let's face it, recruiters are quite competitive, aren't they? So I think it was a real winner with the gamification piece and getting people using it. And again, I suppose that learning and development background came in there as well.
Robert: Well, brilliant. And I think that's the moral of the story for anything that's new that's coming out is to just experiment it and try it out. And actually you don't have to become an expert or even use it in a work setting. I mean, I think the first time I used Chat GPT was to create a rhyming ditty for my mother's day card because I'm hopeless at rhyming and she's got a real knack for it. But Chat GPT was brilliant at being able to, and you learned a little bit about playing around with it to get the best out of it, but it is phenomenally capable feel comfortable with using its power.
And just on that note, of course, once you've becoming familiar with all of this, so is the great student population and everybody else out there becoming more and more familiar with using it in their everyday lives. And so now we find ourselves in a world of the AI-enabled candidate as much as we do the AI-enabled recruiter. One of the things that you've been early to put out and as a change to your career side, is to issue some guidance on, and I think specifically you published something called Understanding Artificial Intelligence in Your Job Application. Could you sort of take us through what prompted you to move from, oh, we're having some fun, to, oh my goodness, then if we're having some fun, then everybody else is, and that's gonna potentially cause a challenge for how recruitment can work in terms of understanding the differences between people.
Siobhan: Absolutely. And we're not unique in this challenge that we are seeing quite a significant rise in the volume of applications that we are receiving. Whether that is, of course, down to the likes of ChatGPT and its release in November 22 or any other AI tools. Whether that's cost of living. whether it's just that there's not as many jobs, so there's more applicants. I'd like to say even maybe down to the great work we've done on our employer value proposition. Employer value, definitely. But regardless of what the reason is, we are seeing those volumes and they are becoming unsustainable in terms of being able to sift them.
So it is of course causing a bit of a challenge.
Robert: So just before you go on to that, if you've got any… metric around that or a feeling for that? Is it a third up or doubling?
Siobhan: Oh, it's more than double. Absolutely. Yeah. And a recent example. So we've just finished recruiting a role in my team, actually. Just one role. And, you know, we received in excess of 800 applications just for one role.
Robert: 800 applications for one role?
Siobhan: Yeah.
Robert: Oh, my goodness.
Siobhan: And this isn't unique across not only HMRC, but other government departments.
Robert: So it's a flood. I mean, I've talked about tsunami, but even I never imagined that we get to a place of 800 applications for one role.
Siobhan: Absolutely, yes.
Robert: So big, big challenge.
Siobhan: It is, it is. And part of that reason will of course be that AI is making it easier for candidates to apply in terms of they can have something up and running within a few minutes and there are some sites out there which I know you've talked about previously that can automatically apply. So, and we are seeing that within a couple of minutes of a role going live, we already had a number of applications. And so you think there's surely no way that someone can- Something going on in the background there and it's all applied or AI that somehow picked it up. Yeah, and you just think there's no way that a candidate can have digested the information on that job advert and really thought about their application and submitted it.
And I suppose what we refer to as that spray and pray approach really, isn't it? So of course that's causing frustration amongst our vacancy holders because they're then having to sift those manually and having to take time out of whatever it is that they do in their day to day. And of course, they're then going through these applications and there is still quite a lot of poorly AI generated applications and I think that's the difference there, which I'll come onto a bit more later on. But so they're very much seeing these samey applications.
Robert: Yes, I call it the sea of sameness. It's just incredible, isn't it?
Siobhan: Yeah, and you can imagine, especially, when we're talking hundreds of applications for one role that you see the same structure, you see the regurgitation of the job advert, you see those same buzzwords, and there's no real substance behind the application. There's no evidence on how that candidate has actually used those skills or where it's maybe experienced thereafter as well. There's just no evidence, there's no metrics behind any of it. So it is very much, as you said, the sea of sameness, which is very dull.
Robert: It must be very dull, isn't it? Because I asked… somebody the other day, how long did they spend looking through a CV? And if you had 50 for a role, you might spend a minute or so on that because it wouldn't be too burdensome. But when you've got 800 to look through, I imagine you can't spend more than five to 10 seconds. So at that point, you're not really digesting. what's going on. I mean, I assume it's still an initial as a CV sift is, is part of all of this. And so you're having to in a very short period of time to look through somebody's CV and find something that is either relevant, or there's just a lot of, oh, this is C of sameness, I'm just chucking it out. Did you, if you, the reason I want to sort of dive into this, because I was talking to my daughter about it last week, who's also going through a recruitment process at where she's at 300 applications for one role.
She was saying, and her team that were reviewing the CVs, you end up rejecting it for reasons that aren't necessarily true of what you know about the candidate. They had a spelling mistake in there. They formatted it in a way, or I could see that they used ChatGPT because they had American spelling. Have you found, I suppose, some of those challenges when you're getting these sort of AI-enabled candidates? How are you then managing how you take the 800 down to the 20 that you can interview?
Siobhan: Yeah, I mean, of course I can't speak for every vacancy holder here, but going off that essential criteria, and really it's coming down to, well one, have they met the essential criteria? Yes. And secondly, have they evidenced how they've met the essential criteria? And I think that's where poorly generated AI applications really fall down. And should they then even make it through that initial sift and they go on to interview, then they are gonna be probed on those examples and they will really need to know, exactly. So that's how we're approaching it for the time being and then of course, with the guidance that we've produced.
Robert: Yes, so tell me about the guidance then. How did you come about that? Because you are one of the first organizations to go and produce guidance out there and I imagine you had to do quite a lot of research and also probably quite a lot of stakeholder persuasion as well. So yeah, share with me how you went about creating guidance on this and why.
Siobhan: Yeah, no, absolutely. So obviously we had the conversation on what are we going to do? We need to do something because this is a real challenge for us and there was the, should we ban the use of Gen.ai in the application process? And that led us onto, well, how would we then know that they've used it? How do you enforce it? Well, this is it. And of course there are tools on the market that purport to detect AI and it's, I think it's a risky way to go in terms of, well, as far as I'm aware anyway, there's no real tools on the market that are.
Robert: They're not fully reliable, no. And false positives. Can you imagine public body rejecting somebody who is neurodiverse on a false positive use of software?
Siobhan: Well, this is it. And I think it opens not only us, but any organization that looked to go down that route to legal challenges really, because how can you be telling a candidate that they can't use AI, but then you're potentially using some form of AI tool to then assess whether they've used AI.
Robert: It doesn't seem fair in any way, does it? Yes.
Siobhan: So I think we, and of course we were very much exploring the use of AI ourselves within the talent acquisition process, more, you know, for our recruiters and to enable them. And it was a bit like, well, how can we turn around and say we're gonna ban the use when we are using it ourselves? And not only that you know, thinking sort of future skills here, or even now really, we're gonna expect people in our organization to be effective with these AI tools to be comfortable using them in their day-to-day. So why would we ban the use when we're, you know, we're looking to be an innovative.
Robert: You're embracing it yourself. Yeah, a tech-forward organization.
Siobhan: So it just made no sense to go down that route. So that's where we thought, well, okay, we're not going to ban it. And what we actually done initially was it's just an extension of our plagiarism policy. So I'm sure there's many organisations that will have their own plagiarism policies and we very much took the line of, you know, so long as it's a true and accurate representation of that candidate's skills and experience then fine, why not use a tool that could help them to be more succinct in their writing, to maybe just gather those initial thoughts and get everything together as a first draft for them to be able to then really work through and use it that way.
Robert: Because that could level the playing field. And for people, if you're dyslexic, or maybe you haven't, you don't, school hasn't given you great grammar skills for whatever reason. And so you know, a chat GPT type of tool could be incredibly helpful of just refining and improving your application.
Siobhan: Absolutely, and you know, it was great to see the research that Arctic Shores have done. There's other research as well. And I've, you know, I've certainly spent the time to be able to collate that together to really, again, coming back to the data-driven side to say, hey, look, actually, there's evidence here that this is going to level the playing field for a wider range of candidates. And so I think that's been really key in us being able to drive that forward as well. And we've done a lot of work on diversity on our employer brand and really wanting to expand our diverse talent pool. So it would make no sense to shut off a significant amount of talent.
Robert: By saying you can't use this tool which is gonna level the playing field for you too. I totally agree on that.
Siobhan: Absolutely. And then I suppose coming back to that human element again and the sort of that drive to help people, to coach people, I thought, well actually, the challenge then that we've got is AI is making it easier for people to apply, but we're then receiving poor quality applications. So let's coach people through how to use Gen.ai. Similarly to what we've done with our internal teams, our recruiters for example, we've had these lunch and learn sessions.
We've got people comfortable in using it and guiding, almost bit of best practice really. So let's do the same. And I think there's a real key thing here around, we are a people function and therefore we need to support people. This is, it's such a seismic shift in technology and it's a bit of an adjustment period really. People are still only understanding what it all means. There's some people that still haven't even heard of these tools. And so they're just exploring as much as we did, you know, almost two years ago now. And so we need to allow for that adjustment period, I think. So by providing this guidance, we hope that eventually, I mean, it's only been out for a few weeks but we should hopefully see possibly the application numbers slightly reduce, but then the quality of applications increase. And of course we will be measuring that in the new year as well. So I'm quite excited to see how that does pan out because I think as well we're quite clear in our guidance that don't use it to just, you know, a copy and paste job because you're not going to stand out in the sea of sameness.
And so therefore make sure that it's really representing you and your authentic self. And I think once candidates realize actually, they have got to put some effort into their applications. They can't just, you know, a quick two minute job on an AI tool and they think they're gonna get the role of their dreams, it's not going to happen. And I think once they see that they're not getting that success and they're not getting either the interviews or they're falling down interviews, they'll start to go.
Robert: Yeah, actually, I should have followed that guidance. And I really like your point on that too, that TA is a people function and we need to look at it through the lens of how do we guide and mentor people on that. And that for me is a bit about where I think, you know, the career website is so important. A lot of good work has been done on that one, giving people access to, oh, here's a practice assessment area or here's the type of things we're going to talk to you about.
So it seems absolutely obvious that you should be giving guidance on good and bad use of gen AI. And I think the point that you make there, which is so important, if you don't give guidance then people don't know what they should or shouldn't be doing.
And so that's just totally not fair for them too and I was talking to some UCL students a few weeks ago who said, I just want to know what I'm allowed to do and what I'm not allowed to do on this because then I know where the boundaries are and you talked about that earlier, where the boundaries are on this one because without that guidance then I don't know and that actually causes anxiety rather than just leaving people to work it out. So you then came up, and I think you've come up with a really nice on your website of acceptable and unacceptable, and you've done that both for the application and the interview. Was that easy to come up with those things, or how did you come up with what's acceptable and unacceptable?
Siobhan: I suppose from playing around with the tools ourselves and being both on the exploratory side, but also receiving those applications. I think it was quite, I think it was quite straightforward to put together initially. And of course I did ask others if there was any other additions that they thought. And I think it was really good to include other people, particularly some of the vacancy holders that we've worked with recently, just to understand, you know, was there any other angles, any other insights that actually this would be really good to put in as well.
And of course, you know, again, Arctic Shores, there was some really good guidance there as a bit of a starter for 10, which was really useful. And then just from just wider conversations really with the industry. And I think there's a lot to be said for that networking and understanding what other organizations are doing and what are their challenges and how are they overcoming it. And so from that really, that was how, you know, I put together the that guidance and split it out both for the application and the interview process as well.
Robert: And I think you're right, it needs to be for both those things. And I really like the way that you, at the end of your web page on this, you finished with a sort of final thought on that, and I'll quote it, we value authenticity. If you want your application to stand out, using a generic AI-generated response isn't going to separate you out from the crowd. And I think that's what I really like about that, is that you're making it very, very, it's not just good use and bad use, but you're helping the candidates to understand that A, what you want, which is authenticity, you want to see the real person in there, but also kind of explaining that if you just use the same as what everybody else is, think what that means for me as a recruiter.
I am just going to get a lot of things which I can't see the difference between any of them. So how am I going to decide whether you are the right person or somebody else is? It's something else has to jump out from that. And I suppose it's still sort of early days for you yet. So you don't quite know how that's gonna play out. But have you had any feedback or responses in the short time that's been out from anybody or any of your peers?
Siobhan: So we have been approached by another government department to say, we've come across your guidance, we think it's really great, could we please replicate it? So I'm really pleased that they've approached us and of course I was more than happy for them to do that. Hopefully, we might see some more guidance coming out as well from other departments, but of course that's up to them but equally from other organizations outside of the public sector as well.
Also, I did actually see, or I was alerted to a thread on Reddit. And so people are talking about our guidance there as well. I mean, you can't please everyone.
Robert: You definitely can't.
Siobhan: But there was some really positive comments actually around how we are actually guiding and just the fact that we've taken the time to guide people, of course, there's gonna be some people that might be like, oh, well, maybe you should have this or maybe I don't agree with that element. But I think just the fact that we've bothered to put some guidance out speaks volumes.
Robert: It does, Siobhan, and well done for doing all of that because I find one of the hardest things about being a pioneer, particularly in this very public social media world that we live in now, is that anytime you stick your head above the parapet and do something slightly different from what's been done in the past. All sorts of people come out of the woodwork, some of whom are largely positive because you're doing it for positive reasons. And everybody has got as well their little opinion about how it could be done better. But you rightly point out that we have to make some change to improve things on this. And if you're doing it for the right reasons and it's well researched and it's well thought through.
You can't please everybody, but you are making a big difference. And so I commend you for being so willing to pioneer on that one. And it's always hard with things like Reddit. I try and avoid sometimes some of these things where, you know, you want to get the feedback because you want to improve, but sometimes you look at it and you just go, actually, this is not helping in any way at all. But I suppose on that, do you, having done that, do you think every organisation should be giving some guidance now? I mean, it's having done this and put it out there, would you recommend that every recruiter should have this on their career site?
Siobhan: Well, yeah, I'd say so. Most organizations will already be providing some form of guidance to their candidates on their application process. So it's just adding a few, even if it's just a few extra lines in, off the back of that, I think it's powerful. Candidates are expecting it.
And from the research that Arctic Shores has done as well, they are using it and they want to use it, but some of them are anxious on, are they using it in the right way or are they allowed to use it? So I think just being clear and having that education piece really for people in terms of developing them through the process and guiding them through your recruitment processes. Again, it really adds to you as an employer and it just showcases that actually, you do care about the candidates and you want them to do well. Because let's think back, the whole point of recruitment is to get the right person in that role. So why would you not do everything you can to help that person showcase that they are the right person and maybe not fall? all down the generic AI trap of going, oh, well, this is how you do AI applications now. And actually there could have been a missed opportunity there.
Robert: Great advice, Siobhan, and I think something that, you know, that's important to reflect on, because quite often, we're so lost in the, particularly with the sort of flood of applications, we're just trying to process it all, and you start to forget that there are real people behind those applications, and therefore, one of the roles in TA, there are many in there, but one of them should be how we find the right candidate and encourage the right candidate to apply in the right way.
I suppose the next bit, because we've talked a lot about candidates now, what about, and you referred to it a bit earlier, that, oh, recruiters are using it now. So how are you using AI? I know one of the things quite often candidates want to know too is, well, how are you using AI in the recruitment process? And what are some of the things that you've had to work through in terms of good use and bad use of AI as a recruiter?
Siobhan: We initially looked at the whole of our talent acquisition process and thought, where could we use GenAI? Where is it going to help us the most? And we focused very much on our pain points to begin with and our sort of challenges. And one of those was the time that it's taken, not only the recruiter, but the hiring manager to produce job adverts. Our vacancy holders, they are not recruiters, but they will have to recruit and they will come to our team with, most of the time anyway, a job advert. And one challenge that our recruiters were finding was that there was a bit of back and forth and challenging like the job advert itself, the vacancy holder would either spent quite a bit of time on it and thought, oh, well, you know, you've just given me all this feedback on how I can maybe change it, I don't want to now. Or equally, they've pulled something out from three years and go, oh, well, it worked last time.
So that was a real challenge and a pain point for our recruiters. So we thought, right, well, let's used Gen.ai to help us with our job adverts, as well as maybe our interview questions as well. So we built a tool, myself and the team.
Robert: You built a tool? So you didn't just use ChatGPT in its raw setup?
Siobhan: No, I mean, of course we experimented with that at first and what we found that there was quite a lot of thoughtful prompting going into it. We also drawn upon things like research in bias. We've also.
Robert: That's right, because that's the big worry, isn't it? Absolutely. If you're using ChatGPT and the great unwashed of the internet to come up with interview questions, then we know there's going to be a ton of bias.
Siobhan: Well, this is it. So we wanted to have a tool where actually we could provide it in the background with additional context and sort of system prompting in the background. So, you know, we added to things in like research, our recruitment commissioners principles, our success profile framework, which is what the civil service use as a framework for recruitment. We also included our employer value proposition and our core branding really, that messaging that we are trying to send and as I said earlier, we've done a lot of really great work on our employer brand and we socialized our EVP. So this was a great way to be able to really bring that into job adverts as well.
Robert: So you were creating the content then from which Gen.ai was going to sort of support in how you write a job advert, rather than, so this was an internal tool, or was this something that was still reaching out to the internet to get some information, but then it was pushed through your own internal guardrails, as it were, to come up with something that was suitable?
Siobhan: So it is an internal tool, and it will still draw on the AI model, and we experimented with a few different ones. But then we also just added in this data really, which it was all, it's all open source public data. There was no sensitive information in there, which so that was a real quick win for us to be able to build and develop so quickly and more of a sort of a low risk as well. So we then created this tool, Skillshare we've called it. And it will then generate job adverts and the interview questions that are inclusive and a brand aligned to what we're obviously trying to offer.
Robert: And how did you check that they're inclusive? Because that's obviously the big worry is that something gets created and we know that generative AI can hallucinate as it were.
Siobhan: Absolutely. Yeah, and there is a number of checks that we've done to that. And of course, it won't always be perfect and we produce a lot of user guides that say, hey, it's AI generated content, do check again, it's just a good first draft. You will then need to edit this as well. Absolutely. To put your own expertise or perspective on it. Yeah, but we did use a number of different AI models to check itself. We also did a lot of testing with things like temperature, and I won't go too technical now, to sort of test those outputs as well.
and get the user feedback, because we piloted it first. We didn't just send this out into the open. We took a lot of time to really make sure that it was producing good quality.
Robert: I'm sorry, just to add on there, you used the word temperature, which is, I know, a sort of technical term around this. It's not actually the Celsius or Fahrenheit temperature of the ChatGPT setup, but rather the way that it's, you can set almost a setting, isn't it, for its sort of approach to whether it's going to be positive, negative?
Siobhan: So, yeah, the temperature, so between naught and one, and the closer it is to one, the more it's likely to hallucinate. It'll be much more creative, and it might go a bit haywire, whereas bringing the temperature further down, it will be more stable is probably the right word.
Robert: But it is that these are some of the things that you have to think about and understand if you're gonna be using these tools internally.
Siobhan: Exactly, and we were using that then also in parallel with another tool, Textio, which I'm sure you're very familiar with.
Robert:Yes, really important for checking the languages inclusive, isn't it?
Siobhan: Exactly that, so there was that there as well. And then of course, human oversight.
Robert: Yes can't ignore that or, you know, miss, or I say miss, but it's so important to make sure that that is the final check in a good process.
Siobhan: Absolutely, and this tool then enabled the user to, it would put all this content into a text editor. And again, the user could then edit it in there and use AI as well. So one thing that we put in our user guide for this particular tool was around, maybe that we want to change the tone of a particular advert to appeal to a broader, more diverse talent pool. And so therefore they were able to change it that way. But what that did is because that was the sort of the end state of this tool, it then promoted the fact that you as the user do now need to edit this. And you need to have that final overview and sign off of that particular output.
And so therefore, again, sort of just thinking back to sort of that mitigation in any bias language. I know we can't 100% eradicate, but, and humans inherently bias themselves. So there's a lot to play with, but by putting in those steps, we thought actually that's probably the right way to take. And of course we still do periodically review the outputs and ensure that, you know, that there's nothing untoward being produced as well.
Robert: Sure, and often that gets, you know, thrown out there that, well, humans are biased too, and therefore is, you know, why are we grumbling about whether AI is biased? The big difference for me is that humans are aware of their bias or can be aware of their bias. So it's obviously, we hold separate talk track around unconscious bias, but the self-aware human will be aware of bias on that one. So I think a human final piece of oversight is absolutely right.
One area, just as a sort of final sort of challenge question on this, because I like the way that you talk about using AI in a controlled way for something that's very specific that is well thought through and has lots of oversight on it. There's, and I'd just be interested as a sort of general question for you as a sort of subject matter expert on this. There's a lot of discussion about people using AI to screen CVs and application forms. With what you've learned in the last year or so, what do you think about that? Because it's very tempting. I get 800 applications. I've not got the time to do this. So I'm just gonna put all those applications or those 800 CVs into chat GPT and ask it to rank them.
Siobhan: Yeah, and that is absolutely a challenge. And of course, it's very contentious subject as well, isn't it? There's lots of pros and cons and it's trying to get that right balance, isn't it? I think any decision to go down that route will need to be thoroughly thought through this, you know, there's a lot of people or teams that will need to be involved in that, you know, right from your legal teams, your AI ethics boards, your data scientists. There's multiple people that would need to really be on board to work that through and it's I'm not sure if there's any organization that's done it successfully as of yet, or at least have sort of been able to sort of show their success. But it's, I mean, it's a potential option, it's a potential use case, but I think it will just need to be really thought through going forwards.
Robert: And that's great advice, Siobhan, on that one. Because I think it's not just, that's not just something that's important for you as being part of a government backed organization. But if, and this is the point around this, if you're going to use AI to make a decision as opposed to create a job advert, nobody's impacted if that job advert, other than you can correct it. But nobody's impacted in whether they get a job or don't get a job because of the way somebody's constructed an advert. But if you use it for a CV to then say, okay, this person is allowed to come through and this isn't. There is a decision that's being made there. And so anything around AI where a decision is being made, you want to be really thorough. I think that's your advice on this in getting legal you might want compliance. There are various other stakeholders that you'd want to get together that just say, before we do this, we need to be 100% sure we're doing it in the right way for the right reasons.
Siobhan: Absolutely, and AI is, there's not just any gen AI, there's so much more to it than that. And so actually, it's really thinking what is the best use for this particular use case? It might be a mix of different technologies as well that that will be the one and only solution. And I suppose it really comes back to the whole point of responsible AI. And I think by instilling that into an organisation and to candidate. So if I link this back to where our guidance is, in essence, we're getting candidates to be responsible in their use of AI, because they could potentially be our employees one day. And so by having that mindset right from the beginning of we need to be responsible with AI, hopefully that will then that will be instilled into everyone so that any other applications and use cases, responsible AI will be at the forefront of that as well.
Robert: That is a wonderful way to finish off the discussion. So it's a term that I've heard banded around responsible AI, but you've brought it to light in such a well-defined and practical way. And I think ultimately that's where we need to is how do we use it responsibly. Siobhan has been fascinating to have you on the podcast, really enjoyed the discussion.
Siobhan: Thank you very much.
Robert: Thank you.