Ask a room of TA leaders whether they use automated decision-making, and the answer is usually the same: "that's not us, we've always got a person involved."

It's an understandable answer. It's also the one most likely to catch teams out, because the law in this area has already changed. Since February 2026, using tools to score, rank, or sift candidates at volume sits squarely inside a set of rules that most hiring teams haven't yet mapped onto their own process.

The good news, and it is good news, is that nothing here is banned. The rules are an invitation to do what strong hiring should already do: make decisions you can explain, defend, and trust. The teams who get ahead of this now will be in a far stronger position than those who wait to be told.

To make sense of it, we sat down at our first TA Disruptors Summit with Natalie Farmer, a data and technology partner at the law firm Fieldfisher, in conversation with our CEO Estelle McCartney. Natalie spends her days advising organisations on exactly this. Here's what TA leaders hiring at scale need to know.

What's changed in the law?

For years, automated decision-making in recruitment sat under a restrictive regime. The rule of thumb was that you couldn't let a machine decide, unless a narrow exception applied.

The Data (Use and Access) Act, in force since February 2026, flipped that. Natalie described the shift in plain terms:

"It took it from a prohibited regime, where we say you cannot take automated decisions, to a permissive regime where we say you can, absolutely. It's now permitted with safeguards, as opposed to prohibited with certain exceptions."

She was clear about the significance. This is one of the biggest changes to hit hiring since GDPR landed in 2018. The regulators aren't trying to dampen the technology; they want it used in the right way.

Sitting alongside the Act is new guidance from the UK's Information Commissioner's Office, the regulator for this area. A draft landed in March 2026, and the final version is expected around winter, though Natalie was candid that the date may move. In the EU the picture differs again, with the EU AI Act treating candidate evaluation as a high-risk use, but the direction of travel on both sides is the same: transparency, fairness, and trust.

Does being classed as ADM mean you're using AI?

No, and this is where a lot of confusion sits. Being in scope turns on how a decision gets made, not on whether AI is involved. A tool that scores or ranks at volume is automated decision-making with or without any AI behind it.

That distinction matters for a practical reason. The rules ask whether you can explain a decision, show it's fair, and stand it up if a candidate challenges it. That's a straightforward question to answer when your process rests on validated measurement you can open up and account for. It's a much harder one where the logic sits inside a black box that can't be fully unpacked.

Our own research shows how little trust the opaque options have earned. When candidates and hiring managers were asked how much they trust different parts of the process to predict performance, AI-driven CV screening came last on both sides, at 28% each. The tools most exposed by the new rules are the ones that can't show their working. The tools best placed are the ones built to be explained from the start.

What safeguards does the new regime require?

Where a process involves automated decision-making, a set of safeguards is what makes it lawful. Natalie walked through four, and they map neatly onto a hiring funnel:

• Tell candidates what's happening. Before and at the point of decision, be clear that automated decision-making is being used, what data it draws on, and why.
  

• Let them add context. Give candidates a real opportunity to provide information that could change the outcome, rather than a form that goes nowhere.
  

• Offer a human review that means something. On request, a person with real authority looks at the individual case and can change the result.
  

• Allow a challenge. Give candidates a clear, documented route to formally contest a decision, with timelines and records kept.
  

On that third point, Natalie was reassuring about what a compliant human review actually involves:

"It does not mean you are doing a forensic deep dive on every aspect of the decision. It's about being able to present the individual with the logic: this is the assessment and the criteria, this is where you scored or ranked, and this is why."

Her wider theme was that transparency pays for itself. Teams that explain clearly at the front end face far fewer complaints at the back. Most disputes she sees come down to candidates feeling misled, confused, or kept in the dark, and clear messaging early heads most of that off.

What about bias and discrimination?

The new rules don't sit on their own. Natalie pointed to a convergence of data protection, employment law, and the right to be free from discriminatory outcomes. If an automated output produces a discriminatory result, the organisation deploying it can attract liability much as it would for a biased human decision.

Her advice followed from that: treat the outputs of your tools with the same scrutiny you'd apply to a human decision-maker, and work with vendors who understand these issues and build for them. It's another reason the ability to test a tool for fairness, and to explain how it reaches its conclusions, is quickly becoming the thing that separates a safe choice from a risky one. 

What should TA teams do now?

Natalie's headline message was calm: you can do all of this, the technology is there to be used, and the law doesn't prohibit it. The work is in doing it properly. A few practical starting points came out of the session:

• Map your process to see where automated decision-making applies.
  

• Bring your legal team in early.
  

• Update your data protection impact assessment to reflect ADM use.
  

• Refresh your candidate communications so they're transparent about how decisions are made.
  

• Ask your assessment and recruitment vendors how they're preparing to support you, and how much of the logic they can actually explain.
  

There's also a timing choice. You can move now, working from the draft guidance, and be ahead of the curve, accepting you may fine-tune once the final version lands. Or you can wait for the final guidance and implement once, with the full picture. Both are defensible, and your legal team is the right partner for that call. What's harder to defend is assuming none of it applies to you.

Key takeaways

• The rules flipped from restriction to permission with safeguards. Since February 2026, automated decision-making in recruitment is lawful, provided the right protections are in place.
  

• "Automated" now catches far more than teams think. If a score or ranking makes the call and no one can meaningfully overrule it, that's ADM, even with humans elsewhere in the process.
  

• ADM is not the same as AI. Being in scope depends on how a decision is made, not on the technology behind it.
  

• Explainability is the dividing line. Tools built on validated measurement you can open up are well placed. Tools whose logic can't be unpacked are the ones most exposed.
  

• Transparency is a trust-builder, not just a compliance task. Clear communication up front reduces complaints and earns candidate confidence.
  

• Start now. Map where ADM applies, involve legal, update your DPIA, and press your vendors on what they can explain.
  

Watch the full session

This article covers the essentials. The full session with Natalie Farmer and Estelle McCartney goes deeper, and includes the audience questions on keeping pace with vendor product updates, what a proportionate approach might look like, and how anti-discrimination law fits in.

Watch it below. 👇

The direction of this regulation rewards exactly what good hiring has always needed: measurement that's valid, fair, and easy to explain. If you're working out what it means for your process, and how to build one that stands up to scrutiny, that's a conversation we're glad to have. Click here to speak to the team today.

Transcript:

This transcript is AI-generated and may contain mistakes.

Natalie Farmer (00:00.16)
In the UK, have obviously Brexited, we've got the same data protection regime in the UK, but now the discretion to change it. We've just started seeing those changes. So earlier this year, we had a big landmark piece of data protection legislation in the UK, the Data Use and Access Act, which updated our UK version of the GDPR. One of the things it did, and it didn't do a lot, it wasn't looking to overhaul that piece of legislation, but it looked at automated decision-making rules. And it made a really important change.

It took it from a prohibited regime where we say you cannot take automated decisions to a permissive regime where we said you can, absolutely. In fact, we want you to use this technology. That's the signal I got. But we want you to use this technology safely and fairly. And so it's now permitted with safeguards as opposed to prohibited with certain exceptions.

I'm sure come on in more detail to what those safeguards look like at the moment, but that is one of the core pieces of legislation that impacts our use of technologies to take decisions. That's definitely the sort of UK. Do we want to talk about EU at the same time? Go for it. Right. So EU is a bit different. We have the old GDPR in the EU. If you are pan-Europe, then you have to comply with now two diverging regimes. So the EU retains that less permissive regime, that prohibited regime that says you can't do it unless you fit into an exception.

One of the exceptions is you've got consent of the individual. So it's not a no, it's just trickier. The same safeguards then apply. So actually, while these two regimes now look a little bit different, we're still pulling towards those same principles of transparency, fairness, control to the data subject, instilling trust in those decision-making processes. Now, the UK doesn't have a separate piece of legislation governing AI, whereas we know in the EU, we've got the EU AI app.

So it's not surprising that the legislators in Europe didn't really look at the GDPR the way that they have done in the UK because they've got a whole other regime that you have to think about for Europe. So in the EU, the AI Act tells us that certain systems are considered high risk. And one of those system classifications is the use of AI systems, AI technologies, to take decisions that affect employment-related relationships. And they include, as an example, candidate evaluation.

Natalie Farmer (02:20.814)
Again, high risk AI systems sound scary and they are regulated relatively highly, but from a policy perspective, they are looking to achieve exactly the same things. The EU AI Act can be considered a piece of safety legislation, essentially, not that different to safety components in a car, the way that they're regulated. You can put it on the road, but it's got to be safe. So there is a sort of detailed list of requirements under the Act that says you can definitely use the tech.

but just make sure that you're using technology that is robust, that you can audit, that has a logic behind it, and that you can ensure will be free from bias and discriminatory outcomes, or at least tested for that purpose.

Estelle McCartney (03:02.466)
So the questions move from can I use ADM? No. It's now, yes you can, but you've got to make sure that your processes meet the requirements of legislation.

Natalie Farmer (03:16.782)
Quite right, I think we've shifted, I think we're going with the idea that the regulators aren't trying to dampen this technology, but they want it to be used in the right way.

Estelle McCartney (03:26.934)
Okay, so I guess can we move then maybe to I think the question that might be on people's minds in the room and maybe where part of the confusion lies. So when people hear this phrase EDM automated decision-making I guess most people are kind of picturing a bit of an extreme version of that so I don't know you take a CV and put it into something at one end and out it pops and there's absolutely no human intervention there and

You might be there sitting thinking, well, we're not doing that. So we're fine. We're not using ADM. But my sense from what you've said, is that actually the ground has kind of moved with this new legislation. So there's two things I'd love you to take us through now. The first is after the new act, the Data Use and Access Act and the changes at land, what counts as ADM now? Like, how has that definition shifted? And secondly, for the...

tools that many of the people in this room are using, assessments included, where does the guidance land today and what settled and where is there still a bit of uncertainty?

Natalie Farmer (04:35.532)
Yeah, that's a great question because I think it's an area of the law that's a little bit misunderstood because when we think about automated, we think solely automated. In fact, the law actually uses the word solely. And now we have a situation where in the UK, that law has been clarified to say that solely automated means the absence of meaningful human involvement, not the absence of human involvement. So why they ever use the word solely is a little bit beyond me, but they've now clarified, I think, because the technology has shifted.

The idea that something is end-to-end automated is a little improbable. In fact, the reality is that there is normally a human element and it's a qualitative assessment of that human element. Now, where the legislation is a little bit vague, but where the guidance now steps in and picks up in some of the grey area, is what exactly meaningful human involvement is and whether we can draw any bright lines to assist people in figuring out when this does or does not apply.

Historically, a lot of time was spent trying to avoid the application of the rule, because if you could just say it's not an automated decision, great, the story ends there. And frankly, given the way the law was previously drafted and the lack of guidance, that was a fairly robust approach. Now we've got guidance that says, OK, we're talking about meaningful human involvement. What does that look like? Well, it's not about end-to-end automation. It's about whether you are only including the human involvement at the beginning and perhaps not at the end.

if you are essentially using something to narrow a large pool of candidates down, filter out, score and rank, and only have a certain percentage proceed to second phase, the regulator sees that as an absence of meaningful human involvement. But again, not that you can't do it, just that you ought to do it compliantly. The EU has a slightly different situation. They don't have that level of clarity, but there is some case law that...

I won't go into the detail, but the case law essentially gets you to the same place. That's very recent case law. And in a way, it's of quite timely because the UK were going in one direction and the case law in the EU has kind of led us in the same place.

Estelle McCartney (06:39.994)
So I guess that kind of reframe matters enormously, not just legally, but strategically as well. So for every tool that's scoring or ranking a candidate volume, and this is not a niche issue just for one vendor, it's a shift across the whole category that probably most of us haven't fully mapped yet. And Natalie, you referenced some guidance. Could you say a little bit about where that guidance has come from and what the status of that guidance is?

Natalie Farmer (07:08.846)
So the UK, we've got a regulator in the UK called the UKICO, the Information Commissioners Office. It's a pretty pragmatic regulator. It likes to engage with stakeholders and industry. And it saw, I think, the divergence of the technology, the sort of employment law issues, the lack of an EU AI Act in the UK, and the changes in the UK regime as an opportunity to look at the use of automated technologies in recruitment specifically.

When it looked at that, one of the things it identified was that there was some understanding of the rules, but perhaps there needed to be more guidance around the intentions, the policy intentions and how the law was meant to land. And that guidance has helped enormously understand really what the scope of the legislation is. We've got draft guidance at the moment that really is answering a lot of the questions that were presented through that stakeholder review that they did. We should get final guidance in the summer.

And it helps us a lot because it tells us what an automated decision is much more clearly than we had before. And that does include things like certain rankings and scoring and screening where there is an output that essentially doesn't involve meaningful human involvement, especially sort of field narrowing tools. Now, a lot of where this lands is in how you use technology. So it's not a one size fits all and the draft guidance doesn't

identify any specific technology that is in scope. So we always have to go back to basic principles. But what we have now, I think, are much firmer rules of the road, which is really helpful for us in this space.

Estelle McCartney (08:48.216)
Brilliant, thank you. you've painted the of the landscape, the intent, the mechanics of it from the UK and the EU. We've got a room full of TA leaders who are probably sitting there thinking, my goodness, I'm not quite sure about all of this. And I think some of them might sort of walk out of room, hopefully not right this instant, but when they go back to their office, you know, gosh, I need to speak to my legal team or get some advice. Could you give them some

practical advice, like what should they be doing next and how urgent is it?

Natalie Farmer (09:25.506)
Yeah, of course. So my main message is you can do anything. There's nothing that's prohibited entirely. When we're using technology, there's enormous commercial upside. It's just about ensuring that it's done properly and compliantly. And there are some things you can do. There are some things you can spot. And it really depends on who has the conversations in your business with the technologists. But there'll be tech vendors out there who are on top of this.

and in my experience there'll be a large number of them who aren't. When it comes to complying with these rules, it's firstly not entirely for you to do on your own. You will need to be supported by your vendors. It's a bit of a partnership because you only have half of the picture. There's the tech that's plugged in and there's the outcome and what you do with it. So you're looking for vendors who can support you in your compliance and your lawyers will want to do an appropriate amount of due diligence on the vendor to understand whether they've thought about these issues.

There is a shift here in thinking from ADM rules don't apply to yes they do and we need to do it right. What that looks like, there's an acronym I use which is TRIC. So the requirements that you need to bake in are transparency, we need to have the ability to represent. So the data subject is able to make representations about what they think needs to be factored into an automated decision. There needs to be scope for intervention.

and there needs to be the ability to contest a decision, so TRIC. Your lawyers will know about that and they'll expect your vendors to understand what that looks like and how that can be implemented in your business as well. So one of it, the starting point is you can do anything you want, decent legal due diligence of your vendors, hope that your vendors are partners with you in that process, hope they're working with you and that they've got a sense for what all of this means. And then I'd say lastly, think carefully about what you are saying to your candidates.

and what trust you're attempting to build with your candidates. In my experience, the better you do that part of the project at the front end, the less aggravation you'll have at the back end. Individuals are more likely to complain, and we see this because I deal with lot of access requests that really go to the heart of this, where ADM hasn't necessarily even been acknowledged. Those sorts of complaints are, I didn't understand, I feel misled, I didn't realise that this was going on, why was that technology not explained to me?

Natalie Farmer (11:51.724)
I feel confused. You can beat that away very quickly by having good messaging, decent levels of transparency at the front end, and it really pays off.

Estelle McCartney (12:01.362)
Thank you. your trick analogy, which I like, is for safeguards that are set out in the legislation. One of those safeguards gives the candidates a right to a human review.

What does a compliant human review actually look like? Because before you got here at lunchtime, the subject has been about volume, number of applications, et cetera. So we're kind of all here going, gosh, what does that look like in practice? I know that the guidance is still under consultation, but could you give us your take on what would meet the requirements for that human review piece?

Natalie Farmer (12:46.794)
Yeah, it's tricky, isn't it? Because we've got a piece of legislation that says use the tech, you can do it, but you have to let people have human review. And does that necessarily undermine all of the efficiency that you got from using the tech in the first place? We are in a situation where I think we're going to see the guidance evolve here. We're also going to see practice evolve. And we have to pay attention to both. My view is that human intervention or human review

does not mean you are doing a forensic deep dive on every aspect of the decision made. It's about being able to present the individual with logic and being able to say this is the assessment and the criteria, this is where you scored or where you ranked and this is why. But that is not giving away proprietary information, that is not compromising IP trade secrets. So the GDPR is quite clear, there are rights that individuals have but they should not compromise the rights of others.

and that can be other individuals or businesses. So you need to be able to, in order to stand up a human review process or this idea that you have a right to intervene as a data subject, you need to understand how your decision process was meant to operate and whether you've spotted any flaws. Or can you go back to the individual and say, this is how our system is intended to work, and on this occasion it worked as intended, this is where you scored or ranked, and this is the outcome.

And that is what it is, as opposed to, okay, there was a flaw. We missed a piece of information. Your technology, when you were using a particular testing apparatus, failed on that occasion. You were timed out too early. We see that now. Yes, it was an automated output that said, hey, you failed this test, or whatever it is, but actually you've brought to our attention the fact that there was a technical error, and we're happy now to think about redoing that, or providing you at least with an explanation of what happened.

It's not as scary as it sounds, but it does require an understanding of the technology you use, an understanding of the logic involved in the output, and being able to talk very plainly, concisely, and transparently in lay terms to individuals about what that looks like. And again, it brings me back to what I said earlier. If you're providing that sort of information almost at the front end, you're very unlikely to get the same level of questioning at the back end. You're always going to get one disgruntled individual who feels hard done by. But if you can say, we've got some

Natalie Farmer (15:07.756)
very clear messaging here about how this works. Happy to look at whether it worked in that instance. And if it does, that's the review done.

Estelle McCartney (15:16.078)
Brilliant, thank you. Final question, we've thrown quite a lot at the room today. If you've got one piece of advice or kind of one moment to take stock, what would that piece of advice be? And I suppose, there any parallels with this that you've come across?

Natalie Farmer (15:33.784)
Yeah, think one is, I mean, I've said it already, but you can do all of this. The tech is there to be used and the law doesn't prohibit you from using it. But also you do have, this is a good moment to take stock because we're seeing a change in the law in the UK. We've got focus in the EU on the EU AI Act, which isn't enforceable until December of next year. So you've got runway in the EU to comply with the EU AI Act. You have a slightly different landscape for the GDPR.

You've got UK legislation just changing and guidance not fully landed yet. So now is the time to go and think what tech do we want to use? What vendors support as well? What are our legal team saying? And how can we sell this internally and get the most out of the tech that we use?

Estelle McCartney (16:16.686)
Thank you, Natalie. So obviously the compliance piece matters enormously, but I think you'll have gathered from the conversations and the report that's on your tables that, you know, our view is that the prize is actually bigger than compliance. you know, candidates trust processes that they understand, hiring managers trust outcomes that they can defend and regulators, whatever the final guidance looks like, are looking for evidence of

good faith and sound judgment. And so if you can bring that thinking to this lens, not just as a compliance, but actually how do you also use this to build the trust and to have that transparent, defensible, explainable situation. And just to close up before we move to a couple of questions, obviously from Arctic Shores' perspective, as you can tell, we're fully engaged with this. We've been...

speaking to the ICO, we've submitted to the consultation that they are running, we are building out the guidance and the content and the cons that you will need for this. And so please do come and speak to us as you start to get your heads into this and navigate it and we will be there to support and guide you as well. Natalie, thank you so much for sharing your amazing expertise and insights.

Natalie Farmer (17:45.538)
All right, brilliant stuff. I flashed a slide out early because the questions that we have.

Estelle McCartney (17:47.966)
asked Ben to see the slides, I'm sure there's tons of questions for Natalie in particular here, so I'm just going to go straight to the see

Natalie Farmer (17:56.648)
them and see where we're at. Whilst I get

Estelle McCartney (18:02.124)
the questions up, Natalie. I got a question.

Natalie Farmer (18:04.43)
sort of from myself. We're reasonably smart people in the room, I reckon. Ever keep pace with product innovation from the vendors that we use. So what if we kind of interact with the vendor, they pass our tests, yes, they're compliant, we love them, in, but their product iteration is so fast, how do we make sure that they stay within compliance?

Estelle McCartney (18:10.574)
But we're not going to.

Estelle McCartney (18:30.102)
Is that something we

Natalie Farmer (18:30.798)
just got to trust there are not, is there a way in which we can do this? That's a great question. It's incredibly difficult to do it well, I think. You've got, I mean, you should have when you're on board a vendor, there's certain things that you'll do, I mean, it's a bit boring, but contractually, to be able to keep a bit of an eye on that, where their technology moves sufficiently, I would expect a good vendor to be coming to you and saying that these are features that we think you can enable, and we think you can enable them.

Compliantly and then you'd be in a process of a really due diligence of a feature Where there are updates to products that happen automatically if you're unaware of them It can be quite tricky and you can get caught out. We see this particularly with Technology that say compliant in another jurisdiction and particularly my experiences with clients in the US They'll be moving very quickly with a know Legislative environment that it looks very different to the one we're in

And there I would expect your legal team to have some way of checking in with them occasionally and saying, there anything we need to know? Normally a vendor will help you produce something called a DPIA or a data protection impact assessment. And that is your sort of regulatory document that says why you think you can use the tech. I mean, to put it sort of plainly. If there are changes that would update that document, you'd want to be able to go back to the vendor and say, help us fill this out.

So again, what I was mentioning earlier is being in partnership with your vendors is really important. Work with vendors who are aware of these rules and who can flag things to you. And then the fallback is your legal teams checking in occasionally and saying, is there anything we need to know? Do we need to update our DPIA? Yeah, great. A question here, already for folks who are just listening in. Guidance, what is your view of candidate management?

Estelle McCartney (20:13.016)
Thank you.

Estelle McCartney (20:17.779)
In light of ICO.

matching and ranking tools in a world of mass applications to help recruit.

Natalie Farmer (20:23.842)
needed to produce SIF candidates. So I guess this is about the narrowing. We've all heard that yes, a human is in the loop, but what if we never see the candidates that were... How does all that fit in? Yeah, this is where we see the sea change. I think it's clear now from the way that the ICO is posturing itself that if you are essentially...

Estelle McCartney (20:26.958)
feel aspect.

Estelle McCartney (20:34.574)
sort of deranged

Natalie Farmer (20:47.438)
hiding from view, if you like, a group of candidates who never really get surfaced for second level review or whatever the level of review is that would actually have that human, meaningful human review involved, then you were in ADM territory. But that is not, again, it's not the end of the road. So you may be engaging the MDM doors. Previously, there was a much more, I think, creative way of looking at whether ADM was engaged and human in the loop could mean humans at the beginning of the process, designing the process.

being on the back end ad hoc reviewing the outputs, now it's much clearer that ad hoc review, humans involved in design process are insufficient to cut it or insufficient to take you out of ADM. Any sort of candidate screening, filtering or suppression if you like will be ADM because there is no individual human review of an output that determines that this candidate does not move on to the next round.

and that's what they're trying to capture. What we don't yet see in the guidance, but I think we'll see in practice and possibly be reinforced later on, is a proportionate approach to the way that gets regulated. There is definitely a recognition, I think, in the UK that tools that allow for great efficiencies when it comes to high volume candidate review should be allowed to evolve, should be allowed to provide efficiencies to businesses.

but there would need to be a proportionate approach to their regulation. And that includes when it comes to things like intervention in human review. And I think we will see that coming out. We haven't seen it, I think, explicitly enough yet. But my guess is that we will. And when I talk about proportionality, that's baked into the legal concept. Things need to be proportionate. So if we are talking about latter stage review, where you're getting a machine to essentially do a very qualitative assessment of a high value candidate,

instead of any human review at all. I'd expect the regulatory scrutiny of that to look a little bit different to the sort of volume and filtering that we get maybe at the earlier stages. The guidance never provides us with that level of specificity, but I can expect to see that coming out through maybe some decisions, potentially case law, and perhaps sort of additional guidance that we see tacked on on the sides. Yeah, really interesting. I'm getting more...

Estelle McCartney (23:07.064)
Where is...

Natalie Farmer (23:09.91)
the opposite

Estelle McCartney (23:12.406)
so many things that I'm just thinking of like very well known

Natalie Farmer (23:15.182)
tools that we use, like the LinkedIn's of this world and so on. It's like, okay, what does it mean when you've got an AI system recommending candidates and pipelining candidates for you? I'm presuming we're never gonna see the ones that they didn't pipeline. So yes, lots of big companies might be following a lot of this. Okay, another question from the crowd. What metrics would you encourage vendors?

Estelle McCartney (23:21.334)
and reading a bunch of can-

Natalie Farmer (23:39.054)
to be documenting in order to be able to explain that there's meaningful human involvement? Yeah, so, I mean, in terms of the vendor side of things, it's a bit of a tricky question because when we think about what the vendors do, we might be thinking about the technology that they put out there that allows for a particular output that may or may not be subject to meaningful human review.

So as I mentioned, this idea of when the human review happens is now really relevant. The design of the system, the design of the technology, which inevitably involves humans thinking carefully about the calibration and the weights and the way the algorithm works, is now less relevant than the output and whether the output is reviewed by a human or if the output, which is you scored X and that's below threshold, therefore you don't move on to second round. That's the important part. That's where the human review sits.

Now if you have a human looking at every output and judging it themselves, you're not talking about automated decision making, but that completely undermines the efficiency of the technology. So the main message here is if you've got a vendor who's got a great piece of tech and saves you lot of time, resource, money in your decision making process, that might be worthy of the investment on the compliance piece, which actually goes back to GDPR principles and isn't as scary as you think.

In terms of the actual vendor itself, what I think the better question is, or maybe the more helpful question is, what is that vendor doing to tell you about the logic so that if you get an individual requesting intervention and human review, you can actually present them with a set of facts that's sufficient to check that box and say that you gave human review a good shake. There, I think that it involves good, transparent output from the vendor that you can then utilize.

as a potential employer, which explains that we have potentially worked with the vendor to set certain parameters. This is how we understand the waiting works. This should not be overly technical, by the way. In fact, the legislation requires conciseness and brevity, but so that they can understand that if there has been something missed, if they think there's a piece of information that they should be able to supplement, they have the opportunity to do that. So I'd be looking for good collateral from the vendor.

Natalie Farmer (25:58.286)
that you can read as the customer that allows you to understand how a decision gets made so that if you get the question, you can be responsive. Yeah, great. OK, final one from the crowd. Where does anti-discrimination legislation come into this landscape? I guess we're talking about adverse impact and stuff like this. Things may, like you might produce outcomes that are within the letter of the law with regards to GDPR and so on.

Estelle McCartney (26:15.63)
for instance

Natalie Farmer (26:27.192)
but actually it's producing these outcomes that you think actually this is bad. Yeah, this is one of the reasons I think that the UK regulator decided to scrutinise this area of technology use in particular. And it's because there's a convergence here of data protection, employment and fundamental rights. So the right to be free from biased outcomes, from harms, from discriminatory harms. We obviously have a completely different regime in the EU that tries to tackle those under the AI Act. In the UK,

the outputs of a piece of technology to the extent they cause harm, arguably will attract liability in the same way that a human decision could if in fact that that decision is determined to be discriminatory. So we have a little bit of a hole in the legal landscape at the minute around liability for automated outputs. If you go back and, sorry, this is a little bit legally technical, but if you go back to of basic legal principles and we think about liability, when you use an agent, for example,

you might be liable for the actions of your agent where they're acting on your instruction, where they're doing what you told them to do. We'd have to look at tech in a similar way, but it's potentially a little bit different because it depends how truly autonomous the technology is, whether it's doing what you asked it to do or acting roguely. So if you think again about the EU AI Act, it's a piece of safety legislation, as I mentioned, some of the requirements under that piece of legislation are to stop AI going rogue.

and taking decisions that could cause harm, that could be discrimination, that you couldn't have foreseen, where liability wouldn't necessarily attract to the deployer because it was not foreseeable. So in the UK, I think when we're talking about ADM, we should think carefully about liability because it's relevant, but ultimately you as a deployer of technology should be thinking about the outputs in the same way that you would think about human review if it causes discriminatory outcome. That's why you want to work with trusted vendors.

who can talk to those issues and who are aware of those issues and are baking them into the creation of their tech. Yeah, fantastic. I've got a final question from me. Vibe coding your ATS, what? I mean, I think vibe coding is a no-no anyway, but I'm a lawyer, so you're probably asking the wrong person. Give it a go, see what happens. Yeah, fine. That's not legal advice. That's not advice. That's brilliant. Okay.

Estelle McCartney (28:34.648)
That's a no-no.

Estelle McCartney (28:50.126)
and

Natalie Farmer (28:50.562)
Natalie and Estelle, you so much. Round of applause for these two.