Howdy Recruiters!
The bajillion AI recruitment vendors on the planet tell us the same thing.
"Don't worry. There's a human in the loop."
It sounds reassuring. It's supposed to make you feel safe.
"oooh human in the loop, its ok, theres human in the loop, don't ask questions...human in the loop!!!!"
And for a lot of companies right now, it is COMPLETE NONSENSE.
Because most people, including the vendors saying it, have no idea what "human in the loop" actually means in law.
Or what it takes to prove it. Or what happens to your company if you can't.
This issue is your practical guide to where you actually stand. its a big boring bastard of a newsletter but its necessary, so strap in, or just save it for later
I'm not a lawyer. Nothing in here is legal advice. But this stuff is your job to understand, and I'm going to make it as clear as I can.
The Law: What It Actually Says
Let's start at the top.
GDPR Article 22 (EU and UK) says that individuals have the right NOT to be subject to a decision made "solely by automated processing" that produces a "legal or similarly significant effect" on them.
Recruitment is explicitly named in the regulation's own supporting text (Recital 71) as a use case. Specifically: "e-recruiting practices without any human intervention."
So if your AI tool screens, ranks, or rejects candidates — that's in scope. Full stop.
The EU AI Act, which is already live and phasing into full enforcement from August 2026, goes further. It classifies AI used for recruiting, filtering applications, and evaluating candidates as high-risk AI. That brings a whole additional layer of requirements on top of GDPR.
The UK is broadly aligned with the EU via UK GDPR, though the Data (Use and Access) Act 2025 does relax some restrictions on automated decision-making for non-special category data. The principle of meaningful human review still applies.
In the US, it's a patchwork:
- New York City (Local Law 144): mandatory annual independent bias audits for any automated employment decision tool (AEDT), plus 10 business days' advance notice to candidates. Already enforceable. Illinois (HB 3773): prohibits AI-based employment discrimination under the Illinois Human Rights Act. California: employers must retain "automated decision data" for four years. New rules from October 2025. The trend is clear. More states are following. If you're hiring across the US, assume this will apply to you eventually.
So...What Actually Counts as "Human in the Loop"?
Under GDPR Article 22, a decision doesn't count as "solely automated" IF there is meaningful human involvement. That's the escape hatch everyone's using.
But the EDPB (European Data Protection Board) is very specific about what "meaningful" means. And most companies are not meeting this bar.
For human review to count, the reviewer must have:
- The actual authority to override the automated decision Access to ALL the relevant data the system used Enough understanding of the logic to be able to interrogate it The ability to factor in information the system DIDN'T consider
So If your recruiter is just looking at an AI score and clicking "approve" or "reject" in an ATS, that is not meaningful review. That is rubber-stamping. And regulators have said, explicitly, that rubber-stamping does not get you out of Article 22.
But wait, the dumpster fire gets even more...dumpster fiery
The ICO guidance says that "the sequencing of human and AI factors is crucial." A human uploading data into a system that then makes a decision is NOT a human in the loop for the purposes of the decision. You were in the loop for the data entry. You were not in the loop for the decision. These are different things.
And if your recruiter is approving 200 AI-screened applications per hour under KPI pressure? One legal commentary put it bluntly: "Speed is evidence against meaningful oversight." A human forced to approve decisions in seconds is not exercising judgment. Courts and data protection authorities are starting to look at the ENTIRE organisational environment — including reporting lines, training standards, and whether overrides actually happen — when determining if a decision was truly automated or not.
The Gap Between Vendor Claims and Reality
Here is a direct quote from a major HR software company's public legal position:
"[Our] AI does not make hiring decisions… Customers retain full control and human oversight."
This is now standard vendor language. And it is technically designed to push ALL the compliance responsibility onto YOU.
When a vendor says "you retain control," what they mean is: you are the data controller. You are the deployer. The liability is yours.
The research bears this out. When a NY State Comptroller audit reviewed 32 companies for Local Law 144 compliance, the city's enforcement body found ONE instance of non-compliance. Independent auditors reviewing the SAME companies found SEVENTEEN potential violations. The enforcement gap is enormous right now. But it won't stay that way.
And the Workday lawsuit (basically like a soap opera for recruiters...i love the drama) is a preview of what escalation looks like. If plaintiffs succeed, every AI recruiting company will have to answer questions about training data, validation methodology, and the actual gap between marketing claims and operational reality.
The vendors will not be standing beside you when that happens. You will be.
What This Means for Your Decision-Making Process
If you are using AI to screen, rank, score, or reject candidates, you need to answer these questions. Now.
1️⃣ Can your reviewers actually override the AI?
Not theoretically. In practice. Does your system allow it? Is there a workflow for it? Are there any audit logs showing it HAS happened?
If the answer is "the AI recommendation almost always stands," you may be operating a solely automated system with a human sticker on top.
2️⃣ Do your reviewers understand the logic?
Not the marketing version. The actual logic. If a candidate was ranked low, WHY? Can your recruiter articulate that reasoning independently? Or are they just seeing a score?
3️⃣ Have you done a DPIA?
Under GDPR, automated decision-making processes that produce significant effects require a Data Protection Impact Assessment. This is not optional. It's a documented process where you assess the risk, the necessity, the proportionality, and the safeguards. If you haven't done one, you're already out of compliance.
4️⃣ Are your candidates being told?
Under GDPR Articles 13 and 14, candidates have the right to be informed that automated processing is taking place. Under NYC Local Law 144, they need 10 BUSINESS DAYS' notice before an AEDT is used on them. Under the EU AI Act, they have the right to know AI played a significant role in a decision about them.
"We use AI tools in our hiring process" buried in a privacy policy is probably not going to cut it.
5️⃣ Is your vendor contract actually protecting you?
Review it. Specifically look for:
- Who bears liability if the tool produces discriminatory outcomes Whether the vendor will support an independent bias audit What data the tool was trained on and whether they'll document it What their obligations are if the tool is found to be non-compliant
If "the vendor did it" is your compliance strategy, that is not a compliance strategy.
What Good Human in the Loop Actually Looks Like
Here's the practical model. It's not complicated. It IS operationally demanding.
- Recruiters reviewing AI recommendations document their reasoning. Not just "approved" — WHY. Override rates are tracked. If no one ever overrides the AI, that's a flag for your DPO, not a celebration of tool accuracy. Reviewers receive training on how the system works, where it's likely to fail, and what bias signals to look for. Candidates can request human review of automated decisions — and that process is real, not a dead email address. You have a DPIA. It's reviewed annually. If you're in NYC, you have an independent bias audit completed in the last 12 months, published on your website.
That's the bar. It's doable. Most companies are nowhere near it.
The Timeline You Need to Know
Feb 2025 — EU AI Act: emotion recognition in hiring banned. AI literacy obligations for employees using AI systems kick in.
Aug 2025 — EU AI Act: GPAI (general-purpose AI) obligations go live for vendors.
Oct 2025 — California: automated decision data retention rules active.
Aug 2026 — EU AI Act: full high-risk AI requirements enforceable. This includes ALL AI used in recruitment. This is the big one.
Aug 2027 — AI systems embedded in regulated products: extended compliance deadline.
Jeez, I'm sweating after that...SO.
The law on this is not ambiguous. It's just uncomfortable.
If your AI tool is making or substantially influencing hiring decisions, and you cannot demonstrate meaningful human review of those decisions, you are exposed. In the EU, that's potential GDPR enforcement. Under the EU AI Act, fines for high-risk AI non-compliance go up to €30 million or 6% of global annual turnover. In NYC, it's compounding daily fines and the reputational exposure of a public bias audit you didn't prepare for.
And if a candidate challenges a decision? You need to be able to explain the logic. Not the vendor's marketing. The actual logic. If you can't, that's your problem.
The vendors will keep saying "human in the loop." That's good marketing.
Luke Update
- For enterprise businesses, how do we deploy AI safely? How do we build good governance? And then how do we build a narrative that the whole business will respect, rather than specific building tools
Well... that's issue sixty six! If you have any questions about it, or any feedback on this issue of The Data Driven Recruiter, grab me on LinkedIn for a chat.
I'll see you next week!