How to Assess AI Skills in Non-Technical Roles (Without a Coding Test)

You can't assess AI skills in an accountant, a recruiter or a marketing manager with a coding test, and you can't assess them by asking "how are your AI skills?" Either you get a confident answer you can't verify, or you get silence from the excellent candidate who doesn't describe themselves that way. Both paths lead to the same mistake: hiring someone who talks well about AI and can't use it.

A 2026 TestGorilla survey reported that 59% of organisations had made a bad AI hire — someone who spoke confidently about AI in an interview but couldn't apply it on the job. The fix isn't a harder interview question. It's a work sample with a trap in it, plus a rubric that scores what the trap reveals. Here's the whole method.

What you're actually assessing

Strip away the tool vocabulary and there are four things worth scoring in a non-technical role:

1. Delegation — can they look at a task and split it into "machine does this, human does that"?
2. Verification — do they check output, and do they know specifically where these systems fail?
3. Judgment — do they know what should never be automated, and will they say so out loud?
4. Transfer — can they explain the technique to a colleague so the capability doesn't die with them?

Notice that none of these are about prompts. Prompt syntax changes quarterly; these four behaviours are the durable thing. They're the same behaviours we screen for in the six pillars of AI-first talent, expressed as evidence rather than as personality.

The rubric

Score each dimension 1–5 against a work sample. The anchors matter — without them, five interviewers produce five different scores.

Hiring bar: 3+ on Delegation and Verification, and no dimension at 1. A 5 on Transfer with a 3 elsewhere is often the better hire — they raise the floor of the whole team.

Three work samples that work

Give a real artefact from the role, a two-hour window, and ask them to work it out loud or in writing. Each one has a planted flaw. You're scoring whether they catch it and what they delegate.

Finance: the month-end memo

A messy ledger extract with three planted problems — a duplicated vendor payment, an invoice booked in the wrong period, and an unusual manual journal with no source document. Ask for the close memo and the exceptions list.

• Strong: flags all three, delegates the matching and the drafting, and explicitly refuses to let AI decide the accrual estimate.
• Weak: produces a clean, confident memo that repeats the planted error. This is the bad AI hire, caught before you make it.

Marketing: the campaign brief

A vague one-paragraph brief plus a contradictory email from a stakeholder. Ask for the campaign plan, the audience definition, and what they'd hand to a machine.

• Strong: asks the clarifying question first, uses AI to generate angles and competitor angles, then kills most of them with a reason, and flags that a generated claim about a competitor needs sourcing before it goes anywhere public.
• Weak: ships the AI's five bullet points as strategy.

Recruiting or ops: the screening queue

Twenty anonymised CVs and a role brief with a deliberately fuzzy requirement. Ask for a shortlist with reasons.

• Strong: builds a rubric before looking at CVs, uses AI to rank, then overrides two rankings with reasons, and raises the fuzzy requirement as a decision the hiring manager needs to make.
• Weak: accepts the ranking, or ranks by gut and calls it AI.

The take-home challenge designer skill turns any of these into a bounded, non-burnout-inducing assignment in ten minutes.

Red flags that predict a bad AI hire

• Tool lists instead of stories. "I use ChatGPT, Copilot, Notion AI and Claude" with no example of a mistake caught.
• No failure story. Everyone who uses these systems daily has one where the output was confidently wrong. Nobody with one has either never used them or is hiding it.
• Delegating the decision. If the machine picks the shortlist, writes the customer message and sets the forecast, there's no human left accountable.
• Speed as the only claim. "It saves me four hours a week" is a workflow claim only if you can show what the four hours went into.
• Rehearsed answers to the ethics question. A prepared sentence about bias, with no example of an actual adverse-impact check they ran.

What good looks like in practice

The best non-technical AI-first candidates share a shape: they talk about their work as a process with named stages, they know which stages are safe to hand over and which are not, they have a written-down checklist somewhere, and they've taught the thing to at least one colleague. That's not a personality type — it's a workflow owner, and it's the profile that compounds once they're inside your organisation.

For the interview structure that surfaces this in 45 minutes rather than two hours, see the AI-first hiring framework and the screening call script. The verbal version of these tests — the questions that separate rehearsed from real — is in AI interview questions that reveal judgment.

Frequently asked questions

Do non-technical candidates need to understand how models work?

No. They need to know the failure modes: hallucinated citations, plausible-but-wrong numbers, quiet truncation, sycophancy, and stale data. That's a checklist, not a course. Understanding attention mechanisms adds nothing to hiring judgment.

Should I use an AI-skills certification as a screen?

Treat it as a weak positive signal at best. Certificates prove someone completed a course, not that they can delegate and verify inside your actual workflow. A 30-minute work sample beats any certificate.

What if the candidate is excellent but genuinely doesn't use AI?

Hire them and train. Levels 1→2 and 2→3 are trainable in weeks for someone with strong domain judgment and a habit of checking their own work. What isn't trainable is the refusal to verify — so test for that specifically.

How do I stop candidates from just pasting AI output into the take-home?

Let them. Then ask them to walk you through it live, explain the two decisions they'd change, and fix a deliberate flaw you introduce. Pasting survives a document review; it doesn't survive a conversation about the document.