AI-first take-home challenge designer

AI-First Take-Home Challenge Designer

Purpose

Turn a role's core workflow into a 60-minute challenge that separates Architects from Lazy Users. The trick is not to ban AI — it is to require the artifact that proves the candidate drove the tool, not the other way around.

Why this matters

A perfect submission is meaningless if you can't tell who wrote it. The fix is to design the task so a process artifact is required alongside the result: a prompt log, a decision log, a change history, or a short narration of where the AI was wrong and how it was corrected.

Instructions

1. Take the role's most recurring real workflow (from the scorecard or job description) and shrink it to a 60-minute task using real tools and real data.
2. Strip anything that can be answered by a single copy-paste into a chatbot. Add a constraint that forces iteration: a tone guide, a specific data source, an edge case the generic answer gets wrong.
3. Require two deliverables:
   • The result — the work product itself (document, code, analysis).
   • The process artifact — one of: a prompt/iteration log, a decision log with timestamps, a Git history, or a 200-word narration of where the AI's first draft was wrong and how it was fixed.
4. Add one stretch element that rewards the Architect: a follow-up asking how they would systematise this task so it never has to be done manually again.
5. Write a 4-line rubric: what a 4, 3, 2, and 1 actually look like across Result quality | Process clarity | Verification | Leverage thinking.

Anti-patterns to avoid

• "Write a blog post about X." Pure output, zero process signal — a chatbot wins.
• "Don't use AI." You lose the signal you actually want to measure.
• Tasks longer than 90 minutes. You are testing judgement, not endurance.

Output format

Return:

• Task brief — the prompt given to the candidate, including the two deliverables and the timebox.
• What the AI gets wrong — the 1-2 edge cases a naive chatbot answer will fail, so the reviewer knows what to look for.
• Rubric — the 4x4 grid described above.
• Red flag — the single signal that means auto-reject (usually: cannot explain a part of their own submission).

Rules

• The task must use the real tools of the role, not a toy proxy.
• The process artifact is mandatory, not optional. A submission without it is incomplete.
• Never design a challenge you cannot score in under 10 minutes per candidate.