Hiring in the Age of AI Agents

Sometime in 2025, I was interviewing candidates for one of my clients. I opened one of the sessions the way I always do, by explaining the rules: we’re going to do a small coding task; you can use any language you want, any framework you want, any IDE you want - but no AI tools. The candidate paused for a moment, said “you know what? I don’t want this job”, and hung up the Zoom call.

I sat there in shock. What was this guy thinking? It took me a while to admit that the more interesting question was what I was thinking. But to explain that, I first need to talk about what a hiring process is actually for.

Why Interview At All?

The purpose of a hiring process is to predict, using as little time and as little information as possible, whether a person will be a good fit. A more precise phrasing would be: to get the closest approximation possible of what it would be like to actually work with that person. Every interview is a proxy for months of working together, compressed into a few hours, and every step in the process should be judged by how good a proxy it is.

This yardstick sounds obvious, but most common interview formats fail it, because they were designed to be easy to administer and easy to score rather than to approximate the actual work.

Fibonacci, Whiteboards and Take-Homes

Consider the classic formats. Leetcode-style questions can be useful as a cheap filter; a candidate who cannot solve a trivial one probably cannot code. Beyond that, and with all due respect to corporates like Google or Meta, I don’t think these questions are really useful today. Nobody reverses a linked list for a living.

Whiteboard coding is worse. The risk of a false negative is very high: a candidate who is anxious or stressed out will not be presenting their best thinking mind, and standing at a whiteboard while strangers judge your handwriting is about as stressful as our profession gets. You end up measuring composure under artificial pressure, a skill your workplace hopefully never exercises.

And then there are home assignments. Some 18 years ago, a distant friend asked me for help with one; they had to build a blog server in Java, a language they had never used. I told them which libraries to use and what to Google, and they solved the assignment and got hired. The hire was a good one - they really were a good engineer - but the question was flawed, because I had solved half of it for them, and the process had no way of knowing. Keep this story in mind; we’ll come back to it.

How I Used To Do It

At Wix, where I was a founding member of the engineering organization, we interviewed in pairs: two interviewers, one candidate. We would start with a small whiteboard programming task - Fibonacci, reversing a linked list - then move on to a project question, and finish with a system design question. Later, we replaced the whiteboard task with an actual pair programming session: interviewer and interviewee sitting at one computer, building something like a small HTTP server, with the interviewer evolving the task over an hour or an hour and a half.

I took a similar process with me to Orbs, and later into my consulting practice. For some clients I run only the first part: a pair programming session of under an hour, over Zoom, in the candidate’s language of choice, platform of choice, IDE of choice. The task itself is trivial on purpose - an HTTP server that accepts this request and returns that response. If the domain is complex, the hour gets spent on the candidate deciphering the domain; if the domain is trivial, the hour gets spent on me watching how the person works, which is the whole point.

And what I watch for has very little to do with syntax. How do they learn? How do they figure out what needs to be done? What queries do they put into Google, and which links do they click? How do they respond to my feedback? Is their thinking incremental, or do they try to swallow the whole problem in one bite?

Candidates were always allowed to use whatever tools they wanted, including Google. But when ChatGPT came along, and later GitHub Copilot, I forbade them; I thought I needed to see how the candidate actually codes.

Which brings us back to the candidate who hung up on me.

The Candidate Was Right

Once the shock passed, I got to thinking. This is not going to get better; more and more people are going to expect to be interviewed the way they work. And the entire point of my one-hour session - the reason I let candidates choose their language and their IDE, and interview from their own comfortable chair - was to observe them in an environment as close as possible to their day-to-day work. By banning AI tools from a person whose day-to-day work is full of them, I was doing a disservice to both myself and my clients: I was measuring the candidate in an environment where they never operate.

So I changed the rule to the only version of it that makes sense: if you’re used to working with AI agents, use AI agents. If you’re not, don’t. Interview in your native environment, whatever it is.

This is also the point where the home assignment quietly died. My friend with the Java blog server needed a well-connected friend to invisibly do half the work; today, every candidate has such a friend on tap, and it never sleeps. If your process cannot tell who did the work, it isn’t measuring anything.

Same Signals, New Medium

To my surprise, very little else changed. I used to watch what a candidate typed into Google and which links they clicked; now I watch what prompts they write and what they do with what comes back. It’s almost the same signal. A well-formed query and a well-formed prompt come from the same place: understanding what you’re trying to achieve.

The bigger surprise was the pace. I expected candidates using agents to get much farther in the hour than candidates coding by hand. Nobody did. The good candidates moved fast, but they spent their time setting up their environment, explaining to me what they were doing, reading the code the agent produced and making sure they understood it. The unfitting candidates would write a bad prompt, get themselves into a mess, and then spend a long time getting out of it. If anything, the agent made the difference between the two groups easier to see. The session remained exactly what it had always been: a very good proxy for what it would be like to work with this person.

What Is FastAPI?

In November 2025, Opus 4.6 came out, and in my opinion it revolutionized the way people write software. These days, when I run the pairing session, the agent writes most of the code, and my attention has shifted accordingly. The task is still the same: build me an HTTP server that accepts this request and returns this response. I look at how the candidate writes their prompts, what tools they use and how they use them. And whenever a step is done, I ask them to explain what they got from the agent.

This is where candidates diverge. Do they understand what the agent produced? How deep are they willing to dive into the platform and the framework underneath it? If the agent chose to write Python, I ask: what is FastAPI? What is Flask? What does this class do? What does this function do? I expect someone who’s versed in software engineering to understand the inner workings of the frameworks they use, even in the era of AI agents. The agent may have written the code, but the candidate is the one shipping it, and somebody has to know whether it’s right.

What Didn’t Change

The second and third parts of my process - the project overview and the system design question - survived the AI revolution untouched.

The project question is simple: tell me about a project you’re proud of. I find this to be a strong question precisely because of the word proud. Many candidates have a prepared project description, polished over dozens of interviews, and have never stopped to consider whether they’re actually proud of what they built. What a person is proud of, and why, tells you a lot about what they value.

It also tests something subtler: communication. Good candidates make sure I understand what the system does, and what problem it solves, before diving into technical details; they build a shared model of understanding between us. The not-so-good ones plunge straight into the internals, and I find myself interrupting: wait, what does this system do? Why does it work that way? What problem were you trying to solve? A person who cannot notice that their listener is lost will do the same thing in every design discussion you’ll ever have with them.

The design question is some variation of “build me a copy of Twitter” or “build me a copy of WhatsApp”, and its essential property is that it’s incremental. I start by asking for a very naive solution: no scale, no concurrency, nothing clever. Then I grow the complexity as time permits. Many candidates fail at the first step - they’re so used to design interviews where all the specs are given upfront that they cannot bring themselves to give me a naive solution; they overthink, adding queues and caches to a problem that fits in a single process. What I want to see is a person responding to evolving requirements: adapting their solution when the requirements change, staying pragmatic, and not building anything the current problem doesn’t need.

Why did these two parts survive when the coding session changed so much? Because agents changed the cost of producing code, and these questions never measured code production. They measure how a person thinks, how they communicate, and how they respond to change - which are exactly the things I’m still hiring for.

In Summary

The purpose of a hiring process is to approximate, in a few hours, what it would be like to work with a person for years, and every format should be judged against that yardstick. Leetcode and whiteboards fail it because nobody works like that. Home assignments fail it because you don’t know who did the work - and today, nobody works alone. The pair programming session survives because it is the yardstick: watch the person work, in their environment, with their tools - and today their tools include agents. The project and design questions survive because thinking, communication and pragmatism are what I’m actually hiring for.

As for the candidate who hung up on me: it was a rude way to end an interview, but he was right. I owe him one.