The interview where answers were random, and the interviewers still felt they understood the candidate

In 2013 three researchers, Jason Dana, Robyn Dawes and Nathanial Peterson, published an experiment with a slightly cruel design. They recruited 76 students at Carnegie Mellon to act as interviewers and predict another student's grade point average for the coming semester. Interviewers saw a file: past grades, courses enrolled in. Some also got to interview the student¹.

The cruel part: in one condition the interviewee had been instructed to answer at random. They said yes or no according to a secret rule that had nothing to do with the question¹.

First finding: the interviewers did not notice. Their confidence in their own judgement was essentially the same whether the answers were honest or random¹. People are very good at making sense of almost anything they hear.

Second finding, and the heavier one: those who conducted interviews predicted less accurately than those who only read the file. They even did worse than simply using the student's previous GPA¹. The conversation did not add information; it diluted the information already there.

In a third study, participants were asked whether they wanted an interview, knowing the answers might be random. Most still said yes. They believed that talking to someone would improve their judgement¹.

So what does predict performance?

For two decades the familiar ranking in recruitment rested on Schmidt and Hunter's 1998 meta-analysis, in which cognitive ability tests topped the list at around .51.

In 2022 Paul Sackett's team published a paper in the Journal of Applied Psychology showing that earlier analyses had over-corrected for range restriction, inflating every estimate². Recalculated, the order changes: structured interviews come first at about .42, biodata .38, cognitive ability falls from .51 to .31, integrity tests .31, conscientiousness measures .19^2,3.

One detail matters more than the ranking. For structured interviews, the 80 percent credibility interval runs from .18 to .66². Two companies that both say they run "structured interviews" can get very different results. The execution decides, not the label.

Google dropped the brainteasers

For years Google was famous for puzzle questions such as how many times a day a clock's hands overlap. In 2013 Laszlo Bock, then running Google's people function, told the New York Times plainly that the brainteasers were "a complete waste of time": they predicted nothing and mainly served to make the interviewer feel smart⁴.

Google had done what few companies do: compared tens of thousands of interview scores with later job performance. According to Bock, the relationship was essentially zero. The company moved to behavioural interviews with a consistent rubric, and stopped asking most candidates for academic transcripts⁴.

What "structured" actually means

Most companies believe they already interview in a structured way because they have a list of questions. That is one part of it. The other four make the difference:

1. The same questions in the same order for every candidate for a role. Everyone faces the same hard question.
2. Questions drawn from the real job, describing a situation that has happened or will happen in that role, rather than generic personality questions.
3. A rating scale with behavioural anchors: what a 3 looks like, what a 5 looks like, written before meeting anyone.
4. Independent scoring before discussion. Each interviewer records a score and the evidence for it, and only then do they talk. Discuss first and the loudest voice decides.

In Vietnam there is a further practical reason to drop free-form questioning. The familiar questions about age, marital status or plans to have children predict nothing about performance and amount to discrimination in hiring. A prepared, reviewed question set removes them automatically.