Live coding interviews measure stress, not coding skills

I'm now a successful, self-employed indie developer. One of the main reasons I stuck with indie development through the hard times—the maxim that it takes 5 years to become an overnight success was true for me—was that I became practically unhireable. There are multiple strikes against me: I'm middle age in an industry rife with age discrimination, I don't have a computer science degree, and I experience brain freeze during live coding interviews.

I would note that not all stress is the same. Firefighters rush into burning buildings for a living, and what could be more stressful, yet many of them would panic at the idea of giving a speech to a non-burning roomful of strangers. I have no problem with stress on the job, and I've successfully navigated many work emergencies during my career, but something about strangers standing over my shoulder judging me, determining my financial future by providing or withholding a job, like the sword of Damocles, turns my stomach inside out. And like the article author, I can revisit the coding problems and solve them afterward, as soon as the interview is over. Interviewers must think I'm a fraud who can't code, yet all evidence from my career, now almost 2 decades long, suggests otherwise.

A lot of commenters causally speak of "false negatives" as if they were random, but some people, myself included, are always the false negative. I am consistently filtered out by audition-style interviews. I'm not a stage performer.

[Edit:] I didn't expect my comment to rise to the top of an already crowded discussion. I feel a little self-conscious about it. ;-)

I’ve worked in the assessment space for 6 years and have seen many hiring processes, from Fortune 10 companies to startups hiring their first engineers. The range of signal required, "how much time can my engineering team spend with a candidate", and how much candidate experience you can get away with, is huge. I’ve also been a candidate myself and failed many live coding interviews. It made me feel terrible about myself. The last time for a role at Ycombinator (the interviewer was super nice).

When I work on my product, I try to view it through the lens of empowering candidates to show their skills and potential. I encourage our customers to use assessments that somewhat resemble on-the-job skills. I don’t like the phrasing “real work” anymore. An assessment shouldn’t be unpaid labor, it should be a way for candidates to demonstrate that they can do the job and handle future work thrown at them, and for hiring managers to feel confident extending what are often very high salaries in tech.

With AI, unfortunately, short take-homes (what I prefer as a candidate, using my own tools and editor) are becoming harder to maintain as a fair signal due to AI assistance. I’ve seen companies move back to onsite, and competitors deploy all kinds of proctoring and invasive monitoring.

The perfect solution, in my view, would be an assessment where the candidate feels relaxed and able to perform at their best, with their own editor and configuration, knowing that every other candidate in the pool has the same constraints in terms of time and tooling. It’s a tough problem to solve. I think about it daily and have not come up with a solution.

Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding.

But also, someone can fail a live coding interview for reasons other than belonging to that group.

Let's say you need someone who can lift 10 kilograms:

- Good interview: "Please lift this 10 kilogram bucket by the handle."

- Not Good interview: "As a proxy for your overall strength, please take off your pants, squat, pick up this 1 kg bucket by clenching the handle with your butt cheeks, and then stand. We know this isn't a real test of your strength, but we want to see how you perform under pressure."

EDIT: what I mean by doing the job, I mean test the skills used on the job. See if a chef knows their way around a kitchen & actually cook something. See if a customer support rep has good written & verbal communication skills in a mock support interaction. See if a phone screening can do phone screens. Stuff like that.

I get stuck in ways I’ve never experienced in my life; my brain feels like it stopped working (but has reserved the bandwidth to shout at me the whole time “you’re blowing this!”). Then shortly after the call ends, sitting quietly, a path forward opens and suddenly I’m working on a solution, which it’s own special kind of hell—to know you could do it but didn’t and now some stranger out there thinks you’re a real idiot.

Studying/practicing has mixed results as you are taking away time from your partner and kid, this causes feelings of guilt. I found the more time I spent practicing, the more leetcode hards I solved, the worse I got. It was paradoxical! It was additionally demoralizing!

I only share all of this to say that circumstances change; you change. What once was easy becomes hard—maybe just for now, maybe forever, I don’t know. What I do know from my time at Big Co., is that this style of interview (often aped at smaller co.) is employed as one of the stated goals is to remove opportunity for biases (test everyone the same "quantifiable" way). However, my experiences lead me to believe that it falls short in that regard. As I said, people change, and a candidate who becomes a coworker with a high tolerance for stress today might not have the same appetite in a few years. Surely removing biases is about seeking balance, but I think certain types of interviews are blind to the imbalances they might cause.

I once went to a job interview with Google. I built one of the first local (Global to the Netherlands) search engines of the Netherlands, but the guy in the cowboy hat at Google asked me to write a binary search with a marker and a whiteboard. I never write with my hand, I always use keyboards. Plus I'm being insulted to write a binary search when I designed and build a search index and retrieval engine.

[I did the binary search but was not happy with the whole process that did not want to even look at what you had actually done before, because that would take away the baseline they wanted].

I guess they must have been looking for cowboys. Tip for interviews, take your cowboy hat, just in case..

Why?

Purely being a "good programmer" isn't just enough for the job. It's important that we can develop a good working rapport, and communicate.

For example: Someone who's an expert programmer might misinterpret a discussion and build the wrong thing; or someone might be a great programmer but otherwise be impossible to actually conduct a technical discussion.

IE, this is why "Fizzbuz"-style questions work so well: It's not really screening that you're a great programmer, it's screening your ability to have a technical discussion.

One thing that’s helped me, as both an occasional interviewer and a frequent interviewee, is recreating the stress loop on my own terms. Friends rarely push hard enough, and a paid coach is $150+ an hour. Lately I’ve been working on a little side-project Tough Tongue AI: a voice-driven agent that drops you into a live code editor, throws follow-up questions, even interrupts when you go off-road, then gives a feedback at end. It’s not magic, but after a few sessions the “somebody’s watching me” adrenaline spike starts to feel familiar.

If live coding interviews are here to stay, a repeatable way to train the physiological side of the test, not just the algorithms would be helpful!

Really great interview and what I realized from that experience is that this format is so good because it really measures for both breadth and depth without a lot of pressure. Here, you're not measuring so much for right and wrong, but how experienced an engineer is in real-world scenarios; everyone can find something to fix in the code, but more senior engineers will find more, faster, with less hints.

I think it also reflected day-to-day responsibilities more and in this AI agent era, reflects the needs for an engineer to carefully review AI generated code for correctness, performance, security, and fit-for-purpose.

I enjoyed the exercise so much, I ended up building a simple, OSS tool to facilitate this kind of interview using code reviews:

https://coderev.app (https://github.com/CharlieDigital/coderev)

Younger engineers trying to find jobs in the industry should consider how much the other parts of the hiring process privilege older engineers regardless of ability. I'm sure many of the people I worked with who lacked basic coding competence twenty years ago still lack competence and are employed right now in jobs that require it, jobs that could go to younger, more capable people except that their resume, their experience, and their ability to say the right-sounding thing don't match up.

Hiring someone to write code without ever seeing them do it is like hiring a guitar player for a band by verifying their MFA in music theory, talking to them about music for an hour, and checking their references. All that is fine, but can they play the guitar? You can acquire everything else, the lingo, the knowledge of music theory, the pros and cons of competing techniques, the familiarity with current and past great performers and performances, without ever putting your hands on the instrument. And some people are built like that. Some people are guitar connoisseurs: they gave up their hopes of playing the instrument long ago, but they're huge fans and nerd out over the art and science of it. If those folks needed to be in a band to have health insurance, and they didn't have to audition, you can bet some of them could talk their way into it.

Finally, when a team interviews someone for a position that requires writing code, they are inevitably making a judgment about whether they think they can do it. If you ask someone to guess without any direct evidence, you're inviting them, almost forcing them, to fill the gap with bias. Do they collect subtle little hints and make fragile deductions like they're Sherlock Holmes? Do they trust their instincts about whether the person in front of them looks and acts like the kind of person who could code? Either one is guaranteed to be rife with bias and problematic preconceptions.

I have done the entry interview as well. For example have had very good conversations with managers who were applying for senior IC roles. They then go and fail the tech interview basic "can you do write and execute a python hello world script from the command line".

That being said, IMO there is no good replacement. Especially with LLMs being as good as they are you simply cannot trust someone to solve a problem without supervision and rely on the result alone as a signal. Not just because they might cheat, but moreso because the main signal you get from these interviews is watching someone think through the problem and talking through it with them.

These should not be pass/fail tests, there's so much more that goes into a good evaluation.

I was interested by the author's statement: "Working memory is the most reliable proxy (I know of) for fluid intelligence, your ability to reason, solve novel problems, and think abstractly." and the linked study (https://pubmed.ncbi.nlm.nih.gov/21037165/). My working memory is not so great, but it degrades less under stress.

Question worth considering for hiring managers: do you prefer stress-capable employees, or greater working-memory employees? Is my model a false dichotomy?

I once had a coding interview where I was given a laptop with no internet access and 30 minutes by myself to solve a couple questions a step or two above FizzBuzz, in pseudocode. The interviewer then came back into the room and we discussed my solutions, using them as a jumping-off point into a more open-ended discussion (e.g. "why did you use a hash table here? can you think of another approach?" which then led to a general discussion on the suitability of hash tables versus other approaches for related problems.)

It was one of the best technical interviews I've had. I'm surprised more places don't do this. It's a win-win-win: it relieves stress on the interviewee, gives the interviewer back half an hour of valuable time, and makes for a much more productive assessment of the candidate — it's a lot easier for a candidate to explain code they've already written, and thus a lot easier to an interviewer to assess the candidate's thought process.

A live coding exercise should be like 15 minutes of time at the start of an interview. If it goes well, tell the candidate they did great, and then get to the real interview. If it doesn't go well, you can cut the interview short, and thank the candidate for their time. There's no point in talking to someone about a programming role if they can't program. e.g. basic control flow in a language of their choosing.

People who can't program will sneak through your hiring funnel and into contact with your engineers. It's a question of rates, not absolutes. The cost of not catching them quickly can be 1000s of dollars in engineer time.

I wrote about this here: https://acjay.com/2023/11/12/everything-all-at-once-inside-a...

There are certainly jobs where you need someone who is cool under intense pressure, where what happens inside an hour matters. That's what these interviews are revealing. But I think this is a tiny minority of dev jobs.

However, I also think when people talk about hiring practices, the elephant in the room is that the true purpose our interview processes have evolved to serve is taking the candidate pool from a lot of people to something that is decidable. We can't fully evaluate 500 candidates, but we can evaluate 5. The process of weeding out 99% is designed to be inexpensive, at the cost of accuracy. The hope is that you'll have one person squeeze through the filters that is a good fit. If that happens, then all the false negatives are tolerable.

But we really, really don't like to admit this. We tell ourselves that we are running interview processes that are predictive of performance when there is actually very little evidence to support that, and plenty of reason to doubt that it's true.

If quality were paramount, I think we'd do hiring completely differently: https://acjay.com/2019/05/16/hiring-developers-is-easy/

Did y'all not have to take a load of exams in high school and college?

I'm sure there are variations between education systems, but by the time I graduated college I'd done at least a hundred high-stakes, time-pressured hour long written tests. All of them substantially more difficult than summing the even entries in a list.

Is this not what everyone experiences, in every education system?

At least reading this post has helped me regain some of that lost self-esteem. Thanks for sharing it. <3

Yes, we can. Don't do them. But, we have to replace them with something that works. That means none of these poorly constructed take home projects that are almost universally either drastically over scoped, criminally under specified, or both.

There was one company whose process I really appreciated:

- A take-home test, more of a puzzle-style challenge. - The interviewer does a quick review of your submission. - If they like what they see, you're invited to a follow-up session where you explain your code and make a few simple changes to it.

This approach proves that the work is genuinely yours, shows your thought process, and still gives the opportunity to demonstrate live coding on code you're already familiar with.

Now, that doesn't mean that stress management cannot be part of the job: I've worked in the hellish places where an on-call rotation meant at least 4 calls off-hours, and some which needed resolutions in 10 minutes or less from the pagerduty call. Someone with bad stress response is not going to be doing great at that kind of live diagnosing, with possibly many millions for the company on the line. But the number of positions like that, where you are a cross between a developer and an ER doctor are much less common than places that have none of those demands, yet give you a series of leetcode mediums and hards. They might as well be testing for height, or how much you can bench press. It filters, but it does not help.

That said, does anyone have a good alternative? If somebody has open-source work and portfolios, there's a great window into their work, but that's probably an unfair expectation.

They gave me a take home, said use whatever AI you want, just tell us which.

The take home was the equivalent of a simple TODO app using their API (key provided). It took an hour to build.

After I submitted it, they had a follow up session where I had to explain the code to them and answer questions about why I did things the way I did.

Simple, easy, and something any developer should be able to do.

For example, for the senior frontend engineer, I think we went with "create a real-time comments panel using SSE that works in multiple browser tabs", and created a ready-made git repository with a dev server that includes the SSE endpoint. That way, they only had to focus on the actual task.

Now I thought that was nice because it should be very easy to solve for someone with a bit experience, but we get lots of opportunities to discuss their choices and the technology. It's hard to confidently bullshit an answer to if you think server-sent events or web sockets are superior, and why. So from everything we tried so far, this format gave me the most accurate impression of someone yet.

Dunno if the study has enough statistical power but that's an absolutely horrific result. A great talent pool is mostly discarded because of a faulty process...

I recall reading advice that you should just verbalize all your thoughts and have found that this is not optimal.

It's okay to take some moments of silence and talk strategically. For example, I'll tell the interviewer that I'll be silent for a few minutes while I read and understand the question. I'll then talk out loud to confirm with them that I've correctly understood it. From there I'll talk on and off as I narrow towards a solution.

While writing code and debugging, I'll start talking if I get stuck on something.

So the idea is to use talking in a way that doesn't slow you down and may even help you solve the problem faster.

I won't use live coding questions (usually) because I don't see much value in trying to code under that much stress.

But I will ask hard questions to see how well a candidate communicates in a stressful situation (which I think is a far more valuable indicator of their brain's "default" mode in stress). I want a candidate that talks honestly about stuff -- especially the stuff they don't understand -- even in a stressful situation. Sometimes stress can bring out the asshole in somebody -- and that's an immediate red flag.

- Past: ~8 years in big tech as an IC - Currently: ~4 as CTO a < 10 person startup.

I always hated doing and conducting coding interviews. When I became in charge of the interview process, I swore to myself not to have them.

I had to let go of a person after ~3 weeks in because he just didn't write any code. This was about 1 year post the ChatGPT moment.

He had lots of experience. Great communicator. Culture fit (or so I thought). Etc...

Interviews can provide a signal, but nothing trumps a month of working with someone. If both parties are open to it, a well paid short term contracting gig goes a long way.

This is something that someone familiar with a language should be able to do as naturally as answering "tell me about yourself" or some other non-coding question. (Personally, way more naturally, nothing makes me more uncomfortable than "tell me about yourself")

I agree that for something more involved, nerves can become more of a factor, certainly any "trick" question, but these seem more like just checking if whatever language is second nature, which it should be, not about algorithms etc.

Since then, I've found that I need to practice extensively so that I can still function when my IQ drops due to the stress. It reminds me of, "Under pressure, you don't rise to the occasion, you fall to your level of training."

int total = 0;

for (int i=0;i<arr.length;i++) {

   if ((arr[i]%2) == 0) {

      total += arr[i];

   }

return total;

I am good at whiteboard coding but I am also good at compensating for what I don’t know and even bluffing. In a data science interview they asked me about “regularization” which I didn’t recognize but when they defined it I could describe many different approaches to regularization I had used in projects. For a long time it seemed that any question in an interview could be answered answered with “look it up in the hashtable” or “look it up in the literature”

I haven't worked at a FAANG, but I've heard they have such gruelling interviews. Is that how they build technically superior teams, or do they just have access to a strong candidate pool to begin with and use these tests to eliminate candidates to get the number they need?

Which is funnily something opensource doesn't suffer from. You just have to prove identity to the commits you signed in previous contributions, so the problem of proving technical skills in the opensource community is replaced by the effort of building trust.

The answer therefore could be to replace technical interviews by sending out assignments to candidates to have them contribute to opensource and come back with results. This has the upside for candidates that if multiple companies do such assignments they only have to contribute once to be eligible for multiple hiring processes.

Everybody wins.

I have been working in code for over 12 years. (self.taught(php,python))

I refused to take these types of tests. (yet I have no problem writing profitable financial algorithms) Hence I went to the management side.

I will be hiring some developers soon, I will be looking for desire to work, personality and previous verifiable experience. I will also ask if they believe Kim Jung Un is fat.

At my previous job, I didn't get a chance to hire people they were sent to me. It was my job to ensure they were trained and a team player. I would like to thank the Marine Corps for 20 years of experience.

Ironically, I wrote about this topic (https://loudwhisper.me/blog/code-interview/) months ago when I failed my first -and last - coding interview. The more interesting aspect is that it was not even for a SWE position, but for a platform security engineering position. Honestly, I can't ever imagine being on the hiring side and thinking this is a good use of anybody's time.

Thing is, I reckon those candidates aren't a good fit anyways. The biggest mistake I see engineers who join startups make is that they pursue excessively sophisticated solutions, with robustness that does not justify the added complexity. I'm sure these candidates are smart, but they're not good fits for the high-velocity, simplicity-obsessed technical environment of a product-play startup.

Not sure what the answer should be apart from hiring people you know - personally or because they have a personal brand.

Our engineers don't have a pre-set of questions. They asked what came to their heads, they picked at stories I told, they were interested in technical problems I had solved in the past.

I find we've dodged some bullets using this methodology, and the people we've picked up using this process have been pretty excellent and tend to gel well with the team.

What's the alternative? Take home problems come with their own problems, and select for people with no family and lots of time. Going by general vibe alone also works, kind of. But we're going to work together, so having something that at least faintly resembles work is kind of useful.

The leetcode type problems are useless however, fully agree on that. But a trivial loop with conditional, perhaps iterating over a list of lines shouldn't be that bad, especially if you get help with the syntax.

a test where everybody taking it feels happy and a successful winner? that's showing the limits of the test. live coding interviews test your limits.

This is why I like to do a lot of talking with my interviewer during these tests. It sorta tricks my brain into seeing the interviewer as a friend that I’m solving a fun problem with. It really reduces stress.

To date, I've never gone on to regret hiring someone who blitzed the in person coding exercise.

Who are these self-important employers?? I mean, outside of Ops and live incident response, are there really that many software engineer roles that require someone to perform well under pressure? It seems a false and egotistic narrative. If you need a software builder to perform under pressure then you've got systemic issues of a type not solved with software. You need better management or better work practices.

Then some agent turns up and says "If you can do the shot, I'll give you [life changing money]". If you miss, you get nothing.

If you miss that one shot, does that mean that you're essentially just a fake player who can't shoot for shit?

I don't remember, because the interviewer was kind enough to point it out for me to help me move forward.

It's not a puzzle, it's not a trick question. It's simply a matter of 1) can you do the job and 2) do you get along with the team.

The last interview I did, we had the candidate build a game of marbles in Unity. None of us knew the rules, so it turned into a collaborative process between the candidate and our engineers. We learned that the candidate can think on his feet and fluently adapt his program to requirements as they manifest. Plus it was just a lot of fun. He was one of our best hires, too.

Should it be that way? No. Was it that way? Yes.

a related thing here is that I've long believed that remote work positively impacted my career for precisely this reason.

Often times coding interviews are academic and only prove basic understanding.

Those that are most successful are ones related to the business/product and that have no expectations of being finished or solved. So much so that I will simply bring up code as a reference to discuss and talk about -- not actually have anyone do any coding. The value is in understanding how someone approaches a challenge or simple day to day work. What are the questions they have? They should have questions. How engaged are they? Is there any past experiences with something similar? Etc.

Using code as a talking point or guide in an interview isn't bad. It's the awkward and stressful academic mental gymnastics that is. Most people giving interviews simply get this wrong. As a result, they have a lot of employee turnover. The funny thing is many people still sit there scratching their heads over why people don't work out if they passed the coding challenges.

At every place I've worked at, spanning a whole spectrum of places (Google, Discord, tiny, huge), its been a truth universally acknowledged that interviewing is a garbage metric full of false negatives. But we'd all rather put candidates through the gauntlet than the whole team through the firing gauntlet.

Make it easier to fire and interviews will be less insane.

2 years ago given a coding assessment "Roman Numeral Conversion". Was given a set of test cases. I got near the solution. I think if I had another 5-10 minutes I would have solved it fine. Sat down that evening with the same test cases and did it from scratch in 20 minutes.

5 years ago, I didn't get a job because I while I did well technically. The reason for the rejection was that I appeared "stressed" while being assessed. What do they expect?

10 years ago. I walked out of an interview essentially after about 10 minutes. The interviewer(s) interrupted me the moment I started writing any code, constantly interrupting my train of thought. I think it was intentionally done to irritate me. I've had companies play these stupid games in interviews before.

Being a senior engineer means having confronted a shitload of different minutia type problems from network stuff, compiler bugs, language nuances, threading nuances, etc. and having spent time figuring each one out. Not that senior engineers can reiterate all that minutia off the top of their head, but it gives them a good intuition for how to solve problems when they hit them making them significantly more productive than junior devs that have to figure them out from scratch.

I don't understand what's so hard about this, testing algorithmic knowledge tests if the individual studies and implements algorithms. It's very simple. And yes, people have stress reactions in test type settings (I don't for paper tests in school but apparently I do for live coding mostly because I don't know how to implement different algs).

Stress during interviews is insane. Once I was at the tail end of solving an interview problem but it came down to multiplying 9 * 3 and my brain wouldn't fucking do it (apparently I can't fucking multiply).

I actually like live coding much more than any other type of interview. There are a few reasons for this.

First, I don’t really know what to say to many of the standard interview questions.

“Give an example of a challenging task you had to do in your previous job.”

Eh, I don’t remember? I probably will remember and draw on my experience if a similar task comes up. But right off the bat? No idea.

“A question about some obscure feature of X,” where X can be a programming language, library, framework, etc.

What’s the point of this question? If I need to use that feature, I’ll check the docs. I’m very good at quickly skimming through the docs and finding what I need.

“Do you have experience with <the exact software stack that we use>?”

Maybe not, but I understand the underlying concepts and I’m good at reading the docs.

Second reason is, of course, that I tend to perform much better in live coding interviews than in any other type. I get better offers, more positive feedback, and interviewers clearly feel much more confident in my abilities.

It feels like one of those intractable industry standards where nearly everyone beyond management is unhappy about, yet can't or won't do anything about. Other examples- restrictions on remote work, office open floor plans, Agile (not as unpopular, but you see some fierce criticism of it), etc.

It feels like preaching to a choir that never changes. Only external forces can shift these traditions. Like how the pandemic mandated teleconferencing, perhaps the proliferation of generative AI interview cheating software might eventually force companies to rethink their processes. And, most likely, they will just then simply make candidates interview on-site.

Why would I design a global scale app without docs in 30 minutes?

At least coding interviews you can practice leet code and play the game.

In my opinion after having a long term job go bad, just always be interviewing. Then they won't be stressful, your ready to be jump and nothing feels like it's high stakes.

I still do bad at interviews though lol

Take someone who is absolutely calm under pressure (an astronaut, say), but who has never so much as seen a line of code, and give them a live coding exercise. They’re going to fail completely.

They measure coding skills and ability to function under stress. If you’re looking for coding skills, it’s a useful albeit imperfect proxy. And you may well be interested in performance under stress too.

Most people, even the good ones, show a little hesitance when starting. Which isn't really necessary, most people do fine. I'm not trying to get them to fail, I'm trying to get them to succeed. I want to see if they're smart and understand the problem and the direction of a solution. Not if they miss any semicolons or don't recall some arcane data structure.

She was one of the best hires I made. Coding interviews can also measure attitude and confidence.

The post describes what candidates can do, but there's a lot of responsibility on the end of the interviewer. Most of the responsibility, if not all of the responsibility here, is on the interviewer. If you're seeing 75% of your candidate bomb your livecoding interview, your process is very, very broken.

- Never spring a livecoding question on someone as a surprise. Candidates should know well in advance which portion of an interview has a livecoding question in it.

- Interviewers are responsible for helping the candidate feel comfortable. I always chat a bit with candidates, explain the schedule of the timeslot up front, read through the problem with them, answer any questions about the problem they have to make sure they understand what is being asked, etc.

- Always give more than enough time. A question that you expect to take 10-15 minutes to solve requires an hour long interview slot. For candidates that finish very early, have a bonus question lined up in advance, or use the time to talk and answer candidate questions. E.g., for a 10-15 minute problem, you could do a 5 minutes intro, 40-45 or minutes coding, and a 10-15 minute outro for candidate questions.

- Don't make the problem too small. If it's too small, you're quizzing them on knowing a single thing and not gathering any signal on whether or not they can break a problem down. One medium problem that breaks down into two smaller subproblems is better than two small problems that have nothing to do with one another.

- I tell candidates they can pick what language they want and I also tell them what languages I know.

- Frame it more as a pair programming exercise. If the candidate wants to do something that requires a lot of typing, I'll pitch in. If they make a typo in a variable name, an indentation error in python, or drop a parenthesis, I just tell them. It's not a typing test.

- I tell people they can look stuff up. I even tell people they can ask me. If someone asks me a question about the Python or Go standard library I just answer it. It's not a test of your memory of the layout of the Python standard library. If you vaguely know "the standard library has this thing that I need" and can name it, I'll just tell you where it is.

- I tell people they can run the code as many times as they want; if they want to run their code a bunch of times and solve it in an iterative fashion that's totally normal. Without this instruction, sometimes candidates think they are expected to get it right on the first run, or in as few runs as possible. We're not golfing here.

etc. Things like that. Sometimes people tell me what they're going to do and ask me if it will work and I'll say "ah I think that's giving too much away", so there are limits. Obviously don't solve the problem for people, but making sure the candidate is comfortable is something the interviewer must actively engage in and manage, and when candidates are getting stressed out, it is the interviewer's job to do their best to help lower the stress level of the situation. And yes, sometimes you'll still get people who bomb, but if you're seeing that happen 75% of the time, that's likely signaling that your process is not working.

I didn’t need the job that bad.

I’ve worked on one man projects that were hard realtime C image analysis (Fourier, calc, stats), large data science eval projects, devops stuff. CS degree and used it for published research.

I also had a five rounds of interviews a MANGA corp. Didn’t bother with the second because I didn’t have two months to waste.

Went hired gun and never looked back. Homey network uber alles, I make 10% less but I call all the shots, and am way healthier.

It will probably get worse before it gets better out there, between RTO and whatever LinkedIn is doing to CEOs.

The reality is that some sort of interview must be conducted, and it's impractical (and simply unfeasible for some companies) to just hire and then fire during the probation period. It's also wasteful and unfair for the team where the candidate is supposed to work. Interviewing is an exhausting process for the interviewers as well, not just for the interviewee.

Live coding is a good, if flawed, step in the interview process.

Here's how I do it when I'm the one conducting the interview:

- I approach the interview from the mindset that I want the candidate to succeed. Not only it's more fair to the candidate, we're also not Google or Meta that we get so many candidates that we need to weed them.

- I take into consideration their stress levels, because I'm a shy person myself. I never discount points for a candidate that freezes for a moment, or requires help to get out of momentary panic, as long as they eventually solve the challenge.

- I ease them as much as I can, explain the coding exercise has no tricks or "aha!" moments, and that they are free to ask as many questions as they want, and also google anything they don't remember. I just ask them NOT to use AI assistance, and explain that it's just an artificial requirement because I want to listen to them think, not blindly use AI. I also trust them not to use AI, I don't double-check; I explain I simply trust them to be honorable.

- I help them whenever I see them stuck. If I see them going down what I think is the wrong road, I wait in case I'm mistaken, but when they get stuck I gently prod them in the right direction.

- Live coding is not the moment for "thinking outside the box"; I hate that kind of FAANG challenges. We never ask one of the "Cracking The Coding Interview" style of questions. The exercise itself can be solved with basic Python, similar to the example in TFA. If you know dicts, lists, for-comprehensions, etc, you can solve it.

Aaaand... about 50% of candidates fail it, spectacularly. Like, they cannot get basic syntax right (and remember I'm helping them along the way, e.g. "this is how you access a key in a python dict"). Some are supposed seniors, some are juniors. There are people who struggle but get it right eventually with some help (I consider they passed the challenge), but others struggle and simply cannot make any progress even with a lot of help. The seniority level is no predictor.

Maybe for these candidates that really don't go anywhere, there could be a better interview format that is not live coding. Sure. But the interview process cannot accommodate every person, it simply won't scale. Designing interviews is difficult, too. So unfortunately, these people will fail the interview. It doesn't mean they cannot be good professionals, it simply means given the time and effort we can devote to interviewing, they will fail our process.

Some people do badly with take-home challenges. Some people do badly in other kinds of interviews. No process can accommodate everyone, and it's a balancing act which must also take into consideration that designing good interviews is hard.

Except humans can do well at live coding. Especially for trivial cases like the start of the article (sum . filter even). Yes, companies are looking to hire humans as opposed to ChatGPT, but they are looking to hire a subset of humans. Just being human is not good enough. Failing this test might not provide a reliable signal, but passing it does. If you are getting stressed over simple question or existing stress is impairing your thinking enough to not answer simple questions that seems like an issue to me.

A good employer takes care of their worker even if they have problems, but they also want to weed out as much of them before hiring. The value of worker is not revealed when they are at their best. Fragile workers with anxiety perform well only small amount of time.

Live interviews are generally excellent at revealing how individuals perform when placed outside their comfort zone. If you perform well in a live interview, be it for coding or any other skill, you are likely to perform even better reality.