95% of Companies See 'Zero Return' on $30B Generative AI Spend

Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever, delete the oldest audio.

There, I've saved you more millions.

We use Google meet and it has Gemini transcriptions of our meetings.

They are hilariously inaccurate. They confuse who said what. They often invert the meaning "Joe said we should go with approach x" where Joe actually said we should not do X. It also lacks context causing it to "mishear" all of our internal jargon to "shit my iPhone said" levels.

I wonder if the human agents agree the AI summaries are better than their summaries. I was nodding as I read and then told myself "yeah but it wouldn't be able to summarize the meetings I have", so I wonder if this only works in 3rd person.

At work we've tried AI summaries for meetings, but we spent so much time fixing those summaries that we started writing our own again.

Is there some training you applied or something specific to your use case that makes it work for you?

> agents were previously spending 3-5 minutes after each call writing manual summaries of the calls

Why were they doing this at all? It may not be what is happening in this specific case but a lot of the AI business cases I've seen are good automations of useless things. Which makes sense because if you're automating a report that no one reads the quality of the output is not a problem and it doesn't matter if the AI gets things wrong.

In operations optimization there's a saying to not go about automating waste, cut it out instead. A lot of AI I suspect is being used to paper over wasteful organization of labor. Which is fine if it turns out we just aren't able to do those optimizations anyway.

I'm curious, have you noticed an impact on agent morale with this?

Specifically: Do they spend more time actually taking calls now? I guess as long as you're not at the burnout point with utilization it's probably fine, but when I was still supporting call centers I can't count the number of projects I saw trying to push utilization up not realizing how real burnout is at call centers.

I assume that's not news to you, of course. At a certain utilization threshold we'd always start to see AHTs creep up as agents got burned out and consciously or not started trying to stay on good calls.

Guess it also partly depends on if you're in more of a cust serv call center or sales.

I hated working as an actual agent on the phones, but call center ops and strategy at scale has always been fascinating.

So long as precision isn't important, I suppose. Hallucination within summaries is the issue I keep running into which prevents me from incorporating it into any of our systems.

Who reads the summaries? Are they even useful to begin with? Or did this just save everyone 3-5 minutes of meaningless work?

This highlights the potentially unrealistic spend on AI, where the proposal is to spend 10s of millions to save... marginal millions... that could have also been saved by a change in process with limited additional spend.

I also would assume that there are far more significant behavioral or human factors that consume the time writing those minutes, i.e. an easy spot to kill 5-10 min before opening the line for the next inbound call, but the 5-10 minute break will persist anyway.

I fully believe AI will create a lot of value and is revolutionary, especially for industries where value is hidden within data. Its the pace of value creation that stands out to me (how long til its actually useful and better and creates more value than it costs??) but the bubble factor is not ignorable on the near term.

That’s the thing. There’s value in AI, it’s just not worth half a trillion dollars to train a new model that’s 0.4% better on benchmarks. Meta is never going to get a worthwhile return on spending $100M on individual engineers.

But that doesn’t mean AI is without its uses. We’re just in that painful phase where the hype needs to die down and we treat LLMs as what they really are; an interesting new tool in the toolkit that provides some new ways to solve problems. It’s almost certainly not going to turn into AGI any time soon. It’s not worth trillions. It’s certainly worth something, though.

I think the financials on developing new frontier models are terrible. But I’ve already built multiple AI projects for my company that are making money and we’ve got extremely happy customers.

Investors thought one company was going to win the AI Wars and make a quadrillion dollars. Instead it’s probably going to be 10,000 startups that will build interesting products based on AI, and training new models won’t actually be a good financial move.

This makes sense. AI is obviously useful for many things. But people wouldn't invest tens of billions to summarize data center calls and similar tasks. Replacing data center workers isnt where the money is - it's replacing 100K-200K/year workers.

> It's not going to replace anyone's job.

Is it not, in the scenario you are describing? You are saying the agents are free now to do higher-value work. Why were there not enough agents before, especially if higher-value work was not done?

so you're saving 3-5 minutes per agent per call. I'm guessing calls come into a queue and then the next available agent starts to handle it. If an average call takes about 20min until the agent hangs up and is free for another then after about 5 calls they've saved enough time to take an extra call they wouldn't have before. 5 calls is 1.4 hrs on the phone, i'm guessing with breaks and call center reps not being 100% on the ball all the time then your agents probably will take maybe 3-4 more calls per day with the AI than without (assuming call volume is such that there are always more calls than agents can handle)

Is that really millions of savings annually? Maybe it is but I always hesitate when a process change that saves one person a few minutes is extrapolated all the way out to dollars/year. What you'll probably see is the agents using those 3-5 minutes to check their phone.

Where is the money being saved? Are you reducing the number of agents? Otherwise, it should actually cost more, before you simply had customers wait longer to speak to the next agent no? Or do you sell "support calls" so you're able to sell more of them given the same number of agents?

> it's a huge, measurable efficiency gain.

> It's not going to replace anyone's job

Mechanically, more efficiency means less people required for the same output.

I understand there is no evidence that any other sentence can be written about jobs. Still, you should put more text in between those two sentences. Reading them so close together creates audible dissonance.

I’d like to see a competent AI replace the time that doctors and nurses spend tediously transcribing notes into a medical record system. More time spent doing the actual job is good for pretty much everyone.

I wonder how this change affects what the agents remember about the calls, and how that affects their performance on future calls. And I wonder whether agent performance, as measured by customer satisfaction, will decline over time, and whether that will affect the bottom line.

I lead an AI engineering team that automated key parts of an interviewing process, saving thousands of hours each month by handling thousands of interviews. This reduced repetitive, time-consuming tasks and freed human resources to focus on higher-value work

I think the biggest issue is accurately estimating the LLM failure risk, and what impacts the company is willing to tolerate in the long term. (As distinct from what the company is liable to permit through haste and ignorance.)

With LLMs the risk is particularly hard to characterize, especially when it comes to adversarial inputs.

This is a great use-case of AI.

However I strongly doubt your point about "It's not going to replace anyone's job" and that "they also free up the human agents to do higher-value work". The reality in most places is that fewer agents are now needed to do the same work as before, so some downsizing will likely occur. Even if they are able to switch to higher-value work, some amount of work is being displaced somewhere in the chain.

And to be clear I'm not saying this is bad at all, I'm just surprised to see so many deluded by the "it won't replace jobs" take.

I wouldn't allow myself to be held accountable for anything in a summary I didn't write.

But I doubt it justifies the billions of dollars getting burned for training language models and building power plants.

And are full transcriptions not the better option?

> recently switched to using AI to transcribe and write these summaries

Did users knew that conversation was recorded?

Previously discussed here: https://news.ycombinator.com/item?id=44941118

It's also disappointing that MIT requires you to fill out a form (and wait for) access to the report. I read four separate stories based on the report, and they all provide a different perspective.

Here's the original pdf before MIT started gating it: https://web.archive.org/web/20250818145714/https://nanda.med...

i was pretty sure you were going to say "meeting summaries" ( which apparently is poster child of LLM application) .

my guess was wrong but not really.

Given that people skimp on work that is viewed as trash anyway, how were you getting value out of the summaries in the first place?

> We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls.

This is a tiny fraction of all work done. This is work people were claiming to have solved 15 years ago. Who cares?

> Not only are the summaries better than those produced by our human agents

We have someone using Firefly for note taking, and it's pretty bad. Frequently gets details wrong or extrapolates way too much from a one-off sentence someone said.

How do you verify these are actually better?

Are the summaries reviewed by the agents? And if not how do you handle hallucinations, or transcribe of wrong insurance policy id for example? Like, customer wants to cancel insurance policy AB-2345D and transcribe says wants to cancel insurance policy AD-2345B

That is a good thing, but that's also just a training gap - I worked with tech support agents for years in gigantic settings and taking notes while you take an action is difficult to train but yields tangible results, clarity from the agent on what they are doing step by step, and builds a shared method of documenting things that importantly focuses on the important details and doesn't miss a piece which may be considered trivial by an outsider but (for instance) defines SOP for call escalation and the like.

What’s the actual business value of a “summary” though? A transcript is the record. A tag or structured note (“warranty claim,” “billing dispute,” “out of scope”) is actionable. But a free-form blob of prose? That’s just narrative garnish - which, if wrong or biased, is worse than useless.

Imagine a human agent or AI summarises: “Customer accepted proposed solution.” Did they? Or did they say “I’ll think about it”? Those aren’t the same thing, but in the dashboard they look identical. Summaries can erase nuance, hedge words, emotional tone, or the fact the customer hung up furious.

If you’re running a call centre, the question is: are you using this text to drive decisions, or is it just paperwork to make management feel like something is documented? Because “we saved millions on producing inaccurate metadata nobody really needs” isn’t quite the slam dunk it sounds like.

You could do that offline 20 years ago with Dragon Naturally Speaking and a one-time licence.

How can you double-check the work? Also, what happens when the AI transcription is wrong in a way that would have terminated the employee. You can't fire a model.

Finally, who cares about millions saved (while considering the above introduced risk), when trillions are on the line?

I wouldn’t be surprised if 95% of companies knew this was a money pit but felt obligated to burn a pile of money on it so as not to hurt the stock price.

When OpenAI went from GPT-3.5-Turbo to GPT-4 it seemed massive, and there were no other steps in-between. And nobody else had a meaningful competitive model out yet.

When GPT-5 came out, it wasn't going from GPT-4 to GPT-5. Since GPT-4 there has been: 4o, o1, o3, o3-mini, o4-mini, o4-mini-high, GPT-4.1, and GPT-4.5. And many other models (Llama, DeepSeek, Gemini, etc) from competitors have been released too.

We'll probably never experience a GPT-3.5 to GPT-4 jump again. If GPT-5 was the first reasoning model, it would have seemed like that kind of jump, but it wasn't the first of anything. It is trying to unify all of the kinds of models OpenAI has offered, into one model family.

Sam Altman way oversold GPT-5's capabilities, in that it doesn't feel like a big leap in capability from a user's perspective; however, the a idea of a trainable dynamic router enabling them to run inference using a lot less compute (in aggregate) to me seems like a major win. Just not necessarily a win for the user (a win for the electric grid and making OpenAI's models more cost competitive).

Gemini keeps being rather impressive though, even their iterative updates have improvements, though I'm seeing a significant slowdown in the improvements (both quantity and in how much they improve) suggestion a wall may be approaching.

That said, technologies like this can also go through a rollercoaster pattern itself. Lots of innovation and improvement, followed by very little improvement but lots of research, which then explodes more improvements.

I think LLMs have a better chance at following that pattern than computer vision did when that hype cycle was all the rage

Good point. I wonder if the Windsurf folks saw the writing on the wall and cashed out when they could.

Perhaps just a trough of disillusionment for OpenAI. Anthropic and Google keep delivering.

If GPT-5 had been released last year instead, they could have probably kept the hype manageable. But they waited too long and too greedily, and the launch fell flat. And in some cases even negative as I’m seeing some bad PR about people who got too attached to their GPT-4o lovers and hate the new GPT-5.

Yo what’s the next hype cycle that smart folks like us should be working on?

Prepare for the crash:

"Spending on AI data centers is so massive that it’s taken a bigger chunk of GDP growth than shopping" - https://fortune.com/2025/08/06/data-center-artificial-intell...

AI Customer Service is very frustrating to work with as a end user.

I am working on a project that uses LLM to pull certain pieces of information from semi-structured documents and then categorize/file them under the correct account. it's about 95% accurate and we haven't even begun to fine tune it. i expect it will require human in the loop checks for the foreseeable future, but even with a human approval of each item, its going to save the clerical staff hundreds of hours per year. There are a lot of opportunities in automating/semi-automating processes like this, basically just information extraction and categorization tasks.

One use case I'd love to see an easy plug-and-play solution for is a RAG build around companies vast internal documentation/wikis/codebase to help developers onboard and find information faster. I would love to see less of people trying to replace humans with language models and more of people trying to use language models to make humans jobs less frustrating.

+50% productivity for 200$/mo is outstanding value! Most countries have 0-2% productivity growth per year!

Stock images. I’ve already seen trining courses (for compliance reasons) using AI videos. A bit cringey, but I imagine cheaper than shooting real people.

MS Copilot is quite useful for meeting minutes and summaries etc. Still not nearly as useful as good handwritten notes but saves loads of time.

> Replace junior developers with Claude Code or similar.

I don't know why everyone goes to "replacing". Were a bunch of computer programmers replaced when compilers came out that made writing machine code a lot easier? Of course not, they were more productive and accomplished a lot more, which made them more valuable, not less.

Even “only sort of works” is too generous for point #2. A dozen Claude Code agents spamming out code is… something. But it still does not replace a human, at all, even a junior one. It’s something else entirely.

#5 is an enormous use case that when well implemented will permanently replace prescribed character arcs

It is uniquely susceptible because the gaming market is well acclimated to mediocre writing and one dimensional character development that’s tacked on to a software product, so the improvements of making “thinking” improvisational characters can be immense.

Another revenue potential you’ve missed is visual effects, where AI tools allow what were previously labor intensive and expensive projects to be completed in much less time and with less, but not no, human input per frame

I wouldn't even call it a feature. It's enabling technology. I've never once said "I would like AI in [some product]." I say: "I would like to be able to [do this task]." If the company adds that feature to a product, I'll buy it. I don't care if the company used AI, traditional algorithms, or sorcery to make the feature work--I just care that it does what I want it to do.

Too many companies are just trying to spoon AI into their product somehow, as if AI itself is a desired feature, and are forgetting to find an actual user problem for it to actually solve.

All true, but then there goes your stratospheric valuations and all the crazy hype. This come to jesus moment may very well deflate one of the few remaining hot areas around software engineering..I could see people being reluctant to stop the hype train as then we'd really have to come to terms with the fact that the "industry" as a whole is kind of in the shitter and it's a less good time to be a software engineer across the board than 5 or 10 years ago.

I wouldn't mind it if it were presented as yet another tool in the box. Maybe have a one-time popup saying "Hey, there's this thing, here's a cool use case, go check it out on your own terms."

In reality, AI sparkles and logos and autocompletes are everywhere. It's distracting. It makes itself the star of the show instead of being a backup dancer to my work. It could very well have some useful applications, but that's for users to decide and adapt to their particular needs. The ham-fisted approach of shoving it into every UI front-and-center signals a gross sense of desperation, neediness, and entitlement. These companies need to learn how to STFU sometimes.

1000% agree, so many AI "applications" right now are solutions looking for a problem.

Seriously! The product itself is supposed to be the valuable thing…regardless of the underlying technology.

Agreed. We’ve got the potential to build real bicycles for the mind here and marketing departments are jumping right in to trying to sell people spandex cycling shorts.

I like this take, in fact i feel a little uneasy when i see startups mention "MCP" on their landing pages! Its a protocol and its like saying we use HTTP here.

I could be wrong but, all in all, buy a .com for your "ai" product, such that you survive the Dot-ai bubble [1]

I Love LLM's though!! Amazing math and tech.

[1] - https://en.wikipedia.org/wiki/Dot-com_bubble

Try to click the author name. It doesn't work. This isn't a real author or a real site.

HN is turning into reddit, where people look at the title, come to the comments, and post if they agree with the title or not.

> affecting 39 million positions

Wow, that is crazy. There's 163 million working Americans, that's close to a quarter of the workforce is at risk.

> Lots of people here are jumping to conclusions. AI does not work. I don't think that's what the report says.

Well...

"It is difficult to get a man to understand something when his salary depends upon his not understanding it"

It is rare that a third of the capital in the US stock market (by some recent estimates) is going for essentially the same speculative bet, though.

I’m not sure you can even generalise that much anymore. The US banking system is stuck in the dark ages compared to other countries, China is leading in electric cars, photovoltaics. and lots of other industries.

You could make that claim for the software industry, but I’m pretty sure a big part of the US moat is due to oligopolies, lock-in effects, or corruption in favour of billionaires and their ventures.

> This is how America ends up being ahead of the rest of world with every new technology breakthrough.

Too young to remember GSM?

That must be exactly what Napoleon felt like when he entered Moscow. Unstoppable army, sure riding right into what is obviously hubris, sure having have had a bit of a setback against Kutusov, but man were they on a winning streak!

I suspect there's a lot of nuance you can't really capture in a study like this.

How you use AI will depend on the model, the tools (claude-code vs cursor vs w/e), your familiarity and process (planning phases, vibe coding, etc.), and the team size (solo dev versus large team), your seniority and attention to detail, and hard to measure effects like an increased willingness to tackle harder problems you may have procrastinated on otherwise.

I suspect we're heading to a plateau. I think there's a ton of polish that can be done with existing models to improve the coding experience and interface. I think that we're being massively subsidized by investors racing to own this market, but by the time they can't afford to subsidize it anymore, it'll be such a commodity that the prices won't go up and might even go down regardless of their individual losses.

As someone who knows they are benefitting from AI (study shmuddy), I'm perfectly fine with things slowing down since it's already quite good and stands to be much better with a focus on polish and incremental improvements. I wouldn't invest in these AI companies though!

I sort of wonder if AI pushes people to stay "in the weeds". I've noticed that the speed of my software development hinges a lot on decisions that require me to take a step back (e.g. Am I using the right tool? Is this feature necessary? etc)

Sometimes it's like an autocomplete on steroids and reads my mind. Other times it suggests things that make no sense and completely gets in the way.

An interesting study indeed. Not enough data points but still way better than anecdata and self reporting!

Started an entire consulting practice to get engineering teams and founders out of vibe coded pits. Even got a great domain for it - vibebusters

So far business is booming and clients are happy with both human interactions with senior engineers as well as a final deliverable on best practices for using AI to write code.

Curious to compare notes

Good thing consulting is one thing AI is decent at

Do you sell overpriced generative AI solutions, or do you consult them on how to pivot away from idiotic generative AI?

or steel and wood for making shovels, TSMC

The shovel business is good as long as the gold rush lasts. Once the gold rush is over, you're going to have to deal with a significant decrease in volume, unless you can find other customers.

Crypto's over, gaming isn't a large enough market to fill the hole, the only customers that could fill the demand would be military projects. Considering the arms race with China, and the many military applications of AI, that seems the most likely to me. That's not a pleasant thought, of course.

The alternative is a massive crash of the stock price, and considering the fact that NVIDIA makes up 8% of everyone's favorite index, that's not a very pleasant alternative either.

It seems to me that an ultra-financialized economy has trouble with controlled deceleration, once the hypetrain is on it's full-throttle until you hit a wall.

Are there really that many consultants floating around still? I remember the heyday of the 2000s and kind of thought the outside consultants were largely disappearing and nowadays companies have trained themselves to jump on the hype train without paying consultants to push them?

I don't know?

For some reason, I'm thinking most of the money went to either inferencing costs or NVidia.

It will probably all be used to mine bitcoin.

> AI is improving employee productivity, but that's not leading to more business profit... how does that happen?

One trivial way is that the increase of productivity is less than the added cost of the tools. Which suggests that (either due to their own pricing, or just mis-judgement) the AI companies are mis-pricing their tools. If the tool adds $5000 in productivity, it should be priced at $4999, eventually -- the AI companies have every motivation to capture nearly all of the value, but they need to leave something, even if just a penny, for the purchasing company to motivate adoption. If they're pricing at $5001, there's no motivation to use the tool at all; but of course at $4998 they're leaving money on the table. There's no stable equilibrium here where the purchasing companies end up with a /significant/ increase in (productivity - cost of that productivity), of course.

Maybe they’re measuring productivity by flawed metrics. One could write 10x as much code, but that doesnt mean that will equate to more profit

Possible conclusion: most of the work that employees do has no direct impact on earnings.

Except generative AI is mostly an arms race, with content being generated being processed by some other AI on the other side. And any humans unfortunate enough to be in the middle regretting being born.

> It’s pretty clear to anyone who’s using this technology that it’s significant

I disagree entirely. It’s neat, and it’s a marginal improvement over current-year google, but significant is an overstatement.

I think people want to see the mindless application of crappy LLM chatbots and AI summaries everywhere fail. At least that's my position. I would also like the notion that "development can be sped up 6-700% by applying AI" would go away.

We dont. People just tired of listening to the pipe dreams of these weirdos CEOs who sell AI.

Humans like knowing their next paycheck is safe. AI is a disruption to that feeling of security.

I don’t want to see it fail.

I do want people to understand what they are discarding when they use it to replace human creativity and human contact. All the evidence is that AI bros still think artists are evil gatekeepers.

And I absolutely want to see the bubble burst. I see absolutely no reason to be excited on the behalf of Silicon Valley vampires and their latest, soul-crushing innovation. If it’s making Andreessen Horowitz a stack of money, its value to humanity is already more questionable.

Because it's going to destroy knowledge work and the entire middle class.

Why do people so desperately want to see AI succeed? The financial investment explains it for some.

The Trough of Disillusion. Though we definitely aren't down there yet, it's coming.

You just wait, when we achieve agi it’ll drive hype cycles at a speed humans have no chance of keeping up with. We will always effectively be in all points of the hype cycle at once. Money will wash randomly through the economy like soap through your laundry. You ain’t seen nothing yet.

Man, if people aren’t on the plateau of productivity by now with genai, it’s their own damn fault. Trough of disillusionment was like, 2020 AI projects.

Can you name some of those products?

> who later admitted he had not checked.

Incorrect. He did check, and decided to lie.

The internet is about sharing actual information. LLMs can't share real information, just a flimsy derivative. It's in their DNA. How so many supposedly smart people who understand machine learning and its fundamental entropy failed to acknowledge this reality from day zero, is beyond me.