Do you think that the next big thing in AI is to respond in sub 100ms time?
1 defyonce 1 8/15/2025, 1:19:59 PM
I've been coding in Claude Code recently, and having to wait for it to compute kills all focus and attention. I have to switch constantly, and it makes me tired pretty soon.
I think, if that thing could respond instantly, like my Emacs for its commands, then it would be a supercharged mega boost for productivity.
If it could work with the same speed as a shell, then it would be really awesome!
What do you think?
One of the key issues of cloud gaming that has failed a few times now is that the datacenter servers need to be damned close to get that sub 100ms. Google didnt have enough datacenters to do it and Stadia died.
In a way it's the same problem, they need to get the gpus within probably like 50-100kms of the users. That's plausible to do, but that's doubling or tripling the number of operated datacenters.
If you look at chatgpt's stargate. They essentially will have 2 of them worldwide?
Anthropic/claude is beholden to AWS compute which is us-east-1 and us-west-2 only. I think AWS plans to open in signapore soon?
Elon's Grok is in memphis only; and there's drama around their generators and such?
Far too centralized for all the major players and I dont see any even medium term chances of getting out there enough. It's too much power, you pretty much have to park it near major generation.
What I think they are still doing is training/research, processing current load. Working on energy efficiency. That tops/wattage ratio but also figuring out caching and other super compute limits and constraints. That was the brilliance of MOE; but a major player ought to have a mixture of mixture of experts.
Next big thing for AI is cheaper cost picture and video generation. This is Veo4 or Sora or comfyui.
I'm also expecting a better google home or echo type device. Something that's AI and not hard coded skills; and ideally all local and no built in government tap.