This has been developing for a while... The big players have basically been competing for allocations of a set production, so NVIDIA negotiated into the allocations that some % of the compute capacity they "sell" them is reserved and exclusively leased back to NVIDIA.
So now NVIDIA has a whole bunch of cloud infrastructure hosted by the usual suspects that they can use for the same type of business the usual suspects do.
well played tbh
xhkkffbf · 29d ago
I hate to be cynical, but I've seen such leaseback schemes used to inflate sales. Is that the case here? Or is there enough demand or legit usage to justify this kind of arrangement?
kurthr · 28d ago
It's just another kind of leverage. There's no problem, until there is, and then it's bigger than it would have been. Once all of their sales are leasebacks, you'll know it's about to go boom (of course they won't announce that in their reports).
justahuman74 · 29d ago
I can't see the cloud providers being happy about this, it whitelabels away their branding and customer experience flows.
It puts nvidia on both the vendor and customer side of the relationship, which seems odd
ketzo · 29d ago
Well, what are they gonna do about it?
Nvidia has the most desirable chips in the world, and their insane prices reflect that. Every hyperscaler is already massively incentivized to build their own chips, find some way to take Nvidia down a peg in the value chain.
Everyone in the world who can is already coming for Nvidia’s turf. No reason they can’t repay the favor.
And beyond just margin-taking, Nvidia’s true moat is the CUDA ecosystem. Given that, it’s hugely beneficial to them to make it as easy as possible for every developer in the world to build stuff on top of Nvidia chips — so they never even think about looking elsewhere.
Xevion · 29d ago
While I don't dispute that they're objectively the most desirable at the current moment - I do think your comment implies that they deserve it, or that people WANT Nvidia to be the best.
It almost sounds like you're cheering on Nvidia, framing it as "everyone else trying to reduce the value of Nvidia", meanwhile they have a long, long history of closed-source drivers, proprietary & patented cost-inflated technology that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies.
"Well, what are they gonna do about it?"
When dealing with a bully, you go find friends. They're going to fund other chip manufacturers and push for diversity, fund better drivers and compatibility.
That's the best possible future anyone could hope for.
Dr4kn · 29d ago
I don't think this problem is going to be solved by hyper scalers offering their own accelerators. They probably offer better price to performance, but try to lock you into their ecosystem.
With the Nvidia solution you have at least another option. Vendor agnostic, but Nvidia lock in.
If most ML startups, one hyper scaler and at best also AMD, would go with one common backend, then it might get enough traction to become *the* standard.
almostgotcaught · 29d ago
> that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies.
1. "Identical if not for market share" is a complete contradiction when what we're talking about is the network effect of CUDA
2. What vendor lock in? What are you talking about? They have a software and compiler stack that works with their chips. How is that lock in, that's literally just their product offering. In fact the truth is you can compile CUDA for AMD (using hipify) and guess what - the result sucks because AMD isn't a comparable alternative!
Ygg2 · 29d ago
> In fact the truth is you can compile CUDA for AMD (using hipify)
You can compile x64 to ARM and performance tanks. Does this means ARM isn't a comparable alternative to x64?
It just means their software works badly with said architecture. Could be that AMD acceleration is horrible (but then the FSR would be worse) or it could be that it's just different, or the translation layer is bad.
almostgotcaught · 29d ago
> or the translation layer is bad
There's no translation layer - you don't understand how/what hipify and CUDA are. CUDA is a C/C++ extension and it connotes APIs. 90% of CUDA kernel code (ie the stuff that actually runs on the SMs) does compile for AMD without any changes (intrinsics diff). hipify goes the extra step of remaining APIs to their HIP variants.
Again, all of this is to say there's no vendor lockin like clueless whiny people complain and just a superior product.
Ygg2 · 29d ago
The point was, I don't expect it to work or work smoothly. CUDA was made for Nvidia, same for ROCm on AMD. Comparing CUDA on AMD or ROCm isn't a fair
If Nvidia is better at AI tasks and is superior. Great. Maybe they can finally leave GPU field.
almostgotcaught · 29d ago
> CUDA was made for Nvidia, same for ROCm on AMD
I'm gonna say it again, loud and clear: you don't have any understanding of what you're saying and 90% of the kernel code is exactly the same, transferrable, compilable ie it's just cpp.
Ygg2 · 28d ago
Then elighten me, how is API made to work for Nvidia cards going to work smoothly for AMD.
Nothing you said prevents API makers of biasing their API to favor one hardware platform over the other.
EDIT: Which CUDA to AMD GPU translation project are you referring to? AMD's original efforts or ZLUDA?
cpatil · 28d ago
Getting a bit difficult to understand your point of view here. The simple fact is NVDA executed well, had the strategic vision from 2006-7 onwards to invest in R&D, build complex libraries encompassing various complicated algorithms as well as allocated precious chip area to support these when no one was using the GPUs for those purposes. They took a risk.
I don't use Apple, but I don't complain abt them. You are free to use AMD if you so desire. Why the hate?
Ygg2 · 28d ago
> The simple fact is NVDA executed well, had the strategic vision from 2006-7 onwards to invest in R&D...
The issue here isn't as much as nVidia as it is nVidia fanboism and intellectually dishonest argument.
What determines the speed of an algorithm, all things being similar, is the raw power of hardware underneath. For that, you use the device drivers, use their equivalent level (e.g. low level on both) APIs and let them rip. You want to have as equal as comparison as you want.
What I don't expect is to take nVidia drivers, load them onto the AMD graphics card and then when the thing glitches out or underperforms say - see, it's bad.
The fact is that Hipify on AMD isn't the fastest way to run CUDA code on AMD anymore. Not since ZLUDA was created. Which raises unfortunate implications. Why wasn't Hipify able to reach the same performance? Maybe because it's a shitty translation layer. Who knows?
> I don't use Apple, but I don't complain abt them.
Just because you don't use them, doesn't mean they don't negatively impact the world in a huge way. Looks at the app store, Apple's penchant for proprietary charges, and the constant phone upgrade treadmill.
almostgotcaught · 28d ago
i can't explain things to people that are so steadfastly ignorant. google is free.
Ygg2 · 28d ago
"If you can't explain it simply enough then you don't understand well enough."
jsosshbfn · 29d ago
Yea but you're comparing a mid-level api to an architecture. It's just a category error. It's like saying C is just a PDP frontend.
_zoltan_ · 29d ago
Ugh, that's such a bad take. why wouldn't you cheer for NVIDIA? They had the discipline, the courage and the long term vision that nobody else did for the last 20 years.
Closed source?? Who cares? It's their own products.
Vendor lock in? It's their own chips man. You wouldn't expect Nvidia to develop software for AMD chips would you? That would be insane. I would not do that.
Their tech is superior to everybody else's and Jensen keeps pulling rabbits out of a hat. I hope they keep going strong for the next decade.
Ygg2 · 29d ago
> why wouldn't you cheer for NVIDIA?
Because they are an amoral mass that seeks to make profit and has turned GPU market into a cluster fuck?
> Their tech is superior to everybody
Their only saving grace is CUDA, and DLSS, their hardware has been overvalued for quite some time.
ksec · 29d ago
>their hardware has been overvalued for quite some time.
It is often such a strange thing to see this on HN. From Software developers.
Their Hardware's value is derived from their Software. And Software for GPU is insanely hard. Both the driver and CUDA.
As Jensen once said, their Goal is to make the TCO ( Total Cost of Ownership ) so good, that even if their competitor were selling at cost of giving their GPU away from free they still would not be able to compete with them.
There will also come a point, may be in the next 2-3 years where the volume and margin of those GPU are so good they will be the second in line to take all the Fab capacity for larger die size on leading node. i.e They will always be one node ahead of their competitors. And when that happens both hardware and software will be ahead of everyone else.
Ygg2 · 29d ago
> From Software developers.
As a developer, I have managed to stay outside their (nVidia and Apple both) moats. And what I've seen, as a consumer, has left me wanting. Granted m* battery life is impressive, but I'm not that much of a laptop person.
But I'd love for someone to enlighten me how a 16GB RAM upgrade with $200 dollar tag is any way normal.
> Their Hardware's value is derived from their Software
Their value is derived from their lock-in. If you bought into it, then yeah, it's going to be difficult to switch. OTOH, if you didn't, then there is almost no value.
> As Jensen once said
As Todd Howard once said - Sixteen times the detail![0]
Anecdotes aside, how is that working for nVidia? Oh, they just blackmailed GPU reviewers[1] and their GPU drivers randomly flicker, and cause kernel reboots[2]? Yeah. I definitely feel the TCO getting good, maybe even burnt. Much like their 12VHPWR connections.
But maybe they will fare much better on B2B, I couldn't tell you or care much about it. I honestly wish them a very SGI-experience. And seeing how they weathered the last craze (see cryptocurrency), I wouldn't bet my livelihood on it.
it's funny that you're talking about their consumer fiascos in a thread that discussed enterprise hardware. completely unrelated.
ksec · 28d ago
That is generally a problem on the internet and may be mostly US?
I dont like company X, their product must be shit.
It seems most people dont value product quality anymore.
winterbloom · 29d ago
Couldn't you replace Nvidia with apple here
They have a nice software stack but the hardware is overvalued
Ygg2 · 29d ago
Mostly. I'm not an Apple fan either.
To be fair to Apple their hardware was always overpriced. Their deal is hardware + software combo.
mustyoshi · 29d ago
Arguably, CUDA is the current best in class software for it's market.
amarcheschi · 29d ago
We can also not cheer about apple imho
I mean, we could probably not cheer about big techs that routinely do shady things - or straight illegal things - for their own profit knowing they won't face consequences - or very light ones
chii · 29d ago
> Nvidia’s true moat is the CUDA ecosystem.
it is true, but also not. nvidia is certainly producing a chip that nobody else can replicate (unless they're the likes of google, and even they are not interested in doing so).
The CUDA moat is the same type of moat as intel's x86 instruction set. Plenty of existing programs/software stack have been written against it, and the cost to migrate away is high. These LLM pipelines are similar, and even more costly to migrate away.
But because LLM is still immature right now (it's only been approx. 3 yrs!), there's still room to move the instruction set. And middleware libraries can help (pytorch, for example, has more than just the CUDA backend, even if they're a bit less mature).
The real moat that nvidia has is their hardware capability, and CUDA is the disguised moat.
david-gpu · 29d ago
> The real moat that nvidia has is their hardware capability, and CUDA is the disguised moat.
There is an inmense amount of work behind the cuDNN libraries that outsiders keep ignoring.
These sort of high performance kernels are co-developed in very close collaboration with the hardware architects designing the chip. Speaking of the hardware in isolation of the high performance libraries reveals a deep misunderstanding of how the system was built. This is true of any mature vendor, not just Nvidia.
rvnx · 29d ago
Google is doing the TPUs, isn’t it exactly for that ?
chii · 29d ago
iirc, google's stuff is only for google, and they're not selling it as as something that others can buy.
I suppose this can change.
tester756 · 29d ago
>it is true, but also not. nvidia is certainly producing a chip that nobody else can replicate
AMD, Intel?
_zoltan_ · 29d ago
NVLink says hello. Then rack scale NVLink says hello...
Nobody can touch it. Then that's just the hardware. The software is so much better on Nvidia. The width and breadth of their offering is great and nobody is even close.
tester756 · 29d ago
UALink?
>Ultra Accelerator Link (UALink) is an open specification for a die-to-die interconnect and serial bus between AI accelerators. It is co-developed by Alibaba, AMD, Apple, Astera Labs,[1] AWS, Cisco, Google, Hewlett Packard Enterprise, Intel, Meta, Microsoft and Synopsys.[2]
[Genuine question!] Does NVidia have patents etc on CUDA that prevent a competitor from reverse engineering and producing a compatible clone, or is it just that competitors are incompetent (hey AMD)? Or is it that the task is enormous and rapidly changing, like you have to be bug-for-bug compatible with a large, ill-documented API (the Microsoft Windows moat)?
paulryanrogers · 29d ago
Competitors have tried. It's just chasing a moving target, so you're guaranteed to be behind and always playing catch-up.
imtringued · 29d ago
Tenstorrent is doing pretty well software wise.
imtringued · 29d ago
Google has already mostly exited the CUDA and Nvidia ecosystem. They only offer it for their customers at this point.
mattlondon · 29d ago
Ask Google: TPUs.
roenxi · 29d ago
> Well, what are they gonna do about it?
Ironically, the most effective thing they can do is probably haul up the AMD rep and yell at them.
alexgartrell · 29d ago
The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.
cbg0 · 29d ago
I've always felt that the business model is nickel & diming for things like storage/bandwidth and locking in customers with value-add black box services that you can't easily replace with open source solutions.
Just took a random server: https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=mo... - to get a decent price on it you need to commit to three years at $570 per month(no storage or bandwidth included). Over the course of 3 years that's $20520 for a server that's ~10K to buy outright, and even with colo costs over the same time frame you'll spend a lot less, so not exactly crushing those margins to dust.
shrubble · 29d ago
Cloud is propped up by the tax laws.
Cloud bills can be written off in the month in which they are paid; while buying hardware has to be depreciated over years.
sokoloff · 29d ago
Section 179 allows immediate expensing of equipment including computers, but is limited to $1.25M/yr. That’s enough for many small and medium businesses.
Hilift · 29d ago
> DGX Cloud Lepton, is designed to link artificial intelligence developers with Nvidia’s network of cloud providers, which provide access to its graphics processing units, or GPUs. Some of Nvidia’s cloud provider partners include CoreWeave, Lambda and Crusoe.
> "Nvidia DGX Cloud Lepton connects our network of global GPU cloud providers with AI developers," said Jensen Huang, chief executive of Nvidia in a statement. The news was announced at the Computex conference in Taiwan.
Sounds like a preferred developer resource. The target audience isn't the usual cro-mag that wants to run LLM's for food.
xbmcuser · 29d ago
This is what Nvidia has always done creeping into the margins of its partners and taking over. All it's gpu board partners will tell the same story.
neximo64 · 29d ago
Why? If the GPUs are used and they want more it makes it easy, also its opt-in.
moralestapia · 29d ago
Cool, they're also free to start making their own GPUs ...
seydor · 29d ago
google make their own chips too
londons_explore · 29d ago
TPU's have serious compatibility problems with a good chunk of the ML ecosystem.
That alone means many users will want to use Nvidia hardware even at a decent price premium when the alternative is an extra few months of engineering time in a very fast moving market.
jszymborski · 29d ago
I haven't worked with TPUs, but my understanding is that they are pretty plug-n-play for Google frameworks (JAX, TF) but is also pretty simple to use with PyTorch [0]. That covers nearly all of the marketshare
Certainly not plug-and-play, and perhaps this is just one of many bugs, but if it isn't I can't imagine a scenario where LazyLinear layers are essential.
que-encrypt · 28d ago
pytorch xla is barely supported in the pytorch ecosystem (for instance, pytorch lightning still doesn't easily support tpu pods, with only a singular short page about google colab v2-8 tpus that is out of date. Then there are the various libraries/implementations with pytorch that have a .cuda(), etc. More limitations at: https://lightning.ai/docs/pytorch/stable/accelerators/tpu_fa...). I haven't worked with tensorflow, but I've heard it's a pain even when using gpus. JAX is the real deal, and does make my code transferrable between GPUs/TPUs relatively easily (excluding any custom pallas kernels for flash vs splash attention, but this is usually not a massive code change). However, with JAX, there are often not a bunch of pre-existing implementations due to network effects, etc.
I can see the value of the product, but this seems like an incredibly dangerous offering for smaller clouds. Nvidia has significant leverage to drive prices down to commodity and keep any margin for themselves, while pushing most of the risk onto their partners.
alexgartrell · 29d ago
I’d imagine that these clouds are probably being incentivized to participate
bgwalter · 29d ago
Well, Sun Microsystems launched a cloud shortly before being acquired by Oracle:
But don't let that stop you from going outside your core competency.
londons_explore · 29d ago
Isn't that rather stepping on the toes of your biggest clients - Microsoft, aws, gCloud, etc.
noosphr · 29d ago
All those customers are also building their own chips.
Having been a partner for Microsoft research I've also had them try and patent the stuff we were providing them.
In short with megacorps the only winning move is to fuck them faster than they can fuck you.
mi_lk · 29d ago
That’s a beautiful conclusion
aranchelk · 29d ago
Customers of those services have a lot of considerations, as long as Nvidia doesn’t undercut the prices too much, I think no.
Getting more developers creating more models that can then be run on those services will likely expand business for all of those vendors.
netfortius · 29d ago
Isn't their CEO the guy who called Trump re-industrialisation policies 'visionary'? [1] Maybe that's where the idea on cloud (as all others, nowadays) is coming from?!? ;->
No cloud provider is gonna see further price gouging from the company with the largest market share and think "Yeah, let's disconnect from the only remaining competitor, make sure every nail is in our coffin".
It's probably the opposite. I bet this move will lead to AMD's increased funding towards compatability and TPU development, in the hopes that they'll become a serious competitor to Nvidia.
chii · 29d ago
> AMD's increased funding towards compatability and TPU development
no investor is going to bet on the second-place horse. Because they would've done the betting _before_ nvidia became the winning powerhouse that it has become!
The fact is, AMD's hardware capability is just insufficient to compete, and they're not getting there fast enough - unlike the games industry, there's not a lot of low budget buyers here.
diggan · 29d ago
> Because they would've done the betting _before_ nvidia became the winning powerhouse that it has become!
Right, isn't that an argument to stop investing in nvidia, and hedge your bets by investing in current second-place horse in case it becomes the winning horse?
Assuming of course you think AMD has even a slight chance of becoming that winning horse.
chii · 28d ago
The fact is, AMD's stock price pre-chatgpt kinda tracks nvidia's.
But post chatgpt release, they diverged[0]. And this is what i am talking about regarding the betting before nvidia became the winning horse. Aint nobody betting on AMD any more.
So now NVIDIA has a whole bunch of cloud infrastructure hosted by the usual suspects that they can use for the same type of business the usual suspects do.
well played tbh
It puts nvidia on both the vendor and customer side of the relationship, which seems odd
Nvidia has the most desirable chips in the world, and their insane prices reflect that. Every hyperscaler is already massively incentivized to build their own chips, find some way to take Nvidia down a peg in the value chain.
Everyone in the world who can is already coming for Nvidia’s turf. No reason they can’t repay the favor.
And beyond just margin-taking, Nvidia’s true moat is the CUDA ecosystem. Given that, it’s hugely beneficial to them to make it as easy as possible for every developer in the world to build stuff on top of Nvidia chips — so they never even think about looking elsewhere.
It almost sounds like you're cheering on Nvidia, framing it as "everyone else trying to reduce the value of Nvidia", meanwhile they have a long, long history of closed-source drivers, proprietary & patented cost-inflated technology that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies.
"Well, what are they gonna do about it?" When dealing with a bully, you go find friends. They're going to fund other chip manufacturers and push for diversity, fund better drivers and compatibility. That's the best possible future anyone could hope for.
With the Nvidia solution you have at least another option. Vendor agnostic, but Nvidia lock in.
If most ML startups, one hyper scaler and at best also AMD, would go with one common backend, then it might get enough traction to become *the* standard.
1. "Identical if not for market share" is a complete contradiction when what we're talking about is the network effect of CUDA
2. What vendor lock in? What are you talking about? They have a software and compiler stack that works with their chips. How is that lock in, that's literally just their product offering. In fact the truth is you can compile CUDA for AMD (using hipify) and guess what - the result sucks because AMD isn't a comparable alternative!
You can compile x64 to ARM and performance tanks. Does this means ARM isn't a comparable alternative to x64?
It just means their software works badly with said architecture. Could be that AMD acceleration is horrible (but then the FSR would be worse) or it could be that it's just different, or the translation layer is bad.
There's no translation layer - you don't understand how/what hipify and CUDA are. CUDA is a C/C++ extension and it connotes APIs. 90% of CUDA kernel code (ie the stuff that actually runs on the SMs) does compile for AMD without any changes (intrinsics diff). hipify goes the extra step of remaining APIs to their HIP variants.
Again, all of this is to say there's no vendor lockin like clueless whiny people complain and just a superior product.
If Nvidia is better at AI tasks and is superior. Great. Maybe they can finally leave GPU field.
I'm gonna say it again, loud and clear: you don't have any understanding of what you're saying and 90% of the kernel code is exactly the same, transferrable, compilable ie it's just cpp.
Nothing you said prevents API makers of biasing their API to favor one hardware platform over the other.
EDIT: Which CUDA to AMD GPU translation project are you referring to? AMD's original efforts or ZLUDA?
The issue here isn't as much as nVidia as it is nVidia fanboism and intellectually dishonest argument.
What determines the speed of an algorithm, all things being similar, is the raw power of hardware underneath. For that, you use the device drivers, use their equivalent level (e.g. low level on both) APIs and let them rip. You want to have as equal as comparison as you want.
What I don't expect is to take nVidia drivers, load them onto the AMD graphics card and then when the thing glitches out or underperforms say - see, it's bad.
The fact is that Hipify on AMD isn't the fastest way to run CUDA code on AMD anymore. Not since ZLUDA was created. Which raises unfortunate implications. Why wasn't Hipify able to reach the same performance? Maybe because it's a shitty translation layer. Who knows?
> I don't use Apple, but I don't complain abt them.
Just because you don't use them, doesn't mean they don't negatively impact the world in a huge way. Looks at the app store, Apple's penchant for proprietary charges, and the constant phone upgrade treadmill.
Closed source?? Who cares? It's their own products. Vendor lock in? It's their own chips man. You wouldn't expect Nvidia to develop software for AMD chips would you? That would be insane. I would not do that.
Their tech is superior to everybody else's and Jensen keeps pulling rabbits out of a hat. I hope they keep going strong for the next decade.
Because they are an amoral mass that seeks to make profit and has turned GPU market into a cluster fuck?
> Their tech is superior to everybody
Their only saving grace is CUDA, and DLSS, their hardware has been overvalued for quite some time.
It is often such a strange thing to see this on HN. From Software developers.
Their Hardware's value is derived from their Software. And Software for GPU is insanely hard. Both the driver and CUDA.
As Jensen once said, their Goal is to make the TCO ( Total Cost of Ownership ) so good, that even if their competitor were selling at cost of giving their GPU away from free they still would not be able to compete with them.
There will also come a point, may be in the next 2-3 years where the volume and margin of those GPU are so good they will be the second in line to take all the Fab capacity for larger die size on leading node. i.e They will always be one node ahead of their competitors. And when that happens both hardware and software will be ahead of everyone else.
As a developer, I have managed to stay outside their (nVidia and Apple both) moats. And what I've seen, as a consumer, has left me wanting. Granted m* battery life is impressive, but I'm not that much of a laptop person.
But I'd love for someone to enlighten me how a 16GB RAM upgrade with $200 dollar tag is any way normal.
> Their Hardware's value is derived from their Software
Their value is derived from their lock-in. If you bought into it, then yeah, it's going to be difficult to switch. OTOH, if you didn't, then there is almost no value.
> As Jensen once said
As Todd Howard once said - Sixteen times the detail![0]
Anecdotes aside, how is that working for nVidia? Oh, they just blackmailed GPU reviewers[1] and their GPU drivers randomly flicker, and cause kernel reboots[2]? Yeah. I definitely feel the TCO getting good, maybe even burnt. Much like their 12VHPWR connections.
But maybe they will fare much better on B2B, I couldn't tell you or care much about it. I honestly wish them a very SGI-experience. And seeing how they weathered the last craze (see cryptocurrency), I wouldn't bet my livelihood on it.
[0]https://www.youtube.com/watch?v=r3rXKCT_STM
[1]https://www.youtube.com/watch?v=AiekGcwaIho
[2]https://www.youtube.com/watch?v=NTXoUsdSAnA
I dont like company X, their product must be shit.
It seems most people dont value product quality anymore.
They have a nice software stack but the hardware is overvalued
To be fair to Apple their hardware was always overpriced. Their deal is hardware + software combo.
I mean, we could probably not cheer about big techs that routinely do shady things - or straight illegal things - for their own profit knowing they won't face consequences - or very light ones
it is true, but also not. nvidia is certainly producing a chip that nobody else can replicate (unless they're the likes of google, and even they are not interested in doing so).
The CUDA moat is the same type of moat as intel's x86 instruction set. Plenty of existing programs/software stack have been written against it, and the cost to migrate away is high. These LLM pipelines are similar, and even more costly to migrate away.
But because LLM is still immature right now (it's only been approx. 3 yrs!), there's still room to move the instruction set. And middleware libraries can help (pytorch, for example, has more than just the CUDA backend, even if they're a bit less mature).
The real moat that nvidia has is their hardware capability, and CUDA is the disguised moat.
There is an inmense amount of work behind the cuDNN libraries that outsiders keep ignoring.
These sort of high performance kernels are co-developed in very close collaboration with the hardware architects designing the chip. Speaking of the hardware in isolation of the high performance libraries reveals a deep misunderstanding of how the system was built. This is true of any mature vendor, not just Nvidia.
I suppose this can change.
AMD, Intel?
Nobody can touch it. Then that's just the hardware. The software is so much better on Nvidia. The width and breadth of their offering is great and nobody is even close.
>Ultra Accelerator Link (UALink) is an open specification for a die-to-die interconnect and serial bus between AI accelerators. It is co-developed by Alibaba, AMD, Apple, Astera Labs,[1] AWS, Cisco, Google, Hewlett Packard Enterprise, Intel, Meta, Microsoft and Synopsys.[2]
Ironically, the most effective thing they can do is probably haul up the AMD rep and yell at them.
Just took a random server: https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=mo... - to get a decent price on it you need to commit to three years at $570 per month(no storage or bandwidth included). Over the course of 3 years that's $20520 for a server that's ~10K to buy outright, and even with colo costs over the same time frame you'll spend a lot less, so not exactly crushing those margins to dust.
Cloud bills can be written off in the month in which they are paid; while buying hardware has to be depreciated over years.
> "Nvidia DGX Cloud Lepton connects our network of global GPU cloud providers with AI developers," said Jensen Huang, chief executive of Nvidia in a statement. The news was announced at the Computex conference in Taiwan.
Sounds like a preferred developer resource. The target audience isn't the usual cro-mag that wants to run LLM's for food.
That alone means many users will want to use Nvidia hardware even at a decent price premium when the alternative is an extra few months of engineering time in a very fast moving market.
[0] https://docs.pytorch.org/xla/release/r2.7/learn/xla-overview...
[0] https://github.com/pytorch/xla/issues/9189
https://en.wikipedia.org/wiki/Sun_Cloud
Microsoft's Azure is reportedly a loss leader:
https://www.cnbc.com/2022/12/21/google-leaked-doc-microsoft-...
But don't let that stop you from going outside your core competency.
Having been a partner for Microsoft research I've also had them try and patent the stuff we were providing them.
In short with megacorps the only winning move is to fuck them faster than they can fuck you.
Getting more developers creating more models that can then be run on those services will likely expand business for all of those vendors.
[1] https://www.reuters.com/business/aerospace-defense/nvidia-ce...
No cloud provider is gonna see further price gouging from the company with the largest market share and think "Yeah, let's disconnect from the only remaining competitor, make sure every nail is in our coffin".
It's probably the opposite. I bet this move will lead to AMD's increased funding towards compatability and TPU development, in the hopes that they'll become a serious competitor to Nvidia.
no investor is going to bet on the second-place horse. Because they would've done the betting _before_ nvidia became the winning powerhouse that it has become!
The fact is, AMD's hardware capability is just insufficient to compete, and they're not getting there fast enough - unlike the games industry, there's not a lot of low budget buyers here.
Right, isn't that an argument to stop investing in nvidia, and hedge your bets by investing in current second-place horse in case it becomes the winning horse?
Assuming of course you think AMD has even a slight chance of becoming that winning horse.
But post chatgpt release, they diverged[0]. And this is what i am talking about regarding the betting before nvidia became the winning horse. Aint nobody betting on AMD any more.
[0] https://finance.yahoo.com/chart/AMD#eyJsYXlvdXQiOnsiaW50ZXJ2...