"Alan: Sure, yep, so one of the things that we felt like on MI350 in this timeframe, that it's going into the market and the current state of AI... we felt like that FP6 is a format that has potential to not only be used for inferencing, but potentially for training. And so we wanted to make sure that the capabilities for FP6 were class-leading relative to... what others maybe would have been implementing, or have implemented. And so, as you know, it's a long lead time to design hardware, so we were thinking about this years ago and wanted to make sure that MI350 had leadership in FP6 performance. So we made a decision to implement the FP6 data path at the same throughput as the FP4 data path. Of course, we had to take on a little bit more hardware in order to do that. FP6 has a few more bits, obviously, that's why it's called FP6. But we were able to do that within the area of constraints that we had in the matrix engine, and do that in a very power- and area-efficient way.
treesciencebot · 3h ago
the main question is going to be software stack. NVIDIA is already shipping NVFP4 kernels and perf is looking good. It took a really long time after MI300X's that the FP8 kernels were OK (not even good, compared to almost perfect FP8 support in NVIDIA side of things).
I will doubt that they will be able to reach %60-70 of the FLOPs in majority of the workloads (unless they hand craft and tune a specific GEMM kernel for their benchmark shape). But would be happy to be proven wrong, and go buy a bunch of them
pella · 2h ago
(related)
Tinygrad:
"We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source!"
"Contract is signed! No confidentiality, AMD has leadership that's capable of acting. Let's make this training run happen, we work in public on our Discord.
Does this also ship only in x8 batches? I really liked MI300 and could afford
one of them for my research, but they only come in batches of x8 in a server rack, so I decided to buy an RTX Pro 6000.
jiggawatts · 4h ago
Of course not.
AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers.
This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market.
It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.
rfv6723 · 2h ago
These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.
AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.
AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.
behnamoh · 2h ago
> these people
we're talking about the majority of open source developers (I'm one of them). if researchers don't get access to hardware X, they write their paper using hardware Y (Nvidia). AMD isn't doing fine because most low level research on AI is done purely on CUDA.
qualifiedeephd · 2h ago
Serious researchers use HPC clusters not desktop workstations. Currently the biggest HPC cluster in North America has AMD GPUs. I think it'll be fine.
uniclaude · 40m ago
Neither their revenue nor their market share in the space looks like just fine. What exactly in trailing the market for years is “just fine”?
AMD is very far behind, and their earnings are so low that even with a nonsensical pe ratio they’re still less than a tenth of nvidia. No, they are not doing anywhere near fine.
Are hobbyists the reason for this? I’m not sure. However, what AMD is doing is clearly failing.
creato · 28m ago
When you design software for N customers, where N is very small, and you expect to hold each customers' hand individually, the software is basically guaranteed to be hot garbage that doesn't generalize or actually work except in exactly the use cases you supported (there are exceptions to this, but it requires having exceptional software engineers and leaders that care about doing things correctly and not just closing the next ticket, and in my experience, they are extremely rare).
If you design software for N00000 customers, it can't be shit, because you can't hold the hands of that many people, it's just not possible. By intending to design software for a wide variety of users, it forces you to make your software not suck, or you'll drown in support requests that you cannot possibly handle.
almostgotcaught · 2h ago
> These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.
this guy gets it - absolutely no one cares about the hobby market because it's absolutely not how software development is done (nor is it how software is paid for).
justahuman74 · 50m ago
The hobby market is how you get 'market default' N years later
almostgotcaught · 37m ago
citation please
pstuart · 1h ago
The hobby market should be considered as a pipeline to future customers. It doesn't mean AMD should drop everything and cater specifically to them, but they'd be foolish to ignore them altogether.
no we're not broke! we constantly write grants and receive funding from various sources. guess what hardware we recommend the University to purchase? it's 99.9% Nvidia, and sometimes Mac Studio just to play with MLX.
teleforce · 4h ago
This 8-combo MI350 is a beauty with 2304 GB VRAM of HMB3E memory on each UBB [1].
I've got the MI300x and I can't wait to deploy a bunch of the MI355's.
AbuAssar · 2h ago
If MI350 employs CDNA, which is based on the VEGA (GCN) architecture, does that imply that MI400, when introduced next year, will skip the 2020 GCN and directly transition to RDNA 5 equivalent?
A solid 40% of George's questions were deemed great. (Not counting some fluff like what's your job.)
jonfromsf · 3h ago
NVDAs advantage is software, not just hardware. Would be amazing to have a competitive market but better hardware won't be enough to make it happen.
deadbabe · 4h ago
Will AMD catch up to Nvidia?
AzzyHN · 23m ago
On the consumer side, almost certainly not. Nvidia is a HUGE brand name, it doesn't matter how good and cheap AMD makes their consumer GPUs, people will buy Nvidia GPUs for the brand and prebuilts will stick with Nvidia for the name.
For AI chips... also probably not, unless AMD can compete with CUDA (or CUDA becomes irrelevant)
mobilio · 3h ago
If they improve software quality and providing some low budget versions then - Yes.
I will doubt that they will be able to reach %60-70 of the FLOPs in majority of the workloads (unless they hand craft and tune a specific GEMM kernel for their benchmark shape). But would be happy to be proven wrong, and go buy a bunch of them
Tinygrad:
" https://x.com/__tinygrad__/status/1935364905949110532See https://arxiv.org/abs/2402.17764
AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers.
This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market.
It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.
AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.
AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.
we're talking about the majority of open source developers (I'm one of them). if researchers don't get access to hardware X, they write their paper using hardware Y (Nvidia). AMD isn't doing fine because most low level research on AI is done purely on CUDA.
AMD is very far behind, and their earnings are so low that even with a nonsensical pe ratio they’re still less than a tenth of nvidia. No, they are not doing anywhere near fine.
Are hobbyists the reason for this? I’m not sure. However, what AMD is doing is clearly failing.
If you design software for N00000 customers, it can't be shit, because you can't hold the hands of that many people, it's just not possible. By intending to design software for a wide variety of users, it forces you to make your software not suck, or you'll drown in support requests that you cannot possibly handle.
this guy gets it - absolutely no one cares about the hobby market because it's absolutely not how software development is done (nor is it how software is paid for).
[1] This is the AMD Instinct MI350:
https://www.servethehome.com/this-is-the-amd-instinct-mi350/
https://www.tomshardware.com/pc-components/gpus/amd-says-ins...
AMD went down the wrong path by focusing on traditional rendering instead of machine learning.
I think future AMD consumer GPUs would go back to GCN.
https://llvm.org/docs/AMDGPUUsage.html#id38
For AI chips... also probably not, unless AMD can compete with CUDA (or CUDA becomes irrelevant)