If MI350 employs CDNA, which is based on the VEGA (GCN) architecture, does that imply that MI400, when introduced next year, will skip the 2020 GCN and directly transition to RDNA 5 equivalent?
"Alan: Sure, yep, so one of the things that we felt like on MI350 in this timeframe, that it's going into the market and the current state of AI... we felt like that FP6 is a format that has potential to not only be used for inferencing, but potentially for training. And so we wanted to make sure that the capabilities for FP6 were class-leading relative to... what others maybe would have been implementing, or have implemented. And so, as you know, it's a long lead time to design hardware, so we were thinking about this years ago and wanted to make sure that MI350 had leadership in FP6 performance. So we made a decision to implement the FP6 data path at the same throughput as the FP4 data path. Of course, we had to take on a little bit more hardware in order to do that. FP6 has a few more bits, obviously, that's why it's called FP6. But we were able to do that within the area of constraints that we had in the matrix engine, and do that in a very power- and area-efficient way.
treesciencebot · 25m ago
the main question is going to be software stack. NVIDIA is already shipping NVFP4 kernels and perf is looking good. It took a really long time after MI300X's that the FP8 kernels were OK (not even good, compared to almost perfect FP8 support in NVIDIA side of things).
I will doubt that they will be able to reach %60-70 of the FLOPs in majority of the workloads (unless they hand craft and tune a specific GEMM kernel for their benchmark shape). But would be happy to be proven wrong, and go buy a bunch of them
kristianp · 1h ago
Will 1.58 bits be in the MI400? Or is it not established as a widely useful technology yet?
Does this also ship only in x8 batches? I really liked MI300 and could afford
one of them for my research, but they only come in batches of x8 in a server rack, so I decided to buy an RTX Pro 6000.
jiggawatts · 1h ago
Of course not.
AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers.
This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market.
It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.
rfv6723 · 19m ago
These ppl are very loud online, but they don't make decisions for hyperscalers which are biggest spenders on AI chips.
AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.
AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.
https://www.tomshardware.com/pc-components/gpus/amd-says-ins...
I will doubt that they will be able to reach %60-70 of the FLOPs in majority of the workloads (unless they hand craft and tune a specific GEMM kernel for their benchmark shape). But would be happy to be proven wrong, and go buy a bunch of them
See https://arxiv.org/abs/2402.17764
[1] This is the AMD Instinct MI350:
https://www.servethehome.com/this-is-the-amd-instinct-mi350/
AMD stubbornly refuses to recognise the huge numbers of low- or medium- budget researchers, hobbyists, and open source developers.
This ignorance of how software development is done has resulted in them losing out on a multi-trillion-dollar market.
It's incredible to me how obstinate certain segments of the industry (such as hardware design) can be.
AMD is doing just fine, Oracle just announced an AI cluster with up to 131,072 of AMD's new MI355X GPUs.
AMD needs to focus on bringing rack-scale mi400 as quickly as possible to market, rather than those hobbyists always find something to complain instead of spending money.