Faster small matrix, for AI. Yup, that seems like good fit for what folks want.
Supercharging the Local Data Share (LDS) that's shared by threads is really cool to hear about. 64 -> 160KB size. Writes into LDS go from 32B max to 128B, increasing throughout. Transposes, to help get the data in the right shape for its next use.
Really really curious to see what the UDNA unified next gen architectures look like, if they really stick to merging Compute and Radeon CDNA and RDNA, as promised. If consumers end up getting multi-die compute solutions that would be neat & also intimidatingly hard (lots of energy spent keeping bits in sync across cores/coherency). After Navi 4X ended up having its flagship cancelled way back now, been wondering. I sort of expect that this won't scale as nicely as Epyc being a bunch of Ryzen dies. https://wccftech.com/amd-enthusiast-radeon-rx-8000-gpus-alle...
robjeiter · 1d ago
When looking at inference is AMD already on par with Nvidia?
moondistance · 1d ago
Yes, for many applications.
Meta, OpenAI, Crusoe, and xAI recently announced large purchases of MI300 chips for inference.
MI400, which will be available next year, also looks to be at least on par with Nvidia's roadmap.
moondistance · 1d ago
(this is also why AMD popped 10% at open yesterday - this is a new development and talks from their 2025 "Advancing AI" event were published late last week + over the weekend)
christkv · 1d ago
Is the software stack still lacking?
OneDeuxTriSeiGo · 1d ago
Yeah it's still a few years behind but it's getting better. They are hiring software and tooling engineers like crazy. I keep tabs on some of the job slots companies have in our area and every time I check AMD they always have tons of new slots for software, firmware, and tooling (and this has been the case for ~3 years now).
They've been playing catch up after "the bad old days" when they had to let a bunch of people go to avoid going under but it looks like they are catching back up to speed. Now it's just a matter of giving all those new engineers a few years to get their software world in order.
storus · 1d ago
They pay hardware rates to software engineers (principal engineer at the salary level of a decent fresh graduate) so I won't be too optimistic about them attracting software people that would propel them forward.
OneDeuxTriSeiGo · 1d ago
At least where I live (very much not west coast), their SW and HW rates are at or above what we normally see in this area.
latchkey · 1d ago
Stock is undervalued. If you get in now and it pops over the next few years, it'll likely make up for lower compensation.
MegaButts · 1d ago
You don't need to work at AMD to buy their stock.
latchkey · 1d ago
True, but if you don’t have a job, where’s the money for buying stock coming from?
alemanek · 23h ago
If you are what AMD needs to catch up then you can just go work for NVidia for 3x the pay. This market sucks but top tier engineers in the niche they need are not a dime a dozen.
latchkey · 21h ago
It isn't always about the money.
MegaButts · 7h ago
Then why is your original comment about compensation?
latchkey · 7h ago
What I said was: “it’ll likely make up for lower compensation.”
The point is, someone might join AMD because they believe in the mission, not just for the paycheck. I followed that with: “It isn’t always about the money,” which is consistent with my original comment.
The real subtext is something I care deeply about: Nvidia is a monopoly. If AI is truly a transformative technology, we can’t rely on a single company for all the hardware and software. Viable alternatives are essential. I believe in this vision so strongly that I started a company to give developers access to enterprise grade AMD compute, back when no one was taking AMD seriously in AI. (Queue the HN troll saying that nobody still does.)
If the stock goes up while they’re there, great, that’s a bonus.
MegaButts · 6h ago
Your original comment only talked about compensation and why AMD's stock might make up for lower pay. This thread tangent (starting with your comment) was about compensation and explaining why working for AMD, if your goal is to maximize profit, is dumb. Adding "well there are other reasons" after your original comment doesn't change your original comment where you had none of that context.
In the context of maximizing compensation, working for AMD is dumb. Your own comments support this.
latchkey · 6h ago
"Then why is your original comment about compensation?"
I answered your very specific question, even gave you my own additional context as a friendly thing to do, and now you are going off on some sort of maximizing rant for what purpose?
¯\_(ツ)_/¯
MegaButts · 6h ago
I guess what I was politely trying to say is your original comment is misleading and pointless if your actual point is that things besides what you mentioned in your original comment are what really matter. So my point was - why make the original comment at all if it wasn't what you meant?
latchkey · 5h ago
Calling my comments pointless is unnecessary and not polite at all.
We’ll have to agree to disagree, I don’t think I was misleading. I’ve been clear and even took the time to explain my reasoning when you asked.
Take care.
MegaButts · 5h ago
You don't think it's misleading to talk about how working at AMD can make you more money because of the stock appreciation, despite this being objectively incorrect due to it being a public company, when you don't think the reason to work for AMD is because of money?
Sure, agree to disagree. You have been anything but clear.
latchkey · 5h ago
You go to work at ANY company, public, private, whatever.
You probably get stock options.
Those stock options can gain or lose value over time.
Generally, the incentive for options is that by working at the company, you're contributing to the overall value of the company, which makes those options more valuable.
I believe there’s more to choosing a company than just money. I don’t work at my own company for the paycheck. I’ve put in decades of hard work and I’m fortunate enough not to need the money. I’m driven by a bigger mission: helping AMD become a real alternative to a monopoly. Nothing more, nothing less.
Sorry if I wasn’t clear to you. But when I was, you called my comment pointless and misleading. Now you’re just making it personal in an effort to psychopathically prove I’m somehow unclear. That’s not okay. Get over yourself, go outside, touch grass, whatever. I’m done engaging with you.
MegaButts · 5h ago
You do not need to work for a public company to own stock in that company. You can work at another company and simply buy that stock and get the same benefit.
I think you should take your own advice and go outside and touch grass. Cheers.
latchkey · 5h ago
> You do not need to work for a public company to own stock in that company. You can work at another company and simply buy that stock and get the same benefit.
You have to work for the company to truly affect change within the company. That's the point you're missing.
MegaButts · 5h ago
If you really think whatever contributions you make to AMD with tens of thousands of employees are going to move the stock price so much that it's a better investment than working somewhere else that pays 3x better, then you're the one that needs to get over yourself.
latchkey · 5h ago
Again, it isn't always about the money. 100% yes, a single individual can have a huge effect on a company and the stock price. Look at what Anush is doing and the stock price action since he started his new role.
Why you keep trying to make this personal, is beyond me. Feels aggressive and uncalled for.
MegaButts · 5h ago
I'm just responding to things you said. You told me to get over myself, so I'm just throwing it back at you. I think the only one taking this personally is you.
Let's just go back to "agree to disagree" because we're both tired of each other at this point.
latchkey · 4h ago
I said that after you called my comments pointless and misleading, which is a personal attack.
I tried twice to politely end this, and you keep it going. Let's see if you can stop now?
MegaButts · 4h ago
Your original comment is pointless and misleading and I explained why. I didn't attack you or even your comment, I just pointed out what's wrong with it. Anyway sure, I'm happy to continue if you want.
> I’m done engaging with you.
You're the one that said this, not me. I was just trying to be kind and keep giving you an exit that you said you wanted. If you want to keep getting yourself worked up over a comment thread on HN, I'm happy to oblige. After all, according to you I'm acting psychopathically.
latchkey · 4h ago
I knew you couldn't stop.
MegaButts · 4h ago
I won't lie, your projection is pretty funny.
latchkey · 4h ago
¯\_(ツ)_/¯
iszomer · 23h ago
We're forbidden to trading our own stock anyway, SEC regulation on insider trading and all.
vlovich123 · 11h ago
You’re forbidden from shorting. Buying is completely allowed unless you are classified an insider and even then trades are open for I believe a month after quarterly results.
almostgotcaught · 1d ago
You're "talking your book".
zombiwoof · 1d ago
They pay terrible and still have legacy old guard managers. If you try to innovate on software you should look elsewhere or really make sure your manager knows what’s what
martinald · 1d ago
FWIW for the first time in 2+ years I managed to compile llama.cpp with ROCm out of the box and run a model with no problems* on Linux (actually under WSL2 as well), with no weirdness or errors.
Every time I have tried this previously it has failed with some cryptic errors.
So from this very small test it has got way better recently.
*Did have problems enabling the WMMA extensions though. So not perfect yet.
halJordan · 1d ago
If this has been an issue for two years, then it's not rocm or llama.cpp problem.
vlovich123 · 11h ago
You can blame the user for not using the tools correctly or the manufacturer for making difficult to use tools that aren’t straightforward or don’t work in various non happy path conditions (ie unreliable installers).
For example, to this day installing MSVC doesn’t make a default sane compiler available in a terminal - you have to open their shortcut that sets up environment variables and you have to just know this is how MSVC works. Is this a user problem or Microsoft failing to follow same conventions ever other toolchain installer follows?
martinald · 14h ago
Oh I'm sure you are right its operator error, but I'd always have some issue installing rocm and getting the paths right or something. This is the first time I've managed to install rocm following the commands exactly and then compile llama.cpp without having to adjust anything.
BTW, this kind of dev experience does really matter. I'm sure it was possible to get working previously; but I didn't have the level of interest to make it work - even if it was somewhat trivial. Being able to compile out of the box makes a big difference. And AFIAK this new version is the first to properly support WSL2, which means I don't have to dual boot to even try and get it working. It's a big improvement.
moondistance · 1d ago
Yes, big time, but there continues to be lots of progress.
Most importantly, models are maturing, and this means less custom optimization is required.
martinald · 1d ago
Yes I'd agree with that. There is so much demand for inference which is maturing rapidly that even if a lot of the "R&D" is done on NVidia cards because of their (vastly, let's be fair) software stack, if AMD is competitive on the inference side (and perhaps more importantly have shorter lead times) then doing the inference on AMD is still an enormous market.
I suspect we will (or already are?) at a point where 95%+ of GPUs are used for inference, not training.
It's also worth noting Ultra Ethernet isn't just an AMD thing. The steering committee for the UEC is made up of basically every hardware manufacturer in the space except Nvidia. And of course Nvidia is a general contributor as well (presumably so they don't get left behind).
Supercharging the Local Data Share (LDS) that's shared by threads is really cool to hear about. 64 -> 160KB size. Writes into LDS go from 32B max to 128B, increasing throughout. Transposes, to help get the data in the right shape for its next use.
Really really curious to see what the UDNA unified next gen architectures look like, if they really stick to merging Compute and Radeon CDNA and RDNA, as promised. If consumers end up getting multi-die compute solutions that would be neat & also intimidatingly hard (lots of energy spent keeping bits in sync across cores/coherency). After Navi 4X ended up having its flagship cancelled way back now, been wondering. I sort of expect that this won't scale as nicely as Epyc being a bunch of Ryzen dies. https://wccftech.com/amd-enthusiast-radeon-rx-8000-gpus-alle...
Meta, OpenAI, Crusoe, and xAI recently announced large purchases of MI300 chips for inference.
MI400, which will be available next year, also looks to be at least on par with Nvidia's roadmap.
They've been playing catch up after "the bad old days" when they had to let a bunch of people go to avoid going under but it looks like they are catching back up to speed. Now it's just a matter of giving all those new engineers a few years to get their software world in order.
The point is, someone might join AMD because they believe in the mission, not just for the paycheck. I followed that with: “It isn’t always about the money,” which is consistent with my original comment.
The real subtext is something I care deeply about: Nvidia is a monopoly. If AI is truly a transformative technology, we can’t rely on a single company for all the hardware and software. Viable alternatives are essential. I believe in this vision so strongly that I started a company to give developers access to enterprise grade AMD compute, back when no one was taking AMD seriously in AI. (Queue the HN troll saying that nobody still does.)
If the stock goes up while they’re there, great, that’s a bonus.
In the context of maximizing compensation, working for AMD is dumb. Your own comments support this.
I answered your very specific question, even gave you my own additional context as a friendly thing to do, and now you are going off on some sort of maximizing rant for what purpose?
¯\_(ツ)_/¯
We’ll have to agree to disagree, I don’t think I was misleading. I’ve been clear and even took the time to explain my reasoning when you asked.
Take care.
Sure, agree to disagree. You have been anything but clear.
You probably get stock options.
Those stock options can gain or lose value over time.
Generally, the incentive for options is that by working at the company, you're contributing to the overall value of the company, which makes those options more valuable.
I believe there’s more to choosing a company than just money. I don’t work at my own company for the paycheck. I’ve put in decades of hard work and I’m fortunate enough not to need the money. I’m driven by a bigger mission: helping AMD become a real alternative to a monopoly. Nothing more, nothing less.
Sorry if I wasn’t clear to you. But when I was, you called my comment pointless and misleading. Now you’re just making it personal in an effort to psychopathically prove I’m somehow unclear. That’s not okay. Get over yourself, go outside, touch grass, whatever. I’m done engaging with you.
I think you should take your own advice and go outside and touch grass. Cheers.
You have to work for the company to truly affect change within the company. That's the point you're missing.
Why you keep trying to make this personal, is beyond me. Feels aggressive and uncalled for.
Let's just go back to "agree to disagree" because we're both tired of each other at this point.
I tried twice to politely end this, and you keep it going. Let's see if you can stop now?
> I’m done engaging with you.
You're the one that said this, not me. I was just trying to be kind and keep giving you an exit that you said you wanted. If you want to keep getting yourself worked up over a comment thread on HN, I'm happy to oblige. After all, according to you I'm acting psychopathically.
Every time I have tried this previously it has failed with some cryptic errors.
So from this very small test it has got way better recently.
*Did have problems enabling the WMMA extensions though. So not perfect yet.
For example, to this day installing MSVC doesn’t make a default sane compiler available in a terminal - you have to open their shortcut that sets up environment variables and you have to just know this is how MSVC works. Is this a user problem or Microsoft failing to follow same conventions ever other toolchain installer follows?
BTW, this kind of dev experience does really matter. I'm sure it was possible to get working previously; but I didn't have the level of interest to make it work - even if it was somewhat trivial. Being able to compile out of the box makes a big difference. And AFIAK this new version is the first to properly support WSL2, which means I don't have to dual boot to even try and get it working. It's a big improvement.
Most importantly, models are maturing, and this means less custom optimization is required.
I suspect we will (or already are?) at a point where 95%+ of GPUs are used for inference, not training.
BEWARE: I was running fully patched ubuntu 24 LTS and I needed to upgrade to ubuntu 24.10 and then ubuntu 25 before the drivers worked. Painful.
But, does AMD just own the whole HPC stack at this point? (Or would they, if the software was there?).
At least the individual nodes. What’s their equivalent to Infiniband?
https://www.tomshardware.com/networking/amd-deploys-its-firs...
https://semianalysis.com/2025/06/11/the-new-ai-networks-ultr...
https://ultraethernet.org/
Externally, it is 8x400G NICs, which is the limitation of PCIeV5 anyway.
We had a guy training SOTA models on 9 of our MI300x boxes just fine. Networking wasn't the slow bit.
Now that Nvidia is removing FP64 I assume AMD will have 100% of the HPC market until Fujitsu Monaka comes out.