Layers All the Way Down: The Untold Story of Shader Compilation

38 birdculture 14 5/19/2025, 2:49:25 AM moonside.games ↗

Comments (14)

raphlinus · 2h ago
We tried something like this with piet-gpu-hal. One problem is that spirv-cross is lossy, though gaps in target language support are getting better. For example, a device scoped barrier is just dropped on the floor before Metal 3.2. Atomics are also a source of friction.

But the main problem is not the shader language itself, but the binding model. It's a pretty big mess, and things change as you go in the direction of bindless (descriptor indexing). There are a few approaches to this, certainly reinventing WebGPU is one. Another intriguing approach is blade[1] by Dzmitry Malyshau.

I wish the authors well, but this is a hard road, especially if the goal is to enable more advanced features including compute.

[1]: https://github.com/kvark/blade

pjmlp · 2h ago
As expected, they don't touch the issue of how shaders work in PlayStation (LibGNM, LibGNMX) and Switch (NVN).
gmueckl · 55m ago
These systems are highly proprietary and Inam reasonably certain that stating anything about them publicly would break some NDAs.
pjmlp · 4m ago
As someone still having an Nintendo Developer Portal account, holding SCEE content back when the London Soho office used to have a developer site (aka Team Soho), and PS2Linux owner, there is plenty of material that can be discussed publicly without breaking NDAs.
riggsdk · 1h ago
BGFX (https://github.com/bkaradzic/bgfx) uses a different approach. You basically write your shader in a GLSL-like language but it's all just (either very clever or very horrible) macro expansions that handles all the platform differences. With that you get a very performant backend for OpenGL, WebGL, Vulkan, Metal, Direct3D 11 and 12 and Playstation. It's shader compiler program basically does minimal source level transformations before handing it over to the platforms' own shader compiler if available.
shmerl · 2h ago
> Rendering, by comparison, is a huge can of worms. Every platform has their own unique support matrix.

GPU APIs landscape is stuck in the dumb NIH mentality of the '90s because stuck up lock-in proponents refuse to support Vulkan on their walled gardens.

The only recent somewhat positive development about that was MS deciding to use SPIR-V and eventually ditch DXIL. A small step in the right direction, but not really enough yet.

alexk101 · 1h ago
I have been playing around with slang, which is supposed to be more cross platform. They have a neural rendering slant, and I have yet to fully test on all platforms, but I think it's a welcome move to consolidate all these apis. https://shader-slang.org/
raphlinus · 1h ago
Yup, I think slang is the future. Anyone on this thread willing to fund a Rust implementation?
gmueckl · 45m ago
Why? Unless you need to embed the slang transpiler (unlikely), the language it is written in literally doesn't matter.
shmerl · 26m ago
I think Rust itself as a language for GPU programming is another interesting alternative:

https://github.com/Rust-GPU/rust-gpu/

mvdtnz · 1h ago
This is a very interesting article, it was fun to learn what shaders are and why I have to wait for them to compile every time I play Call of Duty.

What I'd like to know is why game developers can't provide a service / daemon that will perform shader compilation when shaders or drivers change so I don't need to waste 25 minutes of my precious gaming time when I load CoD and see they need compiling. My computer has been sitting there unused all week. Steam was smart enough to download game updates in that time, why isn't your game smart enough to recompile my shaders on an idle machine?

Negitivefrags · 33m ago
Steam actually shares compiled shaders between users with the same hardware / driver version, but only for Vulkan I believe.
gmueckl · 48m ago
Imagine the outcry if every game came with a background process that just sits there in the background.

Also - and this may sound a little bonkers - some renderers are so complex and flexible that the actual set of required shaders is only discovered when it tries to render a frame and the full set of possible shader configuration permutations is so large that compiling them all in advance is pointless or borderline infeasible.

mvdtnz · 31m ago
Lots of games come with background processes.