WebGPU enables local LLM in the browser. Demo site with AI chat

Looks like this is a wrapper around: https://github.com/mlc-ai/web-llm

Which has a full web demo: https://chat.webllm.ai/

This demo only works if you have the webgpu feature "f16". You can find out if you have this by checking for the feature list in https://webgpureport.org/ . The page itself can of course check for this but since f16 support is common they probably just didnt bother.

Browser LLM demo working on JavaScript and WebGPU. WebGPU is already supported in Chrome, Safari, Firefox, iOS (v26) and Android.

Demo, similar to ChatGPT https://andreinwald.github.io/browser-llm/

Code https://github.com/andreinwald/browser-llm

- No need to use your OPENAI_API_KEY - its local model that runs on your device

- No network requests to any API

- No need to install any program

- No need to download files on your device (model is cached in browser)

- Site will ask before downloading large files (llm model) to browser cache

- Hosted on Github Pages from this repo - secure, because you see what you are running

What's the performance of a model like vs an OpenAI API? What's the comparable here? Edit: I see it's same models locally that you'd run using Ollama or something else. So basically just constrained by the size of the model, GPU and perf of the machine.

Beware of opening this on mobile Internet.

very cool. improvement would be if the input text box is always on screen, rather than having to manually scroll down as the screen fills.