Beyond Text: On-Demand UI Generation for Better Conversational Experiences

35 fka 23 5/16/2025, 9:23:51 AM blog.fka.dev ↗

Comments (23)

jFriedensreich · 52m ago
I was working on exactly this in gpt 3 days and still believe ad hoc generation of super specifc and contextual relevant UIs will solve a lot of problems and friction that purely textual or speech based conversational interfaces pose especially if the UI elements like sliders provide some form of live feedback of their effect and are possible to scroll back to or pin and make changes anytime.
WillAdams · 38m ago
This always felt like something which the LCARS interface addressed, at least conceptually (though I've never seen an implementation which was more than just a skin).

I'd love to see folks finding the same sort of energy and innovation which was driving early projects such as Momenta and PenPoint and so forth.

ActionHank · 1h ago
I really believe this is the future.

Conversations are error prone and noisy.

UI distills down the mode of interaction into something defined and well understood by both parties.

Humans have been able to speak to each other for a long time, but we fill out forms for anything formal.

visarga · 59m ago
> Conversations are error prone and noisy.

I thought you'd say not being able to reload the form at a later time from the same URL is bad. This would be a "quantum UI" slightly different every time you load it.

ActionHank · 38m ago
I think that there will be ways to achieve this.

If you look at many of the current innovations around working with llms and agents, they are largely around constraining and tracking context in a structured way. There will likely be emergent patterns for these sorts of things over time, I am implementing my own approach for now with hopefully good abstractions to allow future portability.

fka · 1h ago
Exactly! LLMs can generate UIs according to user needs. E.g. it can generate simplified or translated ones, on-demand. No need for preset forms or long ones. Just the required ones.
joshstrange · 3h ago
Related, it’s crazy to me that OpenAI hasn’t already done something like this for Deep Research.

After your initial question, it always follows up asking some clarifying questions, but it’s completely up to the user to format their responses and I always wonder if people are sloppy if the LLM gets confused. It would make much more sense for OpenAI to break out each question and have a dedicated answer box. That way the user’s response can be consistent and there’s less of a chance they make a mistake or forget to answer a question.

wddlz · 29m ago
Sorry for the shameless plug but, we recently published this research on 'Dynamic Prompt Middleware' (https://www.iandrosos.me/images/chiwork25-27.pdf) as a potential approach for this. Basically, based on the user's prompt (and some other bits of context), we generate UX containing prompt refinements for users to quickly select answers to and do the prompting for the user.
fka · 24m ago
Didn't read the paper but sounds like a similar idea.
fka · 3h ago
OpenAI would implement this within a minute or smth I guess.
banga · 1h ago
Semantic clarity of written prose is hard, but this approach seems like making it easier for the machines rather than the other way around.
aatd86 · 1h ago
that's not a very innovative idea or even better UX. I think that the future wil have to do with voice commands and mcps will be the backend, exposing capabilities.
fka · 22m ago
We don't do most of our jobs with our voice. "Click" interaction is still an important one.
aatd86 · 2m ago
there is no benefit in it being AI generated though. There is a closed set of interaction behaviors.

When you want to order a pizza, you won't have to click. Just browse and ask the AI assistant to place an order as you would in a restaurant. Better UX.

ActionHank · 1h ago
Because we are all going to be in our open planned offices shouting into the void hoping it poops out the app we want?
maxcan · 3h ago
Video isn't loading.
fka · 3h ago
I think it’s because of the video format.

https://x.com/fkadev/status/1923102445799927818?s=46

utku1337 · 3h ago
looks very useful
exe34 · 3h ago
I was hoping to do this over IRC but never got around to implementing it. I hate the idea of implementing a whole website/chat system, when they already exist. I'd like to use it for my (currently in-existent) home automation communication.
fka · 3h ago
Perfect home automation never exists.
casey2 · 3h ago
If it could have been done it would have by now
fka · 3h ago
You can say this for all kind of inventions and new ideas.
revskill · 2h ago
Startups fo not have enough efforts to impriove ux, that is why we have jira.