I gather Google is a great believer in A/B testing.
Yet I consistently find compounding annoyances in their UI's. I suspect they'll jump on this AI bandwagon and that I'll hate the result.
By contrast, I loved the UI on my old PalmOS and its apps.
There is no substitute for good taste, deliberate attention to detail and eating your own dogfood.
dmacfour · 1d ago
You’re setting up a dichotomy where there isn’t one. Measuring the impact of changes isn’t any less necessary if you hand UI work off to genAI, if anything it’s more important.
fka · 1d ago
The response to my AI-generated UI post was incredible.
And it got me thinking: What if we could skip A/B testing entirely?
I've imagined a world with AI creating personalized interfaces for every single user, in real-time.
cheevly · 1d ago
I’ve also gone down this rabbit hole of thinking. Here is the main roadblock I encounter when imagining it in practice:
It might prevent users from having a common ‘language’ for describing their interactions with the product. If everyone perceives the product from a different lens, then everyone’s experience will drastically differ and there isn’t an objective common ground for users to discuss their experience with one another.
This problem seems to be rampant across the user experience with ChatGPT, wherein countless users claim that it offers profound value, and others claim that it is hallucinatory rubbage.
It’s possible that none of this is an actual ‘problem’ though, and just the very nature of probabilistic software. Users and product owners just need to adapt to the mindset that their experience may vary. But this presents a whole new breed of A/B testing considerations.
guestbest · 1d ago
Why not replace the users with agents and then the system could iterate even faster.
achillesheels · 21h ago
I would love this tool for on-boarding users…just inform the machine about components, to develop its own script to help the user…with the parameter being use-time as the success metric…so it can put out 100 prototypes, for UX testers to widdle down to 4-5, for pre-market A/B testing…
Yet I consistently find compounding annoyances in their UI's. I suspect they'll jump on this AI bandwagon and that I'll hate the result.
By contrast, I loved the UI on my old PalmOS and its apps.
There is no substitute for good taste, deliberate attention to detail and eating your own dogfood.
And it got me thinking: What if we could skip A/B testing entirely?
I've imagined a world with AI creating personalized interfaces for every single user, in real-time.
It might prevent users from having a common ‘language’ for describing their interactions with the product. If everyone perceives the product from a different lens, then everyone’s experience will drastically differ and there isn’t an objective common ground for users to discuss their experience with one another.
This problem seems to be rampant across the user experience with ChatGPT, wherein countless users claim that it offers profound value, and others claim that it is hallucinatory rubbage.
It’s possible that none of this is an actual ‘problem’ though, and just the very nature of probabilistic software. Users and product owners just need to adapt to the mindset that their experience may vary. But this presents a whole new breed of A/B testing considerations.