Show HN: I built an AI Agent that uses the iPhone
48 rounak 13 6/2/2025, 2:37:35 AM github.com ↗
It’s powered by OpenAI’s GPT 4.1 model.
Uses Xcode UI tests + accessibility tree to look into apps, and performs swipes, taps, etc to get things done.
Whittaker added that an AI agent powerful enough to do that would "almost certainly" process data off-device by sending it to a cloud server and back.
"So there's a profound issue with security and privacy that is haunting this sort of hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services, muddying their data, and doing things like undermining the privacy of your Signal messages," she said.
--Meredith Whittaker earlier this year.
The interesting thing is that the three laws of robotics says that robots shouldn't harm humans, but I don't really see a way for an AI agent to understand that by "pressing a button" they actually hurt the human.
These design principles make sense when you are talking about a non-sentient object, but intelligent, adaptable beings cannot be so easily constrained.
Essentially the story of the Horizon series of video games: https://en.wikipedia.org/wiki/Horizon_(video_game_series), and I'm sure many other sci-fi novels.
Apparently, not much is planned, per [1]. I'd be very cautious about AI agents like these; from a user level, this has so many security vulnerabilities.
[1] https://www.macrumors.com/2025/05/30/the-macrumors-show-last...
Fool me once...
>iOS apps are sandboxed, so this project uses Xcode's UI testing harness to inspect and interact with apps and the system. (no jailbreak required).
What are practical limitations of this? Maybe you can't submit this app to the store?