AI as the operating system: generative user interfaces 👾

What does it mean when most interfaces you use only exist for a few minutes?

When I used AI to spin up a simple German-language learning app with a single text prompt earlier this year, I was shocked. Hours of tinkering later, and my German-language tutor, Daddy Dativ, was a fact.

I was impressed with how easy it was to generate interfaces on the fly. Then realised I should start thinking about LLMs as operating systems; an operating system being something that launches software to assist the user with tasks.

But what if, for many tasks, pre-installed software isn’t necessary anymore, and the user interfaces (UI) can just be generated by the AI?

“This is not the future of the internet. It is here already. It’s just not how most people use the internet yet.

• Today: widespread ChatGPT use. People spin up mini-apps in the form of prompts to solve basic, everyday problems. Answers are usually returned in a conversational format.

• Tomorrow: people will engage with AI to let LLMs generate custom user interfaces, tailored to their or their team’s specific needs.”

Fast forward to November 18, when Google announced the rollout of ‘generative UI’ to their products, including search. Our media environment is evolving at a breakneck pace.

via Rex Woodbury’s Digital Native newsletter

To get a sense of how astonishing this is, try to rewind time by five years. It’s the first year of the pandemic, and the ‘metaverse’ is popping off (or is it?). The momentum of the VR and online gaming domains was suddenly bolstered by everyone staying at home. People went clubbing online, virtual events became mainstream, and elsewhere, creatives banded together to build exciting infrastructure on blockchain through NFTs and DAOs.

We thought we were seeing the future. Yet nightclubs set in Minecraft still took multiple people months to build. And don’t even get me started on the complexity of building on blockchains.

The question then was: what does the future hold now that online culture has become part of the average person’s daily life?

The question today is much bigger: what does it mean when we primarily connect to the web through operating system-type interfaces, which then spin up personalised and custom interfaces at the tap of a button?

What if solving one’s problem isn’t about finding a good app for it, but simply spawning one?

In a study, Google examined how people evaluate generative UIs for topics such as the history of timekeeping devices, fractals, and teaching basic puppy tricks. While custom-made sites by human experts slightly outperformed AI-generated interfaces, the latter outperformed other AI output and Google’s top search result by a large margin. (link to study PDF)

One important caveat is that the time to serve results wasn’t included in the study participants’ evaluations. This means that if a website takes a second to render, whereas a custom-generated UI takes a minute to appear, the frustration someone experienced while waiting is not captured in the score.

So, we’re not there, in the generative operating system version of the future, quite yet. But given the amount of data Google has access to, it should be trivial for them to quickly render commonly requested interfaces for their users. And from there scale to more niche cases.

While it’s easy to generate a user interface, they’re still a far cry from fully-fledged apps, so don’t expect popular services like Duolingo, Uber, or Spotify to go anywhere.

I want to leave you with a few thought starters to think about the future.

What does it mean when:

• apps become temporary tools instead of destinations?

• most interfaces you use only exist for a few minutes?

• operating systems interpret goals instead of launching apps?

• interfaces are personalised by default?

• two people see different interfaces for the same task?

• the fastest way to solve something is to spawn a UI instead of searching for one?

• the boundary between “searching” and “doing” disappears?

---

ᕱᕱ For your ears ᕱᕱ

I’ve been listening to the intricately experimental electronic album by Oneohtrix Point Never called Tranquilizer, which came out last week. I enjoy that it sounds futuristic and nostalgic simultaneously, likely due to OPN’s smart use of old 90s sample libraries found on the Internet Archive.