Why AI Apps Are Broken: Essential Challenges and a Roadmap for Solutions
In recent years, AI (Artificial Intelligence) has garnered such high expectations that it has been called a "rocket ship for the mind." However, the AI features in many existing apps are actually creating "broken AI apps" that degrade the user experience and increase the workload instead. In this article, based on the essay "AI Apps Are Broken — Here's How To Fix Them" by YC partner and Optimizely founder Pete Koomen, we explore why AI apps are difficult to use and explain concrete approaches toward solving these issues.
1. Challenges Lurking in AI Apps
1-1. Tone Mismatch
Let's consider the email draft feature integrated into Gemini released by the Gmail team as an example. What Koomen presented was the following output:
"Dear Gary I'm writing to inform you that my daughter woke up with the flu this morning as a result I won't be able to come into the office today Thank you for understanding Best regards Pete"
This text has such an unnatural tone that it could even be mistaken for "Pete having been hijacked." Because the generated draft deviates from his own voice, the recipient might even worry that the account has been compromised.
1-2. UI and Prompt Bottlenecks
Furthermore, the prompt entered by the user is almost the same length as the generated draft. In other words,
"Tell my boss Gary that I can't come to the office today because my daughter has the flu this morning."
The effort to type this prompt is no different from the effort to write the email itself. Even though you are supposed to be utilizing AI, the workload has actually increased.
2. Unlocking System Prompts: User-Driven Customization
2-1. What is a System Prompt?
Prompts for Large Language Models (LLMs) are divided into "System Prompts" and "User Prompts." The System Prompt is a function that defines the model's behavior, while the User Prompt corresponds to the input for that function. In many AI apps, the System Prompt is uniformly set by the developers and hidden so that general users cannot see or edit it.
2-2. Implementation Example of a "Pete System Prompt"
Koomen demonstrated that by defining his own custom System Prompt as follows, he could automatically generate emails exactly as he intended.
"You are Pete, a 43-year-old YC partner and a father. Because you are very busy, keep emails short and concise. Omit unnecessary words and strict grammar, and convey the point in about one line."
When you combine this System Prompt with the same User Prompt (notifying about a daughter's illness),
"Gary, my daughter has the flu, so I can't make it in."
A short email close to his own writing style is generated instantly. By allowing users to define and edit System Prompts, you can eliminate the hassle of "prompt hacking" every time and build reusable, self-learning templates.
3. "Horseless Carriage" Thinking in Software Development
3-1. The Limits of Old-World Thinking
Early automobiles were just engines mounted on carriage frames, lacking suspension and offering a terrible ride. Similarly, many AI apps simply 'force AI into legacy software development techniques' and fail to unlock the AI's full potential.
3-2. Shifting to AI-Native Design
The 'automobile' of the AI era should be a new generation of vehicle redesigned from the ground up. Beyond just Gmail, we need to rebuild entire applications as 'AI-automated tools' and design UI/UX that allows AI to handle the repetitive, simple tasks users perform.
4. Separating Agents and Tools: Roles of Users and Developers
4-1. The User's Job: Agent Builder
In most AI apps, the user takes on the role of building the agent (the 'program' that instructs the AI). An interface is required that allows for trial and error in defining System Prompts and combining them with User Prompts to optimize performance.
4-2. The Developer's Job: Providing Tools and UI
Developers provide the toolsets that allow agents to interact with the world (e.g., sending emails, archiving, labeling, etc.) and design the UI/UX to support prompt editing. This enables users to leverage AI safely and powerfully.
5. The Future of AI-Native Applications: The Potential of Email-Reading Agents
5-1. Automating Busy Work
Using an 'email-reading agent' as an example, Koomen released a demo that analyzes an inbox to instantly label, archive, and automatically draft replies as needed. Such agents can even outperform existing filters in spam detection, providing rapid support for user decision-making and action.
5-2. A Paradigm Shift in Productivity
AI-native software provides users with agents that 'actually act in the world,' rather than just generating text. As a result, users are freed from tedious tasks and can devote their time to the creative and strategic work they should be focusing on.
Koomen's assertion that 'when an LLM agent acts on my behalf, I should be able to edit the system prompt myself' is a proposal that shakes the very foundation of AI application design. Developers should break away from the legacy 'horseless carriage' approach and aim to build platforms where users can freely customize their agents. Such AI-native apps are what will realize a future that truly invites users onto a 'rocket ship for the mind.'
Related Articles