Voice AI: What’s Worth Building?
Voice AI feels like a natural next step. But what needs to work beneath the conversation, and where are the opportunities to build?
Talking to technology isn’t new—we’ve had Siri and Alexa for years. But lately, I keep coming across voice AI products and demos that make me wonder what’s possible now. As I look for my next opportunity, that curiosity has become more practical: what’s changing, what’s worth building, and where could I contribute?
I also want to invest time in learning how to build it. Before choosing tools or starting a project, I’m asking myself: what makes a voice experience useful, and what is still difficult to get right?
That curiosity leads to a bigger question: could speaking become our default way of interacting with technology?
For many of us, speaking comes before reading and writing. Yet using software often means translating an intention into menus, forms and carefully phrased searches. Could AI handle more of that translation for us?
Naturally, my mind goes straight to JARVIS. Apparently, years of watching Iron Man have become product research.
What sits beneath the conversation?
The interface may feel simple, but the stack has several layers—and each presents an opportunity:
- Audio input: capturing and recognising speech. Can we serve local languages, different accents and noisy environments better?
- Conversation management: handling pauses, interruptions and context. Does the assistant know I’m thinking, rather than finished?
- Reasoning and action: interpreting intent, retrieving information and using tools. Can it complete a task with appropriate permissions and confirmations?
- Speech output: responding clearly, with suitable pronunciation, pacing and tone.
- Infrastructure: keeping the experience secure, observable, reliable and affordable at scale.
These are useful layers to reason about, even when a model handles several together.
What would work for everyday people?
Voice assistants can already answer questions and help with tasks. What I want to understand is how to build a product people can depend on—even when they speak casually, change their minds or have an unreliable connection.
What architecture would support that?
I’d start with a streaming audio layer, a conversation manager, and a workflow service that keeps track of the task. The AI would interpret the request and propose the next step. Tools would perform permitted actions, while the workflow records what actually happened. That separation matters: sounding confident is not proof that a task succeeded.
Around that, I’d need confirmations for consequential actions, safe retries, clear failure messages and a way to review or correct the result. Testing would measure completed tasks, misunderstandings, response time and cost—not just how convincing the voice sounds.
The opportunity I’m curious about is making that whole experience simple enough for someone who has no interest in learning how AI works. Can they speak normally, get something done, and stay in control?
I want to learn by building one small workflow, then testing interruptions, mistakes and recovery. That feels like a better starting point than trying to build my own JARVIS on day one.
For me, this is both a learning direction and a way to explore new opportunities. The question guiding it is: where does a conversation genuinely make technology easier to use?
GIF via Tenor.
WHAT DID YOU THINK?
A little feedback goes a long way.