Sesame brings its voice-first AI agents to iPhone, with bigger hardware ambitions behind the app

Sesame, the conversational AI startup co-founded by Oculus veterans, has launched a public preview of its iOS app in 39 countries. The app introduces four voice-based AI agents, Maya, Miles, Simone, and Charlie, and puts the company’s main bet in front of regular users: AI assistants should feel less like prompt boxes and closer to spoken companions that can search, reason, remember, and respond without awkward dead air.

The app is free for now, though some users may still see a short waitlist at signup. An Android preview is planned, but Sesame hasn’t given a release date.

The team and roadmap make this release more interesting than the average AI companion launch. Sesame was founded by people tied to Oculus, the VR company Facebook bought in 2014, and the startup has already raised a $250 million Series B led by Sequoia and others. The company’s plan runs past the phone app. Sesame wants these agents to eventually live in intelligent eyewear, with hardware expected in 2027.

Voice assistants get more useful when they’re persistent, hands-free, and aware of context. They also get much harder to build safely and reliably.

The technical pitch: speech that can think while talking

Most AI voice products still feel like chatbots with a microphone attached. You speak, the system transcribes, the model reasons, then a text-to-speech layer reads back the answer. Latency is the enemy. If the system answers too quickly, it may give shallow or stale responses. If it waits for retrieval, tool calls, and reasoning, the conversation starts to feel broken.

Sesame is trying to hide that gap.

According to the company, its agents can run multiple searches in parallel while speaking, then fold new information into the answer as it arrives. A Sesame agent might start responding, pull in fresher context mid-answer, then change direction without stopping and waiting to produce a fully formed paragraph.

That sounds simple in product language. It isn’t.

A system like this needs to coordinate several moving parts:

• Low-latency speech recognition
• Turn-taking and interruption handling
• Retrieval from fresh sources
• Ranking and filtering search results
• Response generation that can revise direction midstream
• Text-to-speech that stays coherent when the underlying content changes
• Memory management across conversations

Calling a search API is the easy part. The hard part is making the agent sound competent while the answer is still being assembled.

Developers who have built retrieval-augmented generation systems know the trade-off. You can wait until retrieval completes, then generate a grounded answer. Or you can start streaming tokens immediately and risk saying something you’ll need to correct seconds later. Sesame appears to be choosing the riskier path because voice UX punishes silence harder than text UX does.

That’s a reasonable bet. It’s also where hallucination, inconsistency, and source quality problems can creep in. If an agent is speaking while search results are still arriving, the orchestration layer has to decide when new evidence is strong enough to change the response. That’s a hard product problem and a hard systems problem.

Why Sesame uses named agents

Sesame’s iOS app includes four agents: Maya, Miles, Simone, and Charlie. Each has its own voice, personality, point of view, and memory. Maya and Miles were already available in Sesame’s earlier Research Preview, which Sequoia said reached over a million users within the first few weeks.

The named-agent approach isn’t new. Character.AI, Inflection, Replika, OpenAI’s voice modes, and plenty of smaller startups have all tried versions of persistent AI personas. Sesame’s version is more ambitious because it connects persona with voice, memory, live retrieval, and eventually hardware.

For technical teams, the design choice is worth watching. A named agent changes user expectations. People don’t treat it like an endpoint. They treat it like a relationship, or at least a semi-consistent interface. That pushes the product toward long-term memory, personalization, and conversational continuity.

Those features bring sharp edges.

Memory needs consent boundaries. It needs deletion controls. It needs predictable behavior. If one agent “remembers” a preference, does another agent know it too? If incognito mode lets an agent use prior context but saves nothing new, how clear is that boundary to users? Does the model infer sensitive traits from repeated conversations? Can users inspect and edit stored memories?

Sesame says the app includes an incognito mode for private conversations. In that mode, agents can access prior context but don’t save the conversation to memory. That’s a useful control, but it depends heavily on implementation. Privacy modes can mislead users if the product doesn’t clearly separate model context, product memory, logs, telemetry, and safety review data.

Senior engineers should care about that distinction. “Not saved to memory” doesn’t automatically mean “not processed,” “not logged,” or “not retained.” The public details don’t fully answer those questions yet.

Search cards, notes, texting, and deep dives

During beta, Sesame added several features based on user feedback: search cards with image results, notes for capturing takeaways, a texting mode, and deeper research-style responses.

Those additions make sense because voice alone is a weak interface for some tasks. If you’re comparing products, reading code snippets, checking a chart, or reviewing source material, audio gets inefficient quickly. A voice-first app still needs visual artifacts.

Search cards give the user something to inspect. Notes turn a conversation into output. Texting mode handles public spaces, meetings, trains, and all the other places where talking to your phone feels awkward. Deep dives cover the other side of the latency problem: sometimes you want the agent to slow down and produce a fuller answer.

This hybrid interface is probably the right shape for consumer AI in the near term. Pure voice is convenient until precision matters. Pure chat is flexible but slow for casual interaction. Better products will move between modalities without making the user manage every mode switch.

The Oculus connection points to hardware

The Oculus pedigree matters less as nostalgia and more as a clue about the product roadmap. Sesame’s team is thinking in terms of embodied computing, not only mobile software. The app is likely a proving ground for the agent layer before the company ships eyewear in 2027.

Smart glasses are a natural target for voice agents. They reduce screen dependence, keep input hands-free, and can eventually add visual context through cameras and sensors. Meta is already pushing in this direction with Ray-Ban smart glasses. Apple, Google, OpenAI partners, and hardware startups are working around the same idea.

Glasses raise the bar for latency and trust. A phone app can be clumsy and still useful. A wearable assistant that talks in your ear has to be fast, discreet, and socially acceptable. If it interrupts too often, it’s annoying. If it misses context, it feels dumb. If it records or infers too much, it becomes creepy fast.

Sesame’s current iOS release gives the company a way to test conversational behavior, memory design, retrieval quality, and personality fit before adding hardware constraints such as battery life, thermal limits, noisy microphones, and on-device processing trade-offs.

Agentic action is the next hard step

Sesame hints that its agents will eventually do more than talk and research. They’ll take action on the user’s behalf.

That’s where the word “agent” starts to matter technically. Today, many so-called agents are chatbots that call tools. Real user-facing agency requires planning, permissions, state tracking, error recovery, and auditability. Booking a flight, sending an email, changing a calendar, buying a product, or modifying a repo issue all require different levels of trust.

Natural conversation could make agentic software easier to use. Prompting is still a weird skill. Many people know the outcome they want but don’t know how to specify the steps. A voice agent that can ask clarifying questions, propose a plan, and execute with approval would reduce friction.

The risk is overreach. Current AI systems still struggle with ambiguous instructions, hidden constraints, and long-running tasks where intermediate errors compound. Developers building agentic systems already see this with tool loops, brittle workflow state, and models that confidently choose the wrong API call. Add voice, personality, and memory, and users may trust the system more than they should.

A responsible implementation needs explicit confirmations for sensitive actions, visible logs, reversible operations where possible, and permission scopes that users can understand. “Just talk to it” is a lovely interface until the assistant buys the wrong thing or sends the wrong message.

What technical teams should watch

Sesame’s app is consumer-facing, but the underlying problems map directly to enterprise AI and developer tooling.

Latency will define adoption. If Sesame can maintain natural speech while doing retrieval and reasoning, that’s meaningful. Many enterprise copilots still feel like ticketing systems with LLMs attached because every interaction requires waiting.

Memory will be both a differentiator and a liability. Persistent context makes agents useful, especially for recurring work. It also creates governance problems. Companies will want controls for retention, access, redaction, and data residency before anything like this gets near regulated workflows.

Retrieval quality will matter more than model personality. A charming voice can hide weak answers for a while, but technical users will notice stale or poorly grounded responses. Parallel search sounds impressive, but source ranking, citation visibility, and conflict handling are where reliability lives.

Multimodal interfaces will beat voice-only systems for serious tasks. Sesame’s search cards and notes are small but important signals. Engineers don’t just need answers spoken back. They need artifacts they can copy, inspect, share, and verify.

The hardware plan raises the stakes. If Sesame ships eyewear in 2027, the company will face a different set of constraints from a mobile app. On-device inference, cloud round trips, wake-word behavior, microphone privacy, camera policies, and battery budgets will all shape the product.

A promising preview with unanswered questions

Sesame’s iOS launch is one of the more technically interesting voice-agent releases because it focuses on conversation flow rather than treating speech as a thin wrapper around chat. An AI that can speak, search, revise, remember, and keep the interaction moving is the right direction for ambient assistants.

The public preview also leaves plenty unresolved. The company hasn’t fully detailed its memory architecture, privacy model, source handling, or future pricing. The app is free for now, which usually means the business model arrives later. Voice AI with live retrieval and persistent personalization can get expensive at scale.

Sesame is worth paying attention to. The first wave of AI chatbots taught users to type better prompts. The next useful interface may ask better follow-up questions, speak naturally, and know when to show its work on screen. Sesame’s iPhone app is an early test of that idea, with smart glasses waiting behind it.