The theory of voice being the next big interface has picked up strong momentum, with investors pouring billions of dollars into voice AI startups working on areas ranging from model makers to enterprise customer service providers, and from meeting note-takers to AI-powered dictation.
Every week there is a new model or a tool release that claims to sound human and converse like one. However, in reality, that might not be the case. Enterprise voice AI platform PolyAI’s CTO Shawn Wen thinks that despite the release of full-duplex models — which can speak while listening to you — voice AI doesn’t have its “ChatGPT moment” yet.
“We have reached the milestone of developing full-duplex models. The next challenge is to make reasoning very fast, so that the models can fetch answers quickly and the conversation feels natural,” he told me on stage at the HumanX conference last month.
He also said that AI agents in customer service should not sound robotic and should give callers enough confidence that they can solve problems.
“I think the next stage will be slightly different because once the voice is good enough, like, and the customer is willing to engage with them for the first two or three turns, they start to build confidence, and over time, they will feel like I probably don’t have to talk to a human if the agent can solve my problem,” he said.
Alex Gay, CMO for meeting notetaker Otter, opined that speaker identification, intent capture, and typing that up with organizational knowledge is a key step for enabling automation. The company is also working on digital twins that might represent people in meetings. For that technology, he said it’s paramount that the output voice gives the same emotive expressions of talking to a human in a meeting.
“If you think about the meetings that you’re in right now, the best conversations that you have are where you can have debate, and strategic discussions, and when you feel like there’s a relationship that underpins it. If you aren’t able to have that with an avatar, then it’s just a q and a chatbot,” Gay noted.
Voice AI’s understanding and transparency
While voice AI models have improved, AI assistants often don’t understand users, or your meeting notetaker shows the wrong transcript or a summary.
Wen thinks that ASR (Automatic Speech Recognition) models often miss important keywords, and that creates an issue in capturing the whole context.
Otter’s Gay agreed with this, adding that the company keeps working on improving transcription. He also said that language is one area where voice models need to improve.
“For Otter, you know, transcription was never the end point. It was just the layer that we could start to drive some of the productivity gains on the back of. But if your original transcription didn’t have the accuracy that you needed, all the follow-up actions that you have become flawed. And the minute that starts to take action, that is wrong. You lose trust in the platform. It is critical for us to continue to improve that ASR model because all of the downstream impacts are significant,” he said.
With the new voice tools, there is also a question of transparency. Tools should declare to customers that they are being recorded or talking to AI. Otter said that it wants to instill trust in people who are in a meeting, so even for a meeting where the bot is not present, it wants to try methods like notifying everyone in the chat that the meeting is being recorded. PolyAI’s Wen also said that it’s important to establish that people are talking to an AI in enterprise calls.
