They’re a notable improvement over the company’s earlier attempts at digital likeness.
I’m almost embarrassed to admit that the first time I saw one of Meta’s new holographic avatars today, during a WhatsApp call with a Meta rep I had just met a few minutes before, I didn’t immediately clock that I was speaking to an avatar. That’s how far the company’s avatar technology has come in the last couple years.
Meta announced at Connect that it’s bringing its holographic avatars to its new VR glasses and to the Meta Ray-Ban Display frames. I got the chance to experience both ahead of the keynote and was surprised at how far the tech has come since the first time I experienced it on Meta’s Orion prototype.
Setting up an avatar is a bit different depending on which device you’re using. With the display-enabled glasses, you create an avatar in the Meta AI app through a process similar to creating a Vision Pro “persona.” Using your phone‘s camera, you’re prompted to turn your head to show different angles of your face. It then asks you a series of questions that are meant to capture how your voice sounds in different interactions.
In my demo, it asked me things like “what does your perfect day look like from start to finish” and “what’s one of your hobbies, and why do you like it.” This part of the demo actually glitched. The Meta AI app kept skipping through the questions as if it couldn’t detect my voice, but the hiccup didn’t seem to affect the process. I was then prompted to “smile and show your teeth” and repeat a phrase as if I was surprised.
In all, creating a holographic avatar for the display glasses should only take about 5 minutes from start to finish. Once set up, you can use it on a video call while wearing your display-enabled glasses, whether you’re chatting
The avatars are powered by Meta’s new Muse Realtime Avatar model, and it uses that brief training period to approximate your actual facial expressions while you speak. Watching my avatar speak was a strange experience. While I was wearing the glasses and speaking as my avatar, I could see a small preview of myself in the bottom corner of the display. It looked exactly like me, but my brain was also able to immediately recognize that it wasn’t me.
But when I tried the experience from the other side, holding the phone and chatting with an avatar, I was fooled at first. It might be because I had only just met the person I was speaking to. So I wasn’t familiar with his mannerisms or expressions, but it looked very similar to how I would have expected him to appear on a typical video call. It was only when looking more closely at his background that I could detect the video wasn’t quite normal. Meta is also adding a transparent watermark to all video calls with an avatar. In my first viewing, I mistook this for a logo on his shirt, but if you know what to look for, it is a clear identifier.
Meta’s plan for holographic avatars in its new VR glasses is even more ambitious. Because the cameras and sensors on that device are much more advanced, you can create a full-body avatar and you don’t need to “teach” it your expressions. It instead relies on the inward facing cameras and sensors to understand your facial expressions.
I was able to chat with a full-size hologram of a Meta product manager while wearing the VR glasses. And it felt less weird than I expected. It was definitely much more natural looking than Apple’s “persona” avatars, which have always looked weirdly smooth and alien-like to me. (Those also don’t show lower bodies or arms, which doesn’t help.)
But the VR avatar I spoke with looked pretty close to the person I had just met, and didn’t feel creepy either. There was some distortion toward her feet, especially if I moved around the room, and I could see some blurring towards the edges of her head. The avatars aren’t fully 3D. They only show a frontal view of a person, so there are some limitations. But I was surprised at how much more realistic it felt than any other interaction I’ve ever had with a VR avatar (a low bar, to be fair).
Holographic avatars are still in an “early preview” stage, and there’s clearly some bugs to be worked out. Availability is also limited to the device you’re using, so someone with Meta Ray-Ban Display glasses can call someone else as their avatar but only when wearing the glasses. And someone on display glasses can’t call someone wearing VR Glasses. VR avatars are limited to glasses to glasses interactions for now.
Meta has said these avatars could help people feel more of a sense of presence when on a video call, and I definitely felt that in my demo. There’s still something about calling a close friend or family member as my avatar that would feel weird and inauthentic, but I could see how some might feel otherwise. The Meta reps I spoke with told me they often call into work meetings as their avatar, which might be a nice alternative when you’re not quite camera-ready.
Holograms will be available to Meta Ray-Ban Display in early access this fall.
