Apple’s FaceTime video calling feature is back with a vengeance with the Apple Vision Pro, but the new mixed reality headset’s method of recreating obscured faces is a haunting uncanny valley that may deter people from talking to anyone strapped into the so-called metaverse.
Out right now, the $3,499 Apple Vision Pro is a remarkable piece of kit. Early Vision Pro reviews describe a fantastic piece of hardware let down by missing software and an incredibly steep price tag, even without the wave of scalpers.
What’s most interesting about the Apple Vision Pro is how it handles FaceTime calling. As users faces are completely obscured by the mixed reality headset, the device instead scans the user and recreates a realistic digital avatar in their stead.
After this digital avatar is generated by the headset, it can track your hand and eye movements to mimic what you’re doing in real life. It’s genuinely fantastic tech, allowing you to be like the floating hologram video caller you’ve seen in countless sci-fi films. However, it also looks ghoulish.
A video shared by iJustine shows her, Marques Brownlee and Brian Tong in a three-way FaceTime conversation, all replicated by the Vision Pro. All three are shown from the shoulders up, placed in a frosted generic valley background as their heads float around like the child version of Heimdall from the atrocious Thor: Love and Thunder.
What’s impressive is what Apple gets right. Skin tones are generally 1:1 and even hair highlights can be replicated properly. Facial movements are also fairly accurate with hand tracking being damn-near perfect, and the rendering of faces is high-quality for something being done in real-time, but it still looks off.
As noted by Marques Brownlee in the video, there’s a huge uncanny valley effect while using Vision Pro’s FaceTime feature. Skin is weirdly smooth whilst being properly lit, offering a bizarre middle ground between realism and CGI. Secondly, hair is also weirdly smooth, replicating the general shape and colour while also lacking any detail.
However, the weirdest part of the Apple Vision Pro avatars is what every realistic CGI face, and even deepfake technology, gets wrong: the eyes. The gateway to the soul, as they’re often called, eyes are the hardest part of a face to generate properly outside of huge budget films like Avatar (which you can watch in 3D on the Vision Pro). Apple’s headset creates eyes that look dead, barely glinting from a non-existent light source. They blink and move realistically, but they look, well, soulless.
Nevertheless, what the Apple Vision Pro is doing is still very impressive, and someday it may be the future. However, in this first generation, Apple’s creation is still quite weak for the general public, and it needs further development to be something truly special.