Every Vision Pro Feature That Is A Total First For Apple
Lots of products have gotten Apple's catchphrase "one more thing" treatment at an Apple keynote. However, Apple Vision Pro's recent unveiling at WWDC 2023 marks the first time in years that the product on the other side of that phrase was a big deal — a long-rumored and innovative device with game-changing potential.
Apple uses the Vision Pro to introduce a slew of new concepts and innovative technologies in an entirely new headset design. The company calls it a spatial computer, but at its heart, Vision Pro is a head-mounted display unlike any we've seen so far. It mixes the virtual and real worlds to provide experiences and interactions beyond what we've come to expect from comparable VR headsets.
Only time will tell whether its tech is worthy of its eye-watering $3,499 price tag when the Apple Vision Pro headset arrives early next year. What we do know, however, is that the Vision Pro is packed with a load of tech that's entirely new to the brand. Here are the innovative features that we can expect to see.
Apple Vision Pro is Apple's first spatial computer
Apple chose not to emphasize Vision Pro as a headset for augmented reality or virtual reality. Instead of playing to those terms, Apple went straight for the familiar, referring to it as a "computer" — a term we've all known for decades.
The Vision Pro is a spatial computer, per Apple's own description. This goggle-style headset is the first computer to provide access to an app environment that literally surrounds you. It blends the real and digital worlds in ways we haven't seen before by interpreting physical movements as navigation inputs.The Vision Pro's mission matches the definition of spatial computing — combining physical actions with a three-dimensional virtual space — as coined in Simon Greenwold's 2003 MIT master's thesis.
As a concept, spatial computing is not new, even if it is first for Apple. Both Microsoft and Magic Leap have referenced spatial computing in recent years. In fact, it even overlaps directly with the concept of mixed-reality headsets first attempted by Microsoft with the Microsoft HoloLens.
It's Apple's first computer that lets you control your view of the real world
Apple's Vision Pro is a true mixed-reality device. The Vision Pro's occluded goggles can become more transparent at will with a turn of the Digital Crown, which is found on the right side of the goggles. This way, you can control how much of the real world you see beyond the 3D visionOS environment. If you want full immersion in a gorgeous 360-degree landscape or a movie, you can do so. If you want to see the room beyond while you work on a document or place a FaceTime call, you can do that, too.
Apple is not the first to mix the two worlds, but its video demos make it appear Apple has achieved integration at a whole other level. A clear advantage of Apple's implementation is that you can easily interact with those around you. Dubbed EyeSight, this new feature makes the goggles more transparent automatically. This way, you can engage someone who approaches you and maintain eye contact without removing the headset.
As described by Apple, this strikes us as the vision equivalent to noise-canceling headphones' hear-through feature that automatically detects someone talking to you and reduces the sound so you can have a conversation without removing your earbuds or headphones.
Vision Pro is the first device to use Apple's visionOS platform
There's been plenty of digital ink spilled about the Vision Pro headset itself. Equally of note, though, is the Vision Pro's software and its visionOS platform. Apple isn't cavalier about introducing new operating systems, so the new software platform bodes well for Apple's long-term investment in visionOS. Apple is also one of the few tech companies around with the clout to build out a new software platform credibly.
By design, visionOS uses familiar design constructs in apps, icons, and even familiar features like FaceTime. They may behave differently in the visionOS environment, but there's no mistaking their familiar veneer. That familiarity gives visionOS an edge over competitors. There's plenty to learn, but also enough is similar to feel welcoming to anyone who's used an iPhone or iPad recently. VisionOS is chock-full of new concepts, too — immersive environments, infinite workspaces that let you set up multiple workspaces however you wish, and collaborative FaceTime calls — all enabled by the Vision Pro's hardware and software.
Apple's core strength lies in its walled-garden approach, which lets the company precisely match its hardware with its software. As a new platform, visionOS has that same technical advantage since the OS shares frameworks with other Apple operating systems. This means it will be easier for developers to bring existing iPadOS and iOS apps to visionOS using the visionOS SDK. Adapting an app could be as simple as updating an app's settings in Xcode and recompiling that app's code for visionOS. It also means that when the Vision Pro ships next year, we can expect its app store to have a dramatic jumpstart on content.
Apple Vision Pro is Apple's first augmented reality device
As we noted above, at its launch Apple pointedly steered clear of pushing Vision Pro as an AR/VR device. CEO Tim Cook only mentioned augmented reality during his verbose introduction of the Vision Pro at WWDC 2023, stating, "I believe that augmented reality is a profound technology. Blending digital content with the real world can unlock experiences like nothing we've ever seen. So today I'm excited to announce an entirely new AR platform with a revolutionary new product."
However, the Vision Pro is very much poised for virtual reality — and especially augmented reality — experiences. The goggles' ability to open your view to the real world while interacting with on-screen elements is the very definition of augmented reality. More critically, Apple opened the door wide to development for visionOS and the Vision Pro platform with its ARKit, which was first introduced at WWDC in 2017. Since then, it's been a programming language waiting for a platform beyond smartphones or tablets to showcase what it can do. In fact, Apple's Happy Beam sample game for visionOS was created using ARKit.
ARKit lets developers bring the digital and real worlds together by adding 2D and 3D graphical elements superimposed over your device's live view. Those elements can then work with the device's sensors and software to provide augmented reality experiences. It's hard to imagine such apps because we haven't seen them yet, but we're willing to bet that will change once Vision Pro ships next year.
Vision Pro is Apple's first head-mounted display
If you break the Vision Pro down to basics, it's a souped-up head-mounted display. It's the first such head-mounted display from Apple. It can stand on its own or even work with your laptop's actual MacOS environment. Apple's launch went into detail about its attention to comfort and industrial design. The company used lightweight, breathable materials and fittings on the Vision Pro so it could be a comfortable and ergonomic wearable. Many head-mounted displays before it have failed because of poor fit, heavy designs, and lack of comfort over extended use.
It uses micro OLED technology to pack 23 million pixels across the display. That equates to more pixels than a 4K TV for each eye. Apple describes each display as "the size of a postage stamp."
Apple reportedly was collaborating with TSMC, its Taiwanese iPhone processor supplier, on the manufacture of micro OLED displays as far back as 2021. However, the technology remains expensive to produce, let alone to produce at scale. A more pixel-dense variant of the OLED technology in televisions and other devices, micro OLED displays require less power than other display technologies.
Vision Pro is Apple's first personal cinema
At one point, there'd been chatter about Apple actually producing its own television set. That never happened, and we got the Apple TV streaming box instead. However, it seems that Apple never abandoned the idea of a personal home theater — it simply scaled that vision to fit with its first head-mounted display, the company's first personal cinema.
Apple's Vision Pro launch spent a lot of time talking about filmed entertainment. Tim Cook even welcomed Disney CEO Bob Iger to talk about how Disney is looking at the Vision Pro as a platform for delivering more personal stories. "The first time I tried Apple Vision Pro, the thing that struck me the most was how it will allow us to create deeply personal experiences that bring our fans closer to the characters they love and more deeply immersed in our stories," Iger gushed. "This platform will allow us to bring Disney to our fans in ways that were previously impossible."
Iger's statements raise lots of questions about what kinds of stories might get this treatment, but are viewers even interested? It's certainly encouraging to hear a major studio interested in exploring the possibilities of a new technology platform, and the Disney+ streaming service will be available on the Vision Pro when it ships in 2024.
Apple says the personal cinema experience feels like you have a 100-foot wide private cinema screen in front of you. The Vision Pro's integrated speakers support Spatial Audio, so soundtracks will sound impactful, too. Apple touted using the screen for movies, TV shows, Apple Immersive Video, and 3D movies.
Apple's first 3D camera
It's not surprising that Apple includes a three-dimensional camera to capture 3D stills and video as one of Vision Pro's dozen on-board cameras. This is the first time Apple has embraced 3D imaging, opening the Vision Pro to a host of immersive experiences.
The examples Apple showed included taking 3D family pictures and videos as well as capturing 360-degree panoramas that you can then share via iCloud and relive in life-size scale through the headset. Apple's description of the resulting images notes you will feel like you're in the middle of the action, which goes beyond what we've seen thus far in standard stereoscopic 3D imaging.
Mainstream users dabble in the novelty of 3D imaging, but rarely has it made more than a blip on the radar. The seeming simplicity of Vision Pro might change that. With Vision Pro, Apple provides an end-to-end solution for capturing and consuming 3D content. And if Apple is talking about sharing 3D content, one has to wonder if Apple is already planning to share beyond Vision Pro's bubble.
Virtual reality and spatial environments rely on three-dimensional content. It only makes sense that Apple will make it easy for you to capture content and use it within the Vision Pro in ways we haven't seen yet. The possibilities go beyond preserving life milestones like first steps and weddings. Imagine doing a walkthrough of a new home or preserving a family homestead before it's sold. Or maybe you want to capture objects in 3D simply to bring those objects into the virtual world.
It will be interesting to see what developers come up with at Vision Pro's launch — and to what degree Apple gives developers access to the 3D camera and the images it captures.
A whole new navigation system with hand, eye, and voice interaction
With Vision Pro and its corresponding visionOS, Apple does what it does best: innovate by precisely matching its software magic with the device's hardware. Forget the mouse and keyboard, the Vision Pro has an entirely new way of interacting with your eyes, hands, and voice. We've seen eye tracking for years on games, but Vision Pro moves eye tracking into the realm of productivity and general navigation.
Vision Pro lets you navigate your environment not only with your eyes but with a tap of your fingers, your voice, or a subtle motion of your wrist. The standard constructs of input and navigation have no place in this world straight out of science fiction. Apple's developer materials for visionOS call out how accessibility friendly its new navigation is, and we agree it has potential.
The visionOS' Pointer Control lets you choose how you navigate through spaces, with options for index finger, wrist, or head. You can move through Vision Pro's three-dimensional space with your eyes, browsing apps and selecting options without ever touching a thing. The longer your eye lingers, Vision Pro detects your interest as intent and can reveal more information. Pinch your fingers together to select something on the screen the same way you'd tap with your finger on a phone. You can also use two hands to zoom and rotate or pinch and drag to scroll through content.
Additionally, Vision Pro supports a virtual keyboard, can work with Apple's Magic Keyboard and Magic Trackpad, and supports popular game controllers, which will be useful for the 100 Apple Arcade games coming at launch.
Vision Pro is the first with optical biometrics
We've seen Face ID unlock devices instantly. Now, the Vision Pro Apple introduces Optic ID, its biometric system for unlocking the headset when you place it on your head. Optic ID also works with Apple Store purchases, Apple Pay, and password autofill. This new optical technology examines your iris and compares that data to the encrypted eye print stored in Vision Pro's Secure Enclave, a secure processor subsystem that's separate from the headset's SoC processor.
Data stored in the Secure Enclave stays local to the device. You don't have to worry about your eye data being shared with Apple or any third-party apps or websites. Optic ID works using the same cameras and LEDs inside the headset as are used for eye tracking. These LEDs send out infrared light onto the eyes, which allows the internal cameras to capture the visuals of your irises and save them into the Secure Enclave.
An optical authentication system is critical for a headset like Vision Pro. However, given the number of cameras and LEDs required to capture the information, don't expect to see this technology filter down to other devices anytime soon.
Vision Pro is Apple's first display with external battery
Since we've already established that Apple Vision Pro is a head-mounted display, it follows that the Vision Pro is also the first Apple display to use a battery pack. Sure, you can plug into an outlet, too, but to use Vision Pro to its fullest and most freeing potential, you'll want to rely on its tethered battery pack.
Apple typically eschews external dongles (or likes to think it does), which is it's a bit ironic that Vision Pro has an external battery pack with a tethered wire to the headset. Unsurprisingly, the battery pack looks proprietary from the pictures, meaning you'd need to buy Apple's battery, but it's also early days and that could change. It's also not hot-swappable, according to early analyst reports.
That's a shame, considering the battery only lasts about two hours, according to Apple. However, it may be less concerning if the battery can fast charge — meaning you can take it off and take a break from the virtual world while it charges up. Apple hasn't released details about the battery yet, so we're sure we'll hear more as we get closer to Vision Pro's release in 2024.