Breaking Down the Apple Vision Pro: A Revolutionary Spatial Computer

With companies striving to offer new, more engaging, and unique ways to experience all kinds of content, we are about to witness innovative products mixing reality with virtuality. Now it’s time for Apple to introduce an all-new device, powered by a new operating system and designed for a new era. Let’s look at the Apple Vision Pro headset, visionOS, and a new way to experience human-computer interaction – Spatial Computing.

I’m a Mobile Developer, and my main focus is to deliver great apps on iPhone and iPad screens. As time progresses and technology evolves, the software I create uses more technology features to involve users in the experience. In my career, I had a chance to work with not-so-popular software technologies like Augmented Reality. My job was to create an app that enabled the user to interact with the real world using the iPhone’s camera. For example, one of the functionalities involved placing a virtual pin with a note in a room so that minor repairs could be made during a renovation. Mixing reality with a visual layer turned out to be a new niche, which Apple recently filled with the introduction of the Vision Pro headset.

Apple’s Vision Pro was launched on February 2, 2024 in the United States. The international launch date is yet to be announced. The price starts from $3,499. That’s no small price to pay for such an enigmatic device, so let’s take a look inside and see what Apple’s new product really is.

What is Apple Vision Pro?

The market that brings virtual reality to consumers isn’t new, but it’s still fresh. For some time, technology had to chase the ambitious ideas of developers. We’ve seen the VR technology in games, from the earliest headsets by Oculus (a company eventually bought by Meta and used to create Metaverse) to the more polished and user-friendly Meta Quest 3.

Competition in the AR/VR market

We’ve seen a lot of products with the goal of mixing real-world environments with a computer-generated one. A lot of companies have tried their best to create devices that will break out in this field. The peak moment for these kinds of devices was when Microsoft launched HoloLens – a headset mixing camera images with 3D objects rendered by the system. At that time, it looked like Microsoft had successfully launched an impressive electronics innovation to revolutionize the market. While the company did follow up with a second generation unit in 2019, Microsoft eventually discontinued its Mixed Reality platform by the end of 2023.

While the idea was impressive, it turned out to be less useful than it appeared. The device was used only in specific niches and the commercial version was never released to the public. As more companies try to dominate this specific market, the niche is still empty and waiting for its big moment.

Apple’s preparations for market entry

Apple has been working to enter the mixed-reality market for a decade. The first take on bringing a new reality to Apple’s devices was with the introduction of a developer’s tool in 2017 called ARKit. It helped developers to place 3D objects on the video feed from the iPhone or iPad’s camera. Apple showcased the feature with dinosaurs walking on a school court recorded from the iPhone X camera. This was no new technology but, for the first time, they showed their interest in adopting the solution to a big market of iPhones and iPads.

As ARKit evolved and enabled users to share their augmented reality session, play games together using different devices, and detect 3D objects, it was clear mixing computer-generated reality with our world is here to stay. The main problem with these kinds of experiences was that it was still only in the form of 2D images shown to us by our smartphones and tablets. We want to be in that action, not standing next to it. 

With the next iterations of iOS, developers had access to technology that can detect movements in the human body, the face, and, more interestingly, could detect very detailed movements of the users’ hands (the developer framework is called Vision, similar right?). The signs were clear – Apple is working on their take to mix augmented and virtual reality. It was rumored to run a completely new operating system, (internally called xrOS) that was about to introduce new and unique ways to interact with a computer environment.

Apple finally announced Apple Vision Pro and visionOS in June 2023 at the WWDC developers conference. They showcased their perspective on a completely new way to work, play, and consume media. What stands out in their idea is that Vision Pro tries to target not only the pro-consumer field (like many products before, such as HoloLens), to be an everyday companion for normal users that use Macbooks as their primary device. Apple wants us to ask “What’s a computer?” in the upcoming years, something they’ve never achieved with iPads.

Design and technical specification

Apple’s Vision Pro is designed to fit the needs of every single person. Users can choose from different head straps and ZEISS optic lenses that fit their requirements. At a glance, Vision Pro can remind you of ski glasses, with a bunch of cameras and proximity sensors hidden underneath, so the user doesn’t have to worry about “how it works” – instead keeping Apple’s iconic “it just works” tagline in mind. At the front of the device, there is a screen that shows the digital representation of the user’s 3D-scanned upper face – thanks to that, we can interact more naturally with a person whose face is covered under the headset. The front screen is also an indicator for showing if a person is currently immersed in an app and can’t see their surroundings.

Inside, Vision Pro is packed with a lot of sensors and cameras that help to create a virtual model of the users’ surroundings. These sensors gain information about users’ movements from head and body tilts to detailed hand and finger movements. All of this is used to control visionOS, which we will talk more about in the next paragraph. Apple says that it uses ground-breaking, almost genius, technology to eliminate lags from video and audio input to user output. With this technology, the user is expected to feel no discomfort during longer sessions with the device. However, not all headset specs are that impressive.

Let’s take a look at what’s inside Apple Vision Pro:

• Two 4K micro-OLED screens supporting a refresh rate of up to 100 Hz
• R1 chip with “photon-to-photon” low 12-ms latency
• M2 processor
• Two high-resolution main cameras
• Six world-tracking cameras
• Four eye-tracking cameras
• LiDAR scanner and TrueDepth camera (well-known from iPhone)
• Secure authentication with Optic ID iris scanning
• Operating system control by voice, face, and hand movements
• Powered by visionOS
• 600 – 650g weight
• Up to 2 hours of battery life using a USB-C power adapter
• 256 GB, 512 GB, or 1 TB of storage

Now, while the sensors, cameras, screens, and latency results are impressive, some of the tech specs look not as good. 

• Apple’s new headset comes with a M2 processor, a technology first announced in 2022 and followed by the M3 family. Launching a ground-breaking device with almost 2-year-old chipsets at the time of launch doesn’t seem good for the longevity of the headset. 
• Some users wearing Vision Pro have pointed out that, after a long session, they can feel neck pain. It looks like weight distribution could be better. 
• The biggest flaw seems to be battery life or the lack of it. Apple sells the device by arguing that it has “all-day battery life while charging the device”. Without charging, and even with a battery pack, we can only use Vision Pro for 2 hours. This is something to improve in the next generation of the device for sure.

What’s inside the box?

Apple Vision Pro comes in a massive box that houses not only the device but also a bunch of peripherals.

• Apple Vision Pro headset
• Light Seal, two Light Seal Cushions in different sizes, and a Solo Knit Band
• Apple Vision Pro Cover for device protection when not in use
• Dual Loop Band
• Polishing Cloth
• 30W USB-C Power Adapter
• USB-C charge cable

visionOS: The heart of Apple’s next-gen AR/VR experience

With the attempt to enter the AR/VR segment, Apple had to choose the system on which the Apple Vision Pro would run. No operating system from Apple’s portfolio was a great fit for this device.

Apple’s operating systems

• macOS – notebooks, desktop computers
• iOS and iPadOS – mobile devices
• watchOS – wearable devices
• tvOS – digital media players
• audioOS – smart speakers

The natural direction was to create a completely new operating system, adapted to the new content control options. Additionally, it needed to remain compatible with macOS and iOS / iPadOS. That’s how visionOS was designed – a unique way to consume new content formats mixing augmented and virtual reality, with support for apps designed for mobile devices and even capable of mirroring your MacBook screen in front of you.

A look at visionOS’ features

When you put the Apple Vision Pro on, you don’t lose touch with your real surroundings. Thanks to the high-quality cameras around the headset, you still see the room you’re standing in, even though a whole new experience is about to begin.

In front of you, you see a simple menu grid containing 13 app icons per page. As you move your hands, you see they can now control the visionOS menu. Your body interacts with the UI, the closer your fingers are to the interface controls, the more UI components glow. You control the menu with intuitive gestures like swipes and pinches, or even voice control.

VisionOS takes advantage of the Apple ecosystem and provides unique ways to consume content from other devices, such as the iPhone and iPad. You can browse photos and videos shot on the iPhone in the Vision Pro. Resize them to see every small detail. Panorama photos are displayed on a curved screen that surrounds the user.

The immersion intensifies when the user turns the digital crown and changes their real surroundings to prepared 360 recordings of the wonders of nature. The system is aware of your body and tracks your eyes, hands, and voice. Where visionOS shines, however, are the apps, primarily those that use platform-specific features to create a compelling experience.

Apple Vision Pro is a completely new product in Apple’s portfolio, so it does not offer a large number of apps at the start. For the platform’s launch, it is still impressive, but it is no match for a device like the iPhone or Mac. The system lacks well-known iPhone and iPad apps like Netflix, Spotify or YouTube. When big companies resist making their mark on a new platform, they give small developers a chance to shine. Apple Vision Pro is a great place to break through with a unique app idea.

You can create breathtaking apps on visionOS that are not possible to code on any other operating system. There are three types of experiences you can give your users that create a three-dimensional interface.

Apps can be displayed as:

• Windows – the classic approach to present content. Two-dimensional boxes that you can rotate and resize.
• Volumes – scenes containing three-dimensional objects placed in your room. You can move those virtual objects around and see them from every angle.
• Spaces – the most immersive experience, where the app changes your environment. When you are inside the app, you can adjust what you see with a Digital Crown, which is well-known from Apple Watch. Rotate it to change the opacity of the app’s space and see more of your room.

visionOS App Store and app development

The systems’ App Store is open to submissions. Developers can write apps that will be available on many Apple platforms simultaneously, with each version of the app having platform-specific features. For example, an iOS app can share a codebase with the visionOS version, so developers don’t have to write separate versions of their software. Some adjustments are always needed, but most of the code can be shared between systems. The time needed to write code for visionOS is not very different from writing applications for other Apple platforms.

The possibilities for visionOS apps are enormous. Familiar apps like Apple TV hold amazing potential, where you can watch 3D movies and TV shows in your living room like you own a 100′ TV screen. You can move around 3D virtual objects like pieces of furniture and place them inside your room.

You can immerse yourself in virtual spaces and feel like you just entered a portal to another dimension or just place the solar system in your living room and learn about the planets. Apple Vision Pro launches with some popular games so you can entertain yourself with Super Fruit Ninja, for example,  specifically for the headset. Besides that, Apple’s new headset supports 100 games available on iOS and iPadOS App Store.

Understanding Spatial Computing

Spatial Computing isn’t a new term. In fact, it was first used in the 1990s to describe the interaction between a user and 3D spaces. Apple saw the opportunity and used it in the context of Apple Vision Pro, which fits perfectly. As a label, Spatial Computing describes advanced human interaction with digital content in a physical world.

Apple previously used the word Spatial in the term Spatial Audio, which refers to surround sound technology. Recently, iOS gained support to record Spatial Videos – 3D movies shot on iPhone 15 Pro. Apple Vision Pro’s headset supports Personalized Spatial Audio and Spatial Video, which makes it their all-in-one device for spatial experiences of any kind.

This perfectly captures what you can do on Apple Vision Pro: placing multiple app windows in a room, listening to surround sound, watching 3D movies, resizing a virtual TV screen to cover your wall, turning your ceiling into a moving sky, or expanding the possibilities of physical devices like using a Macbook screen in visionOS. This is Spatial Computing.

Apple Vision Pro – is it worth it?

Apple Vision Pro is a great opportunity for companies and developers. The App Store, still in its infancy, has a lot of space for new apps to break through. Thanks to the innovative and unique ways to control the headset and its new visionOS operating system, Apple Vision Pro has gained a lot of attention from the media. This gives hope that the product will be a commercial success and that Apple will continue to support it through the years.

As is the case with the first versions of new products, Apple Vision Pro critics have some concerns. We can not ignore that the headset is only available to customers in the United States. The price barrier may be another serious problem, as $3,499 is an enormous price for a basic customer. Battery life represents yet another big flaw as, in order to take advantage of the Apple Vision Pro for longer periods, you have to be plugged in (the provided battery, when not plugged in, only lasts for 2 hours).

At the time of writing this article, three large companies have said that they will not release their applications on the system – these are Spotify, Netflix, and Google’s YouTube. On the other hand, Apple Music and Disney+ are available on day one. The market will verify whether the lack of key applications will affect the success of Apple Vision Pro.

visionOS app development with Droids On Roids

At Droids On Roids, we develop native software for the Apple ecosystem – from iOS and iPadOS to macOS and watchOS. I had my hands on the visionOS operating system as soon as Apple released the Developer Beta. As it turned out, software development for Apple Vision Pro is not very different from developing mobile apps on iOS. Droid On Roids specializes in creating reusable code that allows applications to run on all Apple platforms – including visionOS.

Would you like to create an app that works with the Apple Vision Pro? Let’s talk about your idea!

Wrap-up

What an exciting time it is right now. It’s not often that companies launch new products with as much commitment and budget as Apple has shown when introducing the Vision Pro. Notably, the last time Apple introduced a new platform to the developers was in 2014 with watchOS and the first Apple Watch Series 0 in 2015. Apple Vision Pro is a much more interesting and unique product than smartwatches, with more power and space to sell your product on the App Store.

The headset can not avoid childhood problems such as the lack of most of the well-known iPhone and iPad apps. Moreover, the high price and limited availability will not help it achieve spectacular success quickly. In my opinion, it is worth taking the risk and getting on the Apple Vision Pro hype train so as not to miss the moment when the headset becomes a key player in the software market.

Related posts