Image: Midjourney / DALL-E 2 invited by MIXED
Apple’s possible XR headset is said to have a speed dial to change realities and many other features. The killer app is said to be realistic avatar XR video calls.
The Information claims to have once again received inside information from sources close to Apple familiar with the development of Apple’s XR headphones. The report contains many technical details about the helmet, including new information.
The most important points of the overview:
- Apple’s XR headphones are said to be built from aluminum, glass and carbon fiber and is compact and lightweight.
- The headset is powered by a waist mounted battery, which can be replaced during operation. This design would have been imagined by the former chief designer of Apple, Jony Ive.
- Two integrated processors, one Main SoC (M2) and for video processing (ISP)both manufactured in 5nm and offering a proprietary streaming codec.
- The ISP is said to stream and distort the distorted external image from the cameras built into the headset with low latency. This is crucial for high quality mixed reality apps. Here, Apple’s acquisition of NextVR could pay off; there have been similar rumors before. The chip is said to use fast, custom working memory from SK Hynix.
- Additionally, a H2 chip is integrated for ultra-low latency audio transmission to Apple AirPods Benefits. Conventional Bluetooth headphones aren’t supposed to work optimally, and there’s no headphone jack.
- However, the headband has built-in speakers, but their sound is audible to outsiders, which can be a problem in terms of privacy. It’s said to be made of a material similar to the Apple Watch’s sports bands. For developers, there is supposed to be an alternative headband that can be connected to a Mac.
- A fast dial comparable to the Apple Watch is supposed to allow a quick switch between the virtual and physical world. The button does not offer haptic feedback.
- Corrective lenses can be inserted into the helmet via magnets if the user is visually impaired.
- the the lens distance is automatically adjusted at the distance of the eyes of the wearer of the helmet via an integrated motor.
- the the field of view is said to be 120 degreeswhich would be slightly wider than the Meta Quest Pro’s 106 degrees and significantly wider than open AR headsets like Magic Leap 2 or Hololens (about 50 degrees).
- It is said that Apple prefers hand tracking as an interfacebut experiments with alternative input devices such as a wand and some kind of thimble.
- Games aren’t Apple’s focus, and a game controller isn’t included. Pure VR experiences are to be offered through the Unity game engine as a partner company.
- The Apple headset can display existing iOS apps in 2D.
Apple bets on XR video calls as a ‘killer app’, offers headset face to strangers
The information confirms some earlier rumors, including that the headset will have a outward-facing screen who shows strangers helmet wearer facial expressions. This feature is supposed to reduce the feeling of isolation when putting the helmet on. The screen is said to be low-power and has a high refresh rate, comparable to the Apple Watch’s always-on mode.
The information confirms that Apple will use a Sony 4K micro OLED display for the center of the image per eye. This is complemented by an LG screen in the peripherals, according to earlier rumours. Eye tracking, including foveal rendering, is also built-in, which saves power by rendering only the user’s exact point of focus with full graphic detail.
In total, the helmet would have two LiDAR scanners for short and long distances and over a dozen built-in indoor and outdoor cameraseven detecting the legs of the person using the headset.
In addition to education, one of the objectives of the helmet would be XR video conferencing with realistic avatars which faithfully reflect the movements of the face and body. The avatar’s eyebrow and jaw movements should be estimated by an AI process based on the available sensor data.
Apple also wants to activate a transition between Mac screen and XR screen: If you extract the 2D Maps application from the Mac screen, it can display a 3D model of a city in XR mode.
Depending on the configuration, the headset should costs about $3,000. Apple’s contract manufacturer Pegatron reportedly built several thousand prototypes last year, and the launch was originally scheduled for 2022.