According to recent revelations and rumors, Apple will announce the AR/VR headset for the first time at WWDC2023 in June. In this regard, AR/VR optics expert Karl Guttag is skeptical. He previously wrote in a DisplayDaily article that Apple's development of AR/VR headsets is more like worrying about missing out on new technology trends. Looking back at some key AR products in the past, from Google Glass, Magic Leap, and HoloLens, although a lot of research and development costs have been invested, the market has not given the expected feedback.
As far as Apple is concerned, no matter how much research and development costs are invested, AR hardware still faces many insurmountable limitations, such as the limitations of physics and human factors (the wavelength of light perceivable by the human eye is limited, and the size of the display screen is small). Guttag said: The laws of physics make the development of all-weather AR glasses difficult, and related technologies still need major breakthroughs. In addition, due to the small size of the display/optical module used in the AR/VR head-mounted display, it also faces problems such as diffraction and etendue. Even VST cannot solve the problem of AR optics, because VST mixed reality headsets still have limitations in social, safety, and ergonomics. It is true that AR/VR has a slower learning curve than semiconductors, but many related technologies are approaching the limits of physics, so technological breakthroughs require greater investment.
Guttag believes that although AR/VR has shown actual advantages in enterprise-level scenarios, it is difficult for its sales to reach the level of the iPhone. In the field of mobile AR, the success of "Pokemon Go" does not mean that AR is expected to be widely accepted, because it is difficult for subsequent LBS AR games to break through. On the contrary, if only a few technology giants such as Apple enter the AR market, it will still be difficult to promote widespread purchase by consumers.
Recently, in an interview with the podcast AR Show, Guttag further shared his views on Apple's AR/VR and VST mixed reality headsets replacing AR glasses. Let's take a look if you are interested.
The price of AR headset is difficult
KG: There are more and more technology and hardware choices, and the cost is getting lower and lower. For example, you can take professional photos and videos with a smartphone, which obviously brings market competition for professional cameras. The same is true for AR. Smartphones have powerful functions and many price options. In contrast, emerging computing devices such as AR glasses lack advantages in terms of price and content.
People have become accustomed to spending less money and getting more technical experience, so it is difficult to pay for expensive AR headsets/glasses. In other words, the high cost of AR actually creates a threshold for new users to buy, especially at a price of around US$3,000. Consumer expectations will be very high, and AR technology may not be able to meet such high expectations in the short term.
AR Show: Apple is the promoter of the smart phone industry. In the early days of smart phone development, telecom operators provided a lot of subsidies, including discounted phone charges to attract consumers to buy, which further lowered the price of mobile phones. For consumers, $3,000 is a price inflection point. If the price of Apple’s AR/VR headset is at this level, it may lack an advantage in price compared to the iPhone.
Apple's AR/VR Strategy
AR Show: In a recent blog, you have suggested that Apple may postpone the AR glasses project indefinitely and develop VST VR headset instead?
KG: Yes. I have always felt that Apple and Meta have the same mentality for AR/VR, namely: VR technology is real and feasible. But the market is limited and not big enough to get mainstream consumers excited. Although AR technology is immature, its market potential seems to be large, and there are no known limitations.
Most companies believe that VR is only a subset of the game market. Like games, VR has loyal supporters, a real market, and profit opportunities, but the market size is too small. On the other hand, although many people like VR, VR is not sticky enough. Often people don't use VR headsets long after they buy them, and put them in the cabinet. And when the new VR headsets are released, consumers are not very willing to buy new VR headsets because they already have an idle VR at home. I heard that VR sales even dropped by 2-3% last year.
With its brand influence and user scale, Apple is indeed expected to boost sales of high-end VR headsets and improve VR headset technology, but this will not fundamentally change market demand.
In fact, VR is not yet a "four-corner market", that is, a market with comprehensive development. For example, Harry Potter IP has a four-corner market, seniors, adults, teenagers, children, men, women like it, covering almost all groups of people. In contrast, the VR audience is more teenagers and young men, and although there are more female users than a few years ago, the VR market is still not well developed.
Especially for big companies like Apple and Amazon, low-return, money-losing projects are a nuisance.
AR Show: Apple's entry into the AR/VR hardware market, can it bring different application scenarios for this technology and expand it to a wider market?
KG: AR/VR is just as difficult for Apple. Apple also needs to solve the limitations of AR/VR in physics and other aspects. Also, if Apple builds a $3,000 high-end AR/VR headset, it will need to provide subsidies to attract people to buy it, which means that Apple may not have much profit margin on AR/VR headsets. And typically, Apple charges higher margins on its products than any other brand.
On the other hand, even a $3,000 AR/VR head-mounted display device may not solve the technical limitations of AR/VR in the short term, such as vertigo, visual quality, human eye dynamic zoom, and so on. Even raising the cost of an AR/VR headset to $10,000 would hardly solve these problems.
In addition, while the application scenarios and markets for AR/VR are real, the market size is still small. In the enterprise market, AR headset shipments may be 200,000 units per year, but the question is, how to reach millions of units? How to really change the AR/VR market?
AR Show: Why do you think Apple puts so much time and energy into developing AR/VR?
KG: Maybe Apple is building technical barriers, worrying that AR/VR will become mainstream, but it has missed the opportunity. If Meta spends $12 billion a year on AR/VR, Apple will have to spend at least a few billion to catch up, and this scale of investment is acceptable for Apple, even if the return is not high.
According to people familiar with the matter, Apple engineers believe that the AR/VR headset is immature, but management wants to launch the product as soon as possible to verify that the multi-billion dollar investment is effective so that it can continue to invest. At this stage, while Apple AR/VR may be a good headset product, it is likely to not meet Apple's usual quality, like a laboratory technology that has not yet been perfected.
This has happened several times, and the result is almost always a colossal failure. If Apple's AR/VR headset is mature enough, the engineers will definitely be willing to bring it to market as soon as possible. But if a technology hits the market too early, it's likely to fail.
Yes, Apple is capable of building a premium AR/VR headset, but the company's strategy is different than in the past. For example, in the Jobs era, after a mobile phone was released, it could be shipped within half a year. According to rumors, the first Apple AR/VR may be more focused on developers before being widely released to consumers.
AR Show: AR/VR and early mobile phones are in different situations. When Apple first launched the iPhone, the mobile communication market had existed for many years, with 25 billion mobile device users, and the annual sales of smartphones had reached 100 million units. Compared with the mobile market, the VR market is much smaller, and it is not growing as fast, and may even shrink.
KG: Apple may improve AR/VR in some aspects, such as optimizing hardware and software design, and improving UI and UX. In the early days of smartphones, mobile phones such as BlackBerry already supported touch screen technology, but the experience was not yet mature. Apple uses predictive algorithms to improve touchscreen interactions, making browsing the web on mobile phones smoother. As screen sizes get bigger and phones do more and more, web browsing adds a smarter experience.
However, considering that AR/VR is a wearable device, there are obvious contradictions in ergonomics, such as vision adjustment solutions suitable for different people. Due to the different vision conditions of the wearer, the vision adjustment mechanism is critical for AR/VR headsets.
At present, there are three common vision adjustment mechanisms for AR/VR headsets: 1) large eye distance, which can be superimposed on frame glasses; 2) built-in mechanical refraction adjustment system; 3) with prescription lens inserts. These mechanisms have their own advantages and disadvantages. For example, HoloLens 2 can be worn directly on frame glasses, which means that most of them can be used directly. However, this design also means that the side of the head display is thick, making it difficult to make glasses.
If you want to make AR into the form of glasses, you will need to shorten the eye distance, which means that the lenses are closer to the face, and you cannot wear a frame underneath. If you customize insert lenses for AR glasses, or customize optical waveguide lenses with diopters, on the one hand, the cost is high, on the other hand, it is not suitable for multiple people to share, and it is not easy to be accepted by more people, so the potential market size is limited. The effect of the diopter adjustment mechanism is not good enough. On the one hand, the accuracy is difficult to guarantee, and on the other hand, other vision problems such as astigmatism cannot be corrected.
Only the problem of vision correction will bring thresholds for AR/VR glasses. If AR glasses are not easy to use, it will be difficult to open the market.
In my opinion, the reason why Apple and Meta entered VR may be to be optimistic about AR. The purpose of Meta's acquisition of Oculus may be to accumulate experience in VR and lay the foundation for AR. In fact, no matter how it changes, the VR market cannot be as big as mobile phones. In theory, when AR/VR realizes the form of glasses, it can replace mobile phones in some form, but this is very difficult and is a long-term vision, and it may not even be realized. After all, some people never wear glasses, or don't upgrade as often as they buy cell phones.
As far as Apple is concerned, I have a question, namely: Since the AR glasses project has been postponed indefinitely, why should it enter the smaller VR market? This is not very reasonable.
Indeed, in addition to the consumer market, AR/VR has potential application value in enterprise-level scenarios such as museums, factories, logistics, and training. For example, usually a factory needs to pay each worker more than US$100,000 in wages and benefits per year, compared to the price of HoloLens 2, which only costs about US$3,000. If HoloLens 2 can improve worker efficiency by 10%, then a few months It will pay back, and it will bring more returns in the future.
Although the HoloLens 2 hardware is expensive, its built-in software services may be worth more, and it is cost-effective enough for enterprises.
Even so, the sales of AR/VR headsets on the enterprise side may only be between 300,000 and 400,000, which is far behind the sales of smartphones (billions). Instead of developing AR/VR glasses, it is better to make a car AR HUD, at least it may be installed on every car.
Advantages and risks of VST scheme
AR Show: It is rumored that Apple is developing a VR headset that supports VST perspective. The advantage is that it can switch between AR and VR modes. Lynx has also launched a similar head-mounted display. The company's CEO Stan Larroque once believed that VST perspective may even replace HoloLens and Magic Leap in the future. What do you think of this? Can VST see-through VR headsets replace high-end optical AR?
KG: Whether it is VST mixed reality or optical AR, they each face their own problems that need to be solved. Technologies that are easy to implement in optical AR are almost impossible to implement in VST mixed reality headsets, and vice versa.
Stan underestimated the difficulties of MR, such as latency, dynamic zoom (screen and camera), saccades (saccade), lens thickness, camera distance (matching IPD), vertigo, etc. When using the camera to see through the surrounding environment, the camera should be located directly in front of the user's left and right eyes (and change the viewing angle according to the change of the gaze point). However, Quest Pro adopts a rather strange design, equipped with a single RGB camera and two black and white tracking cameras. During the perspective process, the system maps the RGB camera data on the tracking camera data to synthesize a three-dimensional, color perspective image. The perspective effect of this solution is not good, and the image quality is poor.
Why does Quest Pro use such a camera solution? Explain that Meta should be worried that if dual RGB cameras or more cameras are used, it will bring a computing burden to the VR head display, and the delay will be higher.
In addition, VST perspective will block the peripheral vision of the human eye to a certain extent, so it is not suitable for use when walking outdoors, or it is limited to a specific space like VR.
In short, VST headset technology is not yet mature enough in the short term, and requires high-resolution light field cameras, high-resolution screens, and more efficient computing, which may take 20 years to develop.
In addition, from the perspective of safety, the ideal AR glasses/head-mounted display needs to have high light transmittance, and only display a small amount of information without blocking the line of sight, which is difficult to achieve in the perspective mode of VR. Of course, there can still be intersections between AR and VR. For example, in some industries, AR or VR headsets can be used in different scenarios: use VR in training, and use AR outdoors. However, judging from the IVAS project of HoloLens 2, there are still limitations to using AR outdoors (weight, eyeglow, durability, etc.), especially for actual combat scenarios, and it will take at least decades to perfect the technology.
AR Show: Indeed, VST perspective is not a trend of AR, but an enrichment of VR functions, which can improve the safety of VR. As far as Lynx is concerned, the flip-up design of the R1 headset is borrowed from the HoloLens 2, which is a very practical solution.
KG: Yes, the flip-up design of Lynx R1 allows you to quickly switch between virtual and reality, which is a design suitable for MR headsets. Still, the R1's image quality is probably the worst of any VR headset on the market.