In the previous course, we introduced the development process of AR and its underlying logic in detail . But if AR technicians want to become king from bronze, they still need to overcome the following five challenges.
Challenge one:
Interface issues-lack of universal UI metaphors
Imagine if you brought a person from the 19th century into today’s society, let him sit in front of a computer, and would he be able to surf the Internet without any prompts? Does he know how to use the mouse and keyboard? Can he even turn on the computer successfully? The answer should be no.
It's like in front of AR, we are all novices. Although we know what we can do with it, we are not sure how to use this breakthrough technology and how to use a common language to help our dreams shine into reality. IT practitioners will certainly be familiar with this concept-User Interface Metaphor, or UI metaphor for short. It is a common interface type suitable for multiple technology platforms. For example, mobile phones, computers and even refrigerators can be equipped with QWERTY keyboards (also known as "full keyboards"). As long as we see such a keyboard, we know what it is and how to use it. However, the difficulty with AR is that it is a brand-new technology and cannot really benefit from any interface metaphors. We cannot learn from precedents, and without too much background knowledge, it is difficult to seize the unique digital opportunities that AR brings us.
Like the smart phone AR application introduced earlier, this is currently the main way people experience AR. Many developers or users are still exploring the AR technology and applications, such as how to reflect the real environment, virtual environment, and let users understand mobile phones, scan the environment, interact or watch virtual objects, and prompt actions to avoid danger, etc. No common, intuitive UI habits have been formed.
Another example is the Google Glass optical head-mounted display, as shown in the figure below. What should we do when selecting an option on the menu? Tap, tap, or use voice? Or is it really necessary to use the menu? For AR interface designers, this requires research from the ground up, designing and testing the most suitable operating mode for this platform.
Challenge 2: High power consumption, easy to heat up
In detail, the technical factors that make AR extremely challenging are complex. Here, it can be simply boiled down to three words: size, power consumption, and heat.
Size
Although we have made great progress in miniaturizing processors and graphics cards, we still haven't reached the level of making high-end AR a reality. In the process of continuous evolution of AR devices, developers found that wearing the processor on the head is convenient but bulky. Therefore, some AR prototype designers believe that an external computing package the size of a hard disk can be used to accommodate AR equipment that handles complicated tasks and reduce the weight of the frame. This will be more ergonomic.
Power consumption (Power)
In addition, the process of rendering the AR experience is very power-intensive. Think about it, your phone consumes a lot of power when you watch videos online. However, AR wants to create an immersive experience. It needs to generate images while streaming images and track other objects in the room. In addition, the device must be recalibrated every time the user's body moves, which consumes a lot of power on the device. To solve this problem, some people think that an external battery pack can be hung on the belt. This idea is currently feasible, but if we want AR to realize its full potential, we must solve the power consumption problem in the device frame.
Heat (Heat)
As far as existing electronic equipment is concerned, a lot of heat is generated during the calculation process. Whether it is a notebook or a desktop computer, there is a built-in fan to help the computer dissipate heat. In fact, the greater the power consumption, the more heat is generated, and the smaller the device, the slower the heat dissipation. AR rendering is a very complicated process, so it is inevitable that it will get hot. The heat generated will in turn reduce the speed of the processor, or even short-circuit it completely. In addition, the limited size and structural requirements of AR devices make it more difficult to solve the heating problem. It is conceivable that it will be extremely difficult to realize this vision in goggles weighing only 100 grams, leaving little space for heat sinks and fans. Some AR headset manufacturers package the required processing power in the frame of the goggles themselves, while others use external hardware such as battery packs to solve this problem. Both methods have their own advantages and disadvantages, which requires AR designers to balance the three aspects of device size, power consumption, and heat dissipation.
Challenge 3: High barriers to 3D content design
Today, photography has become a way for people to record their daily lives. Various softwares for taking pictures and pictures are emerging in endlessly. Almost everyone knows how to take pictures, but once in a while this was a skill that professionals only had. There are similar skills in the AR field, which we call-three-dimensional design. Things in AR mainly exist in three-dimensional space. Although not all of these things are in 3D form, most of the resources, applications, and experiences require some knowledge of 3D design in the process of building.
At present, the base of technical personnel with this ability is relatively small. Unless they are professional animators, graphic designers, mechanical engineers or video game producers, it is difficult to have much experience in this area. If AR wants to develop, it needs to widely popularize the concepts, technologies, and language of 3D design. At the same time, intuitive creation tools and applications such as AR stickers and Google Blocks should also lower the technical threshold for producing 3D content. In the follow-up ARCore development introductory series of courses, we will tell you how to apply these tools to practice.
Challenge 4: "Computer Vision" recognition is difficult
The last obstacle that AR needs to overcome on the road to maturity is "computer vision." It refers to the process of allowing the computer to see and understand the hardware and software in the real world. For example, if you search for the keyword "dog" in Google Chrome, you can find pictures related to dogs. This is because Google has unique search algorithms and tools that have classified these pictures under the category of dogs.
However, the computer vision process can really allow search engines to see the captured pictures and automatically recognize whether there is a dog in the view. Don't underestimate this simple example, it has great significance. For example, the computer can see and recognize pedestrians and traffic lights, thereby ensuring that driverless cars can run safely on the road. In addition, it can also view the surrounding environment and place digital objects in appropriate locations. This idea is great, but it is difficult to realize. At present, computer vision is a rapidly developing technology that still has many limitations. It is a long way for developers to make computers recognize everything in the world and classify them into categories at any time.
Challenge 5: Real-time occlusion of dynamic objects is difficult to achieve
We introduced the concept of "occlusion" in the previous course. Before the ARCore upgrade, virtual objects could not be occluded by real objects, and users often played out instantly. But in 2020, ARCore Deep API successfully helped developers realize this function. In the future, the goal of AR is to naturally achieve occlusion during rendering. Therefore, as a developer, you still need to understand the limitations of technology and continue to innovate solutions. In addition, ARCore has started to support the "shadow" function in 2019. 3D game engines such as Unity can also successfully do this. This can undoubtedly help AR developers create more authentic content. In the following courses, we will introduce to you how to use Unity.
The development of AR technology from budding to maturity requires developers to continuously improve their knowledge reserves and overcome the above-mentioned difficulties. In the next course, we will help you understand the advantages of the ARCore platform and prepare for the creation of AR applications.
Further reading: