This article is reproduced from OpenHarmony TSC official WeChat public account " Summit Review Issue 8 | OpenHarmony Distributed Hardware Key Technologies "
Speakers | Li Gang
Review and arrangement | Liao Tao
Typesetting proofreading | Li Pingping
Guest profile
Li Gang, Huawei OpenHarmony technical expert, OpenHarmony distributed hardware technology leader. Mainly responsible for OpenHarmony distributed hardware architecture design and Huawei multi-device collaboration research.
content source
The 1st Open Atom Open Source Foundation OpenHarmony Technology Summit - Ecology and Interconnection Sub-Forum
video review
Video Link: Summit Review Issue 8 | Key Technologies of OpenHarmony Distributed Hardware - Gang Li_哔哩哔哩_bilibili
Contents
OpenHarmony is an operating system for the future Internet of Everything scenario, and its design adopts a distributed architecture. So what key distributed technologies does OpenHarmony have compared to traditional operating systems? Li Gang, Huawei's distributed hardware technology expert, shared a few points with you at the first OpenHarmony Technology Summit.
01 ►Distributed hardware design concept
From the perspective of the development trend of smart terminals, it has become increasingly difficult for a single smart terminal hardware to meet the requirements of users for all scenarios, and it is facing a development bottleneck: due to size limitations, it is impossible to add all the hardware involved in all scenarios into one device , and a single device cannot meet the needs of all scenarios. Based on this real pain point, the "super terminal" formed by the "combination" of multiple smart terminals came into being. According to user expectations, HyperTerminal can combine multiple devices through distributed technology, share hardware resources among devices, realize "free" expansion of hardware, and combine hardware resources on demand in different business scenarios, providing Better user experience. For example, through the combination of smart terminals such as mobile phones, tablets, watches, large screens, and computers, functions such as multi-screen series connection, multi-camera and microphone interaction, and professional sensor arrangement can be realized.
Smart Terminal Evolution
Under such a trend, it is difficult for traditional operating systems to meet the requirements of developers. Because the traditional operating system can only use the hardware on a single device, each piece of hardware runs separately, and the application can only work in the vertical field of a single device, so the cost and complexity of cross-device experience are extremely high. For developers, they hope to be able to share hardware "cross-terminal" and break the hardware PCB boundary, so as to define hardware through software and build a "super terminal" with multiple devices in all scenarios. Distributed hardware technology can help developers meet the above needs, because it can build a hardware resource pool, provide the ability to define hyperterminal hardware on demand, support multi-channel hardware coordination and scheduling, and enable hardware capabilities to be self-adaptive.
What kind of new experience can distributed hardware bring? For example, in office scenarios, users can easily link various smart terminal devices to realize hardware capability sharing, cross-device and cross-system application operations, and seamless data transmission; in travel scenarios, users can connect mobile phones and cars Combined with the mobile phone, the application can share the hardware of the two, realize the seamless operation of functions such as navigation, music and calling, and achieve "smart travel". For developers, to control a remote device through a program, they only need to select its corresponding ID, and other operations are exactly the same as using the local device.
Smart Scenario Example
02 ►The core challenge of cross-terminal distributed hardware
To realize the above functions and experiences, what challenges exist in cross-end distributed hardware technology? As the HyperTerminal contains more and more devices, the complexity of hardware management also increases. The hardware of each device not only provides hardware capabilities for the device, but also empowers other devices in the HyperTerminal. Therefore, the operating system must provide management capabilities for each device. For example, the update and synchronization of each hardware state, the processing of hardware conflicts, the processing of multi-channel concurrency, etc. The management technology between multiple devices is one of the core challenges of current cross-device distributed hardware.
Cross-terminal multi-hardware management
In the wireless network environment, the bandwidth is limited, and the delay and effect of hardware calls are difficult to guarantee. For example, the delay and shooting effect of the local camera are determined by the hardware bus, which can generally reach a delay of tens of milliseconds and a resolution of 4K or higher, and is very stable. When controlling the camera remotely through the application, in addition to the hardware bus, it is also affected by the network signal transmission, the delay can only reach a few hundred milliseconds at the lowest, and the resolution can only reach 1080P, and the fluctuation is large. The unreliable wireless network brings great uncertainty to the hardware delay and effect.
HyperTerminal hardware calls
When multiple hardware calls are made concurrently across terminals, it is very difficult to synchronize hardware synchronously. As mentioned above, the delay of cross-end hardware scheduling itself is difficult to guarantee. When multiple hardware devices are called across the end at the same time, the synchronization between devices is even more difficult to guarantee. For example, when the application needs to operate the camera and the microphone at the same time, the consistency can be determined locally through the debugging work of the two when they leave the factory; when the remote operation is performed across terminals, due to the uncertainty of the delay, the operating system needs to be installed on multiple devices. The hardware collaboration between them greatly increases the complexity of the software.
Hyperterminal multi-channel hardware concurrent calls
In addition, hardware compatibility and fault tolerance among heterogeneous smart terminals are also very difficult. Different devices lead to large differences in their respective system resources, processing capabilities, supported hardware data processing types, driver I/O, etc., and further consideration needs to be given to compatibility and fault tolerance between hardware. For example, the processing power of the watch is relatively weak, and it is difficult to use the 4K screen of the TV, the high-channel stereo speakers and the ultra-high-definition camera.
03 ►Key technology of distributed hardware platform
First, OpenHarmony adopted a distributed hardware pooling architecture at the beginning of its design. By abstracting the capabilities of each device, an abstract model of a global hardware resource pool is constructed, and a set of unified hardware abstract interfaces is provided to realize unified management and plug-and-play. In addition, the architecture also supports the expansion and on-demand deployment of hardware types, decoupling logical and physical resources, and realizing the indiscriminate use of local and distributed hardware. The hardware is used by the application, and the application developer only needs to call the API of the upper layer service to use it, achieving the effect of software-defined hardware.
Distributed hardware pooling architecture
Second, the distributed hardware platform also provides a unified device discovery and authentication framework. Supports device authentication between devices by touching, scanning, and leaning. Once the device is discovered and authenticated, the hardware of the device will automatically enter the hardware resource pool and can be shared with other devices.
Device Discovery and Authentication Framework
Third, the distributed hardware platform provides hardware adaptive technology. It can automatically negotiate hardware capabilities, and can dynamically adjust hardware effects when the hardware is invoked through the perception of network bandwidth and delay. In addition, adaptive conversion technology is also provided to realize the enhancement of hardware effect through enhancement algorithm.
hardware adaptive
Fourth, the distributed hardware platform also provides hardware synchronous technology. It can provide millisecond-level clock synchronization capability in multiple channels to ensure the consistency of multiple hardware devices. Through the hardware delay dynamic perception technology, the synchronization policy is dynamically issued to multiple devices to ensure the synchronization experience between multiple hardware.
hardware co-sync
In OpenHarmony3.2, the distributed hardware platform provides a new hardware resource pooling architecture, which can realize the "super terminal" hardware mutual assistance ability of cameras and screens. In addition, technologies such as distributed hardware pooling architecture, device discovery and authentication framework have also been implemented. For application developers, it is more convenient to realize the development of multi-device collaborative scenarios, and it also provides a huge room for imagination.
OpenHarmony3.2 distributed hardware function
04 ►Innovative ideas and prospects
In the future, distributed hardware technology can provide a new experience for multiple scenarios. For example, in the conference scene, the conference application can be connected to the cameras of multiple devices synchronously, providing panoramic images, and realizing a full range of video conferencing; in the audio-visual entertainment scene, it is easy to put the audio and video of the mobile phone on the TV and speakers Playing, you can also make the lights at home automatically change with movies and music, achieving a very shocking home theater effect.
We look forward to more and more developers participating in the OpenHarmony ecosystem, jointly researching and discussing the technical problems of distributed hardware, and empowering new scenarios for the Internet of Everything in the future.