one. introduction
In the field of autonomous driving and robotics, algorithms have always been at the core of research. Whether it is navigation technology, control technology, or recognition technology, it is an important part of its technology stack. However, with the development of technology, developers gradually realized a problem, that is, the organizational structure and effectiveness of the program itself also had a great impact on the correctness and accuracy of the system. The quality and performance indicators of data transmission, such as low latency, high load capacity, and high usability, have gradually attracted the attention of engineers. This makes the importance of middleware and architecture design gradually highlighted. At present, there are mainly two types of solutions in China: one is to develop based on existing open source middleware; the other is to develop middleware by oneself or secondary research and development based on open source middleware.
two. Robot Architecture Design
Robot architecture design mainly faces four directions
(1) The overall design and operating environment of the robot operating system.
(2) A robot operating system kernel that supports real-time and intelligent computing.
(3) Robot functional components and visual integrated development environment.
(4) Key technology of robot network distributed processing.
Complete the technical coverage of the four directions through the following four aspects
(1) Operating system
(2) Real-time operating system kernel
(3) Robot Operating System (ROS)
(4) Robotics Middleware Framework Robotics Middleware Framework (RMF)
Among them, the ubuntu operating system is used to support the operation of the entire structure, RT-Thread supports the real-time kernel, and ROS and RMF support the functional components and distributed processing of the robot
In the following, we will elaborate on these aspects respectively.
1. Operating system
At present, many robots use ubuntu 1804 as the basic operating system of the robot host computer. This is mainly because, first, the upper layer software is developed based on the ROS robot operating system, and the ubuntu1804 operating system is currently the longest and most Stable operating system, most embedded hardware manufacturers will provide this operating system for free, and it integrates the ROS Melodic version. Using the ubuntu1804 operating system will undoubtedly reduce the development workload at the system level. The following is the corresponding relationship between the ubuntu system and ros
version code
The corresponding relationship between some versions of Ubuntu and ROS1 and the end time of official support are shown in the table below.
| Ubuntu |
ROS1 |
release time |
Support period |
| 14.04 LTS |
Indigo Igloo |
May 2014 |
April 2019 |
| 16.04 LTS |
Kinetic Kame |
May 2016 |
April 2021 |
| 18.04 LTS |
Melodic Morenia |
May 2018 |
May 2023 |
| 20.04 LTS |
Noetic Ninjemys |
May 2020 |
May 2025 |
| 22.04 LTS |
- |
- |
- |
The corresponding relationship between some versions of Ubuntu and ROS2 and the end time of official support are shown in the table below.
| Ubuntu |
ROS2 |
release time |
Support period |
| 18.04 LTS |
Crystal Clemmys |
December 2018 |
December 2019 |
| 18.04 LTS |
Dashing Diademata |
May 2019 |
May 2021 |
| 20.04 LTS |
Foxy Fitzroy |
June 2020 |
May 2023 |
| 22.04 LTS |
Humble Hawksbill |
May 2022 |
May 2027 |
But recently, due to the authorization problem of the ubuntu system in the commercial field, many hardware manufacturers have turned their attention to the open source system debain. The reason why the robot field chooses debain instead of centos or redhat is because Ubuntu is derived from Debian. This means that Ubuntu uses the same APT package management system as Debian and shares a large number of packages and libraries from the Debian repositories. On the other hand, the domestic operating system adapted to domestic chips in my country is also transformed from the ubuntu operating system. For robot manufacturers, the use of the ubuntu operating system will undoubtedly make our products easier to integrate into the current domestic software and hardware ecology. In the chain, it can greatly reduce the development workload, and more quickly adapt to the migration workload brought by the country's security requirements for basic hardware and basic operating systems.
Because the operating system is closely related to ROS, choosing an appropriate operating system will help us reduce the work of system adaptation and accelerate the development process
At present, the operating systems we can use include ubuntu1804, ubuntu2004, debain11,
Consider the following issues when choosing:
1. Copyright issues. The ubuntu1804 system is the operating system we are currently using. Using this operating system, the ROS version and other dependent library versions that our upper-level software depends on are consistent with the current products we release, and the compiler gcc version is also Consistent, can greatly reduce our software migration work and overall machine testing work, but on the official website of ubuntu, there is such an explanation on copyright issues: Must not require royalty payments or any other fee for redistribution or modification.It's important that you can Exercise your rights to this software without having to pay for the privilege, and that you can pass these rights on to other people on exactly the same basis. That is to say, you cannot
charge for republishing or modifying the software. Whether we want to circumvent such authorization restrictions is to be considered
2. Ubuntu18 and ubuntu20, some of the new functions that robot manufacturers want to use are easier to compile and pass on ubuntu20. Of course, this is not the most fundamental basis for us to choose the ubuntu version. Considering the mainstream version of ros in the next few years is our key consideration. Yes, choose the ubuntu operating system version according to ros, to ensure that the stable version that we have spent a year developing can be used stably for at least the next 3-5 years
2.RTOS
When discussing RTOS, we must first understand real-time operating systems and time-sharing operating systems, as well as related concepts such as soft real-time and hard real-time
A real-time operating system (Real Time Operating System, referred to as RTOS) means that when an external event or data is generated, it can accept and process it at a fast enough speed, and the processing result can control the production process or process within the specified time. An operating system that responds quickly to the processing system, schedules all available resources to complete real-time tasks, and controls the coordinated operation of all real-time tasks. Providing timely response and high reliability are its main features.
A real-time operating system is an operating system that is guaranteed to complete a specific function within a certain time limit. The real-time operating system can be divided into hard real-time and soft real-time. Hard real-time requires that the operation must be completed within the specified time, which is guaranteed during the design of the operating system; soft real-time only needs to complete the operation as quickly as possible according to the priority of the task. That's it. The operating system we usually use can become a real-time operating system after certain changes.
14 kinds of mainstream RTOS, namely μClinux, μC/OS-II, eCos, FreeRTOS, mbed OS, RTX, Vxworks, QNX, NuttX; and domestic embedded operating systems include Dujiangyan operating system (djyos), Alios Things, Huawei LiteOS, RT-Thread, SylixOS.
ROS is a set of software frameworks running on Linux. Linux itself is not a real-time system (it can be converted into a real-time operating system by patching). Some tasks of robots that require high real-time performance are not suitable for Linux, such as RT-Thread is a real-time operating system (RTOS) kernel developed and maintained by a domestic team. Although it is a real-time operating system (RTOS), after all, MCU has limited resources and cannot directly run a complete ROS.
Therefore, the usual practice is to use the rich software packages of Linux to implement some top-level algorithms, while RT-Thread is responsible for real-time control-related tasks, and the communication between them can be through rosserial and micro_ros. The common goal of rosserial and micro_ros is to connect MCU to ROS, so that MCU can communicate with ROS, and call ROS API on MCU. The main difference is that rosserial is for ROS1, while micro_ros is for ROS2.
Whether the robot needs to introduce soft real-time or hard real-time kernel functions is closely related to the product.
Here we only list the comparison between FreeRTOS and RT-Thread
Freertos is a mini real-time operating system kernel launched abroad. It is open source. Its functions include: task management, time management, semaphore, message queue, memory management, recording function, software timer, coroutine, etc., which can basically meet the needs of small System needs.
RT-Thread is a technology platform launched by the Chinese that integrates real-time operating system (RTOS) kernel, middleware components and developer community. It is an open source os. In addition to the functions of conventional RTOS, RT-Thread also has an IoT OS All key components required by the platform.
Such as GUI, network protocol stack, secure transmission, low power consumption components and so on.
Let's compare Freertos and RT-Thread in the kernel, supported components, and drivers:
3. Robot Operating System (ROS)
ROS is an open source meta-operating system for robotics. In fact, it is not a real operating system. Its underlying task scheduling, compilation, addressing and other tasks are still completed by the Linux operating system, which means that ROS is actually a secondary operating system running on Linux. However, ROS provides various services for operating system applications (such as: hardware abstraction, low-level device control, common function implementation, inter-process message passing, software package management, etc.), and also provides code for acquisition, compilation, and cross-platform operation. tools and functions.
At present, a large number of manufacturers still use ROS1, and the architecture and functions in the ROS1 skeleton framework cannot meet the needs of today's rapidly developing robot industry. ROS2 has been developed with an enhanced architecture for use in industry and academia. To avoid deprecating current applications developed with ROS, ROS 2 was developed not as an update to ROS1, but as software to run alongside existing versions.
(1) Process management: In ROS1, the central node manager Master needs to be turned on to manage all nodes in a unified manner. If the Master node fails, it will seriously affect the ROS system function. In ROS2, the system introduces a node self-discovery mechanism, which can effectively improve the robustness of the system. In ROS1, before running a node, the ROS master node needs to be started. The ROS1 master node will act as a DNS server for the nodes so that they can retrieve each other.
In ROS2, there are no more ROS master nodes! This is no longer a centralized system. Each node has the ability to discover other nodes. One can simply start a node without worrying about whether there is a master node running or not. Fully distributed systems can be created.
(2) Inter-process communication: ROS1 developed the TCPROS and UDPROS protocols based on the TCP and UDP protocols. In ROS2, the communication protocol was replaced with a more complex but more complete DDS system.
(3) In-process communication: If a large amount of data communication is required in the process, both ROS1 and ROS2 provide communication methods based on shared memory, which are Nodelet and Intra-process modules respectively.
(4) Cross-platform: The bottom is the system layer, which is the operating system on which ROS can be installed. ROS1 is mainly installed on Linux, and ROS2 can be cross-platform, such as Linux, windows, MacOS, RTOS.
(5) Multi-robot team application scenarios that cannot be solved by ROS1. In the future, robots will not be independent individuals. Communication and collaboration between robots are also required. Although ROS can be used to build multi-robot systems today, there is no standard method. And they are all based on the single host structure of ROS. ROS2 provides standard methods and communication mechanisms for the application of multi-robot systems.
(6) We want to support real-time control directly in ROS, including inter-process and machine-to-machine communication (assuming suitable OS and/or hardware support). Robot motion control and many behavior strategies require the robot to be real-time. For example, the robot must reliably detect pedestrians in front of it within 100ms, or complete kinematics and dynamics calculations within 1ms. ROS1 does not support real-time systems. ROS2 Provides a basic guarantee for real-time requirements like this.
(7) On non-ideal networks: We want ROS to perform as well as possible when the network connection is degraded by packet loss and/or latency (from poor WiFi to ground-to-air communication links), regardless of the In the network environment, ROS2 can try to ensure the integrity and security of a large amount of robot data. For example, when the wifi signal is not good, the data should be sent as much as possible. In the scenario where there is a risk of hacking, the data should be encrypted and decrypted.
(8) In ROS1, services are synchronous. When a service client makes a request to the server, the request is stuck until the server responds (or fails). In ROS2, services are asynchronous. When calling a service, you can add a callback function that will fire when the server responds. Meanwhile, your main thread won't get stuck.
(9) The interface is more unified. In ROS1, for Cpp, you use roscpp, and for Python, you use rospy. Both libraries are completely self-contained and built from scratch. Some functional interfaces are not uniform.
(10) ROS2 has more layers. There is only one basic library called rcl, which is implemented in C language. This is the base that contains all ROS2 core functionality. Build Cpp and Python interfaces on this basis, and the underlying interfaces are unified
(11) Write nodes using OOP, in ROS1 there is no specific structure telling you how you should write node functions. You can decide to add callback functions anywhere in your program, or use OOP if you want, but the implementation of each function can be unique. In ROS2, things are different. There is a convention on how to write nodes. You must create a class that inherits from the Node object. This will save everyone a lot of time. You already have a nice modular structure to write nodes. This will make your program cleaner and it will be easier for developers to collaborate on different projects.
(12) The development and transplantation of Navigation software in ROS 2 is critical to the improvement of the core functions of ROS 2 and the application of commercial products. More meaningful packaging ensures the adaptability of Navigation2 to future needs, such as support for different map formats, robot bodies, path and obstacle avoidance algorithms, etc.
Generally speaking, in addition to functional enhancements, ROS2 has also been greatly enhanced in the following aspects of performance and security
(1) Real-time enhancement: data must be updated before the deadline;
(2) Continuous enhancement: DDS can provide data history service for ROS 2, and newly joined nodes can also obtain all historical data released by publishers;
(3) Reliability Enhancement: To configure the reliability principle, users can choose performance mode (BEST_EFFORT) or stable mode (RELIABLE) according to their needs.
(4) Security enhancement: Encrypt and decrypt data in scenarios where there is a risk of hacking
(5) The performance in several aspects such as delay, throughput, number of threads, and memory consumption has also been greatly improved
About the communication problem between ROS1 and ROS2
ROS2 provides a ROS2 package that provides a bridge to exchange messages between ROS 1 and ROS 2.
Generally speaking, it is inevitable for ROS2 to replace ROS1, but whether to use ROS2 to replace ROS1, I think it still needs to be considered for Chinese robot manufacturers. Why do I say that? Because both ROS1 and ROS2 are output from universities Yes, its post-maintenance and version updates cannot keep up with the rhythm of the enterprise, especially ROS1, which still has hundreds of bugs that have not been resolved. Although this situation has improved on ROS2, it is obviously supporting not enough.
Let's talk about Baidu's Apollo Cyber RT. Apollo Cyber RT is an open source, high-performance runtime framework designed for autonomous driving scenarios. Based on the centralized computing model, the main value is to improve the high concurrency, low latency, and high throughput of the automatic driving system.
Apollo did not use CyberRT from the very beginning, and before v3.0, it was developed based on the ROS framework. However, many problems were found in the previous version. With the development of Apollo, the demand for the highest level of robustness and performance, Apollo Cyber RT came into being, which meets the basic needs of a commercially oriented autonomous driving solution .
From the perspective of technical details, I feel that Apollo Cyber RT and ROS2 have roughly the same optimization direction for ROS1. If the appollo community and developers are more powerful, there is no doubt that appollo will be chosen.
4. Robotics Middleware Framework (RMF)
Imagine a world we live in where robots from different companies and manufacturers can co-exist in the same facility; elegantly sharing important resources such as hallways, lifts (elevators), doors and other infrastructure for greater efficiency the overall system. Imagine being integrated in a way that allows any robot to use shared resources in a reliably controlled and safe manner. Imagine a world without the dilemma of robots in shared corridors. Today, these ideas are possible using an amazing system called RMF.
Today, current-generation robots in production environments are capable of providing services including batch and piece-flow delivery, cleaning, sanitizing, security, monitoring, and more. These diverse use cases will likely mean that the best robot for each task will come from different robot providers or system integrators. This modern reality is crucial to use an appropriate common software framework to manage these heterogeneous resources and ensure effective utilization of information on different platforms to improve overall system efficiency.
Without a plan for an overall efficient robotic system, end users may face significant but hidden risks when committing to a single system or platform provider. Hidden risks may force end users to limit user selection of future solutions from that particular provider in order to minimize operational risk and avoid redundant integration costs. This problem is exacerbated as robot deployments expand in scope and scale, leaving customers feeling that there are no good alternatives to staying with their current provider.
In addition to the added cost risk of scaling deployments with different providers, there is an inherent aspect of sharing resources (e.g. elevators, doorways, hallways, network bandwidth, chargers, operations center screen “real estate”) and human resources (e.g. IT staff) Conflict and maintenance technicians. As bots scale, it becomes more cumbersome for operations teams to consider managing large, heterogeneous, multi-vendor bot environments.
These problem statements are the fundamental motivation for the development of RMF. Historically, ROS development has focused on software that runs on or near a single robot. RMF is designed to operate at a higher level of abstraction to create teams of networked robots that interoperate with building infrastructure systems, enterprise services, IoT devices, and human-machine interfaces.
Using RMF unconstrains developers' choices, devices, and futures.
Use robot operating system ROS2 and robot middleware framework system RMF for multi-robot system integration, high-level planning and its application, etc.
RMF is a collection of reusable, scalable libraries and tools built on top of ROS 2 that enable interoperability of heterogeneous teams of robotic systems of any kind. RMF utilizes standardized communication protocols with the infrastructure, environment and automation in which robots are deployed to optimize the use of critical resources (i.e. robots, elevators, doors, pathways, etc.). It adds intelligence to the system through resource allocation and preventing conflicts between shared resources through the RMF core, which will be described in detail later in this blog post.
RMF is flexible and robust enough to run on almost any communication layer and can be integrated with any number of IoT devices. RMF's architecture is designed to allow scalability as the level of automation increases in the environment. Systems and users can interact with RMF in a variety of ways through APIs and customizable user interfaces. Once deployed in an environment, RMF will save costs by allowing shared resources and minimizing integration. This is what robot developers and robot customers have been looking for. In short, the RMF architecture is as follows:
Concepts such as maps, floors, paths, and waypoints are applicable to almost all mobile robots, so RMF can be used directly. But what other concepts are shared between different types of robots? RMF currently abstracts the tasks of mobile robots into three types: cleaning, transportation, and circulation. Of course, other types may be added in the future. You may be surprised that these three types can cover almost all types of mobile robots at present. The cleaning task is full coverage of an area, so not only vacuum robots, but also autonomous lawn mowers if they are in the system. Loop means going back and forth between fixed waypoints, which is very suitable for security patrol tasks. And there are many different robot tasks that can be abstracted as delivery tasks. Delivery is a common need, and the robot picks up the goods at one point, and then moves to another point to put them down. The most important of these is the abstraction of the two concepts of taking and putting down. The system only gives such commands to the robot and monitors the execution status, regardless of how it is implemented. For example, when an automatic forklift picks up goods, it first recognizes the pallet and then docks it; the composite robot grabs the goods through the mechanical arm; and some robots rely on manual cooperation to pick up the goods. These are all taken. Similarly, the robot can put down the goods in various ways. Even a welcome robot taking a guest from one place to another can be seen as a delivery task, even though the guest just follows the robot, and there is no physical pick-up and drop-off. These specific implementations are done by the supplier, but after being abstracted, RMF can manage them in a standard way.
The specific implementation of the task is still completed by the supplier, but it is subject to the management of RMF. Therefore, it is necessary to look at the connection level between RMF and the supplier system. RMF does not care who the supplier of the robot is, but only what type of robot it is and what tasks it can perform. Therefore, the same robots are generally put into a fleet and managed at the fleet level. Of course, in most cases at present, the supplier's system can become the fleet's system as long as some adaptations are made. Of course, in view of the various situations described above, complexity cannot be completely avoided, depending on whether the previous system design is weak or not. RMF would of course like to have complete control over the fleet, ideally with individual control over the routes of each robot. But the worst case is that a fleet is completely unschedulable, we only know its running status, but there is no interface to modify tasks, or even to suspend. RMF also has a response to this. It can be used as a read-only fleet, and other fleets must avoid it. Obviously, there can only be one read-only fleet in the system, but this at least guarantees that multiple fleets can be used simultaneously. A better level than this is the control of traffic lights, and the route still cannot be modified, but at least the task can be suspended and resumed at this time, which is more convenient in resolving resource conflicts. The ideal situation is when all fleets are fully in control and the system can be most efficient.
3. Summary
According to the above summary, the following are just my suggestions
-
Use the debain11 system to replace the ubunutu18 operating system, and customize the system, such as replacing the existing gnome desktop with a lightweight desktop, reducing unnecessary system services, and reducing the unnecessary load of the graphical interface on the system
-
In view of the fact that service robots do not require absolute hard real-time, the method of injecting real-time patch packages into the kernel allows task priority scheduling to achieve soft real-time and increase the real-time performance of robot actions
-
Using Apollo Cyber RT to replace ROS1 can greatly improve the competitiveness of the product in terms of system functions, performance and security
4. Use the RMF architecture to improve the position in the field of product integration and improve the ecological chain