HuNavSim: a ROS 2 human navigation simulator for benchmarking human-aware robot navigation Paper Reading

Paper information

Title : HuNavSim A ROS 2 Human Navigation Simulator for Benchmarking Human-Aware Robot Navigation
Authors : Noé Pérez-Higueras, and Roberto Otero
Source : arXiv
Time : 2023
Project Address : https://github.com/robotics-upo/hunav_sim

Abstract

This work presents the Human Navigation Simulator (HuNavSim), a novel open source tool for simulating different human agent navigation behaviors in mobile robot scenarios. The tool is programmed under ROS 2 framework for the first time and can be used with different well-known robot simulators such as Gazebo. The main goal is to simplify the development and evaluation of human-perceived robot navigation systems in simulation. In addition to general human navigation models, HuNavSim is novel and includes a rich set of personal and realistic human navigation behaviors and a comprehensive set of metrics for social navigation benchmarking.

Introduction

Assessing a robot's skill at navigating a scene shared with humans is critical to conceiving new types of service robots that work alongside humans.

The development of such a mobile social robot presents two main challenges:

1. First, real experiments on humans are costly and difficult to perform and replicate except in very limited controlled scenarios. In addition, human participants may be at risk, mainly during the initial stages of development.
2. Second, the evaluation should not only evaluate the navigation efficiency, but also the safety and comfort of people. The latter requirement is a human feeling, which is difficult to quantify through mathematical equations.

With HuNavSim, we aim to provide solutions to the two mentioned problems: providing individual agents with a diverse set of real-world behaviors and providing a compilation of metrics used in the literature. In brief, we present the following contributions:
i) An open-source and flexible human navigation simulation tool under the ROS2 framework [1] that can be used with different robot simulators.
ii) A rich set of navigational behaviors for human agents, which includes a set of realistic personal responses to the robot's presence.
iii) A complete compilation of metrics from the literature into a tool for evaluating human perceptual navigation that is configurable and extensible.
iv) Use this tool with a wrapper for the famous Gazebo simulator used in robotics (see Figure 1).

Related Work

PedSimROS1 is a base library adaptation of ROS that uses the social force model (SFM) [2] to guide crowd movement. MengeROS2 [3] is a more complete tool that uses A* and potential fields for global path planning and allows selection among a set of collision avoidance strategies such as different variants based on SFM, Optimal Mutual Collision Avoidance (ORCA ) [4] and Pedestrian Velocity Obstacles (PedVO) [5].
However, both PedsimROS and MengeROS are deprecated software integrated into unmaintained ROS 1 releases. Also, they do not contain any navigation evaluation options.

CrowdBot3 [6] and SEAN (Social Environment for Autonomous Navigation) 4 [7], [8] are newer, advanced and ambitious tools. They have similar characteristics. Both are based on the game engine Unity and ROS 1, and both aim to be the standard for evaluating robot navigation in densely populated environments.

In contrast to them, HuNavSim offers a more flexible approach, allowing the tool to be used with different simulators and providing a set of personalized and realistic human responses to the presence of robots. Additionally, these tools provide a closed set of indicators, while HuNavSim includes a larger compilation of indicators that can be easily configured and extended.

An interesting simulator is the Intelligent Human Simulator (InHuS) [9], [10]. This simulator is designed to control the movement of a human agent in another simulator. It also includes a small subset of individual human behaviors, as HuNavSim does. Compared with InHuS, our simulator adopts a powerful and flexible tool - behavior tree [11] to define and drive specific human behaviors. These trees can also be easily modified and extended. In addition, InHuS is based on ROS 1 and mainly employs HATEB2 [12], a navigation system designed for robots, to control human motion. This can lead to more unrealistic crowd movements than simulators based on specific crowd movement models.

SocNavBench5 [13] takes a different approach. It is a simulator-based benchmark that replays pre-recorded real-world pedestrian data. The main disadvantage of this tool is that pedestrian trajectories are replayed from an open-source dataset, so the impact of robot motion on pedestrian paths is not considered. This makes realistic evaluations of human perceptual navigation difficult.

Other methods are designed to train, test, and benchmark learning-based navigation agents in dynamic environments. For example, Arena-Rosnav [14] allows training deep reinforcement learning based obstacle avoidance methods. It utilizes the Pedsim library to guide the movement of pedestrians. Arena-Rosnav was then expanded to include navigation benchmarking, Arena-Bench [15], and a larger set of utilities for development and benchmarking (Arena-rosnav 2.0 [16]). for

To simulate and train reinforcement learning agents in a space shared with humans, we found SocialGym 2.0 [17], [18], which also uses the Pedsim library to simulate human navigation.

These tools do not include specific human behavior nor a complete and specific set of metrics for human perceptual navigation.

human Navigation Simulator(HuNavSim)

Architeture of the simulator

The overall architecture of the simulator is shown in Figure 2. HuNavSim is responsible for properly controlling human agents generated in another base simulator such as Gazebo, Morse or Webots, which must also simulate the scene and the robot. Therefore, a wrapper that communicates with the underlying simulator is required.

Initially, the user has to provide the number and characteristics of the agents to be simulated (e.g. individual behavior, target list, etc.). It can be specified via a configuration yaml file or a GUI based on the ROS 2 RViz panel.

Then, at each execution step, the simulator wrapper sends the current state of the human agent to the hunav manager module via a ROS 2 service. Based on the current state, the system decides the next state of the agent, which is returned to the wrapper and thus updated in the base simulator.

Finally, the hunav estimator module records the experimental data and computes the evaluation metrics at the end of the simulation. Information about each simulation and the desired set of metrics to compute can be specified via a yaml file or via a list of checkboxes displayed in the ROS 2 RViz panel. The module produces a set of output results files that contain simulation information and the names and values ​​of calculated metrics.

Human navigation modeling

HuNavSim is primarily based on using the well-known social force model [2] and its extensions to crowds [21], [22] to lead movements of human agents, like other crowd simulators. However, we extend this model to provide a realistic set of individual navigational responses of the agent to the presence of the robot. This can enrich navigation scenarios and challenge navigation algorithms with more diverse and realistic human behaviors.

The included behavior sets are as follows:
• $regular$ general: This is the general navigation based on SFM. The robot is seen as another human agent, so humans treat the robot like another pedestrian.

• $impassiveindifference\; :\; In\; this\; case,the$ robot is not included in the SFM as a human agent. Therefore, humans treat robots like obstacles.

• $superised$ surprise: current goal navigation stops when the human sees the$robot$$.$The human stops walking and starts looking at the robot (walking towards the robot position).

• $CuriousCuriosity$ : When a robot is detected, the human abandons$the$$current$$navigation$$goal$$.$Then, humans slowly approach the robot. After a short time (30 seconds by default), people lose interest and return to the targeted navigation.

• $Scared\; :\; Whenthe$ agent detects a robot, the agent tries to move away from$it$$.$In this case we add to the agent an extra repulsive force from the robot and reduce the max speed. In this way we simulate the human behavior of fear.

• $Threatening$ threat: When the agent sees the robot, it approaches the robot and tries to block the robot's path by walking in front of it$.$The goals in front of the robot are constantly calculated and commanded. After some time (40 seconds by default), the agent gets bored and continues with its original navigation.

An interesting feature is that all these behaviors are controlled by behavior trees [11]. They allow switching between different tasks or actions to efficiently build autonomous agents. Additionally, they are easy to program and new behaviors can be easily added or modified. In particular, we use the engine BehaviorTree.CPP8 for behavior creation/editing.

Wrapper for Gazebo Simulator

Wrappers for using the HuNavSim and Gazebo simulators are also provided. Figure 3 shows the wrapper module (red square) and the communication with the Gazebo simulator and HuNavSim.

Control of agent movement is programmed into the Gazebo world plugin and must be included in the Gazebo world file. Additionally, human agents are also included in the same file as Gazebo actors.

Therefore, we wrote two modules:
• A world generator module, responsible for reading the human agent's configuration parameters (defined in the agent's configuration yaml file) and writing the appropriate plugins and agent bower world files in the base.
• Gazebo then loads this world file and the HuNav plugin communicates with HuNavSim to update the agent's state during simulation execution.

The wrapper contains a set of typical scenes shared with humans: cafeteria, warehouse, and house.

In addition, we have developed a new set of different 3D human body models with different animations based on individual behavior. Users can select a 3D model of the human body and associated behaviors.

Navigation Evaluation

To address the problem of choosing the best metric to evaluate human perceptual navigation, we decided to keep the evaluation system as open as possible.

First, we reviewed the literature to collect most of the metrics applicable to this problem. We then let the user choose which metrics to calculate for each simulation. Finally, the system also allows new indicators to be easily added to the assessment. The inputs to all metric functions are two arrays containing the pose, velocity and other data of the HuNavSim agent and robot at each time step.

In addition to computing metrics for all agents in the scene, evaluators perform computations about specific behaviors of agents. In this way, we can obtain interesting indicators about the situations in which the agent exhibits a certain behavior in response to the presence of the bot.

Thus, HuNavSim provides a reliable and flexible evaluation system compared to the fixed evaluation systems in the literature.

Metrics description

The implemented metrics are the ones used in our previous work [23], the SEAN simulator [7], some SocNavBench [13] and some metrics related to the social force model (SFM) used According to the compilation by Gao et al., in in different works [26]. This provides a total of 28 indicators, the most extensive compilation of indicators to our knowledge. A list of available metrics is shown in Table I. A detailed description of the indicator and its mathematical equation can be found on the Github page of HuNavEvaluator9

Output metrics data

As output, HuNavEvaluator will generate a set of metrics files. They are formed as text files, beginning with a header indicating the name of the metric, followed by the data organized in rows and columns divided into tables:

• A general metric file, which contains the calculated final metric values.

• A list of metrics calculated for each time step in the simulation. This is useful for plotting and visualizing the behavior of certain metrics over simulated time or at specific moments.

• A file set that contains the final metrics, but only considers groups of people with the same behavior. Therefore, we will get as many files as there are specific behaviors used in the simulation.

• Finally, the tool will generate metrics calculated at each timestamp for each previous behavior group.

Valid Experiments

Scenarios

According to the research by Gao et al. [26], commonly used scenarios for evaluating human perceptual navigation are: Passing, Crossing, Overtaking, Approching, Folling/Accompany (overtaking, crossing, overtaking, approaching, following/accompanying) and combinations of these scenarios.

In this evaluation, we employ basic scenarios of overtaking and crossing, as well as combined scenarios in which humans exhibit different individual behaviors. In Figure 4, we can see the Passing scenario on the left, where the robot has to reach the other side of the room, and there are two people walking in opposite directions. In the intersection scenario, as shown in the central image of Figure 4, the robot has to go to the other side of the room again when intersecting four people: two of them walk perpendicular to the robot direction, and the other two walk diagonally. Finally, the image on the right presents a combined scenario in which three people exhibit different responses to the robot's presence: "regular" navigation, which sees the robot as another person; the "curious" response of a human agent cautiously approaching the robot; and the "threat ” response, in which the human agent would persist in trying to prevent the passage of the robot.

Planners

In the previous social navigation scenario, we will compare the metrics obtained by three planners with different characteristics:

• DWBs. The basic ROS 2 navigation stack using the DWB local planner is mainly based on the dynamic window approach [27].

• SCLs. The same system using the social cost layer (a custom ROS 2 version of the social navigation layer implemented in ROS 1).

• SFW. A social local planner we develop based on a dynamic window approach. It uses a social force model as a human pose predictor and a trajectory scoring function that incorporates social work.

Results

Conclusion

We have launched a new open source software to simulate human navigation behavior, HuNavSim. The tool is programmed under the new ROS 2 framework and can be used to control human agents in different generic robot simulators. A wrapper for using this tool with the famous Gazebo Simulator is also presented.

Furthermore, it presents other novelties such as a set of realistic individual human navigation behaviors guided by behavior trees. These behaviors can be easily configured and extended through behavior trees. The tool also includes a complete, flexible and scalable compilation of evaluation metrics for robotic human perception navigation from the literature. A comparison of three different planners shows that this tool is useful for the development and evaluation of navigation algorithms. Different responses to the robot's presence also bring a novelty that helps simulate more realistic navigation situations in spaces shared with humans.

Future work includes completing the full set of indicators reviewed. Other metrics from the work of Crowdbot [6] and Gau et al. [26] will be investigated and included. Wrappers for other simulators such as Webots or Isaac Sim are currently under consideration. In addition, augmentation of the set of personal navigation behaviors and the addition of other human navigation models in addition to the social force model will be investigated.

Finally, on a technical level, we will be working on updating the software to use the ROS2 Humble distribution (currently in development) and the latest version of the Behavior Tree library (v4)