Author: Lin Jiangbin, Yue Peilin, Li Jinyang
Unit: School of Mechanical and Vehicle Engineering, Beijing Institute of Technology
Instructor: Li Zhongxin and Zhu Jie
1. Introduction to the work
1. Design background
This work is based on the application background of many large supermarkets near Beijing Institute of Technology since the outbreak of the epidemic, and combined with the epidemic prevention and control work and customer service, a realistic, flexible, practical, multi-functional and integrated campus welcome service robot was developed. . It mainly has functions such as face image recognition, disinfection, temperature measurement, issuing gift packages to new students, guiding isolation, tracking supermarkets, automatic obstacle avoidance and Bluetooth remote control, etc. It can contribute to the school's orientation work and epidemic prevention and control work. This article mainly introduces the application of ideas and advanced functional technologies to robots through the research and design of robot systems, making robots have high artificial intelligence characteristics.
2. Innovation points of robots
2.1 High functional integration
Most traditional robots have single functions and can basically only realize a single function. The supermarket service robot designed in this work integrates various functions, and functions such as guiding customers, disinfection and temperature measurement, and tracking guidance are all concentrated on one robot, ensuring that the robot It is more efficient and more practical to work in a system that performs a series of supermarket services.
2.2 High degree of artificial intelligence
The current society is still under the epidemic, and epidemic prevention has become the norm. This group of supermarket service robots is fully unmanned, which greatly reduces the risk of infection among supermarket staff and greatly reduces supermarket labor costs. It can grip, telescope, and avoid obstacles at the same time. , tracking, face image recognition and voice broadcast are completed in the specified order.
2.3 All-round motion of three-degree-of-freedom robotic arm
The main body of the robot is mainly composed of a smart car, a robotic arm and a display screen. It is shaped like a humanoid robot. When the car is moving, the robotic arm can also grasp and telescope. Take the robotic arm as an example: Each robotic arm is controlled by three servos and can move in all directions and at multiple angles (one is a series type with three degrees of freedom, and the other is a linkage type with three degrees of freedom).
2.4 Clever combination of remote control and intelligence
Since the sensor sensitivity of the car during tracking is not particularly accurate, in order to prevent the actual walking route from deviating from the designated tracking route, we used a Bluetooth module on the original basic training. When the car deviates from the route, it can be reduced by remote control. Minimize or avoid deviation (treat grayscale sensing as a sub-function of the Bluetooth module), while improving the flexibility of the intelligent robot.
3. Difficulties in the work and solutions
3.1 Searching for creative ideas
Check the market information of relevant service robots, such as "2021 Service Robot Application Prospects", etc., and check the relevant supermarket epidemic prevention situation online, and then contact the shortcomings of supermarket service work under the epidemic, and finally finalize the supermarket service robot based on epidemic prevention.
3.2 The Raspberry Pi identification function is difficult to implement
Due to the limited performance of the Raspberry Pi, the number of frames of the video is very low, only about 5 frames, when only one CPU core is used, and the library functions and other issues are not very effective. By using different XML files at the same time, you can realize the function of identifying different objects at the same time. For example, the following code can identify faces and black mobile phones at the same time. The XML files required to identify mobile phones were produced by Radamés Ajna and Thiago Hersan. The source is here . Furthermore, we can write our own Cascade files according to our own needs. Naotoshi Seo gives a detailed tutorial here. A relatively simple one is the improvement of Thorsten Ball's banana recognition tutorial.
3.3 Finalize the structure of the robotic arm
In order to have a high enough degree of freedom, we discussed and decided to use two different forms of robotic arms: one is a series type with three degrees of freedom; the other is a linkage type with three degrees of freedom.
4. Simulated supermarket environment test
The robot as a whole is divided into the upper part (mainly the robotic arm and display screen), and the debugging process of the lower part (car) is separate debugging, and the tracking, Bluetooth, and ultrasonic modules of the car part are separately tested. The upper part mainly tests whether the robotic arm can complete specified actions in a coherent manner, whether the display screen and camera can complete recognition normally, and whether the voice module and temperature sensor can normally broadcast and measure temperature. The campus environment is scaled down using the North Campus of Beijing Institute of Technology as an example, which greatly restores the complex route environment of the supermarket.
5. Application prospect analysis
The size of my country's service robot market is expected to reach 75.18 billion yuan in 2023, and industry development is also facing many favorable factors. First, the current increase in the elderly population provides development opportunities for service robots in medical care and elderly care; second, service robots are a carrier integrated with multiple technologies such as artificial intelligence and human-computer interaction. With the development of technology, service robots Performance will be improved and user experience will be upgraded, which may further tap market demand; third, a top-down policy system has been formed in China, which is accelerating the development of the industry; fourth, normalized epidemic prevention and control will serve Robots provide development opportunities in epidemic prevention and public services. ----"2021 China Robot Special Forum"
In 2020, my country's service robot market size was 38.38 billion yuan, a year-on-year increase of 37.4%. Service robots are mainly household service, medical service and public service robots. According to statistics from the China Institute of Electronics, service robots account for 47.7% of the market. Medical service robots and public service robots are equally divided, accounting for more than 20% of the market, 28.2% and 24.1% respectively. With the rapid development of 5G, artificial intelligence, Internet of Things and other technologies, it will further promote the technological upgrading and cost reduction of the service robot industry, and help the application fields of the service robot industry continue to expand. The future of the service robot industry is bright. ----"2021-2025 China Service Robot Industry Research Prospects Forecast Report"
6. Conclusion
There are certain limitations in the design materials of this work. It is difficult for the robot to complete the entire process accurately. In the future, more precise materials and parts can be used to make the campus welcome robot more perfect. At the same time, we hope that the campus epidemic prevention and welcome integrated robot can provide Campus orientation and epidemic prevention work have brought help.
2. Technical description
1. Overall structure design of the robot
The main function of supermarket epidemic prevention and control and guidance robots is to ensure supermarket epidemic prevention and control while also being able to lead customers in a friendly manner and guide them to browse the supermarket. In order to realize intelligent monitoring, intelligent services and autonomous movement, a rough model of the robot is designed (as shown in the figure below):
Some improvements are made during the actual assembly process to form the final robot (as shown in the picture below):
The overall design of the robot is a humanoid robot, which consists of an LED display connected to the Raspberry Pi, a camera, a temperature sensor, a grayscale sensor, an ultrasonic sensor, an infrared sensor, a Bluetooth module, a voice module, two robotic arms, a self-made car, and a battery. and other components. Its functional modules include: disinfection temperature measurement module, text face recognition module, voice output module, robotic arm movement module, car movement and other modules.
2. Functional design of the robot
The specific functions of supermarket service robots are as follows (in no particular order):
① The customer’s face will be recognized to determine whether the customer is wearing a mask. The wearer will be allowed to pass. If the customer does not comply, the car will hand over the mask to the customer;
② Carry out disinfection and temperature measurement on customers, those who pass the test will pass, and those with high temperature will be taken to the isolation point;
③ Let visiting customers scan the Beijing Health Treasure QR code;
④ Voice broadcast;
⑤ Guide customers to browse the supermarket;
⑥ The robot car can be controlled by Bluetooth;
⑦ Automatic obstacle avoidance function.
The workflow diagram of the campus welcome robot is as follows:
3. Implementation of relevant functional modules
3.1 Face mask recognition module
The basic parts mainly use the display screen, camera, and Raspberry Pi. First, download the SDK to start the camera to collect and detect faces and specified images, then preprocess the images, and finally extract, match and recognize face image features. , where OpenCV provides a local classification trainer.
Python implements facial image recognition:
Start v412 - compile OpenCV2.4.9 - install Pyqt4 - execute the sudo apt-get install realvnc-vnc-server command - enter the command systemctl enable vncserver-virtuald.service (as shown in the figure below), and then click on the face image to enter It can be executed.
3.2 Disinfection temperature measurement module
It is mainly composed of grayscale sensor, infrared sensor, air pump, temperature sensor, telescopic mechanical arm and other components.
When the infrared sensor detects that someone is approaching, by giving the temperature sensor a value, the critical condition we adopt is 38 degrees Celsius (+-2 degrees Celsius). While measuring the temperature, the control board is connected to the control air pump to spray and disinfect the person. If the temperature is too high, the condition of the grayscale sensor will be triggered, and the high temperature person will be brought to our designated isolation point through black line tracking.
3.3 Voice output module
It is mainly composed of voice module, speaker and other components.
That is, it is driven through the TF card and directly uses the computer to update the contents of the spi flash. Simple sounds can be completed through simple serial port commands (the level can be kept looping). We let the voice module complete simple voice commands such as "The body temperature is abnormal, follow me to the isolation point" and "The body temperature is normal."
3.4 Robotic arm motion module
It is mainly composed of servos and other metal parts. It has two mechanical arms, each of which is equipped with three servos (as shown in the figure below). Functions such as telescoping, swinging, and grabbing are realized through a high degree of freedom. Graphical programming is performed through aurdino, and the steering gear movement angle and sequence are set to complete the tasks together.
3.5 Sports Car Module
It is mainly composed of Bluetooth module, grayscale sensor, ultrasonic sensor, motor and other metal components.
The Bluetooth module controls the movement of the car by controlling the motor. Two motors need to be controlled. The motor includes two parameters: steering (forward rotation, reverse rotation, stop) and speed. After testing, it was found that the range of the step speed that the motor can adjust is between 1ms and 30ms. S represents the setting (S:sta), followed by a 1-digit number as the status expression. 0: stop, 2: reverse (go backward), 1: forward (forward). I want to continue to use s to represent the setting speed (s:speed), but I think that two 'S's are not conducive to distinction, and if there are only 3 states, I can express all states directly with one number, and the range of speed is 1-30 requires two digits, so the number is used to represent the set speed, and it also represents the tenth digit of the speed. This is basically completed. Remote control is carried out through the buttons of the mobile phone. The other two sensors realize the functions of tracking and obstacle avoidance by judging if conditions.
3. Program code
1. Sample program
① Robotic arm program
#include <Servo.h>
Servo servo_pin_11;
Servo servo_pin_4;
Servo servo_pin_7;
Servo servo_pin_12;
Servo servo_pin_3;
Servo servo_pin_8;
void setup()
{
pinMode( 16, INPUT);
servo_pin_11.attach(11);
servo_pin_4.attach(4);
servo_pin_7.attach(7);
servo_pin_12.attach(12);
servo_pin_3.attach(3);
servo_pin_8.attach(8);
}
void loop()
{
if (!( digitalRead(16) ))
{
servo_pin_11.write( 15 );
servo_pin_4.write( 40 );
servo_pin_7.write( 150 );
delay( 1000 );
servo_pin_12.write( 180 );
delay( 1000 );
servo_pin_3.write( 50 );
delay( 1000 );
servo_pin_8.write( 90 );
delay( 500 );
servo_pin_8.write( 50 );
delay( 500 );
servo_pin_8.write( 90 );
delay( 500 );
servo_pin_8.write( 50 );
delay( 500 );
servo_pin_8.write( 90 );
delay( 500 );
servo_pin_8.write( 50 );
delay( 500 );
servo_pin_8.write( 90 );
delay( 500 );
servo_pin_8.write( 50 );
delay( 500 );
servo_pin_8.write( 90 );
delay( 500 );
servo_pin_8.write( 70 );
delay( 500 );
servo_pin_3.write( 0 );
delay( 500 );
servo_pin_12.write( 10 );
delay( 1000 );
servo_pin_11.write( 10 );
servo_pin_4.write( 40 );
servo_pin_7.write( 150 );
servo_pin_3.write( 10 );
delay( 1000 );
servo_pin_7.write( 120 );
delay( 500 );
servo_pin_11.write( 40 );
delay( 1000 );
servo_pin_4.write( 70 );
delay( 1000 );
servo_pin_7.write( 150 );
delay( 500 );
servo_pin_11.write( 20 );
delay( 5000 );
servo_pin_12.write( 120 );
delay( 1000 );
servo_pin_8.write( 10 );
delay( 1000 );
servo_pin_3.write( 50 );
delay( 2000 );
servo_pin_3.write( 0 );
delay( 500 );
servo_pin_8.write( 70 );
delay( 500 );
servo_pin_12.write( 10 );
delay( 3000 );
servo_pin_7.write( 20 );
delay( 500 );
servo_pin_11.write( 40 );
delay( 1000 );
servo_pin_4.write( 30 );
delay( 2000 );
servo_pin_4.write( 70 );
delay( 500 );
servo_pin_7.write( 150 );
delay( 500 );
servo_pin_11.write( 15 );
delay( 3000 );
}
else
{
servo_pin_11.write( 15 );
servo_pin_4.write( 40 );
servo_pin_7.write( 150 );
delay( 1000 );
}
}② Bluetooth serial port module program
#include<Servo.h>
const int x1=16;
const int x2=18;
const int x3=17;
const int x4=14;
const int x5=15;
const int y1=3;//左边第一个轮子插口
const int y2=12;//左边第2个轮子插口
const int y4=4;//右边第一个轮子插口
const int y5=11;//右边第二个轮子插口
const int y7=7;
const int p1=180;
const int p2=90;
const int p3=0;
char q;
Servo a1,a2,a3,a4,a5,a6;
void setup() {
Serial.begin(9600);
a1.attach(y1);
a2.attach(y2);
//a3.attach(y3);
a4.attach(y4);
a5.attach(y5);
//a6.attach(y6);
}
void turnright(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p2);
a5.write(p2);
//a6.write(p2);
}
void turnleft(){
a1.write(p2);
a2.write(p2);
//a3.write(p2);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void turnright2(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p3);
a5.write(p3);
//a6.write(p2);
}
void turnleft2(){
a1.write(p1);
a2.write(p1);
//a3.write(p2);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void forward(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void back(){
a1.write(p1);
a2.write(p1);
//a3.write(p1);
a4.write(p3);
a5.write(p3);
//a6.write(p3);
}
void sstop(){
a1.write(p2);
a2.write(p2);
//a3.write(p2);
a4.write(p2);
a5.write(p2);
//a6.write(p2);
}
void loop() {
while(Serial.available()){
q=Serial.read();
if(q=='1') forward();
if(q=='2') turnleft();
if(q=='3') turnright();
if(q=='4') back();
if(q=='5') sstop();
}
}③ Ultrasonic module program
#include<Servo.h>
const int x1=16;//x1,x2,x3分别是前三个灰度传感器，左中右
const int x2=18;
const int x3=17;
const int x4=18;//超声波输入引脚
const int x5=19;//超声波输出c引脚
const int x6=14;
const int x7=15;
const int y1=3;//y1,y2,y3,y4是六个轮子舵机，12左边，45右边
const int y2=12;
const int y4=4;
const int y5=11;
const int y7=8;
const int y8=7;
const int p1=180;//轮子转速
const int p2=90;
const int p3=0;
const int u=88,c=45;
int rightdistance1,rightdistance2;//超声波距离
int leftdistance1,leftdistance2;
int middledistance1,middledistance2;
Servo a1,a2,a3,a4,a5,a6,a7,a8;//a1-a6车轮，a7是超声波旋转
void setup() {
Serial.begin(9600);
pinMode(x1,INPUT);
//pinMode(x2,INPUT);
pinMode(x3,INPUT);
pinMode(x4,INPUT);
pinMode(x5,OUTPUT);
pinMode(x6,INPUT);
pinMode(x7,OUTPUT);
a1.attach(y1);
a2.attach(y2);
//a3.attach(y3);
a4.attach(y4);
a5.attach(y5);
//a6.attach(y6);
a7.attach(y7);
a8.attach(y8);
}
void turnright(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p2);
a5.write(p2);
//a6.write(p2);
}
void turnleft(){
a1.write(p2);
a2.write(p2);
//a3.write(p2);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void turnright2(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p3);
a5.write(p3);
//a6.write(p2);
}
void turnleft2(){
a1.write(p1);
a2.write(p1);
//a3.write(p2);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void forward(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void back(){
a1.write(p1);
a2.write(p1);
//a3.write(p1);
a4.write(p3);
a5.write(p3);
//a6.write(p3);
}
void stop(){
a1.write(p2);
a2.write(p2);
//a3.write(p2);
a4.write(p2);
a5.write(p2);
//a6.write(p2);
}
int Distance_test1(){
digitalWrite(x5,LOW);
delayMicroseconds(2);
digitalWrite(x5,HIGH);
delayMicroseconds(20);
digitalWrite(x5,LOW);
float distance=pulseIn(x4,HIGH);
distance=distance/58;
return int(distance);
}
int Distance_test2(){
digitalWrite(x7,LOW);
delayMicroseconds(2);
digitalWrite(x7,HIGH);
delayMicroseconds(20);
digitalWrite(x7,LOW);
float distance=pulseIn(x6,HIGH);
distance=distance/58;
return int(distance);
}
void chaoshengbo(){
if (a7.read()!=u){
a7.write(u);
delay(300);}
if (a8.read()!=u){
a8.write(u);
delay(300);}
middledistance1=Distance_test1();
middledistance2=Distance_test2();
if(middledistance1<=25 or middledistance2<=25){
stop();
delay(500);
a7.write(u-c);
a8.write(u+c);
delay(500);
rightdistance1=Distance_test1();
leftdistance1=Distance_test2();
delay(500);
a7.write(u);
a8.write(u);
delay(1000);
a7.write(u+c);
a8.write(u-c);
delay(500);
leftdistance2=Distance_test1();
rightdistance2=Distance_test2();
delay(500);
a7.write(u);
a8.write(u);
delay(1000);
if(middledistance2>25 ){
back();
delay(500);
turnright2();
delay(int(500));
}
else if(middledistance1>25){
back();
delay(500);
turnleft2();
delay(int(500));
}
else{
if (rightdistance2>leftdistance2){
back();
delay(500);
turnright2();
delay(int(500));
}
else if(rightdistance2<leftdistance2){
back();
delay(500);
turnleft2();
delay(int(500));
}
else forward();}
}
else forward();
}
void loop() {
//huiduweiyi();
chaoshengbo();
}④ Temperature sensor program
float temp;
int tempPin = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
temp = analogRead(tempPin);
// read analog volt from sensor and save to variable temp
temp = temp * 0.48828125;
// convert the analog volt to its temperature equivalent
Serial.print("TEMPERATURE = ");
Serial.print(temp); // display temperature value
Serial.println();
delay(1000); // update sensor reading each one second
}⑤ Grayscale detection program
#include<Servo.h>
const int x1=17;//x1,x2,x3分别是左中右的灰度传感器
const int x2=16;
const int x3=14;
const int y1=12;//y1,y2为左边第一第二轮子
const int y2=8;
const int y4=11;//y4,y5为右边第一第二轮子
const int y5=7;
const int p1=180;
const int p2=90;
const int p3=0;
Servo a1,a2,a3,a4,a5,a6;
void setup() {
pinMode(x1,INPUT);
pinMode(x2,INPUT);
pinMode(x3,INPUT);
a1.attach(y1);
a2.attach(y2);
//a3.attach(y3);
a4.attach(y4);
a5.attach(y5);
//a6.attach(y6);
}
void turnright(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p2);
a5.write(p2);
//a6.write(p2);
}
void turnleft(){
a1.write(p2);
a2.write(p2);
//a3.write(p2);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void forward(){
a1.write(p3);
a2.write(p3);
//a3.write(p3);
a4.write(p1);
a5.write(p1);
//a6.write(p1);
}
void back(){
a1.write(p1);
a2.write(p1);
// a3.write(p1);
a4.write(p3);
a5.write(p3);
//a6.write(p3);
}
void loop() {
int b1=digitalRead(x1);
int b2=digitalRead(x2);
int b3=digitalRead(x3);
if(b1==0&&b2==0&&b3==0){back();}
if(b1==0&&b2==0&&b3==1){turnright();}
if(b1==0&&b2==1&&b3==0){forward();}
if(b1==1&&b2==0&&b3==0){turnleft();}
if(b1==0&&b2==1&&b3==1){turnright();}
if(b1==1&&b2==1&&b3==0){turnleft();}
if(b1==1&&b2==1&&b3==1){forward();}
}For details of the program source code and prototype stp diagram, see Supermarket Service Robot