Commonly used shell scripts developed by linux

Subsequent additions

• Create a new user in ubuntu and add administrator permissions
  sudo adduser dgliu (dgliu is the new user name I created)
  sudo adduser dgliu sudo Add sudo permissions

---

• My commonly used shell script for linux development

git use

git removes untracked (not in version control) files

• remove untracked files git clean -f

• Even the untracked directory is deleted together git clean -fd

• Even the untrack files/directories of gitignore are deleted together (use with caution, generally this is used to delete files such as compiled .o) git clean -xfd

• Before using the above git clean, it is recommended to add the -n parameter to see which files will be deleted to prevent important files from being deleted by mistake. git clean -nxfd git clean -nf git clean -nfd

• resolve git conflicts

1. git stash    
2. git pull  
3. git stash pop  如果有冲突解决冲突  
4. git push

• git clone --recursive recursively download dependent warehouses

git clone --recursive 用于循环克隆git子项目 
如: git clone --recursive https://github.com/onnx/onnx-tensorrt.git

• git abandons local modifications and forces updates

git fetch --all
git reset --hard origin/master
git fetch 只是下载远程的库的内容，不做任何的合并 git reset 把HEAD指向刚刚下载的最新的版本

• git restore a file to a specific version

git checkout <hash> <filename>
git checkout <filename>

• Delete all local changes that are staged:

git checkout -f

• If you have modified and added to the temporary storage area, then use the following command:

git reset --hard 
git clean -xdf 

• Change the address of Git

git remote set-url origin  http://gitlab.smart-core.cn/liudegui/myinferofflinevideo.git

• View GLIBCXX version strings /usr/lib64/libstdc++.so.6|grep GLIBCXX

• 启动rsync /usr/bin/rsync --daemon --config=/etc/rsyncd.conf

• Check the video memory usage of the Nvidia graphics card in real time. watch -n 5 nvidia-smi Among them, 5 means that the display results of the terminal are refreshed every 5 seconds.

• Linux checks the number of physical CPUs, the number of cores, and the number of logical CPUs. The total number of cores = the number of physical CPUs X the number of cores per physical CPU. number

• Check the number of physical CPUs cat /proc/cpuinfo| grep “physical id”| sort| uniq| wc -l

• View the number of cores in each physical CPU (that is, the number of cores) cat /proc/cpuinfo| grep "cpu cores"| uniq

• View the number of logical CPUs cat /proc/cpuinfo| grep “processor”| wc -l

• When copying files, exclude some unnecessary files ls /src/ |grep -v .gz |xargs -i cp -r /src/{} /dst

• Exclude a directory from searching it: grep -r "test" ./* --exclude-dir=.git

• If you want to search for a specified file type. find . -name “*.c” |xargs grep “test” -n --color

In the Linux environment, when viewing the content of a file, you often need to view the lines before and after the specified keyword. For example, when viewing a log file, if the log file is too large and you want to view it directly in the Linux terminal, you can filter it with grep 'partten' filename , if you want to view the preceding and following lines of the matching line, you can add parameters after grep, as follows:

grep -A 5 'parttern' filename //打印匹配行的后5行
grep -B 5 'parttern' filename //打印匹配行的前5行
grep -C 5 'parttern' filename //打印匹配行的前后5行
grep -5 'parttern' filename //打印匹配行的前后5行 
tail -n 5 filename 查看文件最后5行内容
head -n 5 filename 查看文件前5行内容
sed -n '5,10p' 查看文件5-10行内容 

docker

• docker stop all containers

ps -ef | grep procedure_name | grep -v grep | awk '{print $2}' | xargs kill -9
docker rm -f `docker ps -a | grep -v  CONTAINER | awk '{print $1}'`
docker rmi -f `docker images -a | grep -v REPOSITORY | awk '{print $3}'`

• docker view container IP docker inspect --format='{ {.NetworkSettings.IPAddress}}' 129a18a3810a

导出镜像
docker save -o centos7.tar centos
导入本地镜像
docker load --input centos7.tar

• background launcher

nohup ./testapp >/dev/null 2>&1 &

ffmpeg related commands

• FFMPEG loop streaming command:

    for((;;)); do \
        ./objs/ffmpeg/bin/ffmpeg -re -i ./doc/source.200kbps.768x320.flv \
        -vcodec copy -acodec copy \
        -f flv -y rtmp://192.168.1.170/live/livestream; \
        sleep 1; \
    done

• ffmpeg conversion command

  • mp4转ts： ffmpeg -i input.mp4 output.ts
  • mp4转h264：ffmpeg -i input.mp4 -vcodec copy -an -vbsf h264_mp4toannexb -f h264 output.h264

• Decode debugging related commands

ffprobe -v quiet -show_packets -select_streams v test.ts 2>x|grep pts_time | wc -l
ffprobe -v quiet -show_packets -select_streams v test.ts 2>x|grep pts
ffmpeg -i test.ts
ffplay -stats -f h264 test.h264

• Push command: ffmpeg -re -i ./1080P.h265 -c:a copy -f flv rtmp://192.21.1.61/live/h265 As above, whether to push is h265

• ffmpeg -re -i rtsp://192.21.1.61:554/1080P.h265/playback? -f flv -s 600*400 rtmp://192.21.1.61/live/h265

• ffmpeg -re -i rtsp://192.21.1.235:554/LiveMedia/ch1/Media1 -f flv -an rtmp://192.21.1.61/live/h265 #-an is to remove the audio in your rtsp stream.

• Reduce video playback speed ffmpeg -i output.mp4 -an -filter:v “setpts=10*PTS” output3.mp4

• Convert resolution ffmpeg -i eagle eye camera.mp4 -vf scale=1632:480 output.mp4 -hide_banner

• Convert 0.mp4 in the current directory to 00.mp4, and specify the output frame rate as 15 (-r 15), and the bit rate as 500k (-b 500k)

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i 0.mp4 -c:v h264_nvenc -r 15 -b 500k -y 00.mp4

-hwaccel cuvid: specify the use of cuvid hardware acceleration -c:v h264_cuvid: use h264_cuvid for video decoding -c:v h264_nvenc: use h264_nvenc for video encoding -vf scale_npp=1280:-1: specify the width and height of the output video, note, here It is different from -vf scale=x:x used in soft decoding

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i -c:v h264_nvenc -b:v 2048k -vf scale_npp=1280:-1 -y

• Save rtsp video with ffmpeg ffmpeg -i rtsp://admin:[email protected]:554 -b 300 -s 320x240 -vcodec copy -ab 32 -ar 24000 -acodec aac -strict experimental -f mp4 test.mp4
• FFMPEG loop push ffmpeg -stream_loop -1 -re -i /home/dgliu/vipc_linux_x64/60s_1920_1080.h264 -c copy -f flv rtmp://192.21.1.43/live/livestream
• FFmpeg synthesizes a large number of pictures into video video

1.基本格式
终端输入： ffmpeg -i ./images/image%d.jpg out.mp4
其中./images/images%d.jpg 为图片路径
图片的命名格式为image%d.jpg形式，即：image0 image1 image2 .......
out.mp4为输出视频文件名
2.指定编码格式的使用
终端输入： ffmpeg -i ./images/image%d.jpg  -vcodec libx264  out.mp4
3.指定输出帧率
终端输入：ffmpeg -i ./images/image%d.jpg  -vcodec libx264 -r 10 out.mp4
-r 10 表示定义帧率为10，这样输出的视频就是每秒播放十帧

Testing POST with Curl

• curl -d “leaderboard_id=7778a8143f111272&score=19&app_key=8d49f16fe034b98b&_test_user=test01” “http://172.16.102.208:8089/wiapi/score”

• curl http://127.0.0.1:9002/json -d “{“firstName”: “John”,“lastName”: “Smith”,“age”: 25}”

• curl -L -X POST ‘http://192.21.1.61’ -F ‘uploadFile=@“./README.md”’

• curl -H “Content-Type: application/json” -X POST --data ‘{“id”: “123”, “name”:“test”}’ localhost:8888/user/post

pressure test

• ab stress test

ab -n 10 -c 1 -p get_fea-img_base64_array.json -T application/json http://10.40.27.152:8090/face/get_fea

loop script

while ((1)); do date; echo 'hello'; sleep 1; done

Linux view port occupancy

linux查看端口被哪个进程占用
netstat -tunpl | grep 22
查看那个进程占用了xxx端口
lsof -i:xxx
查看进程号为xxx的进程在哪里
ps -ef|grep xxx

• make -j cpu number

processor_num=`cat /proc/cpuinfo |grep processor |wc -l`
make -j $processor_num

View cpu information View hard disk information More detailed information View network card information

• View cpu information

cat /proc/cpuinfo
cat /proc/meminfo

• View hard disk information df -lh

• For more detailed information cat /proc/scsi/scsi

• View network card information dmesg | grep eth

• More commonly used commands (display system core version number, name, machine type, etc.) uname -a

• The method of creating multiple folders mkdir -p /etc/confd/{conf.d,templates}

vim uses

• vim replacement string: %s/well/good/g (equivalent to: g/well/s//good/g) Replace all wells in each line with good

Statistics total size

du -sh filename (in fact, we often use du -sh * to display all files and their sizes in the current directory, if you want to sort, add | sort -n)

du -sm * | sort -n // count the size of the current directory and sort by size du -sk * | sort -n du -sk * | grep filename // check the size of a folder du -m | cut -d "/ " -f 2 //Look at the text before the second / character