A survival guide for machine learning programmers on the Linux Planet of the Apes

Shanghai Station | GPU CUDA Training for High Performance Computing

April 13-15 640?wx_fmt=jpeg&wxfrom=5&wx_lazy=1 Three days of intensive study to quickly take you to the next level Read more >


The main text contains 3048 words, 8 pictures, and the estimated reading time is 10 minutes.


A small programmer fell into the Linux Planet of the Apes in a flight accident. The difference from the colorful hometown Win Planet of the Apes/Mac Planet of the Apes is that the Linux Planet of the Apes is mostly in the dark [command line/ Shell interface], so most of the time he can only interact with this black world. In order to survive better, he made this basic survival guide.


Scope of the Guidelines


This is not an introduction to Linux terminal commands of a system, but just a few commands that are commonly used on my clipboard, suitable for light Linux users. My use of Linux is mainly to run deep learning models, and to execute some scripts and Hadoop/Spark tasks on online cloud hosts... For more detailed learning materials, please refer to the last reference.


Linux Planet of the Apes was first discovered by a Finn, Linus Torvalds, when he was a student at the University of Helsinki... Since then, a large number of programmers have left their homes to open up wasteland, and now a large number of programmers are still stranded as refugees....


Shell world survival formula


1. Operation folder


pwd //Display the current path

cd folder name //Enter a file, assuming there is a file

Userls //Display the contents of the current directory

ls -a //Display the contents of the current directory, including hidden files [that is, files starting with .

]ls -ahl // Display the contents of the current directory, the file size is displayed in a readable format

du -h // show the size of the current folder


640?wx_fmt=png

ls -aul


640?wx_fmt=png

folder permissions


The permission of a folder is a relatively important concept, which is related to whether you have the right to operate the folder. You can learn commands such as chmod to modify the permissions.


2. Create, copy, move, rename, delete


mkdir newfile: create a new directory

cp source file (source) target file (destination)

cp aa.txt a.txt //Copy a file and rename it to a.txt

cp -r model1 model-3800 //[Folder needs +r parameter] Copy the source folder model1 to the target folder model-3800

mv  source destination

mv aa.txt a.txt //equivalent to renaming the aa.txt document to a.txt


rm file //delete file

rm -r folder //Delete the folder [Use with caution...it's sour with the arrow keys!]

rm *.log //Batch delete


-----In order to solve dangerous commands such as rm -r and other dangerous commands, use history to prevent -------


history //Display the history of commands executed by the terminal

history |grep rm //Prevent deletion by mistake [Lessons of Blood]

history -d 1066 //Manually delete the rm history record to prevent the wrong execution of the rm later... Although it is troublesome, but on the production host, I still have to execute it every time, don't ask me why I am so careful....


3. Inquiry, retrieval, statistics


find . -name "*.py" | wc -l //View the total number of py files under the current file (including subfolders)

grep -o root a.txt | wc -l // Count the number of times the word root appears in the file a.txt

grep root a.txt | wc -l //Count the total number of lines with root appearing


**3. Operate txt and other documents [edit script, configuration file]**

vi a.txt //Create a new a.txt document

head -10 a.txt //Display the first 10 lines of the a.txt file

tail -10 a.txt //Display the last 10 lines of the a.txt file

wc -l a.txt // show how many lines the file has

wc -w a.txt // see how many words are in the file

cat f a.txt | tail -n +3000 | head -n 1000 //that is, display lines 3000~3999 of the file


4. Other commands I commonly use


sh test.sh //Execute the sh script

python test.py //Execute python script

nohup sh test2118.sh> test.log & //Execute the script test.sh in the background and write the output to test.log


Use scripts with:


tail -f test.log //dynamically output the content of test.log to monitor the execution process of the script executed in the background

ps -ef | grep python //View the PID of the python program running on the system, according to the PID we can terminate a script that runs the day after tomorrow

kill -9 9208 //kill the process with PID=9208


640?wx_fmt=png

kill native firefox


ps au// show all running processes:


640?wx_fmt=png

ps to


top //Display a dynamic real-time view of the running system:


640?wx_fmt=png

top command


Extension tool Htop


The command to install htop in linux using pt-get: apt-get install htop
htop is an interactive process viewing tool similar to top, but with vertical and horizontal scrolling to view all processes and their command lines. Process related operations (killing, renicing) do not require PID input.


640?wx_fmt=png

Htop


The use of the advanced text editing tool
Awk/Vi/Vim editor requires you to gradually familiarize yourself with those shortcut commands, and there are still many usage scenarios.


640?wx_fmt=png

Shortcuts for Vi/Vim


How to make vi/vim display the line number when opening a file: [when fine-tuning the code online]
1. Only change the current user's vim
cd ~ //Enter
vim .vimrc in the ~ directory --> add a line of set number --> Esc-->wq, save and exit, you can

add some basic learning materials (ordinary user permissions are enough)
  1. Linux Tutorial for Rookie Tutorial (http://www.runoob.com/linux/linux-tutorial.html)

  2. Linux commands that should never be used ( https://linux.cn/article-9206-1.html)

  1. Linux grep command usage (https://www.cnblogs.com/end/archive/2012/02/21/2360965.html)

  1. The use of Linux awk command -- one of JD's internal push interview questions (http://www.runoob.com/linux/linux-comm-awk.html)

  1. Command 1 of Linux common shell (http://www.360doc.com/content/16/0615/15/7044580_567984505.shtml)

  1. Linux commonly used shell commands 2 (https://www.cnblogs.com/BuildingHome/p/4739268.html)

  1. Linux commonly used shell commands 3 (https://segmentfault.com/a/1190000010849424)

  1. Full stack essential Linux foundation (http://blog.jobbole.com/106827/)

  1. If you are a beginner in machine learning, have you mastered such Linux skills (https://mp.weixin.qq.com/s/tvdOEbS-dBBp5YeQM5w-uA), take a
    good look at the following article, you don’t need to pack up and run after accidentally deleting company data on the way...

  1. I got back the data I accidentally deleted on the Linux system! (http://blog.51cto.com/wzlinux/2052835)


finally


It's best, everyone has the painful experience of executing rm -f folders. The comment area is shaking. Let's have a good time. You are not alone on the way to escape... Or, what are some better and commonly used commands? Let's all learn from each other!


Original link: https://www.jianshu.com/p/fae52827a581


For more concise and convenient classified articles and the latest course and product information, please move to the newly presented "LeadAI Academy Official Website":

www.leadai.org


Please pay attention to the artificial intelligence LeadAI public account to view more professional articles

640?wx_fmt=jpeg

Everyone is watching

640.png?

Application of LSTM model in question answering system

TensorFlow-based neural network solves the problem of user churn overview

The Most Common Algorithm Engineer Interview Questions (1)

Sorting out the most common algorithm engineer interview questions (2)

TensorFlow from 1 to 2 | Chapter 3 The Beginning of the Deep Learning Revolution: Convolutional Neural Networks

Decorators | Advanced Programming in Python

Why don't you review the basics of Python today?

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=324521757&siteId=291194637