[Ubuntu/linux server and local file transfer, Screen remote login, common commands in conda environment]

ubuntu/linux server and local file transfer, Screen remote login, common commands in conda environment

1. View GPU instructions

nvidia-smi   ## 查看GPU整体情况
nvidia-smi -l  ## 查看几个GPU在运行
top             ## 查看gpu使用占比

2. Create a new conda environment on the server

• create new environment

  conda create -n 文件夹 python=3.7（指定python的版本）
  

• Activate the created new environment

  conda activate 文件夹 
  

• Delete the new environment created

  conda remove -n 文件夹 --all
  

• Enter the env_mcl7 environment

  source activate env_mcl7
  

3. Conda common commands

• View virtual environment

  conda env list
  conda info --envs
  

• Exit the virtual environment

  conda deactivate
  

• increase source

  conda config --add channels 源的地址
  

• delete source

  conda config --remove channels 源的地址
  

• view source

  conda config --show channels
  

• Add Tsinghua source

  conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
  conda config --set show_channel_urls yes
  

• watercress source

  pip install -r requirement.txt -i https://pypi.douban.com/simple
  

4. Local and server interaction

• ## 将本地文件上传至服务器 （需在本地环境中，且目录是该文件的上级目录，否则需要带路径）
  scp 文件名 [email protected]:/data3/private/mcl
  
  

• ## 将本地文件夹上传至服务器
  scp -r 文件夹名 [email protected]:/data3/private/mcl
  

• ## 将服务器中的文件下载至本地
  scp [email protected]:/data3/private/mcl/test.txt /Users/clmeng/realmadrid
  

• ## 将服务器中的文件夹下载至本地
  scp -r [email protected]:/data3/private/mcl/test /Users/clmeng/realmadrid
  

• ## 显示光标
  echo -e "\033[?25h"
  
  ## 更改sh为shell,解决inux下shell中使用上下键翻出历史命名时出现^[[A^[[A^[[A^[[B^[[B的问题
  chsh -s /bin/bash username  或者使用 usermod -s /bin/bash username
  

5. Common operations

• ## 删除文件夹（-i代表询问是否确定删除该文件夹）
  rm -rf -i test111/
  
  #删除文件（多个）
  rm -i 文件1 文件2
  
  #删除以x开头的文件
  rm -rf -i x*
  
  #删除.log类型的文件
  rm -rf -i *.log
  

•     ## 在linux命令行（终端）不依赖jupyter执行ipynb 文件
                
  pip install runipy
  runipy <YourNotebookName>.ipynb
  

• Package the versions of all functions and libraries in the conda environment server as txt

  pip freeze > requirements.txt
  

• View the version of a package, library, or function

  pip list | grep -i xxx
  

  [External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-MuFtSES4-1681121775380)(/Users/clmeng/Downloads/1_A package/6_Daily illustrations/Screenshot 2022-04-01 pm 7.50.08.png)]

6. Screen remote login

• background

  System administrators often need SSH or telent to log in to Linux servers remotely, and often run some tasks that take a long time to complete, such as system backup, ftp transfer, and so on. Usually we open a remote terminal window for each of these tasks because they take too long to execute. You must wait for them to finish executing, and you cannot close the window or disconnect during this period, otherwise the task will be killed and everything will be abandoned halfway.
  The screen command can solve this problem very well. Realize that in the case of disconnecting SSH, the program can continue to be executed on the server.

• definition

  Screen is called a full-screen window manager through which users can simultaneously connect to multiple local or remote command line sessions and switch between them freely.
  Note that there are two concepts of session and window. After entering the screen session, multiple windows can be created in the session (window)

• Related concepts

  • Default window
    The so-called default window is the clean window when we do not use the screen command, and we cannot use any shortcut keys under the screen command that will be introduced later.
  • Screen session
    A screen session is an interface that occupies a process pid when we enter the screen space. This interface has its own name, its own shortcut keys for internal interaction, and can have many different sub-terminal interfaces. It can also be understood as a window independent of the default window. Note that each screen session can have many sub-windows, and our computer can have many different screen sessions at the same time.
  • Sub-window window
    Each sub-window is attached to a screen session. They do not affect each other and can execute different commands separately. This is the essence of the entire screen command. To manage the window, the management command starts with ctrl + a. When working, all we need to do is to select a certain screen session and select a sub-window. In this way, if we need to change an interface, we only need to change another sub-window.

• three characteristics

  • Session Recovery
    Sessions running inside the Screen can be recovered as long as the Screen itself is not terminated. This is especially useful for users logged in remotely - even if the network connection is interrupted, the user will not lose control of the command line session they have opened. Just log in to the host again and execute screen -r to resume the session. Similarly, when you leave temporarily, you can also execute the detach command detach to suspend the Screen (switch to the background) while ensuring the normal operation of the programs inside.
  • Multi-window
    In the Screen environment, all sessions run independently and have their own numbers, input, output, and window buffers. Users can switch between different windows through shortcut keys, and can freely redirect the input and output of each window. Screen implements basic text operations, such as copy and paste, etc.; it also provides a function similar to a scroll bar, allowing you to view the history of the window status. Windows can also be partitioned and named, and background window activity can be monitored.
  • Session sharing
    Screen allows one or more users to log in to a session multiple times from different terminals and share all the features of the session (for example, they can see exactly the same output). It also provides a window access mechanism, which can password-protect the window.

• Advantage

  Create a new session, and then carry out training in the session. Even if the network is disconnected first, the model in the server will continue to run.

• common command

  screen -S  new_name  ## 新建会话 
  
  ## 退出当前会话
  screen窗口键入C-a d，Screen会给出dettached提示
  exit
  
  ## 远程dettach某个会话 （dettach 表示中断）
  screen -d session_name #session_name也可以是对应的session id
  
  ## 进入dettached的会话
  screen -r session_name #session_name也可以是对应的session id
  
  ## 进入attached的会话
  ## 如果一个会话的状态是Attached的，而你并不在其中，我们就要输入　
  screen -d <screen的pid> ，来使他”Dettached”，
  然后再输入 screen -r <screen的pid>，来进入这个screen。
  
  ## 显示当前所属会话id.name
  echo $STY
  
  ## 列出当前所有的session
  screen -ls
  
  ## 关闭会话
  ## 保持会话退出，这样退出的话，以后还可以通过screen -r （name）恢复会话,快捷命令
  ctrl A D
  exit
  
  ## 删除会话
  screen -X -S session_name quit
  
  ## 清理会话
  screen -wipe #清理那些dead的会话