Written 2 years ago, install a V100 server in the laboratory intranet environment from scratch, use FRP and Alibaba Cloud public network EOS to penetrate the intranet, access the laboratory intranet GPU environment anytime and anywhere, and start Jupyter on the server External network access service + ssh mapping, realize remote and synchronous calling of laboratory GPU resources anytime and anywhere, and run code written by local Pycharm. Mac typing code + calling the V100 server to run the model to achieve an efficient and fast meeting environment. At present, it seems that there is still a certain reference value, so turn it out, and students in need can refer to it~
1. nvidia driver installation
1.1 download
nvidia driver download address
Select the driver installation of your corresponding model:
On the server side, you can use the wget command to download:
1.2 Installation
1.2.1 Give the driver script permission to run:
chmod 777 NVIDIA-Linux-x86_64-440.44.run
777Followed by your own driver script name
1.2.2 Run the installation script:
1.2.3 Installation error:
1.2.4 Enter text mode:
init 3
1.2.5 Install again:
./NVIDIA-Linux-x86_64-440.44.run
1.2.6 Installation complete
1.2.7 Check driver information
nvidia-smi
2. Anaconda installation
2.1 Download the installation script:
2.2 Give the script permission to run
chmod 777 Anaconda3-2019.10-Linux-x86_64.sh
2.3 Run the installation script
./Anaconda3-2019.10-Linux-x86_64.sh
2.4 Crazy press Enter, enteryes
2.5 Set the installation path:
/root/ide/anaconda3
Modify according to my own preferences, I installed it under /root/ide/anaconda3the path
2.6 Whether to write environment variables
yes
2.7 Installation complete
2.8 Close the console and reopen it:
If it appears before the command line base, it means that the environment is already anacondain usepython
2.9 Test condaavailability:
conda --version
3. jupyter setup
3.1 Generate a configuration file
jupyter notebook --generate-config
3.2 Modify the jupyter configuration file
cat jupyter_notebook_config.py
c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = u'sha1:09a6121216a3:146ccafcfbb6288143cf4fba2ef7e29ff79d7ccc'
c.NotebookApp.notebook_dir = '/root/project/'
c.NotebookApp.port = 9999
3.3 Write jupyterbackground running startup script
cat jupyter
#!/bin/sh
nohup sudo /root/ide/anaconda3/bin/jupyter-notebook --ip="0.0.0.0" --config /root/ide/sh/jupyter_config.py --allow-root &
3.4 jupyter LAN access
At this point, you can access jupyter on the server through the internal and external ip and the custom 9999 port, but if we want to directly access the server on the company's intranet at home, we also need to perform the fourth operation below to complete frp penetration.
4. FRP intranet penetration
If the company uses a host internally, it can access the external network, but there is no public network ip. If you can access it at home, buy a host with a public network ip, such as Alibaba Cloud 9.9/month, and then use frp to penetrate the internal network. Realize public network access.
4.1 Download frp to client and server
Note that the client and server use the same version of the compressed package to avoid various version errors.
wget https://github.com/fatedier/frp/releases/download/v0.29.0/frp_0.29.0_linux_amd64.tar.gz
4.2 Decompression Toolkit
tar -zxvf frp_0.29.0_linux_amd64.tar.gz
4.3 Server configuration:
We need to configure two frp penetrations, one for ssh access and one for jupyter access, so here I define two configuration files for frps.inissh access and frps_jupyterjupyter access.
vim ./frps.ini
[common]
bind_port = 20014 #与客户端绑定的进行通信的端口
vhost_http_port = 20013 #:访问客户端web服务自定义的端口号
vim ./frps_jupyter.ini
[common]
bind_port = 20009 #与客户端绑定的进行通信的端口
vhost_http_port = 20008 #:访问客户端web服务自定义的端口号
4.3.1 Simple test of server-side frp
The operation method of frp is very simple, and the operation can be completed by the following command:
./frps -c ./frps.ini, which is -cfollowed by the name of the configuration file.
It works fine without error.
4.4 Client configuration:
Similarly, the client also needs to define two configuration files, one for ssh access and one for jupyter access.
vim ./frpc.ini
[common]
server_addr = 39.108.65.236 #公网服务器ip
server_port = 20014 #与服务端bind_port一致
#公网通过ssh访问内部服务器
[ssh]
type = tcp #连接协议
local_ip = 127.0.0.1 #内网服务器ip
local_port = 22 #ssh默认端口号
remote_port = 20015 #自定义的访问内部ssh端口号:
vim ./frpc_jupyter.ini
[common]
server_addr = 39.108.65.236
server_port = 20009
[ssh]
type = tcp
local_ip = 127.0.0.1
local_port = 9999
remote_port = 20010
4.4.1 Client frp simple test
./frpc -c ./frpc.ini, which is -cfollowed by the configuration file name.
4.5 Client and server connections
If you, the server is running ./frps -c ./frps.ini, the client is running ./frpc -c ./frpc.ini.
Note, run the server first
Open the server background:
It is found that the client server will have connection information without reporting an error. At this time, you 39.108.65.236can 20015connect to the server on the intranet through the port through the public network and through ssh.
4.6 Start script writing
Since multiple configuration files are involved, jupyte configuration, startup, frp configuration, startup, for the convenience of later use, once and for all, we script all startup commands. Then configure it to start automatically at boot.
4.6.1 Server-side scripting:
cat run_frp.sh
#!/bin/sh
nohup /root/tool/frp_0.29.0_linux_amd64/frps -c /root/tool/frp_0.29.0_linux_amd64/frps.ini &
nohup /root/tool/frp_0.29.0_linux_amd64/frps -c /root/tool/frp_0.29.0_linux_amd64/frps_jupyter.ini &
The server only does port mapping, so there are only two running in the background frps.
4.6.2 Test the startup of the server script:
./run_frp.sh
4.6.3 Client scripting:
The client involves the background startup of jupyter, and the startup of shh's frpc and jupyter's frpc,
cat run_config_frp_and_jupyter.sh
#!/bin/sh
nohup sudo /root/ide/anaconda3/bin/jupyter-notebook --ip="0.0.0.0" --config /root/ide/sh/jupyter_config.py --allow-root &
nohup sudo /root/ide/download/frp_0.29.0_linux_amd64/frpc -c /root/ide/download/frp_0.29.0_linux_amd64/frpc.ini &
nohup sudo /root/ide/download/frp_0.29.0_linux_amd64/frpc -c /root/ide/download/frp_0.29.0_linux_amd64/frpc_jupyter.ini &
4.6.4 Test the startup of the client script:
4.7 Setting the startup
4.7.1 Server side add:
Write the script program that needs to be started directly into /etc/rc.d/rc.localthe file
sudo sh /root/ide/sh/run_frp.sh
4.7.2 Client add:
Write the script program that needs to be started directly into /etc/rc.d/rc.localthe file
sudo sh /root/ide/sh/run_config_frp_and_jupyter.sh
4.8 Test connection effect
4.8.1 jupyter can be accessed through the public network:
4.8.2 ssh can be accessed through the public network:
5. Coding environment construction
5.1 Create condaa virtual environment
conda create -n tf2 python=3.6
5.2 Activate the environment
The environment will be switched to tf2, and the installation environment will be installed tf2at this time without affecting other environments.
conda activate tf2
5.3 environment insert jupyter
pip install ipykernel
python -m ipykernel install --user --name tf2 --display-name "tf2"
jupyterto switchkernel
5.4 Installation tensorflow-gpuenvironment
conda install tensorflow-gpu
5.5 jupyter uses tf2 environment
tf.__version__
6. pycharm development configuration
6.1 pycharm configures the remote environment
6.2 Add ssh interpreter
The ip is the previously mapped public network ip, and the port is the bound port. For the internal network, it is 22. I use my internal network penetration server here.
6.3 Interpreter path and code synchronization path
The interpreter selected here uses the one defined in the virtual environment, and which python3.6the path can be viewed by
6.4 View the current interpreter
6.5 Root Directory Settings
6.6 Mappings setting
Like, subscribe and pay attention, become handsome and strong without changing bald~