spyder uses remote python kernel

Although spyder is usually very convenient to use locally, occasionally I still want to use the code and environment on the remote linux server to run remotely.

 

1. The spyder service needs to be started on the remote server

Start spyder core

root@AI# python -m spyder_kernels.console
NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

To read more about this, see https://github.com/ipython/ipython/issues/2049


To connect another client to this kernel, use:
    --existing kernel-26032.json

A json file will be generated. The location of the specific file is not under the path where this command is executed. You need to use this command to find the path:

root@AI# jupyter --runtime-dir
/run/user/0/jupyter

Next, find the json file (kernel-26032.json) in this path, and copy the json file to Windows

Then you can use the spyder on the win side to connect

 

2. Connect spyder

Find this entry link on the lower right side of spyder

 A window will pop up next

Use format of host: root@ip:port number

Don't care about ssh key, fill in the content and click ok to connect, if the following error occurs

 

Then you need to install it on win

pip install Paramiko

Then you can connect again

Note that after installing Paramiko, spyder needs to be restarted to take effect, otherwise the error is likely to be the same

3. Use

The connected interface is equivalent to the command line form of python

For example, try: 

Found that there is no problem

The above is just a command line form, but many times we still want to run a python file more convenient

Note that if there is no way to run the python file on Windows directly, the path will not be found, because after all, our python core is remote, and we did not remotely submit tasks to the server.

Next we run the linux remote file

 

code show as below

import subprocess
top_info = subprocess.Popen(["/data/opt/AAN/bin/python3", "/data/opt/test.py"], stdout=subprocess.PIPE)

top_info.communicate()

This test.py is a linux remote file with the following content:

print(1)
print(1)
print(1)
print(1)

We can see that the result of the previous operation is four 1

 

But the result is still unsatisfactory, because in this case, it is no different from running remotely on xshell, so why use spyder? 

The following can be operated on win:

# -*- coding: utf-8 -*-

print(1)
import torch
print(torch.__version__)
# %%

Add # %% in the last line of code

Make it a cell

Then just do this or press Ctrl+Enter on the keyboard.

operation result:

In this way, the local python file can be run directly

However, there is still a problem, because the data has not been read yet, this mode of operation is equivalent to just sending the code to the console

So if we read the data of the local file like this, there will be

# -*- coding: utf-8 -*-

import pandas as  pd
df=pd.read_csv('C:\\sample-orders.csv')
print(df)
# %%

 

You need to modify the path to the data path on the linux server, that is, the data needs to be uploaded to it:

This will do:

# -*- coding: utf-8 -*-

import pandas as  pd
df=pd.read_csv('/data/opt/sample-orders.csv')
print(df)
# %%

operation result:

However, there will be new problems, such as an error when calling a function of another file, because the cell is equivalent to just sending the code in the cell without knowing where the dependent file is, so an error will be reported

Here is to call the function below Utils.py

# -*- coding: utf-8 -*-

from Utils import get_data

df=get_data('/data/opt/sample-orders.csv')
print(df)
# %%

 

 The solution is to simply upload the dependent file (Utils.py here) to Linux and fix it in a certain path. Here I put Utils.py under /data/opt/test.

# -*- coding: utf-8 -*-
import sys
sys.path.append("/data/opt/test")
from Utils import get_data

df=get_data('/data/opt/sample-orders.csv')
print(df)
# %%

 result: