【Record】Environment｜A complete record of building a Python development and debugging environment in Ubuntu 18.04

All the following steps, personally test, and simply compare the advantages and disadvantages of different solutions, so that there is room for choice in different situations. It mainly includes switching Python version and environment, and VScode/pdb debugging method.

Install Python and switch

reference:

1. [Beginner Linux] How to install and upgrade the python version in ubuntu 18.04. (Pit Climbing Diary) Time 2021/11/10-Engineer-Stephen_Lin-CSDN Blog
2. Relative imports in Python 3-StackFlow
3. 安装python-ldap fatal error: lber.h: No such file or directory
4. Upgrade Python version under Ubuntu18.04 LTS

1 Install a version

Method 1: pyenv installation (strongly recommended)

installation method:

1. Download dependencies:

sudo apt install -y libffi-dev liblzma-dev make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev 

2. Clone pyenv:

git clone git://github.com/yyuu/pyenv.git ~/.pyenv

3. Modify ~/.bashrc, add the following content, save and restart the terminal:

export PYENV_ROOT="${
     
     HOME}/.pyenv"
 
if [ -d "${PYENV_ROOT}" ]; then
  export PATH="${PYENV_ROOT}/bin:${
     
     PATH}"
  eval "$(pyenv init -)"
fi

How to use:

1. Check the version supported by pyenv:

pyenv install --list

2. Install python3.9.12:

pyenv install 3.9.12

3. Check the installed python version:

pyenv versions

Method 2: apt installation (not recommended)

Before installation, it is best not to change the source.

Python3

Python3 takes 3.7 as an example:

It is best to use 3.6 on Ubuntu18, 3.8 on Ubuntu20, and 3.10 on Ubuntu22, which are included with the system release version.

sudo apt install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.7
sudo apt install python3.7-distutils

If you don't install distutils, you will get an error ImportError: cannot import name 'sysconfig' from 'distutils' (/usr/lib/python3.9/distutils/__init__.py).

Note, if you are not using bionic source, but Ubuntu18, then you may not be able to locate the python3.6 package. Because the ppa:deadsnakes/ppa source says that the release version already provides this python, so it does not provide it:

At this point, you also need to run the following command:

1. sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32
2. echo "deb http://archive.ubuntu.com/ubuntu/ bionic main restricted" >> /etc/apt/sources.list
3. sudo apt update

If the original python3 is deleted in advance, software-properties-commonthe following error occurs during installation:

The following packages have unmet dependencies:
 python3 : Depends: libpython3-stdlib (= 3.8.2-0ubuntu2) but 3.10.6-1~22.04 is to be installed
 python3-software-properties : Depends: python3-gi but it is not installable
 software-properties-common : Depends: packagekit but it is not installable
                              Depends: python3-gi but it is not installable
E: Unable to correct problems, you have held broken packages.

Then run sudo apt-get autoremove, you can solve this problem.
But there will be new problems, so don't just delete the original Python3 before installing the new one ! ! !

Python2

Python2 takes python2.7 as an example:

sudo apt install python2
sudo apt-get install -y libsasl2-dev libldap2-dev libssl-dev

If you don't install libsasl2-dev libldap2-dev libssl-dev you will get an error python-ldap fatal error: lber.h: No such file or directory.

View all versions installed by apt

ls /usr/bin | grep python

2 switch python version

Method 1: pyenv (strongly recommended)

How to use:

1. Check the installed python version:

pyenv versions

2. Switch version locally (recommended):

pyenv local 3.9.12

3. Toggle version globally (not recommended):

pyenv global 3.9.12

Method 2: Change the link (not recommended)

Replace the default python3 version with 3.6

sudo ln -s /usr/bin/python3.6 /usr/local/bin/python3

Be careful not to change it /usr/bin/python3, because this is related to system settings and is prone to errors.
Note: After deleting the soft link, it will take effect after restarting the terminal.

Method 3: update-alternatives (not recommended)

Add candidates for python version switching (the larger the number, the greater the priority, and you can also use python3.x as a candidate for python):

sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 3

Add candidates for python3 version switching:

sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1

Remove python3 version candidate python3.6:

sudo update-alternatives --remove python3 /usr/bin/python3.6

Switch python/python3 version:

sudo update-alternatives --config python
sudo update-alternatives --config python3

As shown below:

Enter the number to select.

After switching, reinstall pip, otherwise pip may be abnormal.

python3 -m pip install --upgrade pip

Note: After switching, some behaviors of the system may be abnormal, just switch back.

Create a Python virtual environment

Personally, I think: plan one for daily learning, plan two for development projects, direct plan three for machine learning, and direct plan four for a little obsessive-compulsive disorder in the environment.

Option 1: virtualenv+virtualenvwrapper (recommended)

reference:

1. Install Python virtual environment on Ubuntu 18.04 - JacobHou - 博客园
2. Ubuntu18 installs python virtual environment|lsvirtualenv use-wangju008-博客园
3. virtualenv 1.7.1.2.post1 documentation

Advantages of this solution: it is convenient to switch between multiple python environments, and it is easy to use; it is suitable for Python2 and Python3.

Install virtualenv and virtualenvwrapper:

pip3 install virtualenv
pip3 install virtualenvwrapper
# pip install virtualenv
# pip install virtualenvwrapper

Modify ~/.bashrcthe file and add at the end of the file:

export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source $HOME/.local/bin/virtualenvwrapper.sh # 改成自己的目录

Enable profile:

source ~/.bashrc

Create a virtual environment (where virtual_namethe name of the virtual environment -pindicates the python environment):

mkvirtualenv -p /usr/bin/python3 <virtual_name>

Other instructions:

lsvirtualenv #查看虚拟环境
workon <virtual_name> #切换到某个虚拟环境
deactivate #退出某个虚拟环境
rmvirtualenv <virtual_name> #删除某个虚拟环境
cdvirtualenv #进入到虚拟环境所在的目录

Option 2: venv (obvious advantages and disadvantages)

reference:

1. venv - Liao Xuefeng's official website
2. 12. Virtual environments and packages — Python 3.10.8 documentation

Advantages of this solution: very suitable for independent development of a single project , no installation required, easy to use, and lightweight.
Disadvantages of this solution: It is not convenient to switch between multiple python environments; it is not applicable to Python2, and only applicable to Python3.3 and later versions.

Create an empty directory and enter:

mkdir testenv
cd testenv

Initialize the venv environment (venv usually installs the latest version of Python available to you):

python3 -m venv .

Then a python environment will be generated under this folder.

Activate the environment:

cd bin
source activate

quit:

deactivate

Option 3: Anaconda (not recommended)

Advantages of this solution: suitable for machine learning, pre-installed with a series of commonly used packages such as panda/numpy; with conda, it is easier to manage in detail; it is convenient for switching between multiple Python environments.
Disadvantages of this program: large; difficult to install; complicated to use.

The specific steps are written in [Installation] Installing Anaconda and pytorch under Windows, and modifying the default installation path of pip-shandianchengzi-CSDN blog .

Solution 4: pipenv (not recommended)

Reference: Pipenv Getting Started Tutorial - I'm George - CSDN Blog

Advantages of this solution: additionally integrates the function of pip.
Disadvantages of this solution: This function is useless to the author.

For specific operations, please refer to the reference link or the official introduction. This is only a brief introduction, and no specific instructions for use.

Python debugging method

Personally, it is recommended to be familiar with VScode debugging, and use pdb to assist when analyzing the lower-level content and principles.

Option 1: VScode (recommended)

advantage:

1. It has a graphical interface, is easy to use, and is suitable for most situations.
2. It's easy to hit conditional breakpoints! (If you don’t know what a conditional breakpoint is, my suggestion is that you don’t need to know it, you will fall in love with it once you use it )

Disadvantage: It requires a certain configuration process, not all programs know how to configure it to make it work properly.

Note: Download from the official website! !
Note: Download from the official website! !
Note: Download from the official website! !
Otherwise, the Chinese input method may not be supported.

Switch Python version in VScode (when running and debugging)

Ctrl+Shift+P, select the configuration item, and enter interpreterto select the Python interpreter.

As shown below:

After modification, the interpreter will be automatically adopted when running and debugging.

Debug local modules and third-party libraries

For launch.jsonthe configuration of the file, please refer to the document-Launch configurations on the VScode official website .

Question 1: vscode python debugging flashback

Reference: vscode python debugging flashback

Solution: Go to the official website to install VScode (the castrated version installed in the software store cannot support Chinese input), and downgrade the Python plugin of VScode.

Question 2: Local module debugging method (cannot find the module)

Reference: python [No module named] can't find the module I wrote 3 situations and solutions - Marilyn Chrysanthemum - CSDN Blog

1) How to run the local module

1. When you need to run the module directly in other directories
   For local modules, when you need to run the module directly in other directories, you can add the upper-level directory of the directory where the module is located to the module search directory of the current Python environment:

cd <your_python_path>/
cd lib/python3.6/site-packages #根据自己的版本
sudo echo "模块所在目录的上一级目录" > 随便命名.pth

For example, if __init__.pythe file is /home/s/mycode/mymodule/onemodule/in the directory, then the "upper-level directory of the directory where the module is located" is /home/s/mycode/mymodule/.

After the addition is complete, you can execute python -m onemodule.testand run the module. If you still have other questions, please refer to the reference links in this section .

2. When you don't need to run the module directly in other directories, directly python -m onemodule.test.

2) Debug it in VScode

Create a new file and import this module to breakpoint debugging, as follows:

import onemodule.test

onemodule.test.main() #在此行打断点

Question 3: Debug and enter the code of the third-party library

Reference: vscode breakpoint debugging python third-party package - big imperial envoy - Nuggets

Since the debugger does not enter third-party code by default, it is necessary to modify launch.jsonthe file corresponding to the debugging option.
As shown in the figure below, click Add Configuration to enter launch.jsonthe file:

As shown below, justMyCodechange the options falseto:

Question 4: Passing command line parameters in VScode

As above, click to enter the configuration launch.jsonfile, as shown in the figure below, add the args parameter:

Solution 2: pdb (not recommended)

Reference: How to dynamically debug Python's third-party library - ybdesire-CSDN Blog

Advantages: no configuration is required at all; modules/third-party libraries can be directly debugged; for python programs that already know how to run them on the command line but do not know how to run them in VScode, you may wish to directly debug them pdb.
Disadvantages: No convenient graphical interface.

When you need to debug, just insert the following code before the debugged code:

import pdb
pdb.set_trace()

Add one more command line pdb, such as executing: python -m pdb onemodule.pdbtest, to debug.

Similar to gdb debugging, pdbthe code printed on the terminal is about to be executed rather than executed.

Common commands:

1. The n command (next) allows the code to run in a single step; the s command (single step into, fine running), this command will enter the method.
2. <variable_name>.d (data), check the value of the intermediate variable variable_name. (Note: For most types, just enter the variable name directly)
3. b <line>, breakpoint at the first line; b command, view all breakpoints.
4. c command (continue), let the code run directly.
5. clear <breakpoint_order>, clear the first breakpoint_order breakpoint.
6. l command to view multiple lines of currently running code.
7. Any python code can be directly input, such as print("1"), very convenient.
8. q command, quit.