One, install python3
Install dependent files
yum -y groupinstall "Development tools"
yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
Create a folder to store
mkdir /usr/local/python3
Enter the directory
cd /usr/local/python3
Download different versions of python3 according to your needs
wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
Then unzip the compressed package and install Python3
tar -xvJf Python-3.7.3.tar.xz
cd Python-3.7.3
./configure --prefix=/usr/local/python3
make && make install
Finally create a soft link
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
Enter python3 on the command line to test
python3
exit()
Two, install python3-pip
installation
yum -y install python3-pip
Three, install Chrome
Install the latest version
yum install https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
Fourth, install chromedriver
Install the chromedriver corresponding to the chrome version to
view the chrome version
google-chrome --version
Create a directory to store
mkdir /usr/local/chromedriver
Switch to this directory
cd /usr/local/chromedriver
download
wget http://npm.taobao.org/mirrors/chromedriver/88.0.4324.96/chromedriver_linux64.zip
Unzip the package
unzip chromedriver_linux64.zip
If you can’t find the unzip command, download it, and then execute the previous step after the next step
yum -y install zip unzip
Copy the decompressed chromedriver to the /usr/bin/ directory
cp chromedriver /usr/bin/
Five, install the selenium library
pip3 install selenium
Six, install the requests library
pip3 install requests
Seven, debugging script
Add the following options
#! /usr/bin/python3
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless') #无头启动，无窗口加载
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--disable-gpu') #不开启gpu加速
options.add_argument('--hide-scrollbars') #隐藏滚动条, 应对一些特殊页面
options.add_argument('blink-settings=imagesEnabled=false') #不加载图片, 提升速度
driver = webdriver.Chrome(executable_path = "/usr/bin/chromedriver", chrome_options = options)
driver.get("http://www.baidu.com")
print(driver.page_source)
Eight, run the test
chmod +x ./test.py
./test.py
If the script is copied on the Win platform, it may run incorrectly, most likely due to the different carriage return mechanism of Win and Linux
Solution:
Vim editor to open the script
vim ./test.py
Then enter and press enter
:set ff=unix
Save and exit, enter and press Enter
:wq
Run the script again
./test.py