1. Environmental prerequisites: install and configure python3, configure pip3, and sqlite3 can be used.
You can check centos7 to install python3: https://my.oschina.net/u/3223370/blog/1621511
2. Install scrapy using pip3
pip3 install scrapy
3. Configure scrapy to be accessible globally and delete the existing scrapy (if any)
rm -rf /usr/bin/scrapy
ln /usr/local/python3/bin/scrapy /usr/bin/scrapy
4. Create a scrapy project, test directory: /data/www/
cd / data / www /
scrapy startproject quote
5. Create a spider
cd /data/www/quote
scrapy genspider quotes quotes.toscrape.com
6. Define item
cd /data/www/quote/quote
vim items.py
Save and exit!
7. Write spiders
cd /data/www/quote/quote/spiders
vim quotes.py
Save and exit!
7. Run scrapy
cd /data/www/quote
scrapy crawl quotes -o /data/www/quote/quotes.json
8. At this point, the initial use is completed!
