Scrapy novel reptiles day1

Installation Scrapy

pip install scrapy

 

Configuring a virtual environment

mkvirtualenv Spider

 

Create a project

Spider running in a virtual environment in

scrapy startproject BookSpider

  

Project directory tree

E:.
└─BookSpider
    │  scrapy.cfg
    │
    └─BookSpider
        │  items.py
        │  middlewares.py
        │  pipelines.py
        │  settings.py
        │  __init__.py
        │
        ├─spiders
        │  │  __init__.py
        │  │
        │  └─__pycache__
        └─__pycache__

 

 

Create a reptile

Run into the project directory

scrapy genspider biqubao_spider biqubao.com

 

Create a project directory tree of reptiles

E:.
└─BookSpider
    │  scrapy.cfg
    │
    └─BookSpider
        │  items.py
        │  middlewares.py
        │  pipelines.py
        │  settings.py
        │  __init__.py
        │
        ├─spiders
        │  │  biqubao_spider.py
        │  │  __init__.py
        │  │
        │  └─__pycache__
        │          __init__.cpython-36.pyc
        │
        └─__pycache__
                settings.cpython-36.pyc
                __init__.cpython-36.pyc