log log scrapy framework

debug information in scrapy

Set in scrapy log in

1, set the log level in settings, add a line in settings.py in:

Scrapy offer 5 layer logging level: 

CRITICAL - a critical error (Critical) 

ERROR - General error (Regular errors) 

the WARNING - warning messages (warning messages) 

INFO - General information (informational will messages) 

DEBUG - debugging (debugging messages)

scrapy default log information display DEBUG level

2, the result is saved as log log output, and adds the path in settings.py:

LOG_FILE = './log.log'

3, the display position of the log, in the pipelines.py:

import logging

logger = logging.getLogger(__name__)

def process_item(self, item, spider):
    logger.warning(item)
    ....

4. In spiderintroducing Log log file:

class DcdappSpider (scrapy.Spider): 
    name = ' dcdapp ' 
    allowed_domains = [ ' m.dcdapp.com ' ] 
    custom_settings = {
         # Set conduit download 
        ' ITEM_PIPELINES ' : {
             ' autospider.pipelines.DcdAppPipeline ' : 300 , 
        }, 
        # Set log log 
        ' , LOG_LEVEL, ' : ' the DEBUG ' ,
         ' LOG_FILE ' : './././Log/dcdapp_log.log'
    }