The logging module in Python for getting started with Python

Basic usage

The following code shows the most basic usage of logging.

import logging
import sys
 
#Get the logger instance, if the parameter is empty, return the root logger 
logger = logging.getLogger( " AppName " )
 
#Specify the logger output format 
formatter = logging.Formatter( ' %(asctime)s %(levelname)-8s: %(message)s ' )
 
#File log 
file_handler = logging.FileHandler( " test.log " )
file_handler.setFormatter(formatter) #The   output format can be specified through setFormatter
 
#Console log 
console_handler = logging.StreamHandler(sys.stdout)
console_handler.formatter = formatter #You   can also assign values to formatter directly
 
#Log handler added for logger 
logger.addHandler(file_handler)
logger.addHandler(console_handler)
 
#Specify the minimum output level of the log, the default is WARN level 
logger.setLevel(logging.INFO)
 
#Output different levels of log 
logger.debug( ' this is debug info ' )
logger.info('this is information')
logger.warn('this is warning message')
logger.error('this is error message')
logger.fatal('this is fatal message, it is same as logger.critical')
logger.critical('this is critical message')
 
# 2016-10-08 21:59:19,493 INFO    : this is information
# 2016-10-08 21:59:19,493 WARNING : this is warning message
# 2016-10-08 21:59:19,493 ERROR   : this is error message
# 2016-10-08 21:59:19,493 CRITICAL: this is fatal message, it is same as logger.critical
# 2016-10-08 21:59:19,493 CRITICAL: this is critical message
 
#Remove some log handlers 
logger.removeHandler(file_handler)

In addition to these basic usages, there are some common tips to share.

formatted output log

#Formatted output 
 
service_name = " Booking " 
logger.error( ' %s service is down! ' % service_name)   #Use python's own string formatting, not recommended 
logger.error( ' %s service is down! ' , service_name)   #Use logger format, recommended 
logger.error( ' %s service is %s! ' , service_name, ' down ' )   #Multiple parameter formatting 
logger.error( ' {} service is {} ' .format( service_name, ' down ' )) #Use the format function, recommended
 
# 2018-03-08 21:59:19,493 ERROR   : Booking service is down!

Record exception information

When you use the logging module to record exception information, you do not need to pass in the exception object, as long as you directly call logger.error() or logger.exception()record the current exception.

#Record exception information
 
try :
     1 / 0
 except :
     # Equivalent to the error level, but additionally record the currently thrown exception stack information 
    logger.exception( ' this is an exception message ' )
 
# 2016-10-08 21:59:19,493 ERROR   : this is an exception message
# Traceback (most recent call last):
#   File "D:/Git/py_labs/demo/use_logging.py", line 45, in 
#     1 / 0
# ZeroDivisionError: integer division or modulo by zero

logging configuration points

GetLogger() method

This is the most basic entry. The parameter of this method can be empty. The default logger name is root. If you always use a logger with the same name in the same program, you will actually get the same instance. Using this technique, you can call it across modules. The same logger to log.

In addition, you can also distinguish different modules of the same program by the log name, such as this example.

logger = logging.getLogger("App.UI")
logger = logging.getLogger("App.Service")

Formatter log format

The Formatter object defines the structure and content of the log information, and requires two parameters when constructing:

One is a formatted template , which will contain the most basic and information fmtby defaultlevelmessage
One is the formatted time style datefmt, which defaults to 2003-07-08 16:49:45,896 (%Y-%m-%d %H:%M:%S)

fmtThe variables allowed in can refer to the table below.

%(name)s Logger's name
%(levelno)s log level in numeric form
%(levelname)s log level in text form
%(pathname)s The full pathname of the module calling the log output function, possibly none
%(filename)s The filename of the module that calls the log output function
%(module)s The name of the module that calls the log output function|
%(funcName)s function name of calling log output function|
%(lineno)d The line of code where the statement that calls the log output function is located
%(created)f The current time, expressed as a floating point number representing the time in the UNIX standard|
%(relativeCreated)d When outputting log information, the number of milliseconds since the Logger was created|
%(asctime)s The current time as a string. The default format is "2003-07-08 16:49:45,896". milliseconds after the comma
%(thread)d thread ID. Maybe not
%(threadName)s thread name. Maybe not
%(process)d Process ID. Maybe not
%(message)s message output by user

SetLevel log level

Logging has the following levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
The default level is WARNING, and the logging module will only output logs above the specified level. The advantage of this is that the log used for debug during project development does not need to be annotated one by one in the product release stage. It is only necessary to adjust the level of the logger, which is very convenient.

Handler log handler

The most commonly used are StreamHandler and FileHandler, Handler is used to log to different output terminals.
Logging contains many handlers, the following may be used

StreamHandler instances send error messages to streams (file-like objects).
FileHandler instances send error messages to disk files.
RotatingFileHandler instances send error messages to disk files, with support for maximum log file sizes and log file rotation.
TimedRotatingFileHandler instances send error messages to disk files, rotating the log file at certain timed intervals.
SocketHandler instances send error messages to TCP/IP sockets.
DatagramHandler instances send error messages to UDP sockets.
SMTPHandler instances send error messages to a designated email address.

Configuration configuration method

There are several ways to configure logging.

Complete configuration through code, refer to the example at the beginning, mainly through the getLogger method.
Simple configuration through code, there are examples below, mainly through the basicConfig method.
Through the configuration file, there are examples below, mainly through logging.config.fileConfig(filepath)

logging.basicConfig

basicConfig()Provides a very convenient way for you to configure the logging module and start using it right away. You can refer to the following examples. For specific items that can be configured, please refer to the official documentation

import logging
 
logging.basicConfig(filename='example.log',level=logging.DEBUG)
logging.debug('This message should go to the log file')
 
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logging.debug('This message should appear on the console')
 
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('is when this event was logged.')

备注：其实你甚至可以什么都不配置直接使用默认值在控制台中打log，用这样的方式替换print语句对日后项目维护会有很大帮助。

通过文件配置logging

如果你希望通过配置文件来管理logging，可以参考这个官方文档。在log4net或者log4j中这是很常见的方式。

# logging.conf
[loggers]
keys=root
 
[logger_root]
level=DEBUG
handlers=consoleHandler
#,timedRotateFileHandler,errorTimedRotateFileHandler
 
#################################################
[handlers]
keys=consoleHandler,timedRotateFileHandler,errorTimedRotateFileHandler
 
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
 
[handler_timedRotateFileHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('debug.log', 'H')
 
[handler_errorTimedRotateFileHandler]
class=handlers.TimedRotatingFileHandler
level=WARN
formatter=simpleFormatter
args=('error.log', 'H')
 
#################################################
[formatters]
keys=simpleFormatter, multiLineFormatter
 
[formatter_simpleFormatter]
format= %(levelname)s %(threadName)s %(asctime)s:   %(message)s
datefmt=%H:%M:%S
 
[formatter_multiLineFormatter]
format= ------------------------- %(levelname)s -------------------------
 Time:      %(asctime)s
 Thread:    %(threadName)s
 File:      %(filename)s(line %(lineno)d)
 Message:
 %(message)s
 
datefmt=%Y-%m-%d %H:%M:%S

假设以上的配置文件放在和模块相同的目录，代码中的调用如下。

import os
filepath = os.path.join(os.path.dirname(__file__), 'logging.conf')
logging.config.fileConfig(filepath)
return logging.getLogger()

日志重复输出的坑

你有可能会看到你打的日志会重复显示多次，可能的原因有很多，但总结下来无非就一个，日志中使用了重复的handler。

第一坑

import logging
 
logging.basicConfig(level=logging.DEBUG)
 
fmt = '%(levelname)s:%(message)s'
console_handler = logging.StreamHandler()
console_handler.setFormatter(logging.Formatter(fmt))
logging.getLogger().addHandler(console_handler)
 
logging.info('hello!')
 
# INFO:root:hello!
# INFO:hello!

上面这个例子出现了重复日志，因为在第3行调用basicConfig()方法时系统会默认创建一个handler，如果你再添加一个控制台handler时就会出现重复日志。

第二坑

import logging
 
def get_logger():
    fmt = '%(levelname)s:%(message)s'
    console_handler = logging.StreamHandler()
    console_handler.setFormatter(logging.Formatter(fmt))
    logger = logging.getLogger('App')
    logger.setLevel(logging.INFO)
    logger.addHandler(console_handler)
    return logger
 
def call_me():
    logger = get_logger()
    logger.info('hi')
 
call_me()
call_me()
 
# INFO:hi
# INFO:hi
# INFO:hi

在这个例子里hi居然打印了三次，如果再调用一次call_me()呢？我告诉你会打印6次。why? 因为你每次调用get_logger()方法时都会给它加一个新的handler，你是自作自受。正常的做法应该是全局只配置logger一次。

第三坑

import logging
 
def get_logger():
    fmt = '%(levelname)s: %(message)s'
    console_handler = logging.StreamHandler()
    console_handler.setFormatter(logging.Formatter(fmt))
    logger = logging.getLogger('App')
    logger.setLevel(logging.INFO)
    logger.addHandler(console_handler)
    return logger
 
def foo():
    logging.basicConfig(format='[%(name)s]: %(message)s')
    logging.warn('some module use root logger')
 
def main():
    logger = get_logger()
    logger.info('App start.')
    foo()
    logger.info('App shutdown.')
 
main()
 
# INFO: App start.
# [root]: some module use root logger
# INFO: App shutdown.
# [App]: App shutdown.

为嘛最后的App shutdown打印了两次？所以在Stackoverflow上很多人都问，我应该怎么样把root logger关掉，root logger太坑爹坑妈了。只要你在程序中使用过root logger，那么默认你打印的所有日志都算它一份。上面的例子没有什么很好的办法，我建议你招到那个没有经过大脑就使用root logger的人，乱棍打死他或者开除他。

如果你真的想禁用root logger，有两个不是办法的办法：

logging.getLogger().handlers = []  # 删除所有的handler
logging.getLogger().setLevel(logging.CRITICAL)  # 将它的级别设置到最高

小结

Python中的日志模块作为标准库的一部分，功能还是比较完善的。个人觉得上手简单，另外也支持比如过滤，文件锁等高级功能，能满足大多数项目需求。