[Reserved] On the python duplicate log output problem

Source: https://www.cnblogs.com/huang-yc/p/9209096.html

The origin of the problem:

After learning of functional programming python, they come into contact with such a powerful log logging module. To reduce duplicate code, should many students like me could not wait to write its own log function, such as the following:

import logging
# 这里为了便于理解，简单的展示了一个输出到屏幕的日志函数
def my_log():
    logger = logging.getLogger('mysql.log')

    ch = logging.StreamHandler()
    ch.setLevel(logging.ERROR)
    fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(fmt)
    logger.addHandler(ch)
    return logger

my_log().error('run one')
my_log().error('run two')
my_log().error('run three')

Function written, also seems to be no problem, we have to run it!

The results are as follows:

2018-06-21 13:06:37,569 - mysql.log - ERROR - run one
2018-06-21 13:06:37,569 - mysql.log - ERROR - run two
2018-06-21 13:06:37,569 - mysql.log - ERROR - run two
2018-06-21 13:06:37,569 - mysql.log - ERROR - run three
2018-06-21 13:06:37,569 - mysql.log - ERROR - run three
2018-06-21 13:06:37,569 - mysql.log - ERROR - run three

Repeat actually log output, and increasing amounts.

Problem Analysis

Indeed logger = logging.getLogger ( 'mysql.log'), when executed, is not generated each time a new logger, but first check if there called 'mysql.log' the logger memory objects, there is removed, the new does not exist.
Examples of such a logger object has attribute 'handlers' to store the Handler, the following code shows:

import logging


def my_log():
    logger = logging.getLogger('mysql.log')
    # 每次被调用后打印出logger的handlers列表
    print(logger.handlers)

    ch = logging.StreamHandler()
    ch.setLevel(logging.ERROR)
    fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(fmt)

    logger.addHandler(ch)

    return logger


my_log().error('run one')
my_log().error('run two')
my_log().error('run three')

operation result:

2018-06-21 13:26:14,059 - mysql.log - ERROR - run one
[<StreamHandler <stderr> (ERROR)>]
2018-06-21 13:26:14,060 - mysql.log - ERROR - run two
2018-06-21 13:26:14,060 - mysql.log - ERROR - run two
[<StreamHandler <stderr> (ERROR)>, <StreamHandler <stderr> (ERROR)>]
2018-06-21 13:26:14,060 - mysql.log - ERROR - run three
2018-06-21 13:26:14,060 - mysql.log - ERROR - run three
2018-06-21 13:26:14,060 - mysql.log - ERROR - run three

logger.handlers initially an empty list, perform 'logger.addHandler (ch)' add a 'StreamHandler', output a log
in a second time is called, logger.handlers already exists a 'StreamHandler', execute 'logger again. addHandler (ch) 'will be added again a' StreamHandler ', in this case there are two logger' StreamHandler ', the output of the two log repeated
in the third is called, logger.handlers exist two' StreamHandler ', performing 'logger.addHandler (ch)' will be added again again a, in this case there are three logger 'StreamHandler', the output of three duplicate log

Solution

1. change the name

import logging
# 为日志函数添加一个name，每次调用时传入不同的日志名
def my_log(name):
    logger = logging.getLogger(name)

    ch = logging.StreamHandler()
    ch.setLevel(logging.ERROR)
    fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(fmt)
    logger.addHandler(ch)
    return logger

my_log('log1').error('run one')
my_log('log2').error('run two')
my_log('log3').error('run three')

operation result:

2018-06-21 13:40:51,685 - log1 - ERROR - run one
2018-06-21 13:40:51,685 - log2 - ERROR - run two
2018-06-21 13:40:51,685 - log3 - ERROR - run three

2. cleared up (logger.handlers.clear)

import logging
def my_log():
    logger = logging.getLogger('mysql.log')
    # 每次被调用后，清空已经存在handler
    logger.handlers.clear()
    ch = logging.StreamHandler()
    ch.setLevel(logging.ERROR)
    fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(fmt)
    logger.addHandler(ch)
    return logger

my_log().error('run one')
my_log().error('run two')
my_log().error('run three')

ps: removeHandler method (poor compatibility)

# 这种写法下的可以使用removeHandler方法(logger.handlers.clear也可以使用在这种写法的函数内)
import logging

def my_log(msg):
    logger = logging.getLogger('mysql.log')
    ch = logging.StreamHandler()
    ch.setLevel(logging.ERROR)
    fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    ch.setFormatter(fmt)
    logger.addHandler(ch)
    logger.error(msg)
    
    # 在使用完ch后从移除Handler
    logger.removeHandler(ch)

my_log('run one')
my_log('run two')
my_log('run three')

3. Analyzing the front

import logging
def my_log():
    logger = logging.getLogger('mysql.log')
    # 判断logger是否已经添加过handler，是则直接返回logger对象，否则执行handler设定以及addHandler(ch)
    if not logger.handlers:
        ch = logging.StreamHandler()
        ch.setLevel(logging.ERROR)
        fmt = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
        ch.setFormatter(fmt)
        logger.addHandler(ch)
    return logger

my_log().error('run one')
my_log().error('run two')
my_log().error('run three')

to sum up

Log output first encounter repeated problems, had not yet learned the contents of object-oriented programming, I did not really understand the logging module. After learning object-oriented programming, come back to think about these issues have become clear feeling.
For example, at first not very practical understanding of the principles logging.getLogger in learning object-oriented programming hasattr, getattr, some methods such setattr after dawned. So gentlemen if we still do not understand module for logging, does not wish to tangle in the details, keep learning.
After the expansion of knowledge, some of the problems have been solved naturally :)