Python logging module

Python logging module

heel

• foreword
• Design of logging module
  • Logical inheritance structure of logger
  • Log output to various i/o devices
  • Log format and content rich
  • Log classification is also the classification of logs in the level dimension.
  • You can filter and specify output i/o according to the level of the log object.
  • Logger is mainly responsible for the basic unit of inheritance relationship
  • handler is responsible for the basic unit of output direction
  • formater is responsible for log format
  • The filter is responsible for filtering and can be placed in the logger and handler, with the inheritance and propagation mechanism for flexible filtering.
• logging.basicConfig()
• Logger
• handler
• formatter
• filter
• log recoder

foreword

Python's logging handling is very flexible and feature-complete enough to satisfy all logging needs.
So powerful, of course, the design of the log system is very flawed.
In addition, when understanding its design, it is necessary to sort out the relationship between several important objects in the module and the Mission tasks they are responsible for!

Design of logging module

You can refer to JAVA's log system design . This article uses a humorous language tone, starting from the requirements, to clarify how the log system is designed.
I follow this article here and briefly outline the key points

• need
  • Log messages can be output to a variety of media
  • Log message content can be formatted
  • The program code is organized and classified by package and module. To satisfy the code of different packages and modules, it has its own log processing method. At the same time, it also needs to have log classification (the log is actually the record of an Event event). Because It is an event that triggers the log, so the level of this event should be reflected in the log, so the log should be divided into levels.
  • According to the previous requirement, the log has a level, so there must be articles to do on this log classification. For example, different log levels are handled in different ways, and there are priorities among different log levels. According to the priority, the processing method above a certain log level and the processing method below the level can be set.
  • Log processing and log output can also customize filtering requirements.
• Analyze requirements
  • First, there must be a class to abstract the log message, this class has at least two properties timestamp, the message itself. Log messages are meant to record the occurrence of an event. Then use the LogRecord class to represent log messages.
  • To output LogRecord to different media, abstract a Handler class, and different Handlers process LogRecord to different media
  • To format the output of LogRecord, there must be a Formatter class, and different Formatters can handle different format types.
  • To satisfy different packages and modules, they have their own processing methods. From the point of view of the user's final use, it is too troublesome for me to create multiple Handlers each time, so I also need a handler to integrate these objects. This is an abstract class called Logger. Each package and module can have its own logger, and a unique name can be used to identify the uniqueness of this Logger.
  • The log message LogRecord is divided into levels, and other processing LogRecord can also be processed differently according to its level.
  • Filter at the two log routing locations, logger and handler. That is to add a Filter filter object to the logger and handler.

In this way, according to the needs of the relationship between the core classes,
Logger includes Handler, and Handler includes Formatter. Both handle LogRecord. Both Logger and Handler can add Filter to filter LogRecord.

Inheritance of logger

1.就是通过logger的名字的命名层级来查找其parent logger。使用'.'来表示继承关系。
2.logger的继承其实是继承几个方面
    2.1. level的继承：子logger写日志的时候，优先使用本身设置的level，如果没有设置，则逐层向上级父logger查询，直到查询到为止。最极端的情况是，使用rootlogger的默认日志级别——WARNING.可以使用logger.getEffectivaLeve()获取有效的等级。
    2.2. handler的继承，先将日志对象传递给子logger的所有handler处理，处理完毕，如果孩子的logger的propagate属性设置的是1（TRUE），那么日志record对象会被传播给上级父logger，该父logger的handler都会进行处理，如果父logger的propagate也是1，那么会继续向上。如果子logger的propagate就是0，就不会传播，极端，如果是子logger都没有handler，且不传播，那么就会极端的给logging.lastResort这是一个内置的handler，不会和任何一个logger关联，and acts like StreamHandler which the event description message to the current value of sys.stderr.这个logging.lastResort是没有formatted的，所以我们会看到就message原始格式，输出到标准输出中。
3. 总的来说，其实不是逻辑上的继承，只是利用层级关系，可以来使用父logger的level和handler。

Logical inheritance structure of logger

The construction of a logger is created by naming the logger. Names are unique representations throughout the program.
Such a design makes it possible to combine your own logs with logs generated using third-party modules. There is no need to deliberately rewrite the processing of logs generated by third-party modules in your own code. All are handed over to logging to be integrated and processed. The representation of the inheritance relationship between
Loggers :

• It is indicated by the '.' symbol. For example: 'foo' 'foo.bar' 'foo.bar.baz' 'foo' is the father of 'foo.bar', and 'foo.bar' is the father of 'foo.bar.baz'. The father of 'foo' is actually the 'root' logger.
• The representation of this hierarchical relationship is very similar to the representation of the hierarchical relationship between packages and modules.

logging.basicConfig()

This module-level function is to set the root logger. If the root logger already has a handler, then calling the function has no effect. The following is the source code:

def basicConfig(**kwargs):
    _acquireLock()
    try:
        if len(root.handlers) == 0:
            filename = kwargs.get("filename")
            if filename:
                mode = kwargs.get("filemode", 'a')
                hdlr = FileHandler(filename, mode)
            else:
                stream = kwargs.get("stream")
                hdlr = StreamHandler(stream)
            fs = kwargs.get("format", BASIC_FORMAT)
            dfs = kwargs.get("datefmt", None)
            fmt = Formatter(fs, dfs)
            hdlr.setFormatter(fmt)
            root.addHandler(hdlr)
            level = kwargs.get("level")
            if level is not None:
                root.setLevel(level)
    finally:
        _releaseLock()

Logger Objects

• Loggers are never instantiated directly,but always through the module-level function logging.getLogger(name).
• The logger object cannot be directly passed through the Logger class instance, but is generated by calling the module-level function logging.getLogger(name).
• Mutiple calls to getLogger() with the same name will always return a refernece to the same Logger object.

• In addition to the inheritance relationship of Logger, some things such as level (getEffectiveLevel()) can be inherited from the parent. LogRecords can also be propagated up through inheritance relationships.
  Methods often used by loggers:

method	usage	Notes
logger.info()
logger.debug()
logger.setLevel
logger.getEffectiveLevel ()
logger.getChild(suffix)		Return a child logger, the name of the child logger will be the name of the parent logger plus suffix
logger.addFilter (felt)
logger.removeFilter (felt)
logger.addHandler(hd)
logger.removeHandler(hd)

Handler

Handler is a base class from which other different types of handlers inherit. Likewise, the handler does not instantiate the object directly, but uses its subclasses.
Common type Handler

Handler	The module where it is located	Notes
StreamHandler	logging
FileHandler	logging
NullHandler	loggging
BaseRotatingHandler	logging.handlers
RotatingFileHandler	logging.handlers
TimedRotatingFileHandler	logging.handlers
SocketHandler	logging.handlers

etc. . .

Handler common methods:

• Handler.addFilter ()
• Handler.removeFilter()
• Handler.setFormatter ()
• Handler.setLevel ()

Formatter

This class object is used to format LogRecord.
The initialization of the Formtter object needs to specify the log string format, as well as the time format, and the style used by the format string.
The Fomatter object is added to the Handler object.

Filter

The filter object initialization needs to specify the filter logger name. Only the LogRecord generated by the specified logger name will pass the filter, otherwise it will be filtered out. Logger name filtering means that the specified logger name and its sub-loggers will not be filtered.

LogRecord

The carrier of the log content.
Instances of LogRecord are all through the method of the Logger object.
The main classification of LogRecord is the classification at the log level, which is divided into DEBUG, INFO, WARNING, ERROR, CRITICAL