This article is to disassemble this [ python︱function, for, if, name , iterator, prevention of error reporting, class definition, decorator, argparse module, yield ] to take out the error report + logger prompt
Article Directory
1. Prevent error reporting
1.1 assert assertion
Python's assert is used to check a condition and if it is true, do nothing. If it's false, an AssertError will be thrown with the error message. For example:
py> x = 23
py> assert x > 0, "x is not zero or negative"
py> assert x%2 == 0, "x is not an even number"
Traceback (most recent call last):
File "", line 1, in
AssertionError: x is not an even number
Similar to stopifnot in R
Reference blog: When to use assertions in Python?
.
2 try…except…
There are two ways to display the error exception:
# 第一种
try:
0/0
except Exception as e:
print (e)
>>> division by zero
# 第二种，更加全面
import traceback
import sys
try:
0/0
except:
traceback.print_exc()
>>>
Traceback (most recent call last):
File "<ipython-input-4-17cf35f30609>", line 4, in <module>
0/0
ZeroDivisionError: division by zero
try:
f = open('xxx')
except:
print 'fail to open'
exit(-1)
If the open does not come out in try, then except returns the corresponding content "fail to open"
try:
<语句> #运行别的代码
except <名字>：
<语句> #如果在try部份引发了'name'异常
except <名字>，<数据>:
<语句> #如果引发了'name'异常，获得附加的数据
else:
<语句> #如果没有异常发生
try:
print('I am sure no exception is going to occur!')
except Exception:
print('exception')
else:
# 这里的代码只会在try语句里没有触发异常时运行,
# 但是这里的异常将 *不会* 被捕获
print('This would only run if no exception occurs. And an error here '
'would NOT be caught.')
finally:
print('This would be printed in every case.')
# Output: I am sure no exception is going to occur!
# This would only run if no exception occurs.
# This would be printed in every case.
Report an error and prompt an exception message
Source: Obtaining Exception information in Python
1, str(e)
Returns a string type, only gives exception information, does not include the type of exception information, such as 1/0 exception information
‘integer division or modulo by zero’
2、repr(e)
Give more complete exception information, including the type of exception information, such as 1/0 exception information
“ZeroDivisionError(‘integer division or modulo by zero’,)”
3、e.message
The obtained information is the same as str(e)
4. Use the traceback module
The traceback module needs to be imported, and the information obtained at this time is the most complete, which is consistent with the error message displayed when running the program on the python command line. Use traceback.print_exc() to print the exception message to stderr as if it were not captured, or traceback.format_exc() to capture the same output as a string. You can pass various parameters to these functions to limit the output, or to reprint to objects like file types.
try:
1/0
except Exception, e:
print 'str(Exception):\t', str(Exception)
print 'str(e):\t\t', str(e)
print 'repr(e):\t', repr(e)
print 'e.message:\t', e.message
print 'traceback.print_exc():'; traceback.print_exc()
print 'traceback.format_exc():\n%s' % traceback.format_exc()
or:
def this_fails():
x = 1/0
try:
this_fails()
except :
print('Handling run-time error:')
raise
print(1)
Elegant exception error: suppress
But what often happens is this:
• 我们知道这个异常有可能发生
• 我们不关心这个异常，如果发生了，什么也不用处理，直接忽略就好
If you want to handle exceptions in this situation, you don't have to use try-except. Python's built-in contextlib library provides a function called suppress, which is a more elegant way to handle this exception. Pythonista must try it.
# 单个非0报错
from contextlib import suppress
nums = [3,0,3,0,3]
result = 0
for num in nums:
with suppress(ZeroDivisionError):
result += 1/num
print(result) # 1.0
# 免疫所有报错
for num in nums:
with suppress(Exception):
result += 1/num
# 别这么写
result = 0
nums = [3,0,3,0,3]
with suppress(Exception):
for num in nums:
result += 1/num
print(result) # 0.3333333333333333
3. Error log record: Logger.exception
Log messages at ERROR level, and exception trace information will be automatically added to the log messages. Logger.exception is used in exception handling blocks, such as:
Source: Python module learning: logging logging
import logging
logging.basicConfig(filename = os.path.join(os.getcwd(), 'log.txt'), level = logging.DEBUG)
log = logging.getLogger('root')
try:
a===b
except:
log.exception('exception') #异常信息被自动添加到日志消息中
It will be loaded into Log.txt in the specified directory
Let's take a look at the display in the Log as follows:
ERROR:root:exception
Traceback (most recent call last):
File "test.py", line 12, in <module>
a==b
NameError: name 'a' is not defined
Therefore, some keywords can be simply defined in log.exception() to help locate the problem.
3.1 logger - basicConfig general storage
Source: python logging module prints log to specified file
import logging
import unittest
class lgtest(unittest.TestCase):
logging.basicConfig(filename='../LOG/'+__name__+'.log',format='[%(asctime)s-%(filename)s-%(levelname)s:%(message)s]', level = logging.DEBUG,filemode='a',datefmt='%Y-%m-%d%I:%M:%S %p')
def test(self):
logging.error("这是一条error信息的打印")
logging.info("这是一条info信息的打印")
logging.warning("这是一条warn信息的打印")
logging.debug("这是一条debug信息的打印")
if __name__=='__main__':
unittest.main()
- Filename: The file of the specified path. +—name—+ is used here to name the log as the file name of the current py
- Format: Set the display format of the log (that is, the format seen in the document). They are time + current file name + log output level + output information
- Level: The log level of the output. Those with a priority lower than the set level will not be output and saved in the log file
- Filemode: log open mode
- a: Represents that the log will continue to be written every time the program is run. That is, the previously saved log information will not be overwritten.
- w: Rewrites the log every time the program is run. That is, overwrite the previously saved log information
3.2 常规：logging + handlers + TimedRotatingFileHandler
Reference: Python + logging output to the screen, write the log to a file
Reference: Part 2 of the logging module in python
# 信息写入log
import logging
from logging import handlers
class Logger(object):
level_relations = {
'debug':logging.DEBUG,
'info':logging.INFO,
'warning':logging.WARNING,
'error':logging.ERROR,
'crit':logging.CRITICAL
}#日志级别关系映射
def __init__(self,filename,level='info',when='D',backCount=15,\
fmt='%(asctime)s - %(levelname)s: %(message)s'):
self.logger = logging.getLogger(filename) # 创建log文件
format_str = logging.Formatter(fmt)#设置日志格式
self.logger.setLevel(self.level_relations.get(level))#设置日志级别
if not self.logger.handlers:
sh = logging.StreamHandler() # 初始化 1 ，往屏幕上输出
sh.setFormatter(format_str) # 设置屏幕上显示的格式
th = handlers.TimedRotatingFileHandler(filename=filename,when=when,\
backupCount=backCount,encoding='utf-8')
th.setFormatter(format_str)#设置文件里写入的格式
self.logger.addHandler(sh) #把对象加到logger里
self.logger.addHandler(th)
if __name__ == '__main__':
log = Logger('all.log',level='debug')
log.logger.debug('debug')
log.logger.info('info')
log.logger.warning('警告')
log.logger.error('报错'', exc_info=True)
log.logger.critical('严重')
Logger('error.log', level='error').logger.error('error')
The result on the screen is as follows:
2018-03-13 21:06:46,092 - DEBUG: debug
2018-03-13 21:06:46,092 - INFO: info
2018-03-13 21:06:46,092- WARNING: 警告
2018-03-13 21:06:46,099 - ERROR: 报错
2018-03-13 21:06:46,099- CRITICAL: 严重
2018-03-13 21:06:46,100 - ERROR: error
Because of when=D, the newly generated file name will carry the time, as shown below.
in:
-
level_relations , represents the log level, the default level is WARNING, which means that only this level and above will feedback information, unless the logging module is used for other things.
- Log level: debug < info < warning < error < critical
-
TimedRotatingFileHandler - Automatically split log files according to time
-
interval is the time interval
-
when is the time unit of the interval, and the units are as follows:
- S seconds
- M points
- H hours,
- D day,
- W every week (interval==0 means Monday)
- midnight every morning
-
backupCount is the number of logs to keep. The default value of 0 means that the log will not be deleted automatically. If it is set to 10, the library will judge whether there is more than this 10 during the file creation process, and if it exceeds, it will delete from the first created one.
-
exc_info, True, means to save the error information, the default is False, not to save
At the same time,
you will find that if you do not set it if not self.logger.handlers:, then there will be a problem of repeatedly writing logs. The problem is that he will continue to log.logger.handlersadd handlers. The upper limit is three, and it will appear:
the first record is written once, the second record is written twice, and the third record is written three times.
(The solution comes from: python logging duplicate logging problem )
The principle is that if self.logger.handlersthere is already a class, then don't add it.
# 正常的self.logger.handlers长什么样子：
[<logging.StreamHandler at 0x12eba1e3128>,
<logging.handlers.TimedRotatingFileHandler at 0x12eba1e3160>]
# 不正常的self.logger.handlers长什么样子：
[<logging.StreamHandler at 0x12eba1e3128>,
<logging.handlers.TimedRotatingFileHandler at 0x12eba1e3160>,
[<logging.StreamHandler at 0x12eba1e3128>,
<logging.handlers.TimedRotatingFileHandler at 0x12eba1e3160>,]
The extended error information is loaded into the log together:
import logging
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(message)s',
datefmt='%a, %d %b %Y %H:%M:%S +0000',
filename='my.log')
logging.info('this is a info')
try:
do
except Exception:
logging.error('There are something wrong', exc_info=True)
logging.info('continue')
# 报错信息暴露
Sun, 01 Jul 2018 21:10:53 +0000 There are something wrong
Traceback (most recent call last):
File "learn.py", line 9, in <module>
do
NameError: name 'do' is not defined
Sun, 01 Jul 2018 21:10:53 +0000 continue
Extended application error 2: logger cannot display Chinese
stream.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode character '\ufffd' in position 363: ordinal not in range(128)
Reference: https://github.com/EasyEngine/easyengine/issues/843
The solution is that the system lacks UTF-8 components and needs to be loaded:
This worked for me:
sudo locale-gen en_US.UTF-8
export LANG=en_US.UTF-8
From: http://askubuntu.com/questions/393638/unicodedecodeerror-ascii-codec-cant-decode-byte-0x-in-position-ordinal-n
3.3 Configure Log with .yaml file
The second part of the logging module in python
We can configure logging not only through python code, but also by writing a yaml file. Every time we need to use logging, we only need to call this file to complete the configuration.
The content of the config.yaml file is as follows
version: 1
formatters:
simple:
format: "%(message)s"
more:
format: "%(asctime)s - %(levelname)s - %(message)s"
handlers:
console:
class : logging.StreamHandler
formatter: simple
level: INFO
stream: ext://sys.stdout
file:
class: logging.FileHandler
formatter: more
level: DEBUG
filename: debug.log
loggers:
mainlogger:
level: DEBUG
handlers: [console, file]
root:
level: DEBUG
handlers: [console]
Write the code in the main.py file as follows
import logging
import logging.config
import yaml
import a
with open('config.yaml', 'r', encoding='utf-8') as f:
config = yaml.load(f)
logging.config.dictConfig(config)
logging.info('main file log')
a.run()
4、with…as…
Then with and as are also a way to prevent errors.
When python executes this sentence, it will call the __enter__ function, and then pass the value of the function return to the variable specified after as.
After that, python will execute the statement block of do something below. Finally, no matter what exception occurs in this statement block, __exit__ will be executed when leaving.
ith open("x.txt") as f:
data = f.read()
with open("x.txt") as f1, open('xxx.txt') as f2:
do something with f1,f2
Then try and with can also be combined:
try:
with open( "a.txt" ) as f :
do something
except xxxError:
do something about exception
Extension: How to write ValueError() for error prompt
if pretrained_model not in self._models:
raise ValueError(
'The n_class needs to be supplied as an argument.')
5 Take the initiative to issue an error raise
参考url：https://blog.csdn.net/skullfang/article/details/78820541
raise RuntimeError('testError')
6 A useful new library for Logger: loguru
6.1 File saving
#!pip install loguru
Use it:
from loguru import logger
import sys
# 文件保存
# logger.add("log/file.log", rotation="12:00") # 每天中午12点，自动完成归档，并启用新日志文件
# logger.add('runtime_{time}.log', rotation="500 MB") # 设置超过 500 MB 新创建一个 log 文件
# logger.add('runtime_{time}.log', rotation='1 week') # 设置每隔一个周新创建一个 log 文件
# logger.add('runtime_{time}.log', compression='zip') # 可以配置日志文件的压缩格式，这样可以更加节省存储空间，比如设置使用 zip 文件格式保存
logger.add('log/runtime_{time}.log', rotation='00:00') # 每天0点，自动完成归档，并启用新日志文件
logger.add(sys.stdout, colorize=True) # 将日志输出到控制台，并添加颜色
6.2 File saving and deletion
# 文件保存与删除
logger.remove(log_file) # 删除
logger.add('runtime_{time}.log', retention='15 days') # 可以设置日志的最长保留时间，比如设置日志文件最长保留 15 天
logger.add('runtime_{time}.log', retention=10) # 设置日志文件最多保留 10 个
# 文件定时删除
# 也可以是一个 datetime.timedelta 对象，比如设置日志文件最多保留 5 个小时
import datetime
from loguru import logger
logger.add('runtime_{time}.log', retention=datetime.timedelta(hours=5))
6.3 General usage + exception function tracking - very powerful! !
# 常规用法
logger.info("This is log info!")
logger.warning("This is log warn!")
logger.error("This is log error!")
logger.debug("This is log debug!")
# 异常函数追踪 - 非常给力！！
from loguru import logger
@logger.catch
def my_function(x, y, z):
# An error? It's caught anyway!
return 1 / (x + y + z)
my_function(0, 0, 0)
references:
Loguru: The ultimate solution for Python logging
loguru: Python's out-of-the-box logging library