os module, sys module, json module, pickle module, logging module

os module

Function: to interact with the operating system, you can manipulate files

First, the file operations

1. To determine whether the file

os.path.isfile (r 'path')

1. Delete Files

   os.remove () to delete the file

2. Rename the file

   os.rename(r' ',r' ')

Second, folder manipulation

Whether os.path.isdir () to determine the folder

os.mkdir ( 'm2') to create a folder named folder m2

if not os.path.exists(r'路径')：#检测路径下文件夹是否存在，如果不存在，就创建文件夹
   os.mkdir(r'路径')      

os.listdir (r 'address') lists all the address files and folders (*****)

os.rmdir ( 'm2') to delete a file named folder m2

Auxiliary function

os.getcwd () Gets the path of the current file folder,

os.rename ( 'name1', 'name2') Rename name2

__file__ only pycharm there,

Get the current file specific path

res = os.path.join ( 'path 1', 'path 2') automatic stitching paths , paths of different platforms supported splicing, (windows, ios, andirod, linux, unix) may be a multilayer path splicing, and splicing process automatic '\'

os.path.abspath (__ file__) to obtain the absolute path of the current file, automatically replace the slashes as different operating systems

os.path.dirname and abspath associated with obtaining the upper path of the folder.

os.walk three return path

os.walk () method is used by the migration output file name in the directory in the directory tree, up or down.

os.walk () method is an easy to use file, directory traversal, you can help us efficient processing files, directories aspects of things.

In Unix, Windows effectively.

walk () method syntax is as follows:

os.walk(top[, topdown=True[, onerror=None[, followlinks=False]]])

parameter:

• Top - is the address you want to traverse the directory, returns a triple (root, dirs, files).
  • root referring to the address currently traversing the folder itself
  • dirs is a list, all content is the name of the directory folder (not including subdirectories)
  • files also list, the contents of that folder are all the files (not including subdirectories)
• the topdown - Optional, is True, the first traversal of the top directory, or subdirectory-first traversal top (on by default). If topdown parameter is True, walk will traverse the top folder, a subdirectory for each and top folder.
• onerror - Optional, requires a callable object that needs an exception when the walk, will be called.
• followlinks - Optional If True, Shortcuts in the directory will traverse (under linux is a soft link symbolic link) within the meaning of the actual directory (off by default), if False, the first traversal of the top subdirectories.

return value:

This method has no return value.

sys module

Function: interaction with the python interpreter

sys.argv most commonly used, when operated as a file using the command line, the extra parameters received, received in the form of a list, the first element is the path to the program itself.

sys.path return path search module, using the value of environment variable initialization PYTHONPATH

sys.modules.key () returns a list of all modules has been introduced

json module

Serialization

The objects (variables) into or from the memory may store the serialization process is called

Serialization advantages:

1. Persist status: Memory is not permanently stored data, when the program runs for some time, we power down or restart the program, in-memory data on the program for some time before the (structured) have been cleared. But before power off or restart the program that are currently all the data in memory are saved (saved to a file) to perform data can be loaded from the file before the program to the next, and then continue, which is serialized .
2. Cross-platform data exchange: not only the contents of the serialized write serialization disk, you can also transmitted over the network to another machine, if the two sides agreed to send and receive good practical one sequence of the format, then it broke platform / language difference limit brought achieve a cross-platform data exchange.

json serialization is not unique to python, json serialization will be involved in java and other languages, so use json serialization can achieve the purpose of cross-platform data transmission.

# # 序列化字典为json串,并保存文件
import json
def dic():
    print('func')
with open('test.json', 'w', encoding='utf8') as fw:
    json.dump(dic, fw)

# 反序列化
with open('test.json', 'r', encoding='utf8') as fr:
    data = json.load(fr)
    print(type(data), data)

json Python data types and data type correspondence table

json type	Type python
{}	dict
[]	list
"string"	str
520.13	int or float
true/false	True/False
null	None

From python dictionary (the most common) becomes json string,

Deserialization json into python text from the text.

The json.dump () serialized to a file

the json.load () Gets the string from the hard disk json

json.dumps () of the sequence into memory

json.loads () acquired from the memory string json

pickle module

pickle function is serialized and deserialized, but its drawback is that only a python, and with only non-critical data stored pickle, i.e. not successfully deserialize it does not matter, it has the advantage can be saved python All types of data objects.

logging module

Function: a log generation module

Logging level (if not set, the default display 30 above)

#V1  日志级别(如果不设置,默认显示30以上)
# logging.info('info')  # 10
# logging.debug('debug')  # 20
# logging.warning('wraning')  # 30
# logging.error('error')  # 40
# logging.critical('critical')  # 50

V2 Add Settings

# logging.basicConfig(filename='20190927.log',
#                     format='%(asctime)s - %(name)s - %(levelname)s -%(module)s: %(message)s',
#                     datefmt='%Y-%m-%d %H:%M:%S %p',
#                     level=10)
#上面模板copy过来然后配置level

# username = 'nick'
# goods = 'bianxingjingang'
# logging.info(f'{username}购物{goods}成功')  # 10

V3 custom configuration

# 1. 配置logger对象
nick_logger = logging.Logger('nick')
json_logger = logging.Logger('jason')

# 2. 配置格式
formmater1 = logging.Formatter('%(asctime)s - %(name)s -%(thread)d - %(levelname)s -%(module)s:  %(message)s',
                               datefmt='%Y-%m-%d %H:%M:%S %p ', )

formmater2 = logging.Formatter('%(asctime)s :  %(message)s',
                               datefmt='%Y-%m-%d %H:%M:%S %p', )

formmater3 = logging.Formatter('%(name)s %(message)s', )

# 3. 配置handler --> 往文件打印or往终端打印
h1 = logging.FileHandler('nick.log')
h2 = logging.FileHandler('json.log')
sm = logging.StreamHandler()

# 4. 给handler配置格式
h1.setFormatter(formmater1)
h2.setFormatter(formmater2)
sm.setFormatter(formmater3)

# 5. 把handler绑定给logger对象
nick_logger.addHandler(h1)
nick_logger.addHandler(sm)
json_logger.addHandler(h2)

# 6. 直接使用
nick_logger.info(f'nick 购买 变形金刚 4个')
# json_logger.info(f'json 购买 变形金刚 10个')

logging module contains four roles: logger, Filter, Formatter objects, Handler

1. logger: Object generated logs
2. Filter: Object filtering logs
3. Formatter objects: You can customize different log format object, and then bind to different Handler objects used to control a different log format Handler
4. Handler: receiving print log and the console to different places, FileHandler used to print to a file, StreamHandler for printing to the terminal

#以上可以全部忘记,只要会下面的配置方法
import os
import logging.config

# 定义三种日志输出格式 开始
standard_format = '[%(asctime)s][%(threadName)s:%(thread)d][task_id:%(name)s][%(filename)s:%(lineno)d]' \
                  '[%(levelname)s][%(message)s]'  # 其中name为getLogger()指定的名字；lineno为调用日志输出函数的语句所在的代码行
simple_format = '[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d]%(message)s'
id_simple_format = '[%(levelname)s][%(asctime)s] %(message)s'
# 定义日志输出格式 结束

logfile_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))  # log文件的目录，需要自定义文件路径 # atm
logfile_dir = os.path.join(logfile_dir, 'log')  # C:\Users\oldboy\Desktop\atm\log

logfile_name = 'log.log'  # log文件名，需要自定义路径名

# 如果不存在定义的日志目录就创建一个
if not os.path.isdir(logfile_dir):  # C:\Users\oldboy\Desktop\atm\log
    os.mkdir(logfile_dir)

# log文件的全路径
logfile_path = os.path.join(logfile_dir, logfile_name)  # C:\Users\oldboy\Desktop\atm\log\log.log

if os.path.getsize(logfile_path) > 4000:
    pass

# 定义日志路径 结束

# log配置字典 (不能改)
LOGGING_DIC = {
    'version': 1,
    'disable_existing_loggers': False,
    # 配日志格式
    'formatters': {
        'standard': {
            'format': standard_format
        },
        'simple': {
            'format': simple_format
        },
    },
    'filters': {},  # filter可以不定义
    # 配置打印位置
    'handlers': {
        # 打印到终端的日志
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',  # 打印到屏幕
            'formatter': 'simple'
        },
        # 打印到文件的日志,收集info及以上的日志
        'default': {
            'level': 'INFO',
            'class': 'logging.handlers.RotatingFileHandler',  # 保存到文件
            'formatter': 'standard',
            'filename': logfile_path,  # 日志文件
            'maxBytes': 1024 * 1024 * 5,  # 日志大小 5M  (*****)
            'backupCount': 5,
            'encoding': 'utf-8',  # 日志文件的编码，再也不用担心中文log乱码了
        },
    },
    'loggers': {
        # logging.getLogger(__name__)拿到的logger配置。如果''设置为固定值logger1，则下次导入必须设置成logging.getLogger('logger1')
        '': {
            # 这里把上面定义的两个handler都加上，即log数据既写入文件又打印到屏幕
            'handlers': ['default', 'console'],
            'level': 'DEBUG',
            'propagate': False,  # 向上（更高level的logger）传递
        },
    },
}


def load_my_logging_cfg():
    logging.config.dictConfig(LOGGING_DIC)  # 导入上面定义的logging配置
    logger = logging.getLogger(__name__)  # 生成一个log实例
    logger.info('It works!')  # 记录该文件的运行状态

    return logger


if __name__ == '__main__':#此处是自己加的
    load_my_logging_cfg()