Explain in detail with and context manager in Python

with This keyword is familiar to everyone who learns Python.

When manipulating text objects, almost everyone will let us use them with open. This is an example of context management. You must be quite familiar with it, I won't talk nonsense anymore.

with open('test.txt') as f:
    print f.readlines()

1. what context manager？

Basic grammar

with EXPR as VAR:
    BLOCK

First clarify a few concepts

1. 上下文表达式：with open('test.txt') as f:
2. 上下文管理器：open('test.txt')
3. f 不是上下文管理器，应该是资源对象。

2. how context manager？

To implement such a context management yourself, you must first know the context management protocol.

To put it simply, the __enter__and __exit__method is implemented in a class, and an instance of this class is a context manager.

For example this example:

class Resource():
    def __enter__(self):
        print('===connect to resource===')
        return self
    def __exit__(self, exc_type, exc_val, exc_tb):
        print('===close resource connection===')

    def operate(self):
        print('===in operation===')

with Resource() as res:
    res.operate()

Let's execute it and go through the printing sequence of the log. You can know the execution process.

===connect to resource===
===in operation===
===close resource connection===

It is obvious from this example that when writing code, you can put the connection or acquisition of __enter__the resource in and write the close of the resource __exit__in.

3. why context manager？

Ask yourself a few more whys when studying, and develop thinking about some details, which will help you deepen your understanding of the knowledge points.

Why use a context manager?

In my opinion, this is related to the elegant style advocated by Python.

1. You can manipulate (create/acquire/release) resources in a more elegant way, such as file operations and database connections;
2. You can handle exceptions in a more elegant way;

The first one, we have already taken resource connection as an example.

The second type will be ignored by most people. I will focus on it here.

As we all know, handling exceptions, usually use try...execept..to capture process. One disadvantage of this is that in the main logic of the code, there will be a large number of exception handling agents, which will greatly affect our readability.

A little better thing to do, can be used withto handle exceptions hidden.

Still based on the above code example, we will 1/0this 一定会抛出异常的代码written operatein

class Resource():
    def __enter__(self):
        print('===connect to resource===')
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        print('===close resource connection===')
        return True

    def operate(self):
        1/0

with Resource() as res:
    res.operate()

After running it, I was surprised to find that no error was reported.

This is a powerful aspect of the context management protocol. Exceptions can __exit__be caught in and you can decide how to handle them, whether to throw them or to solve them here. In __exit__in return True(not return to default to return False), equivalent to tell the Python interpreter, we have captured the exception, you do not need to throw out.

When writing a __exit__function, you need to pay attention to it. It must have these three parameters:

• exc_type: exception type
• exc_val: abnormal value
• exc_tb: abnormal error stack information

When the main logic code does not report an exception, these three parameters will all be None.

4. how contextlib?

In the above example, we just wrote a class to build a context manager. If you just want to implement a simple function, writing a class is a bit too complicated. At this time, we thought, if only one function can be written to implement the context manager.

Python has long thought of this point. It provides us with a decorator, as long as you implement the function content according to its code protocol, you can turn this function object into a context manager.

We follow the contextlib protocol to implement a context manager for opening files (with open).

import contextlib

@contextlib.contextmanager
def open_func(file_name):
    # __enter__方法
    print('open file:', file_name, 'in __enter__')
    file_handler = open(file_name, 'r')

    # 【重点】：yield
    yield file_handler

    # __exit__方法
    print('close file:', file_name, 'in __exit__')
    file_handler.close()
    return

with open_func('/Users/MING/mytest.txt') as file_in:
    for line in file_in:
        print(line)

In the decorated function, it must be a generator (with yield), and the code before yield is equivalent __enter__to the content inside. The code after yield is equivalent __exit__to the content in.

The above code can only achieve the first purpose of the context manager (management of resources), but not the second purpose (handling exceptions).

If you want to handle exceptions, you can change it to the following.

import contextlib

@contextlib.contextmanager
def open_func(file_name):
    # __enter__方法
    print('open file:', file_name, 'in __enter__')
    file_handler = open(file_name, 'r')

    try:
        yield file_handler
    except Exception as exc:
        # deal with exception
        print('the exception was thrown')
    finally:
        print('close file:', file_name, 'in __exit__')
        file_handler.close()

        return

with open_func('/Users/MING/mytest.txt') as file_in:
    for line in file_in:
        1/0
        print(line)

It seems that as long as the context manager is mentioned, most people will talk about the classic example of opening files.

But in actual development, there are many examples of context managers that can be used. Let me give an example of my own.

In OpenStack, when you create a snapshot of a virtual machine, you need to create a temporary folder to store the local snapshot image. After the local snapshot image is created, upload the image to Glance. Then delete this temporary directory.

The main logic of this code is 创建快照, and 创建临时目录, belongs to the precondition 删除临时目录, is the finishing touch.

Although the amount of code is small and the logic is not complicated, the " 创建临时目录，使用完后再删除临时目录" function is needed in many places in a project. If this logic processing can be written as a tool function as a context manager, the code reuse The rate is also greatly improved.

The code is like this

In summary, there are three benefits of using a context manager:

1. Improve code reuse rate;
2. Improve the elegance of the code;
3. Improve the readability of the code;

I recommend my original " PyCharm Chinese Guide " e-book, which contains a large number (300) of illustrations . It is well-made and worthy of a collection by every Python engineer.

The address is: http://pycharm.iswbm.com