"Learning Python with You Hand in Hand" 25-List Comprehension

After studying in the previous articles, we have basically mastered the most basic data types in Python, including lists, tuples, dictionaries, and sets. You can learn by clicking the link below.

Today, what we are going to learn is a method called comprehension, which allows us to filter the elements of a container and use a concise expression to transform the elements passed to the filter to generate a new sequence. Specifically, it includes list comprehensions, set comprehensions, and dictionary comprehensions.

1. List comprehension

Take the list comprehension as an example. In the past, when we defined a list, we had to write what elements there are in the list. If the number of elements is large, the input volume is still relatively large and it is not easy to read. At most, the range() function can be used to reduce the amount of input, but after all, this situation is still rare.

The list comprehension is used to solve the problem of the definition of the list or the input of the list element when the list element meets a certain condition.

When using a list comprehension to define a list, you don’t need to fill in the elements of the list one by one in [] like we did before, but write an expression to satisfy a certain condition in [], and finally generate The list of our needs. This process is the concept of "filtering" mentioned in the above definition.

The basic form of the list comprehension can be written as:

list_comp = [expr for val in collection if condition]

From this form of list comprehension, we can see that it is very similar to the process of generating a list using a for loop. The conditions in the for loop are the conditions we wrote in the list comprehension, so the basic form of the above list comprehension , Can also be expressed with the following for loop, they are equivalent:

result = []
for val in collection:
    if condition:
        result.append(expr)

Let's take an example to see why the effects of these two expressions are equivalent, and with practical examples, it will be easier to understand the list comprehension.

The goal of this example is to change all elements whose length is greater than 2 to uppercase for a given list of strings, and output a list that meets this condition.

In [1]: lst = ["Tom", "amy", "h", "James", "we", "Python", "3", (1, 2)]

In [2]: [val.upper() for val in lst if len(val) > 2]   # 使用列表推导式的方法
Out[2]: ['TOM', 'AMY', 'JAMES', 'PYTHON']

In [3]: result = []   # 使用for循环的方法
        for val in lst:
            if len(val) > 2:
                result.append(val.upper())
        result
Out[3]: ['TOM', 'AMY', 'JAMES', 'PYTHON']

As can be seen from the above examples, although the results obtained by the two methods are exactly the same, the amount of code is significantly different. This is where the advantage of comprehension lies, so it is also loved by many Python programmers.

2. Set comprehension

The set comprehension is very similar to the list comprehension, except that the [] of the list is replaced by the {} of the set.

set_comp = {expr for val in collection if condition}

Therefore, you can use the same example as the list comprehension for demonstration.

In [4]: {val.upper() for val in lst if len(val) > 2}   # 使用集合推导式
Out[4]: {'AMY', 'JAMES', 'PYTHON', 'TOM'}

When using the set comprehension, two points should be paid attention to. One is that there may be repetition, and the output result of the set will be "de-duplicated"; the other is that the set is unordered, and the output result is "order" Different from the list.

3. Dictionary deduction

The dictionary is a binary structure, so the dictionary comprehension will be a little bit complicated, but the principle is the same as the list comprehension, and its basic form is:

dict_comp = {key_expr : value_expr for val in collection if condition}

If we want to use all elements greater than 3 in the a list as the key and the square of the element as the value, we can write a dictionary expression like this:

In [5]: a = [1, 2, 3, 4, 5, 6, 7]
        {val : val**2 for val in a if val > 3}
Out[5]: {4: 16, 5: 25, 6: 36, 7: 49}

Because the keys of the dictionary are not repeatable, if the key_expr is fixed, only a dictionary with one element can be generated, for example:

In [6]: a = [1, 2, 3, 4, 5, 6, 7]
        {1 : val**2 for val in a if val > 3}   # 实际上就是后面的值将前面的值覆盖了
Out[6]: {1: 49}

For situations that need to use both index value and element value at the same time, we can use the built-in sequence function emunerate() mentioned in the article "Learning Python with You Hand in Hand" 23-Built-in Sequence Function .

In [7]: a = [1, 2, 3, 4, 5, 6, 7]

In [8]: {val1 : val2**2 for (val1, val2) in enumerate(a) if val2 > 3}
Out[8]: {3: 16, 4: 25, 5: 36, 6: 49}

In [9]: {val1 : val2**2 for (val1, val2) in enumerate(a) if val1 > 3}   # 在这个实例中，改变了一个变量，得到了完全不一样的结果，大家看一下这个表达式的含义又是什么
Out[9]: {4: 25, 5: 36, 6: 49}

4. Simplified derivation

Previously, we discussed the case of complete derivation. For some cases without condition requirements, we can omit the "if condition" part of the derivation to form a simplified derivation.

Take the list comprehension as an example, this time we need to output a list of the length of the elements in the given list.

In [10]: lst = ["Tom", "amy", "h", "James", "we", "Python", "3", (1, 2)]
         [len(val) for val in lst]
Out[10]: [3, 3, 1, 5, 2, 6, 1, 2]

5. The map() function

Another simplified method is to use the map() function. map() is also a built-in function of Python, and its function is to calculate the specified function for the sequence in the parameter. That is to say, both functions and sequences are map() parameters, the name of the function must be placed on the first parameter, the number of sequences is determined by the function, the parameters of the function need several, and there are several sequences in the map.

map(function, iterable, ...)

Using the map() function, the above code can be simplified. Because the len() function has only one parameter, there is only one sequence.

In [11]: lst = ["Tom", "amy", "h", "James", "we", "Python", "3", (1, 2)]
         list(map(len, lst))
Out[11]: [3, 3, 1, 5, 2, 6, 1, 2]

Let us look at a situation where there are multiple sequences as parameters, such as comparing the size of the corresponding positions of the three sequences and returning the maximum value.

In [12]: list(map(max,[8,2,3],(3,4,5,6),{6:'a', 9:'b', 2:'c'}))
Out[12]: [8, 9, 5]

6. Nested list comprehension

Finally, we will end our introduction to list comprehensions with a more complicated content. Before reading this part, please confirm that you have fully understood the previous list comprehension, otherwise the code may look a little dizzy.

Nested list comprehension is to embed a list comprehension in the expression (expr) in the list comprehension to form the embedding of the list comprehension.

In order to facilitate your understanding and comparison, we will continue to use our previous example, lst = ["Tom", "amy", "h", "James", "we", "Python", "3", (1, 2 )]. In this example, we use a list comprehension to filter out elements whose length is greater than 2 and convert them to all capital letters for output.

However, if our initial list is a nested list, can we still use the same list comprehension? Let's try it.

In [13]: lst1 = [["Tom", "amy", "h", "James"], ["we", "Python", "3", (1, 2)]]   # 嵌套列表
         [val.upper() for val in lst1 if len(val) > 2]   # 使用和之前例子一样的列表推导式
Out[13]: ---------------------------------------------------------------------------
          AttributeError                            Traceback (most recent call last)
          <ipython-input-15-0dc770476ed2> in <module>
                1 lst1 = [["Tom", "amy", "h", "James"], ["we", "Python", "3", (1, 2)]]   # 嵌套列表
          ----> 2 [val.upper() for val in lst1 if len(val) > 2]   # 使用和之前例子一样的列表推导式
          <ipython-input-15-0dc770476ed2> in <listcomp>(.0)
                1 lst1 = [["Tom", "amy", "h", "James"], ["we", "Python", "3", (1, 2)]]   # 嵌套列表
          ----> 2 [val.upper() for val in lst1 if len(val) > 2]   # 使用和之前例子一样的列表推导式
               
          AttributeError: 'list' object has no attribute 'upper'

As you can see, when our initial list becomes a nested list, if we still use the original list comprehension, an error will be reported, and the reason for the error is that the list cannot be upper.

Let us analyze the reasons.

For the original list (lst), we filter the length of the elements (a string or tuple), and the final result is to convert elements (strings) with a length greater than 2 into uppercase letters.

For the new list (lst1), we filter the length of the elements (the two nested lists), and the length of the two lists is greater than 2, so there is no problem here. The problem is when the .upper() operation is performed after filtering the two lists, because the list cannot be used for the .upper() method, an error is reported.

So how do you modify it? At this time, we need to use our nested list comprehension. The nested list comprehension can also be understood as a list comprehension on the result of the list comprehension. We can also disassemble this process in language first.

The first step is to filter the new list (lst1) with a list comprehension. Even if it is not filtered, write the conditions for this step-for lsts in lst1. This code can be understood as the traversal process of the for loop, which is to traverse all the lists in lst1. If there are no conditions, all elements in lst1 are retained; if filtering is required, for example, lists with a length less than 3 are retained, you can add judgment conditions, written as-for lsts in lst1 if len(lsts) <3 .

The second step is to perform list comprehension derivation on the filtered list lsts. At this time, it is the same as our original writing-val.upper() for val in lsts if len(val)> 2. This step is exactly the same as the code we wrote before, except that the initial filtered list has changed from lst1 to lsts, which is the list filtered by our previous step.

The third step is to merge the codes of the first two parts, and put the code of the first step before the for of the second step code, which is the part marked in red below. Please note that the variable lsts in the red part is the first filtering process and the result is obtained. It is the same as the initial filtering list in the second step, that is, the part marked in blue below, so the variable names of the two parts must be consistent.

val.upper() for lsts in lst1 for val in lsts if len(val) > 2

In [14]: lst1 = [["Tom", "amy", "h", "James"], ["we", "Python", "3", (1, 2)]]   # 嵌套列表
         [val.upper() for lsts in lst1 for val in lsts if len(val) > 2]
Out[14]: ['TOM', 'AMY', 'JAMES', 'PYTHON']

Your question may be why the code for the first filtering is placed before the for of the second filtering code. In fact, it is like the nesting of a for loop. First, for lsts in lst1 gets a result, and then the result is traversed for val in lsts. You can refer to the following non-standard for loop process, mainly to let everyone better understand the process of nested derivation and the location of the derivation process at different levels.

for lsts in lst1:
    for val in lsts:
        if len(val) > 2:
            val.upper()

The above example is the case without filtering the inner list, which is relatively simple. If the inner list also needs to be filtered, it will be a little more complicated, but it will never change. Just add the condition to the first nested list comprehension.

In the following example, we add a list element with a length of 2 to the initial list, by first filtering the list with a length less than 3, and then converting to uppercase. At this time, the condition of if len(lsts) <3 is added to the first list comprehension.

val.upper() for lsts in lst2 if len(lsts) < 3 for val in lsts if len(val) > 2

In [15]: lst2 = [["Tom", "amy", "h", "James"], ["we", "Python", "3", (1, 2)], ["you", "Good"]]   # 嵌套列表
         [val.upper() for lsts in lst2 if len(lsts) < 3 for val in lsts if len(val) > 2]
Out[15]: ['YOU', 'GOOD']

This process can also be expressed by a similar for loop process that is not standardized, please refer to:

for lsts in lst1:
    if len(lsts) < 3:
        for val in lsts:
            if len(val) > 2:
                val.upper()

According to this rule, the nested list comprehension can theoretically be nested infinitely. However, it is not recommended to use more than two levels of nested comprehensions, because the purpose of the comprehension itself is for code readability. If there are too many levels of nesting, it may be counterproductive.

The above is the explanation of list comprehension, set comprehension and dictionary comprehension. I hope you can use it flexibly on the basis of understanding to improve the readability of our code.

Starting from the next one, we will use several articles to focus on the related content of functions. The functions mentioned here include not only built-in functions in Python, but also the most important and most important custom functions, so stay tuned.

Thanks for reading this article! If you have any questions, please leave a message and discuss together ^_^

To read other articles in the "Learning Python with You Hand in Hand" series, please follow the official account and click on the menu selection, or click the link below to go directly.

"Learning Python with You Hand in Hand" 1-Why learn Python?

"Learning Python with you hand in hand" 2-Python installation

"Learning Python with You Hand in Hand" 3-PyCharm installation and configuration

"Learning Python with You Hand in Hand" 4-Hello World!

"Learning Python with You Hand in Hand" 5-Jupyter Notebook

"Learning Python with You Hand in Hand" 6-String Identification

"Learning Python with You Hand in Hand" 7-Index of Strings

"Learning Python with You Hand in Hand" 8-String Slicing

"Learning Python with You Hand in Hand" 9-String Operations

"Learning Python with You Hand in Hand" 10-String Functions

"Learning Python with You Hand in Hand" 11-Formatted Output of Strings

"Learning Python with You Hand in Hand" 12-Numbers