Table of Contents | Previous Section (6.1 Iteration Protocol) | Next Section (6.3 Producer/Consumer)

6.2 Custom iteration

Rebate website www.cpa5.cn

This section explores how to customize iterations using generator functions.

problem

Suppose you want to customize the iteration mode.

For example: Countdown:

>>> for x in countdown(10):
...   print(x, end=' ')
...
10 9 8 7 6 5 4 3 2 1
>>>

There is a simple way to do this.

Builder

A generator is a function that defines iteration:

def countdown(n):
    while n > 0:
        yield n
        n -= 1

Example:

>>> for x in countdown(10):
...   print(x, end=' ')
...
10 9 8 7 6 5 4 3 2 1
>>>

Any use of a yieldstatement of function known as a generator.

The behavior of generator functions is different from ordinary functions. Calling a generator function creates a generator object instead of executing the function immediately:

def countdown(n):
    # Added a print statement
    print('Counting down from', n)
    while n > 0:
        yield n
        n -= 1

>>> x = countdown(10)
# There is NO PRINT STATEMENT
>>> x
# x is a generator object
<generator object at 0x58490>
>>>

Generator function is only __next__()executed when the method is called:

>>> x = countdown(10)
>>> x
<generator object at 0x58490>
>>> x.__next__()
Counting down from 10
10
>>>

yieldA value is generated, but the suspend function is executed. Builder function in the next call to __next__()resume when the method (resume),

>>> x.__next__()
9
>>> x.__next__()
8

When the generator returns the last value, iterating again will trigger an error.

>>> x.__next__()
1
>>> x.__next__()
Traceback (most recent call last):
File "<stdin>", line 1, in ? StopIteration
>>>

Observation: The protocol implemented by the generator function is the same as the underlying protocol used by the for statement on lists, tuples, dictionaries, and files.

Exercise

Exercise 6.4: A simple generator

If you want a custom iteration, then you should always consider generator functions. Generator function is easy to write - Create a function required to perform iterative logic, and use yieldtransmission value.

For example, to create a generator that finds matching substrings in each line of a file:

>>> def filematch(filename, substr):
        with open(filename, 'r') as f:
            for line in f:
                if substr in line:
                    yield line

>>> for line in open('Data/portfolio.csv'):
        print(line, end='')

name,shares,price
"AA",100,32.20
"IBM",50,91.10
"CAT",150,83.44
"MSFT",200,51.23
"GE",95,40.37
"MSFT",50,65.10
"IBM",100,70.44
>>> for line in filematch('Data/portfolio.csv', 'IBM'):
        print(line, end='')

"IBM",50,91.10
"IBM",100,70.44
>>>

This is an interesting idea-you can hide custom processing in a function and apply the function to a for loop. The next example explores a more unusual situation.

Exercise 6.5: Monitor streaming data sources

The generator can be used to monitor real-time data sources (such as log files, stock market news). In this part, we will explore the idea of "using generators to monitor real-time data sources". First, please strictly follow the instructions below.

Data/stocksim.pyTo mimic stock market data, real-time data will continue to be written to the Data/stocklog.csvfile. Open a separate command line window, enter the Data/directory, then run the stocksim.pyprogram:

bash % python3 stocksim.py

If you are using a Windows system, then locate the stocksim.pyfile, and then double-click the file to run. Then, let's put that aside for the program (it has been running at that), open another command line window, view the simulation program is being (translation: stocksim.py) writing data Data/stocklog.csvfile (translation: If you are using a Linux system you can, go to the Data directory, and then use the tail -f stocklog.csvcommand to view). You should see a new line of text every few seconds is added to the Data/stocklog.csvfile. Similarly, let the program run in the background-the program will run for several hours (don't worry about this).

stocksim.pyAfter running the program, let's write a program to open Data/stocklog.csvthe file, move to the end of the file, and view the new output. Please create at Work directory follow.pyfile and put the following code in which:

# follow.py
import os
import time

f = open('Data/stocklog.csv')
f.seek(0, os.SEEK_END)   # Move file pointer 0 bytes from end of file

while True:
    line = f.readline()
    if line == '':
        time.sleep(0.1)   # Sleep briefly and retry
        continue
    fields = line.split(',')
    name = fields[0].strip('"')
    price = float(fields[1])
    change = float(fields[4])
    if change < 0:
        print(f'{name:>10s} {price:>10.2f} {change:>10.2f}')

Run the follow.py program, you'll see real-time stock quotes (stock ticker). follow.py The code in Unix-like system to view the log file tail -fcommand.

Note: In this example, readline()methods of use and generally read from a file with a slightly different embodiment the (usually forcircular). In this case, we use readline()the end to detect duplicate files, to see if new data is added ( readline()method returns the new data or an empty string).

Exercise 6.6: Use generators to generate data

View Exercise 6.5 code you will find that the first part of the code to generate a few lines of data, while whilecirculation at the end of the statement consumption data. One of the main features of generators is that you can move the code that generates data into reusable functions.

6.5 edit the code of practice, so that through the generator function follow(filename)reads the execution file. Please implement the changes so that the following code will work:

>>> for line in follow('Data/stocklog.csv'):
          print(line, end='')

... Should see lines of output produced here ...

Please modify the stock quote code so that the code looks like the following:

if __name__ == '__main__':
    for line in follow('Data/stocklog.csv'):
        fields = line.split(',')
        name = fields[0].strip('"')
        price = float(fields[1])
        change = float(fields[4])
        if change < 0:
            print(f'{name:>10s} {price:>10.2f} {change:>10.2f}')

Exercise 6.7: View stock portfolio

Modify the follow.pyprogram so that the program can view streaming stock data and print information stock portfolio in those stocks. Example:

if __name__ == '__main__':
    import report

    portfolio = report.read_portfolio('Data/portfolio.csv')

    for line in follow('Data/stocklog.csv'):
        fields = line.split(',')
        name = fields[0].strip('"')
        price = float(fields[1])
        change = float(fields[4])
        if name in portfolio:
            print(f'{name:>10s} {price:>10.2f} {change:>10.2f}')

Note: To be able to run this code, Portfoliothe class must support the inoperator. See Exercise 6.3, to ensure that Portfolioclass implements the __contains__()operator.

discuss

Here, you move an interesting iterative pattern (reading lines at the end of the file) into the function. follow()Functions are now completely universal utilities that can be used in any program. For example, you can use the follow()function to see the server log, debug log, similar to other data sources.