Understanding Python's for loops

In this blog, we will discuss the principle of for loop in Python.

We'll start with a set of basic examples and its syntax, and we'll discuss the usefulness of the else block associated with the for loop.

Then we'll introduce iterator objects, iterators, and the iterator protocol, and learn how to create your own iterator objects and iterators.

After that, we'll discuss how to implement for loops using iterable objects and iterators, as well as implementing for loop logic using the iterator protocol using while loops.

Finally, we'll decompile a simple for loop and walk through the instructions that the Python interpreter executes when the for loop executes, to satisfy your curiosity. These help to understand the inner workings of the for loop runtime.

Python's for loop

The for statement is one of two statements in Python that perform iteration, the other being while. If you are not very familiar with Python iteration, Python iteration: for, while, break, and continue statements are a good starting point.

In Python, for loops are used to iterate over all elements of an iterable object. The statement section inside the loop is executed once for each element item of the iteration object. For the time being, the iteration object can be imagined as a collection of objects, and we can traverse the elements one by one. We will describe iterators and iteration objects in detail in the next section.

A simple for loop

Let's start with a simple for loop that iterates over a list of strings and prints each one.

As you can see, this loop actually goes through every word in the list and prints them. That is, on each iteration of the loop, the variable word is assigned as an element in the list, and then the block of code in the for statement is executed. Since a list is an ordered sequence of elements, the loop also traverses the elements in the same order.

for loop with else clause

A for loop in Python can optionally be associated with an else clause. The code block in the else clause is executed after the for loop has completed, that is, after all the elements in the iterable object have been traversed. Now let's see how to extend the previous example to include an else condition (clause).

When does the else clause apply?

You've noticed that the else clause is not executed until the for loop completes. So what is the point of the else block? Isn't the statement after the for loop also executed?

We often encounter such a situation, when a certain condition is met, the for loop is ended halfway. And if this condition has not been met, another set of statements is expected to be executed. We usually use a boolean type of marker implementation, here is an example.

Call result:

With the else block, we can avoid using the boolean flag found_item. Let's see how the above method can be rewritten using an else clause. Note that the else block is skipped if the break statement in the for loop is triggered.

So the else block is suitable for the case where there is a break statement in the for loop, and we want to execute some statement when the break condition is not triggered.

Otherwise, the statement associated with the else will only be executed at the end of the for loop. You will see this when looking at the decompiled bytecode in the last section of this article.

for loop syntax

Now that we've seen some simple examples, let's end this section with the syntax for a for loop.

Basically, for every element in the iterable, set_of_statements_1 is executed. Once all the elements have been iterated over, the controller jumps to the else block to execute set_of_statements_2.

Note that the else clause is optional. If no else clause is found, the loop ends when all elements have been traversed, and the controller moves to the next statement in the program.

Iterables and Iterators

iterable object

In the previous section, we used the term iterable to refer to the object being iterated over in a loop. Now let's try to understand what an iterable object in Python is.

In Python, an iterable object refers to any object that can be iterated over in a for loop. This means that an iterator should be returned when this object is passed as a parameter to the iter() method. Let's look at some common examples of built-in iteration in Python.

As you can see, when we call iter() on an iterable object, it returns an iterator object.

iterator

So what is an iterator? An iterator is defined in Python as an object representing streaming data. Basically, if we pass an object to the built-in next() method, it should return the next value from the streaming data associated with it. Once all elements have been traversed, it throws a StopIteration exception. Subsequent calls to the next() method will also throw a StopIteration exception.

Let's try it out with a list.

Iterators are also iterables! but…

One interesting thing to keep in mind is that iterators also support (mandatory to support the iterator protocol) the iter() method. This means that we can call the iter() method on an iterator and get its own iterator object.

So we can use it anywhere we expect an iterator. For example, for loops.

Note, however, that calling iter() on a container object like list will return a different iterator each time, while calling iter() on an iterator will simply return the same iterator.


So if you need to do multiple iterations, and replace a normal container or iterable with an iterator, the second time you'll see an empty container.

Iterate over a list twice

Note that this works as we expect.

Iterates over a list iterator twice

Note that the iterator has ended the first time through the loop, and the second time we see an empty container.

iterator protocol

Earlier we saw:

1. An iterable object that returns an iterator when passed as a parameter to the iter() method.

2. An iterator,

1. Returns its next element when passed as an argument to the next() method or throws a StopIteration exception when all elements have been traversed.

2. Returns itself when passed as an argument to the iter() method.

The iteration protocol is just a standard way of defining objects as iterators. We have already seen this protocol in action in the previous section. According to the protocol, iterators should define the following two methods:

1. next()

   1. Each time this method is called, the next element of the iterator should be returned. Once the elements are traversed, it should throw a StopIteration exception.

2. When we call the built-in function next(), this method is actually called internally.

3. iter()

   1. This method returns the iterator itself

4. When we call the built-in function iter(), this method is actually called internally.

Write your own iterator

Now that we know the principle of the iteration protocol, we can write our own iterator. Let's start with an example, below we create a Range class based on a given range and step size.

Let's see how it works in a for loop.

Note that instances of the Range class are both iterators and iterables.

Write an iterable object yourself

We can also create an additional iterable object based on the Range iterator. Its role is to return a new iterator whenever the iter() method is called, in this case, it should return a new Range object.

Use our RangeIterable in a for loop.

How for loops work

Now that we know what iterators and iterables are, let's take a look at how for loops work.

Look again at the previous example.

When we execute the above code block, the following things happen:

1. The iter() method is called on the list ["You", "are", "awesome!"] inside the for statement, and the result is an iterator.

2. Then call the next() method on the iterator and assign its return value to the variable word.

3. After that, the associated block of statements in the for loop is executed. In this example it prints word.

4. Steps 2 and 3 are repeated until the next() method throws StopIteration.

5. Once next() throws StopIteration, the controller jumps to the else clause (if present) and executes the block of statements associated with the else.

Note: If in step 3, the for loop statement encounters a break statement, the else block is skipped.

Use while statement to implement for loop logic

We can implement the previous logic using the while statement as follows.

The behavior of the while loop is actually the same as the for loop, the above code will have the following output.


Decompile the for loop

In this section, we will decompile the for loop and step through the instructions of the interpreter when executing the for loop. Here the dis module is used to decompile the for loop. In detail, we will use the dis.dis method to generate more readable bytecode.

We'll use the simple for loop example we've been using so far. Next write the file to the file for_loop.py.

We can call the dis.dis method to get more readable bytecode. Run the following command on the terminal.

More Python videos, materials, codes and group 660193417 can be obtained for free

Each column of the decompiled output represents the following:

1. Column 1: Number of lines of code.

2. Column 2: If it is a jump instruction, there is a ">>" symbol.

3. Column 3: Bytecode offset in bytes.

4. Column 4: The bytecode instruction itself.

5. Column 5: Displays the parameters of the command. If there is something in the parentheses, it just translates the parameter for better readability.

Now let's step through the decompiled bytecode and try to understand what's actually going on.

1. Line 1, "for word in ["You", "are", "awesome!"]:" translates to:

0 SETUP_LOOP 28 (to 30)

This statement pushes the block of code in the for loop onto the stack. This block of code will span 28 bytes to "30".

This means that if there is a break statement in the for loop, the controller will jump to offset "30". Notice how the else block is skipped when a break statement is encountered.

2 LOAD_CONST 0 ((‘You’, ‘are’, ‘awesome!’))

Next, the list is pushed to the top of the stack (TOS, after which TOS is used to denote the top of the stack or top element).

4 GET_ITER

This instruction implements "TOS = iter(TOS)". This means getting an iterator (currently TOS) from the list, and pushing the iterator to the TOS.

6 FOR_ITER 12 (to 20)

This instruction gets TOS as the current iterator and calls the next() method.

If the next() method produces a value, it is pushed onto the stack as TOS and the startle instruction "8 STORE_NAME" is executed.

Once next() indicates that the iterator has been traversed (ie, a StopIteration exception is thrown), the TOS (iterator) is popped from the stack and the bytecode counter is incremented by 12. This means that the controller jumps to the instruction "20 POP_BLOCK".

8 STORE_NAME 0 (word)

This instruction performs the conversion word = TOS, that is, the value returned by next() is assigned to the variable word.

2. Line 1, i.e. "print(word)" translates to:

10 LOAD_NAME 1 (print)

Push the callable method print onto the stack.

12 LOAD_NAME 0 (word)

Push the word on the stack as an argument to print.

14 CALL_FUNCTION 1

Call a function with positional arguments.

Like the instructions we saw, the parameters associated with the function appear in the TOS. All encountered arguments are popped before getting a callable pair (like print).

Once the callable is obtained, pass all parameters to it and call.

After the callable object is executed, push the return value to TOS, here is None.

16 POP_TOP

TOS (top of stack), which removes (pops) the return value of a function from the stack.

18 JUMP_ABSOLUTE 6

At this point the bytecode counter is "6", which means that the next instruction will execute "6 FOR_ITER". This is how you loop through the elements in the iterator.

Note that the instruction "6 FOR_ITER" ends the loop and jumps to "20 POP_BLOCK" once the elements in the iterator have been traversed.

20 POP_BLOCK

POP_BLOCK removes the code block set by "0 SETUP_LOOP" from the code block's stack.

3. Note that on line 3 (corresponding to else), there are no special instructions associated with it. The program controller sequentially executes the next instruction associated with the else.

4. Line 4, "print("See you later!")" translates to:

22 LOAD_NAME 1 (print)

Push the callable method associated with print onto the stack.

24 LOAD_CONST 1 (‘See you later!’)

Pushes the parameter object of the callable function onto the stack.

26 CALL_FUNCTION 1

The callable function and its parameters are popped off the stack, then the function is executed and its return value is pushed to the TOS.

28 POP_TOP

TOS (top of stack), which removes the return value of the function (None in this case) from the stack.

5. The following two instructions simply load the script's return value (None) onto the stack and return.

30 LOAD_CONST 2 (None)

32 RETURN_VALUE

oh! Now we have seen the decompiled instructions of the for loop. Hope this helps to better understand how the for loop works.

At the end, I recommend a very good learning tutorial to everyone, I hope it will help you learn Python!

Basic introductory tutorial recommendation: More Python video tutorials-Follow Station B: Red Panda Aiqiafan