Python3, it does magic, from the most powerful error retry library.

1 Introduction

Xiao Diaosi : Brother Yu, I have encountered some problems recently.
Xiaoyu : Oh, you have encountered a lot of problems, otherwise you can't take the initiative to tell me.
Xiao Diaosi : Alas~~ On the road of trial and error, the more I go, the more confused I am.
Xiaoyu : That's because you didn't use the right method.
Little Diaosi : Is there a way to do this?
Xiaoyu : You don't understand that.

Xiao Diaosi : Then you can teach me quickly~
Xiaoyu : retry, you can learn about it.
Xiao Diaosi : The usage of retry is single, which does not meet my requirements (wei) and request (kou).
Xiaoyu : Then try tenacity Xiaodiaosi
: No, teach me.
Xiaoyu : ... This is another losing day.

2、 tenacity

When we call web interfaces or crawlers, we often encounter some occasional request failures. At this time, we may think of looping until success; or printing an error message;
this approach is not available in simple programs. Wrong, but in complex or large projects, this approach obviously has many drawbacks.
And we need to use an error retry method, the old bird should think of tenactiy .
That's right, tenacity is probably the best third-party retry library in python at the moment
, so let's take a look at it next.

2.1 Installation

As a third-party library, our first step must be to install

pip install tenacity

Other ways to install :

" Python3, choose Python to automatically install third-party libraries, and say goodbye to pip! ! "
" Python3: I only use one line of code to import all Python libraries! ! 》

2.2 Basic usage

The core function of tenacity's error retry is implemented by its retry decorator.
When no parameters are passed to the retry decorator by default, it will throw errors and retry from time to time during the operation of the function it decorates.

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import random
from tenacity import retry

@retry
def demo_one():
	a = random.random()
	print(f'{
      
      a}')
	
	if a >= 0.1:
		raise Exception
demo_one()

Running results

Through the results, you can see that the function body generates a random number between 0 and 1 each time. When the random number does not exceed 0.1, it will stop throwing errors, otherwise it will be caught by tenacity. Act out and try again now.

2.3 Maximum number of retries

Since our time is precious, the number of retries also needs to be limited.

We can add an endpoint to our "endless" error retry process by taking advantage of the stop_after_attempt
function in tenacity, passed in as the stop parameter in retry(), where stop_after_attempt() accepts an integer input as " maximum " number of retries ":

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

from tenacity import retry,stop_after_attempt

#设置错误重试，3次
@retry(stop = stop_after_attempt(3))

def demo_two():
    print(f'函数执行')

    raise  Exception

demo_two()

Running results

Through the results, it can be seen that the 4th continuation of execution officially throws the corresponding Exception error in the function and ends the retry process.

2.4 Maximum retry time

In addition to setting the number of errors, you can also set the maximum retry time, which is set
by the stop_after_delay() function. If this time is exceeded, the retry process will end.

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import  time
from tenacity import  retry,stop_after_delay

#设置重试最大超时时长为4秒
@retry(stop = stop_after_delay(4))

def demo_three():
	#每次时间间隔为2秒
    time.sleep(2)
    print(f'已过去 {
      
      time.time() - start_time} 秒')
    raise  Exception

start_time = time.time()
demo_three()

operation result

2.5 Combined Retry Stop Conditions

If you need to add the maximum number of retries and the maximum timeout limit at the same time ,
you only need to use the |operator then pass in the stop parameter of retry().

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import  time
import random
from tenacity import retry,stop_after_attempt,stop_after_delay

#函数执行重试超过3秒或次数大于5次时均可以结束重试
@retry(stop =(stop_after_attempt(3) | stop_after_delay(5)))
def  demo_four():
    time.sleep(random.random())
    print(f'已过去 {
      
      time.time() - start_time} 秒')

    raise Exception

#开始时间
start_time = time.time()
demo_four()

operation result

2.6 Adjacent retry interval

Tenacity provides a series of very practical functions, together with the wait parameter of retry(), to help us properly handle the time interval between adjacent retries. The more practical ones are mainly the following two methods:

• fixed time interval ;
• random time interval ;

2.6.1 Fixed time interval

Use **wait_fixed()** in tenacity to set a fixed wait interval in seconds between adjacent retries

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import time
from tenacity import  retry,wait_fixed,stop_after_attempt

# 设置重试等待间隔为1秒
@retry(wait = wait_fixed(1),stop = stop_after_attempt(3))
def demo_five():
    print(f'已过去 {
      
      time.time() - start_time} 秒')

    raise  Exception
#开始时间
start_time = time.time()
demo_five()

operation result

2.6.2 Random time interval

In addition to setting a fixed time interval, tenacity can also set a uniformly distributed random number for adjacent retries through **wait_random()**, just set the range of uniform distribution:

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import time
from tenacity import  retry,wait_random,stop_after_attempt

# 设置重试等待间隔为1到3之间的随机数
@retry(wait = wait_random(min=1,max=3),stop = stop_after_attempt(4))
def demo_six():
    print(f'已过去 {
      
      time.time() - start_time} 秒')

    raise  Exception
#开始时间
start_time = time.time()
demo_six()

operation result

2.7 Customize whether to trigger a retry

The default strategy of retry() in tenacity is to retry when the execution of the function it decorates " throws any error ",
but in some cases, it may be necessary to catch/ignore specific error types, or to Catching of abnormal calculation results.
Related useful functions are also built into tenacity:

• Catch or ignore specific error types ;
• Custom function result condition judgment function ;

2.7.1 Ignore specific error types

Use retry_if_exception_type() and retry_if_not_exception_type() in tenacity, together with the retry parameter of retry(), to catch or ignore specific error types:

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

'''
捕捉或忽略特定的错误类型
'''
from tenacity import retry,retry_if_exception_type,retry_if_not_exception_type

#retry_if_exception_type()
@retry(retry=retry_if_exception_type(FileExistsError))
def demo_seven():
    raise TimeoutError

demo_seven()

#retry_if_not_exception_type()
@retry(retry=retry_if_not_exception_type(FileNotFoundError))
def demo_eight():
    raise FileNotFoundError

demo_eight()

operation result

demo_seven()

demo_eight()

2.7.2 User-defined function result condition judgment function

We can write an additional conditional judgment function, and cooperate with retry_if_result() in tenacity to implement custom conditional judgment on the return result of the function, and the retry operation will be triggered only when True is returned:

code example

# -*- coding:utf-8 -*-
# @Time   : 2022-04-04
# @Author : carl_DJ

import  random
from tenacity import  retry,retry_if_result

@retry(retry = retry_if_result(lambda x: x >= 0.1))
def demo_nine():
    a =  random.random()
    print(f'{
      
      a}')
    return a

demo_nine()

operation result

3. Summary

Today's sharing is here.
Isn't it a strange posture, it has been added again.
The wrong retry pose was also added.
Follow Xiaoyu to learn more about python third-party libraries.