Error Retries and Exponential Backoff in AWS

Error Retries and Exponential Backoff in AWS

https://docs.aws.amazon.com/general/latest/gr/api-retries.html

Numerous components on a network, such as DNS servers, switches, load balancers, and others can generate errors anywhere in the life of a given request. The usual technique for dealing with these error responses in a networked environment is to implement retries in the client application. This technique increases the reliability of the application and reduces operational costs for the developer.

Each AWS SDK implements automatic retry logic. The AWS SDK for Java automatically retries requests, and you can configure the retry settings using the ClientConfiguration class. For example, you might want to turn off the retry logic for a web page that makes a request with minimal latency and no retries. Use theClientConfiguration class and provide a maxErrorRetry value of 0 to turn off the retries.

If you're not using an AWS SDK, you should retry original requests that receive server (5xx) or throttling errors. However, client errors (4xx) indicate that you need to revise the request to correct the problem before trying again.

In addition to simple retries, each AWS SDK implements exponential backoff algorithm for better flow control. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses. You should implement a maximum delay interval, as well as a maximum number of retries. The maximum delay interval and maximum number of retries are not necessarily fixed values, and should be set based on the operation being performed, as well as other local factors, such as network latency.

Most exponential backoff algorithms use jitter (randomized delay) to prevent successive collisions. Because you aren't trying to avoid such collisions in these cases, you don't need to use this random number. However, if you use concurrent clients, jitter can help your requests succeed faster. For more information, see the blog post for Exponential Backoff and Jitter.

The following pseudo code shows one way to poll for a status using an incremental delay.

Do some asynchronous operation.

retries = 0

DO
    wait for (2^retries * 100) milliseconds

    status = Get the result of the asynchronous operation.

    IF status = SUCCESS
        retry = false
    ELSE IF status = NOT_READY
        retry = true
    ELSE IF status = THROTTLED
        retry = true
    ELSE
        Some other error occurred, so stop calling the API.
        retry = false
    END IF

    retries = retries + 1

WHILE (retry AND (retries < MAX_RETRIES))

The following code demonstrates how to implement this incremental delay in Java.

 1 public enum Results {
 2     SUCCESS, 
 3     NOT_READY, 
 4     THROTTLED, 
 5     SERVER_ERROR
 6 }
 7 
 8 /*
 9  * Performs an asynchronous operation, then polls for the result of the
10  * operation using an incremental delay.
11  */
12 public static void doOperationAndWaitForResult() {
13 
14     try {
15         // Do some asynchronous operation.
16         long token = asyncOperation();
17 
18         int retries = 0;
19         boolean retry = false;
20 
21         do {
22             long waitTime = Math.min(getWaitTimeExp(retries), MAX_WAIT_INTERVAL);
23 
24             System.out.print(waitTime + "\n");
25 
26             // Wait for the result.
27             Thread.sleep(waitTime);
28 
29             // Get the result of the asynchronous operation.
30             Results result = getAsyncOperationResult(token);
31 
32             if (Results.SUCCESS == result) {
33                 retry = false;
34             } else if (Results.NOT_READY == result) {
35                 retry = true;
36             } else if (Results.THROTTLED == result) {
37                 retry = true;
38             } else if (Results.SERVER_ERROR == result) {
39                 retry = true;
40             }
41             else {
42                 // Some other error occurred, so stop calling the API.
43                 retry = false;
44             }
45 
46         } while (retry && (retries++ < MAX_RETRIES));
47     }
48 
49     catch (Exception ex) {
50     }
51 }
52 
53 /*
54  * Returns the next wait interval, in milliseconds, using an exponential
55  * backoff algorithm.
56  */
57 public static long getWaitTimeExp(int retryCount) {
58 
59     long waitTime = ((long) Math.pow(2, retryCount) * 100L);
60 
61     return waitTime;
62 }