Retry behavior - AWS SDKs and Tools (original) (raw)

Retry behavior includes settings regarding how the SDKs attempt to recover from failures resulting from requests made to AWS services.

Configure this functionality by using the following:

retry_mode - shared AWS config file setting

AWS_RETRY_MODE - environment variable

aws.retryMode - JVM system property: Java/Kotlin only

Specifies how the SDK or developer tool attempts retries.

Default value: This value is specific to your SDK. Check your specific SDK guide or your SDK's code base for its default retry_mode.

Valid values:

• standard – (Recommended) The recommended set of retry rules across AWS SDKs. This mode includes a standard set of errors that are retried, and automatically adjusts the number of retries to maximize availability and stability. This mode is safe for use in multi-tenant applications. The default maximum number of attempts with this mode is three, unless max_attempts is explicitly configured.
• adaptive – A retry mode, appropriate only for specialized use-cases, that includes the functionality of standard mode as well as automatic client-side rate limiting. This retry mode is not recommended for multi-tenant applications, unless you take care to isolate application tenants. See Choosing between standard and adaptive retry modes for more information. This mode is experimental and it might change behavior in the future.
• legacy – (Not Recommended) Specific to your SDK (check your specific SDK guide or your SDK's code base).

max_attempts - shared AWS config file setting

AWS_MAX_ATTEMPTS - environment variable

aws.maxAttempts - JVM system property: Java/Kotlin only

Specifies the maximum number attempts to make on a request.

Default value: If this value is not specified, its default depends on the value of the retry_mode setting:

• If retry_mode is legacy – Uses a default value specific to your SDK (check your specific SDK guide or your SDK's code base for max_attempts default).
• If retry_mode is standard – Makes three attempts.
• If retry_mode is adaptive – Makes three attempts.

Valid values: Number greater than 0.

Choosing between standard and adaptive retry modes

We recommend you use the standard retry mode unless you are certain that your usage is better suited foradaptive.

Note

The adaptive mode assumes that you are pooling clients based on the scope at which the backend service may throttle requests. If you don't do this, throttles in one resource could delay requests for an unrelated resource if you are using the same client for both resources.

If you choose to use adaptive mode, your application must construct clients that are designed around each resource that might be throttled. A resource, in this case, is finer-tuned than just thinking of each AWS service. AWS services can have additional dimensions that they use to throttle requests. Let's use the Amazon DynamoDB service as an example. DynamoDB uses AWS Region plus the table being accessed to throttle requests. This means that one table that your code is accessing might be throttled more than others. If your code used the same client to access all the tables, and requests to one of those tables is throttled, then adaptive retry mode will reduce the request rate for all tables. Your code should be designed to have one client per Region-and-table pair. If you experience unexpected latency when using adaptive mode, see the specific AWS documentation guide for the service you are using.

Retry mode implementation details

The AWS SDKs make use of token buckets to decide whether a request should be retried and (in the case of theadaptive retry mode) how quickly requests should be sent. Two token buckets are used by the SDK: a retry token bucket and a request rate token bucket.

• The retry token bucket is used to determine whether the SDK should temporarily disable retries in order to protect the upstream and downstream services during outages. Tokens are acquired from the bucket before retries are attempted, and tokens are returned to the bucket when requests succeed. If the bucket is empty when a retry is attempted, the SDK will not retry the request.
• The request rate token bucket is used only in the adaptive retry mode to determine the rate at which to send requests. Tokens are acquired from the bucket before a request is sent, and tokens are returned to the bucket at a dynamically-determined rate based on throttling responses returned by the service.

Following is the high-level pseudocode for both the standard and adaptive retry modes:

MakeSDKRequest() {
  attempts = 0
  loop {
    GetSendToken()
    response = SendHTTPRequest()
    RequestBookkeeping(response)
    if not Retryable(response)
      return response
    attempts += 1
    if attempts >= MAX_ATTEMPTS:
      return response
    if not HasRetryQuota(response)
      return response
    delay = ExponentialBackoff(attempts)
    sleep(delay)
  }
}

Following are more details about the components used in the pseudocode:

GetSendToken:

This step is only used in adaptive retry mode. This step acquires a token from the request rate token bucket. If a token is not available, it will wait for one to become available. Your SDK might have configuration options available to fail the request instead of wait. Tokens in the bucket are refilled at a rate that is determined dynamically, based on the number of throttling responses received by the client.

SendHTTPRequest:

This step sends the request to AWS. Most AWS SDKs use an HTTP library that uses connection pools to reuse an existing connection when making an HTTP request. Generally, connections are reused if a request failed due to throttling errors but not if a request fails due to a transient error.

RequestBookkeeping:

Tokens are added to the token bucket if the request is successful. For adaptive retry mode only, the fill rate of the request rate token bucket is updated based on the type of response received.

Retryable:

This step determines whether a response can be retried based on the following:

• The HTTP status code.
• The error code returned from the service.
• Connection errors, defined as any error received by the SDK in which an HTTP response from the service is not received.

Transient errors (HTTP status codes 400, 408, 500, 502, 503, and 504) and throttling errors (HTTP status codes 400, 403, 429, 502, 503, and 509) can all potentially be retried. SDK retry behavior is determined in combination with error codes or other data from the service.

MAX_ATTEMPTS:

The default number of maximum attempts is set by the retry_mode setting, unless overridden by the max_attempts setting.

HasRetryQuota

This step acquires a token from the retry token bucket. If the retry token bucket is empty, the request will not be retried.

ExponentialBackoff

For an error that can be retried, the retry delay is calculated using truncated exponential backoff. The SDKs use truncated binary exponential backoff with jitter. The following algorithm shows how the amount of time to sleep, in seconds, is defined for a response for request i:

seconds_to_sleep_i = min(b*r^i, MAX_BACKOFF)

In the preceding algorithm, the following values apply:

b = random number within the range of: 0 <= b <= 1

r = 2

MAX_BACKOFF = 20 seconds for most SDKs. See your specific SDK guide or source code for confirmation.

Support by AWS SDKs and tools

The following SDKs support the features and settings described in this topic. Any partial exceptions are noted. Any JVM system property settings are supported by the AWS SDK for Java and the AWS SDK for Kotlin only.