Retry — google-api-core documentation (original) (raw)

As of January 1, 2020 this library no longer supports Python 2 on the latest released version. Library versions released prior to that date will continue to be available. For more information please visit Python 2 support on Google Cloud.

Retry implementation for Google API client libraries.

class google.api_core.retry.AsyncRetry(predicate: typing.Callable[[Exception], bool] = <function if_exception_type..if_exception_type_predicate>, initial: float = 1.0, maximum: float = 60.0, multiplier: float = 2.0, timeout: typing.Optional[float] = 120.0, on_error: typing.Optional[typing.Callable[[Exception], typing.Any]] = None, **kwargs: typing.Any)[source]

Bases: google.api_core.retry.retry_base._BaseRetry

Exponential retry decorator for async coroutines.

This class is a decorator used to add exponential back-off retry behavior to an RPC call.

Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.

Parameters

__call__(func: Callable[..., Awaitable[_R]], on_error: Callable[[Exception], Any] | None = None) → Callable[_P, Awaitable[_R]][source]

Wrap a callable with retry behavior.

Parameters

Returns

A callable that will invoke func with retry

behavior.

Return type

Callable

class google.api_core.retry.AsyncStreamingRetry(predicate: typing.Callable[[Exception], bool] = <function if_exception_type..if_exception_type_predicate>, initial: float = 1.0, maximum: float = 60.0, multiplier: float = 2.0, timeout: typing.Optional[float] = 120.0, on_error: typing.Optional[typing.Callable[[Exception], typing.Any]] = None, **kwargs: typing.Any)[source]

Bases: google.api_core.retry.retry_base._BaseRetry

Exponential retry decorator for async streaming rpcs.

This class returns an AsyncGenerator when called, which wraps the target stream in retry logic. If any exception is raised by the target, the entire stream will be retried within the wrapper.

Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.

Important Note: when a stream is encounters a retryable error, it will silently construct a fresh iterator instance in the background and continue yielding (likely duplicate) values as if no error occurred. This is the most general way to retry a stream, but it often is not the desired behavior. Example: iter([1, 2, 1/0]) -> [1, 2, 1, 2, …]

There are two ways to build more advanced retry logic for streams:

  1. Wrap the target
    Use a target that maintains state between retries, and creates a different generator on each retry call. For example, you can wrap a grpc call in a function that modifies the request based on what has already been returned:
    async def attempt_with_modified_request(target, request, seen_items=[]):

    remove seen items from request on each attempt

    new_request = modify_request(request, seen_items)
    new_generator = await target(new_request)
    async for item in new_generator:
    yield item
    seen_items.append(item)

retry_wrapped = AsyncRetry(is_stream=True,...)(attempt_with_modified_request, target, request, [])

  1. Wrap the retry generator
    Alternatively, you can wrap the retryable generator itself before passing it to the end-user to add a filter on the stream. For example, you can keep track of the items that were successfully yielded in previous retry attempts, and only yield new items when the new attempt surpasses the previous ones:
    async def retryable_with_filter(target):
    stream_idx = 0

    reset stream_idx when the stream is retried

    def on_error(e):
    nonlocal stream_idx
    stream_idx = 0

    build retryable

    retryable_gen = AsyncRetry(is_stream=True, ...)(target)

    keep track of what has been yielded out of filter

    seen_items = []
    async for item in retryable_gen:
    if stream_idx >= len(seen_items):
    yield item
    seen_items.append(item)
    elif item != previous_stream[stream_idx]:
    raise ValueError("Stream differs from last attempt")"
    stream_idx += 1

filter_retry_wrapped = retryable_with_filter(target)

Parameters

__call__(func: Callable[..., AsyncIterable[_Y] | Awaitable[AsyncIterable[_Y]]], on_error: Callable[[Exception], Any] | None = None) → Callable[_P, Awaitable[AsyncGenerator[_Y, None]]][source]

Wrap a callable with retry behavior.

Parameters

Returns

A callable that will invoke func with retry

behavior.

Return type

Callable

class google.api_core.retry.Retry(predicate: typing.Callable[[Exception], bool] = <function if_exception_type..if_exception_type_predicate>, initial: float = 1.0, maximum: float = 60.0, multiplier: float = 2.0, timeout: typing.Optional[float] = 120.0, on_error: typing.Optional[typing.Callable[[Exception], typing.Any]] = None, **kwargs: typing.Any)[source]

Bases: google.api_core.retry.retry_base._BaseRetry

Exponential retry decorator for unary synchronous RPCs.

This class is a decorator used to add retry or polling behavior to an RPC call.

Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.

There are two important concepts that retry/polling behavior may operate on, Deadline and Timeout, which need to be properly defined for the correct usage of this class and the rest of the library.

Deadline: a fixed point in time by which a certain operation must terminate. For example, if a certain operation has a deadline “2022-10-18T23:30:52.123Z” it must terminate (successfully or with an error) by that time, regardless of when it was started or whether it was started at all.

Timeout: the maximum duration of time after which a certain operation must terminate (successfully or with an error). The countdown begins right after an operation was started. For example, if an operation was started at 09:24:00 with timeout of 75 seconds, it must terminate no later than 09:25:15.

Unfortunately, in the past this class (and the api-core library as a whole) has not been properly distinguishing the concepts of “timeout” and “deadline”, and thedeadline parameter has meant timeout. That is whydeadline has been deprecated and timeout should be used instead. If thedeadline parameter is set, it will override the timeout parameter. In other words, retry.deadline should be treated as just a deprecated alias forretry.timeout.

Said another way, it is safe to assume that this class and the rest of this library operate in terms of timeouts (not deadlines) unless explicitly noted the usage of deadline semantics.

It is also important to understand the three most common applications of the Timeout concept in the context of this library.

Usually the generic Timeout term may stand for one of the following actual timeouts: RPC Timeout, Retry Timeout, or Polling Timeout.

RPC Timeout: a value supplied by the client to the server so that the server side knows the maximum amount of time it is expected to spend handling that specific RPC. For example, in the case of gRPC transport, RPC Timeout is represented by setting “grpc-timeout” header in the HTTP2 request. The timeout property of this class normally never represents the RPC Timeout as it is handled separately by the google.api_core.timeoutmodule of this library.

Retry Timeout: this is the most common meaning of the timeout property of this class, and defines how long a certain RPC may be retried in case the server returns an error.

Polling Timeout: defines how long the client side is allowed to call the polling RPC repeatedly to check a status of a long-running operation. Each polling RPC is expected to succeed (its errors are supposed to be handled by the retry logic). The decision as to whether a new polling attempt needs to be made is based not on the RPC status code but on the status of the returned status of an operation. In other words: we will poll a long-running operation until the operation is done or the polling timeout expires. Each poll will inform us of the status of the operation. The poll consists of an RPC to the server that may itself be retried as per the poll-specific retry settings in case of errors. The operation-level retry settings do NOT apply to polling-RPC retries.

With the actual timeout types being defined above, the client libraries often refer to just Timeout without clarifying which type specifically that is. In that case the actual timeout type (sometimes also referred to as Logical Timeout) can be determined from the context. If it is a unary rpc call (i.e. a regular one) Timeout usually stands for the RPC Timeout (if provided directly as a standalone value) or Retry Timeout (if provided asretry.timeout property of the unary RPC’s retry config). ForOperation or PollingFuture in general Timeout stands for Polling Timeout.

Parameters

__call__(func: Callable[_P, _R], on_error: Callable[[Exception], Any] | None = None) → Callable[_P, _R][source]

Wrap a callable with retry behavior.

Parameters

Returns

A callable that will invoke func with retry

behavior.

Return type

Callable

class google.api_core.retry.RetryFailureReason(value)[source]

Bases: enum.Enum

The cause of a failed retry, used when building exceptions

class google.api_core.retry.StreamingRetry(predicate: typing.Callable[[Exception], bool] = <function if_exception_type..if_exception_type_predicate>, initial: float = 1.0, maximum: float = 60.0, multiplier: float = 2.0, timeout: typing.Optional[float] = 120.0, on_error: typing.Optional[typing.Callable[[Exception], typing.Any]] = None, **kwargs: typing.Any)[source]

Bases: google.api_core.retry.retry_base._BaseRetry

Exponential retry decorator for streaming synchronous RPCs.

This class returns a Generator when called, which wraps the target stream in retry logic. If any exception is raised by the target, the entire stream will be retried within the wrapper.

Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.

Important Note: when a stream encounters a retryable error, it will silently construct a fresh iterator instance in the background and continue yielding (likely duplicate) values as if no error occurred. This is the most general way to retry a stream, but it often is not the desired behavior. Example: iter([1, 2, 1/0]) -> [1, 2, 1, 2, …]

There are two ways to build more advanced retry logic for streams:

  1. Wrap the target
    Use a target that maintains state between retries, and creates a different generator on each retry call. For example, you can wrap a network call in a function that modifies the request based on what has already been returned:
    def attempt_with_modified_request(target, request, seen_items=[]):

    remove seen items from request on each attempt

    new_request = modify_request(request, seen_items)
    new_generator = target(new_request)
    for item in new_generator:
    yield item
    seen_items.append(item)

retry_wrapped_fn = StreamingRetry()(attempt_with_modified_request)
retryable_generator = retry_wrapped_fn(target, request) 2. Wrap the retry generator
Alternatively, you can wrap the retryable generator itself before passing it to the end-user to add a filter on the stream. For example, you can keep track of the items that were successfully yielded in previous retry attempts, and only yield new items when the new attempt surpasses the previous ones:
def retryable_with_filter(target):
stream_idx = 0
# reset stream_idx when the stream is retried
def on_error(e):
nonlocal stream_idx
stream_idx = 0
# build retryable
retryable_gen = StreamingRetry(...)(target)
# keep track of what has been yielded out of filter
seen_items = []
for item in retryable_gen():
if stream_idx >= len(seen_items):
seen_items.append(item)
yield item
elif item != seen_items[stream_idx]:
raise ValueError("Stream differs from last attempt")
stream_idx += 1
filter_retry_wrapped = retryable_with_filter(target)

Parameters

__call__(func: Callable[_P, Iterable[_Y]], on_error: Callable[[Exception], Any] | None = None) → Callable[_P, Generator[_Y, Any, None]][source]

Wrap a callable with retry behavior.

Parameters

Returns

A callable that will invoke func with retry

behavior.

Return type

Callable

google.api_core.retry.build_retry_error(exc_list: list[Exception], reason: google.api_core.retry.retry_base.RetryFailureReason, timeout_val: float | None, **kwargs: Any) → tuple[Exception, Exception | None][source]

Default exception_factory implementation.

Returns a RetryError if the failure is due to a timeout, otherwise returns the last exception encountered.

Parameters

Returns

a tuple of the exception to be raised, and the cause exception if any

Return type

google.api_core.retry.exponential_sleep_generator(initial: float, maximum: float, multiplier: float = 2.0)[source]

Generates sleep intervals based on the exponential back-off algorithm.

This implements the Truncated Exponential Back-off algorithm.

Parameters

Yields

float – successive sleep intervals.

google.api_core.retry.if_exception_type(*exception_types: type[Exception]) → Callable[[Exception], bool][source]

Creates a predicate to check if the exception is of a given type.

Parameters

exception_types (Sequence[type()]) – The exception types to check for.

Returns

A predicate that returns True if the provided

exception is of the given type(s).

Return type

Callable[Exception]

google.api_core.retry.if_transient_error(exception: Exception) → bool

Bound predicate for checking an exception type.

google.api_core.retry.retry_target(target: Callable[[], _R], predicate: Callable[[Exception], bool], sleep_generator: Iterable[float], timeout: float | None = None, on_error: Callable[[Exception], None] | None = None, exception_factory: Callable[[list[Exception], RetryFailureReason, float | None], tuple[Exception, Exception | None]] = <function build_retry_error>, **kwargs)[source]

Call a function and retry if it fails.

This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper Retry.

Parameters

Returns

the return value of the target function.

Return type

Any

Raises

async google.api_core.retry.retry_target_async(target: Callable[[], Awaitable[_R]], predicate: Callable[[Exception], bool], sleep_generator: Iterable[float], timeout: float | None = None, on_error: Callable[[Exception], None] | None = None, exception_factory: Callable[[list[Exception], RetryFailureReason, float | None], tuple[Exception, Exception | None]] = <function build_retry_error>, **kwargs)

Await a coroutine and retry if it fails.

This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper Retry.

Parameters

Returns

the return value of the target function.

Return type

Any

Raises

google.api_core.retry.retry_target_stream(target: Callable[_P, Iterable[_Y]], predicate: Callable[[Exception], bool], sleep_generator: Iterable[float], timeout: Optional[float] = None, on_error: Optional[Callable[[Exception], None]] = None, exception_factory: Callable[[List[Exception], RetryFailureReason, Optional[float]], Tuple[Exception, Optional[Exception]]] = <function build_retry_error>, init_args: tuple = (), init_kwargs: dict = {}, **kwargs) → Generator[_Y, Any, None][source]

Create a generator wrapper that retries the wrapped stream if it fails.

This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper Retry.

Parameters

Returns

A retryable generator that wraps the target generator function.

Return type

Generator

Raises

async google.api_core.retry.retry_target_stream_async(target: Callable[_P, AsyncIterable[_Y] | Awaitable[AsyncIterable[_Y]]], predicate: Callable[[Exception], bool], sleep_generator: Iterable[float], timeout: float | None = None, on_error: Callable[[Exception], None] | None = None, exception_factory: Callable[[list[Exception], RetryFailureReason, float | None], tuple[Exception, Exception | None]] = <function build_retry_error>, init_args: tuple = (), init_kwargs: dict = {}, **kwargs) → AsyncGenerator[_Y, None]

Create a generator wrapper that retries the wrapped stream if it fails.

This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper AsyncRetry.

Parameters

Returns

A retryable generator that wraps the target generator function.

Return type

AsyncGenerator

Raises

Retry in AsyncIO

class google.api_core.retry_async.AsyncRetry(predicate: typing.Callable[[Exception], bool] = <function if_exception_type..if_exception_type_predicate>, initial: float = 1.0, maximum: float = 60.0, multiplier: float = 2.0, timeout: typing.Optional[float] = 120.0, on_error: typing.Optional[typing.Callable[[Exception], typing.Any]] = None, **kwargs: typing.Any)[source]

Bases: google.api_core.retry.retry_base._BaseRetry

Exponential retry decorator for async coroutines.

This class is a decorator used to add exponential back-off retry behavior to an RPC call.

Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.

Parameters

__call__(func: Callable[..., Awaitable[_R]], on_error: Callable[[Exception], Any] | None = None) → Callable[_P, Awaitable[_R]][source]

Wrap a callable with retry behavior.

Parameters

Returns

A callable that will invoke func with retry

behavior.

Return type

Callable

google.api_core.retry_async.exponential_sleep_generator(initial: float, maximum: float, multiplier: float = 2.0)[source]

Generates sleep intervals based on the exponential back-off algorithm.

This implements the Truncated Exponential Back-off algorithm.

Parameters

Yields

float – successive sleep intervals.

google.api_core.retry_async.if_exception_type(*exception_types: type[Exception]) → Callable[[Exception], bool][source]

Creates a predicate to check if the exception is of a given type.

Parameters

exception_types (Sequence[type()]) – The exception types to check for.

Returns

A predicate that returns True if the provided

exception is of the given type(s).

Return type

Callable[Exception]

google.api_core.retry_async.if_transient_error(exception: Exception) → bool

Bound predicate for checking an exception type.

async google.api_core.retry_async.retry_target(target: Callable[[], Awaitable[_R]], predicate: Callable[[Exception], bool], sleep_generator: Iterable[float], timeout: float | None = None, on_error: Callable[[Exception], None] | None = None, exception_factory: Callable[[list[Exception], RetryFailureReason, float | None], tuple[Exception, Exception | None]] = <function build_retry_error>, **kwargs)[source]

Await a coroutine and retry if it fails.

This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper Retry.

Parameters

Returns

the return value of the target function.

Return type

Any

Raises