Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions packages/aws-durable-execution-sdk-python-extras/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ Every MicroVM failure is a `MicrovmError`. Catch the base class for all of them,

| Error | When |
|---|---|
| `MicrovmLaunchError` | `RunMicrovm` failed after all retries. A MicroVM can still run the job. The list below the table says when. |
| `MicrovmLaunchError` | `RunMicrovm` failed after all retries, or the launch request was not valid, for example with an empty `image_identifier` or `execution_role_arn`. A MicroVM can still run the job. The list below the table says when. |
| `MicrovmDeliveryError` | HTTP delivery failed after all retries, or the route answered a 4xx status such as 404, or a redirect. |
| `MicrovmNotRunningError` | A `MicrovmDeliveryError`: the MicroVM is terminating, terminated, or removed. It is not retried. A session that checks the MicroVM state before each job reports it. Any job over HTTP also reports it when `CreateMicrovmAuthToken` reports the MicroVM removed. See [Suspending an idle MicroVM](#suspending-an-idle-microvm). |
| `MicrovmJobFailedError` | The job handler in the MicroVM failed, or the MicroVM sent a result that is not valid JSON. The message includes the error type that the MicroVM reported. |
Expand All @@ -123,7 +123,7 @@ except MicrovmError:
... # Any other MicroVM failure.
```

The class is the same on the first run and on every replay, so code that branches on it stays deterministic. A `microvm` call inside your own child context, or inside a `map` or `parallel` item, reaches the code around that context as the SDK's `ChildContextError`. Its `__cause__` is the MicroVM error, so check `isinstance(error.__cause__, MicrovmTimeoutError)` there. An invalid name or config raises the SDK's `ValidationError`, and an input that is not JSON-serializable raises `TypeError`, before any durable operation starts. `Duration` raises the same `ValidationError` for a negative value.
The class is the same on the first run and on every replay, so code that branches on it stays deterministic. A `microvm` call inside your own child context, or inside a `map` or `parallel` item, reaches the code around that context as the SDK's `ChildContextError`. Its `__cause__` is the MicroVM error, so check `isinstance(error.__cause__, MicrovmTimeoutError)` there. An invalid name or config raises the SDK's `ValidationError`, and an input that is not JSON-serializable raises `TypeError`, before any durable operation starts. `Duration` raises the same `ValidationError` for a negative value. The operation does not check `image_identifier` and `execution_role_arn` before it starts. boto3 checks them against the API model before it sends `RunMicrovm`, and the service checks them again. So an invalid value fails the launch step at once with `MicrovmLaunchError`, after the callback step.

A failed AWS call keeps its request ID in the error message, in the form `<Operation> request ID: <id>.`, for example `RunMicrovm request ID: <id>.`. A support case can quote it. This applies to `RunMicrovm`, `GetMicrovm`, `ResumeMicrovm`, `CreateMicrovmAuthToken`, and `TerminateMicrovm`. The SDK checkpoints the message of a failed step, and not its `__cause__`. So the request ID is in the message, the handler sees it on the first run and on replay, and the execution history keeps it. A failed `TerminateMicrovm` call does not fail the operation. So a `TerminateMicrovm` request ID appears only in the terminate log entry and in the terminate step record. A message has no request ID when the answer carries none, for example after a connection failure. A message also has no request ID when the error is not the answer of the Lambda MicroVMs call. For example, a `client` with role-assuming credentials calls STS `AssumeRole` inside each call, and an STS error carries the request ID of STS. The operation recognizes such an error by its `operation_name`. The step raises a copy of the botocore error, of the same class, with the same `response`, and with the request ID in its message. So `default_microvm_retry_strategy` decides as before. A custom `retry_strategy` receives the error that botocore raised, without the request ID. So a strategy that matches the message, such as `create_retry_strategy(RetryStrategyConfig(retryable_errors=[...]))`, decides on the message of the service. A `ResourceNotFoundException` from `GetMicrovm`, `ResumeMicrovm`, or `CreateMicrovmAuthToken` is the exception. It means that the MicroVM no longer exists. So the operation raises `MicrovmStateError` instead, and the strategy receives that error. Its `__cause__` is the botocore error. An error that the operation creates itself, such as `MicrovmStateError` or "RunMicrovm returned no microvmId", has the request ID in its message when the strategy receives it. A custom client can raise one error object for several calls. Each step message then names only its own call, and the object keeps its message.

Expand Down Expand Up @@ -240,7 +240,7 @@ A failed `vm.invoke` raises its MicroVM error inside the handler. The handler ca
1. A MicroVM error that the handler raises itself reaches the caller with its own class, such as `MicrovmJobFailedError`.
2. Any other error reaches the caller as the SDK's `ChildContextError`, as from `run_in_child_context`. This includes a MicroVM error that leaves a child context that the handler created: that child context records `ChildContextError` as its error type. The caller then gets the MicroVM error's message, but not its class. Inside the handler, the `ChildContextError` of that child context has the MicroVM error as its `__cause__`. An SDK error that leaves the handler, such as the `StepError` of a failed step, also reaches the caller as `ChildContextError`, with the SDK error as its `__cause__`. This rule matches `run_in_child_context`. The JavaScript operation differs here: it passes an SDK error to the caller with its own class.

An invalid job raises `ValidationError` or `TypeError` from `vm.invoke`, before the job's durable operations start. The handler can catch it too.
An invalid job raises `ValidationError` or `TypeError` from `vm.invoke`, before the job's durable operations start. The handler can catch it too. A call without the context, `vm.invoke("build", job_input=..., options=...)`, raises Python's `TypeError`. A first argument that is not a durable context, as in `vm.invoke("build", "build", ...)`, raises `AttributeError`.

The handler's return value is checkpointed as the session's result. So it must be serializable by the SDK, and it counts toward the checkpoint size limit.

Expand Down Expand Up @@ -333,14 +333,15 @@ The MicroVM operations handle the network, the service, and the worker lifecycle

### Sessions

1. Pass `vm.invoke` the context that the session handler receives, or a context nested in it. A job created in another context runs, and the next replay fails the execution with `NonDeterministicExecutionError`. `vm.invoke` raises `TypeError` only for the context that `microvm_session` was called with. It does not detect any other wrong context. See [`microvm_session`](#microvm_session-run-several-jobs-in-one-microvm).
2. If you set `auto_suspend_idle_time`, meet three conditions:
1. Call `vm.invoke(ctx, name, job_input=..., options=...)` with the context that the session handler receives, or a context nested in it. A job created in another context runs, and the next replay fails the execution with `NonDeterministicExecutionError`. `vm.invoke` raises `TypeError` only for the context that `microvm_session` was called with. It does not detect any other wrong context. See [`microvm_session`](#microvm_session-run-several-jobs-in-one-microvm).
2. Pass a function as `handler`. `microvm_session` does not check it before it starts. A value that is not callable raises Python's `TypeError` when the session calls it, after the launch. The session then terminates the MicroVM, and the caller receives `ChildContextError`. So the mistake costs one MicroVM boot. mypy reports it before that.
3. If you set `auto_suspend_idle_time`, meet three conditions:
- `auto_suspend_on_idle` is `True`.
- The idle time is at least 10 seconds.
- The idle time is at most the session `timeout`.

A value that breaks any one condition raises `ValidationError` when `microvm_session` is called, before any durable operation starts. See [Suspending an idle MicroVM](#suspending-an-idle-microvm).
3. Change `auto_suspend_on_idle` or `idle_policy` only through a new version, as in [Deploys and versions](#deploys-and-versions). The launch records the resume mode, and a deploy changes it only for sessions that launch after it. Every job of a session that launched with suspending needs `lambda:GetMicrovm` and `lambda:ResumeMicrovm`, also after a deploy turns suspending off. So remove them from the function's role only as [Suspending an idle MicroVM](#suspending-an-idle-microvm) describes.
4. Change `auto_suspend_on_idle` or `idle_policy` only through a new version, as in [Deploys and versions](#deploys-and-versions). The launch records the resume mode, and a deploy changes it only for sessions that launch after it. Every job of a session that launched with suspending needs `lambda:GetMicrovm` and `lambda:ResumeMicrovm`, also after a deploy turns suspending off. So remove them from the function's role only as [Suspending an idle MicroVM](#suspending-an-idle-microvm) describes.

### Retry strategies

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

from __future__ import annotations

from collections.abc import Callable, Mapping
from collections.abc import Callable
from typing import Literal

from aws_durable_execution_sdk_python.retries import RetryDecision
Expand Down Expand Up @@ -58,14 +58,14 @@ def request_id_sentence(operation: MicrovmApiOperation, response: object) -> str

``response`` is the output of a boto3 call, or the ``response`` of a
``ClientError``. Both carry the request ID in
``ResponseMetadata.RequestId``. A custom client can return an output
without it. So the function returns ``""`` when ``response`` has no
request ID that is a non-empty string.
``ResponseMetadata.RequestId``. A custom client or a fake can return an
output that is not a dict, or a dict without ``ResponseMetadata``. So the
function returns ``""`` when ``response`` has no request ID that is a
non-empty string. A ``ResponseMetadata`` value that is neither a dict
nor empty raises ``AttributeError``. boto3 always returns a dict there.
"""
metadata = (
response.get("ResponseMetadata") if isinstance(response, Mapping) else None
)
request_id = metadata.get("RequestId") if isinstance(metadata, Mapping) else None
data = response if isinstance(response, dict) else {}
request_id = (data.get("ResponseMetadata") or {}).get("RequestId")
if isinstance(request_id, str) and request_id:
return f"{operation} request ID: {request_id}."
return ""
Expand Down Expand Up @@ -143,8 +143,11 @@ class of ``error``. A wrapper of another class would change the
- ``error`` is not the answer of ``operation``. See
:func:`is_microvms_service_error`.
- ``error`` has no request ID.
- The class of ``error`` builds ``str()`` from something other than
``args``. The copy would not show the request ID.

A class of ``error`` can build ``str()`` from something other than
``args``. Then neither ``str()`` of the copy nor ``str()`` of ``error``
contains the request ID. The step records the same message either way, and the
strategy still receives ``error``. So the function returns the copy.
"""
if not is_microvms_service_error(operation, error):
return error
Expand All @@ -154,8 +157,6 @@ class of ``error``. A wrapper of another class would change the
labelled = type(error).__new__(type(error))
labelled.__dict__.update(vars(error))
labelled.args = (message,)
if str(labelled) != message:
return error
labelled.__cause__ = error
labelled.__dict__["_original_error"] = error
return labelled
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import time
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING, Any, cast

import boto3
from botocore.config import Config
Expand Down Expand Up @@ -258,7 +258,8 @@ def microvm(
value.
TypeError: When ``job_input`` is not JSON-serializable. No durable
operation starts.
MicrovmLaunchError: When ``RunMicrovm`` fails after all retries.
MicrovmLaunchError: When ``RunMicrovm`` fails after all retries, or
rejects the config, for example an empty ``image_identifier``.
MicrovmNotRunningError: When ``CreateMicrovmAuthToken`` reports that
the MicroVM no longer exists. It is a ``MicrovmDeliveryError``.
MicrovmDeliveryError: When the HTTP delivery fails after all retries,
Expand Down Expand Up @@ -302,9 +303,14 @@ def step_retry_strategy(
)


def _validate_base(name: str, config: MicrovmBaseConfig, timeout: Duration) -> int:
def _validate_base(name: str, timeout: Duration) -> int:
"""Validate the settings that every MicroVM operation shares.

``image_identifier`` and ``execution_role_arn`` are not checked here.
The boto3 client rejects an empty value before it sends ``RunMicrovm``,
and the service rejects any other invalid value. Either way, the launch
step fails, and the operation raises ``MicrovmLaunchError``.

Returns:
The timeout in seconds.

Expand All @@ -314,12 +320,6 @@ def _validate_base(name: str, config: MicrovmBaseConfig, timeout: Duration) -> i
if not isinstance(name, str) or not name:
msg = "a MicroVM operation requires a non-empty name"
raise ValidationError(msg)
if not config.image_identifier:
msg = f'MicroVM "{name}": image_identifier is required'
raise ValidationError(msg)
if not config.execution_role_arn:
msg = f'MicroVM "{name}": execution_role_arn is required'
raise ValidationError(msg)
return _validate_timeout(name, "timeout", timeout)


Expand Down Expand Up @@ -500,20 +500,14 @@ def deliver(
port = DEFAULT_MICROVM_PORT if request.port is None else request.port

def attempt(step_context: StepContext) -> None:
# The launch records an endpoint for every HTTP delivery. So a
# missing endpoint is a defect, not a transient failure. The
# default retry strategy does not retry RuntimeError.
if launched.endpoint is None:
msg = (
f"the launch recorded no endpoint for MicroVM "
f"{launched.microvm_id}, so the job cannot be sent over HTTP"
)
raise RuntimeError(msg)
client = self._client()
# The state check before delivery and the request's retries share
# one window. So the wait for a resume counts toward it.
window_started_at = time.monotonic()
endpoint = launched.endpoint
# _launch_call raises when RunMicrovm returns no endpoint for an
# HTTP delivery. The launch step checkpoints only what
# _launch_call returned. So an HTTP launch always has an endpoint.
endpoint = cast("str", launched.endpoint)
recheck: Callable[[float], str | None] | None = None
if resume:
require_time_for_call(remaining_seconds)
Expand Down Expand Up @@ -727,7 +721,7 @@ def create(
Region cannot be determined.
TypeError: When ``job_input`` is not JSON-serializable.
"""
timeout_seconds = _validate_base(name, config, config.timeout)
timeout_seconds = _validate_base(name, config.timeout)
heartbeat_timeout_seconds = _validate_heartbeat(name, config.heartbeat_timeout)
request = config.request or MicrovmRequestConfig()
retry_window_seconds = _validate_request(name, request)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,16 +165,15 @@ def pipeline(vm: MicrovmSession, ctx: DurableContext) -> Any:
Raises:
ValidationError: When ``name`` or ``config`` is not valid, or the
Region cannot be determined. No durable operation starts.
TypeError: When ``handler`` is not callable. No durable operation
starts.
MicrovmLaunchError: When ``RunMicrovm`` fails after all retries.
MicrovmLaunchError: When ``RunMicrovm`` fails after all retries, or
rejects the config, for example an empty ``image_identifier``.
MicrovmError: A MicroVM error that the handler did not catch.
ChildContextError: Any other error that the handler raised.
ChildContextError: Any other error that the handler raised. A
``handler`` that is not callable raises Python's ``TypeError``
after the launch, inside the child context. So it also reaches
the caller as ``ChildContextError``, after the terminate step.
"""
scope = _SessionScope.create(context, name, config)
if not callable(handler):
msg = f'MicroVM session "{name}": handler must be callable'
raise TypeError(msg)

def run(child: DurableContext) -> _T:
return scope.run_session(child, handler)
Expand Down Expand Up @@ -245,9 +244,12 @@ def invoke(
``SendDurableExecutionCallbackSuccess``, parsed as JSON.

Raises:
TypeError: When ``context`` is not a durable context, or is the
context that ``microvm_session`` was called with, or
``job_input`` is not JSON-serializable.
TypeError: When ``context`` is the context that
``microvm_session`` was called with, or ``job_input`` is not
JSON-serializable.
AttributeError: When ``context`` is not a durable context, for
example a job name passed as the first argument. Python
raises it at ``context.run_in_child_context``.
ValidationError: When ``name`` or ``options`` is not valid.
MicrovmNotRunningError: When the session's MicroVM has ended. The
session reports it when it checks the MicroVM state before
Expand All @@ -260,15 +262,6 @@ def invoke(
MicrovmTimeoutError: When no result or heartbeat arrives in time.
"""
scope = self._scope
# A call in the shape vm.invoke("build", ...) passes a string here.
# This check names the mistake, instead of an AttributeError from
# inside.
if not callable(getattr(context, "run_in_child_context", None)):
msg = (
f'MicroVM session "{scope.name}": invoke requires the durable '
"context to create the job in, as its first argument"
)
raise TypeError(msg)
# The caller's own context is often in scope in the handler's
# closure. A job created in it lands outside the session's child
# context. The core SDK does not refuse that, and the next replay
Expand Down Expand Up @@ -359,7 +352,7 @@ def create(
ValidationError: When ``name`` or ``config`` is not valid, or the
Region cannot be determined.
"""
timeout_seconds = _validate_base(name, config, config.timeout)
timeout_seconds = _validate_base(name, config.timeout)
auto_suspend_idle_seconds = _validate_auto_suspend(
name, config, timeout_seconds
)
Expand Down
Loading
Loading