Skip to content

Commit ee54c16

Browse files
committed
feat(dmap): add distributed map operation
Add ctx.distributed_map with inline, S3, and reader sources, the config, processor, completion, and destination types, the result types, and function-authoring helpers for item and batch handlers. Serialization lives in lambda_service.py with one dataclass per API shape. Translation happens only in the executor, so config.py has no runtime import from lambda_service. Tests are split by layer. Items, Output, and the record body are opaque strings, matching the API model, so any serdes works rather than JSON only. Every terminal state resolves with the summary, and throw_if_error() opts into raising. Missing details or a missing completion reason raise ExecutionError. The processor factories are batch, item_failures, and item_results. Source and destination factories are InlineSource, S3Source, ReaderSource, and S3Destination. Retry is max_retry_attempts and max_retry_duration on the processor. Passing DistributedMapResultConfig selects the DistributedMapResult return type statically. Only the result types and their enums are exported from the package root.
1 parent 9157022 commit ee54c16

20 files changed

Lines changed: 5215 additions & 3 deletions

File tree

‎packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/__init__.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@
1616
CompletionItemStatus,
1717
CompletionOutcome,
1818
CompletionStatus,
19+
DistributedMapCompletionReason,
20+
DistributedMapItemStatus,
21+
DistributedMapStatus,
1922
ParallelBranch,
2023
complete_batch,
2124
continue_batch,
@@ -27,6 +30,19 @@
2730
durable_wait_for_callback,
2831
durable_with_child_context,
2932
)
33+
from aws_durable_execution_sdk_python.dmap.handlers import (
34+
distributed_map_batch_handler,
35+
distributed_map_item_handler,
36+
distributed_map_reader,
37+
durable_distributed_map_batch_handler,
38+
durable_distributed_map_item_handler,
39+
)
40+
from aws_durable_execution_sdk_python.dmap.models import (
41+
DistributedMapItemError,
42+
DistributedMapResult,
43+
DistributedMapResultItem,
44+
DistributedMapSummary,
45+
)
3046

3147
# Most common exceptions - users need to handle these exceptions
3248
from aws_durable_execution_sdk_python.exceptions import (
@@ -35,6 +51,7 @@
3551
CallbackSubmitterError,
3652
CallbackTimeoutError,
3753
ChildContextError,
54+
DistributedMapError,
3855
DurableExecutionsError,
3956
DurableOperationError,
4057
ExecutionError,
@@ -69,6 +86,14 @@
6986
"CompletionItemStatus",
7087
"CompletionOutcome",
7188
"CompletionStatus",
89+
"DistributedMapCompletionReason",
90+
"DistributedMapError",
91+
"DistributedMapItemError",
92+
"DistributedMapItemStatus",
93+
"DistributedMapResult",
94+
"DistributedMapResultItem",
95+
"DistributedMapStatus",
96+
"DistributedMapSummary",
7297
"DurableContext",
7398
"DurableExecutionsError",
7499
"DurableOperationError",
@@ -87,6 +112,11 @@
87112
"__version__",
88113
"complete_batch",
89114
"continue_batch",
115+
"distributed_map_batch_handler",
116+
"distributed_map_item_handler",
117+
"distributed_map_reader",
118+
"durable_distributed_map_batch_handler",
119+
"durable_distributed_map_item_handler",
90120
"durable_execution",
91121
"durable_parallel_branch",
92122
"durable_step",

0 commit comments

Comments
 (0)