Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ three packages:
- When in doubt, write the changelog entry in terms of user impact (what users
can now do or what behavior changed), not implementation mechanism (how it
was implemented internally).
- Keep entries focused on the capability or fix, its relevant default, and
material compatibility or upgrade risks. Preserve actionable replay and
breaking-change warnings, but keep API tutorials, implementation
prescriptions, exhaustive edge cases, and workarounds in linked feature or
API documentation. Narrow an overly broad claim rather than expanding it
into a catalogue of exceptions.
- Changelogs are not covered by the CI Markdown lint step. Review changes to
them manually.
- Use the current unindented changelog style: category labels such as `ADDED`,
Expand Down
59 changes: 23 additions & 36 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,71 +7,58 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

## v1.13.0

ADDED

- Added `OrchestrationQuery.task_hub_names` to filter orchestration instances
by one or more task hubs.
- Added deterministic JSON-array serialization for sets and frozensets.
Generic set reconstruction is opt-in through `JsonDataConverter`; enabling it
for existing orchestrations can change replay behavior.

CHANGED

- Dropped Python 3.10 support. Python 3.11 or later is now required.
- Removed the obsolete PyPI `asyncio` backport dependency; all supported
Python versions use the standard-library module.
- Removed the obsolete PyPI `asyncio` backport dependency.
- Context managers now preserve subclass types in type checkers, including
Azure Managed clients and workers.
- Entity ID copy and pickle hooks now preserve subclass types in type checkers.
- Multiple task completion callback failures now raise the built-in
`ExceptionGroup`, preserving each exception and its traceback in memory and
the first callback error as the explicit cause and serialized inner failure.
A single callback failure is still raised directly.
the first callback error as the cause. A single callback failure is still
raised directly.

> [!WARNING]
> The aggregate exception type and message have changed. Orchestrations that
> branch on the old aggregate type or message can take a different path during
> replay; drain affected instances on the previous SDK or use a separate task
> hub for new instances. Branches on the aggregate's explicit first-error cause
> retain their previous behavior.
> hub for new instances.

FIXED

- Activity and sub-orchestration retry policies now honor
`failureDetails.isNonRetriable`, surfacing the failure immediately without
scheduling a retry timer or another attempt. Existing histories that already
retried such failures can replay differently; drain affected instances on the
previous SDK before upgrading.
retrying. Existing histories that already retried such failures can replay
differently; drain affected instances on the previous SDK before upgrading.
- Grouped failures now retain nested exception messages, tracebacks, and notes
in failure details, including `ExceptionGroup` failures from task callbacks.
Traceback rendering is limited to 15 children per group and 10 group levels,
with truncation markers for omitted diagnostics. Failure details do not
serialize a complete structured tree of group children.
- Optimized set and frozenset serialization, particularly for deeply nested
frozensets and large singleton elements. JSON object names within sets are now
ordered consistently after conversion to strings, including numeric names.
Equivalent surrogate-pair and non-BMP object names retain the last value
rather than emitting duplicate JSON properties.
Set reconstruction remains opt-in.
- With set reconstruction enabled, annotated sets of typed tuples are now
recognized during input and activity-result discovery, and bare
`collections.abc.Set` annotations reconstruct JSON arrays as concrete sets.
- Implemented `__hash__` and `__repr__` for `EntityInstanceId`, making identity attributes immutable to guarantee hash stability when used in sets or dict keys.
- Large-payload externalization now checks every eligible payload's maximum
size before uploading or replacing any protobuf values, including carryover
events and entity payloads, for both synchronous and asynchronous calls.
Oversized messages remain unchanged and cause no uploads; their size error
now takes precedence over hypothetical earlier storage failures.
- Improved set and frozenset serialization performance and consistent JSON
object-key handling.
- `EntityInstanceId` is now hashable and has a diagnostic `repr`, allowing use
in sets and as dictionary keys. Entity IDs are now immutable.
- Oversized payloads now fail validation before any payload uploads or
replacements occur.
- Orchestrations that exceed the configured external payload storage limit now
fail without retrying and release their entity locks, allowing other callers to
progress. Transient payload-store failures remain retryable.
- Previously unhandled non-cancellation `BaseException` exits, including result
serialization and generator teardown failures, now fail with a clear reason instead of leaving an accepted dispatch
unfinished. Cancellation and process-control signals retain their semantics.
The gRPC worker abandons work that exits before an outcome is sent and resets
the originating work-item stream after an uncertain completion or abandonment
RPC, allowing scheduler ownership recovery without retrying an uncertain outcome.
- Unexpected orchestration execution, result serialization, and teardown
failures no longer leave accepted work unfinished. Cancellation and
process-control signals retain their semantics.
- Orchestrator generators are now closed at the end of each dispatch, including
while waiting for durable work, releasing resources held by async bridges.
`finally` blocks and context-manager exits can run before orchestration completion
and repeat across dispatches; see [cleanup semantics](docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks).
while waiting for durable work. `finally` blocks and context-manager exits can
run before orchestration completion and repeat across dispatches; see
[cleanup semantics](docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks).
- Async cancellation raised by generator teardown no longer replaces dispatch
responses or version-rejection abandonment.

Expand Down
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,47 @@ The Azure Functions Durable provider requires Python 3.13 or later.

## Optional Features

### Task Hub Query Filters

Set `OrchestrationQuery.task_hub_names` to filter orchestration queries by one or
more task hubs. The filter is available through synchronous and asynchronous
core and Azure Managed clients; filtering support depends on the backend.

```python
from durabletask.client import OrchestrationQuery

query = OrchestrationQuery(task_hub_names=["orders", "billing"])
```

### Set Serialization

The default `JsonDataConverter` serializes sets and frozensets as JSON arrays
ordered by their canonical JSON representation, not numeric order. For example,
`{2, 10}` serializes as `[10, 2]`. To reconstruct annotated sets and frozensets
instead of receiving lists, configure the same converter on clients and workers:
Comment thread
andystaples marked this conversation as resolved.
Resolved

```python
from durabletask.serialization import JsonDataConverter

converter = JsonDataConverter(enable_generic_sets=True)
```

Pass it using the `data_converter` parameter on core or Azure Managed clients
and workers. `enable_set_coercion=True` is an alias for the same option.
Azure Functions uses its own shared codec rather than this converter.

> [!WARNING]
> Set reconstruction is disabled by default to preserve the order and duplicates
> of recorded JSON arrays. Enabling it for existing orchestrations can change
Comment thread
andystaples marked this conversation as resolved.
Resolved
> replay behavior. Iteration over reconstructed sets is not replay-safe; sort
> values explicitly before using their order to schedule durable work.
Comment thread
andystaples marked this conversation as resolved.
Resolved

### Entity IDs

`EntityInstanceId` values are immutable, hashable, and usable as dictionary keys
or set elements. Construct a new ID to change an entity name or key; attribute
assignment and deletion raise `AttributeError`.

### Large Payload Externalization

Install the `azure-blob-payloads` extra to automatically offload oversized orchestration payloads to
Expand Down
60 changes: 27 additions & 33 deletions azure-functions-durable/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,49 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased

## v2.0.0rc4

ADDED

- Added inherited `OrchestrationQuery.task_hub_names` support to filter
orchestration instances by one or more task hubs.
- Added `DFApp.configure_serialization(ensure_ascii=False)` to opt in to compact
Unicode JSON across durable clients, orchestration and entity workers, activity
bindings, and registered blueprints. The default remains ASCII-escaped JSON.
The opt-in uses the shared codec's `ensure_ascii` option available in
`azure-functions` v2.4.0b1; unsupported SDKs fail explicitly at configuration time.
Unicode JSON for durable payloads. The default remains ASCII-escaped JSON.

CHANGED

- Updated the minimum `durabletask` dependency to v1.13.0.
- Updated the minimum `azure-functions` dependency to v2.4.0b1.
- With the corresponding core SDK update, client and worker context managers
preserve Functions subclass types in type checkers. Python 3.13 remains the
minimum supported version for this package.
- With the corresponding core SDK update, multiple task completion callback
failures raise the built-in `ExceptionGroup`; a single failure is still raised
directly. The first callback error remains the explicit cause and serialized
inner failure. Branches on the previous aggregate exception type or message
can change during replay; see the [core upgrade warning](../CHANGELOG.md#unreleased).
- Client and worker context managers now preserve Functions subclass types in
type checkers. Python 3.13 remains the minimum supported version.
- Multiple task completion callback failures now raise the built-in
`ExceptionGroup`; a single failure is still raised directly. Branches on the
previous aggregate type or message can change during replay; see the
[core upgrade warning](../CHANGELOG.md#v1130).

FIXED

- With the corresponding core SDK update, activity and sub-orchestration retry
policies, including compatibility `RetryOptions`, honor
`failureDetails.isNonRetriable` and surface the failure immediately without retrying.
Histories that already retried such failures can replay differently; drain
affected instances on the previous SDK before upgrading.
- With the corresponding core SDK update, grouped failures retain nested
exception messages, tracebacks, and notes in failure details. Tracebacks are
limited to 15 children per group and 10 group levels with truncation markers;
group children are not serialized as a complete structured tree.
- With the corresponding core `durabletask` SDK fix, oversized messages fail
large-payload validation before any uploads or payload replacements. Size
errors now take precedence over hypothetical earlier storage failures.
- With the corresponding core `durabletask` SDK fix, previously unhandled non-cancellation
`BaseException` exits from orchestration execution, result serialization, or teardown now produce a failed response
with the exception's type, message, and stack trace instead of losing the response.
- With the corresponding core `durabletask` SDK fix, orchestrator generators
are closed at the end of each dispatch, including while waiting for durable work.
See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks)
for repeated finalizers, discarded teardown actions/status, and yielding restrictions.
- With the corresponding core `durabletask` SDK fix, async cancellation raised by
generator teardown no longer replaces dispatch responses or version-rejection abandonment.
- `EntityInstanceId` and the compatibility `EntityId` are now immutable and
hashable, with a diagnostic `repr`.
- Activity and sub-orchestration retry policies, including compatibility
`RetryOptions`, now honor `failureDetails.isNonRetriable`. Histories that already
retried such failures can replay differently; drain affected instances on the
previous SDK before upgrading.
- Grouped failures now retain nested exception messages, tracebacks, and notes.
- Oversized payloads now fail validation before any uploads or replacements.
- Unexpected orchestration execution, result serialization, and teardown
failures now produce a failed response instead of losing the response.
- Orchestrator generators are now closed at each dispatch boundary, so
`finally` blocks and context-manager exits may run repeatedly before completion.
See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks).
- Async cancellation during generator teardown no longer replaces dispatch
responses or version-rejection abandonment.

## v2.0.0rc3

Expand Down
8 changes: 8 additions & 0 deletions azure-functions-durable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ Key capabilities include durable orchestrations and sub-orchestrations, durable
timers, external events, durable entities, retries, versioning, durable HTTP
calls (`context.call_http(...)`), recurring scheduled tasks, and history export.

`DurableFunctionsClient` and `SyncDurableFunctionsClient` inherit
`OrchestrationQuery.task_hub_names` for filtering orchestration queries by one
or more task hubs. Filtering support depends on the backend.

`EntityInstanceId` and the compatibility `EntityId` are immutable and hashable,
so they can be used as dictionary keys or set elements. Construct a new ID
instead of assigning or deleting attributes on an existing ID.

## Unicode JSON serialization

By default, durable payloads preserve the shared `azure-functions` codec's
Expand Down
4 changes: 2 additions & 2 deletions azure-functions-durable/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "azure-functions-durable"
version = "2.0.0rc3"
version = "2.0.0rc4"
description = "Durable Task Python SDK provider implementation for Durable Azure Functions"
keywords = [
"durable",
Expand All @@ -30,7 +30,7 @@ requires-python = ">=3.13"
license = {file = "LICENSE"}
readme = "README.md"
dependencies = [
"durabletask[opentelemetry]>=1.12.0",
"durabletask[opentelemetry]>=1.13.0",
"azure-identity>=1.19.0",
"azure-functions>=2.4.0b1"
]
Expand Down
66 changes: 34 additions & 32 deletions durabletask-azuremanaged/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,44 +7,46 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

## v1.13.0

ADDED

- Added inherited `OrchestrationQuery.task_hub_names` support to filter
orchestration instances by one or more task hubs.
- Added inherited deterministic set and frozenset serialization.
Generic set reconstruction is opt-in through `JsonDataConverter`; enabling it
for existing orchestrations can change replay behavior.

CHANGED

- Updated the minimum `durabletask` dependency to v1.13.0, including the
`azure-blob-payloads` extra.
- Dropped Python 3.10 support. Python 3.11 or later is now required.
- With the corresponding core SDK update, client and worker context managers
preserve Azure Managed subclass types in type checkers.
- With the corresponding core SDK update, multiple task completion callback
failures raise the built-in `ExceptionGroup`; a single failure is still raised
directly. The first callback error remains the explicit cause and serialized
inner failure. Branches on the previous aggregate exception type or message
can change during replay; see the [core upgrade warning](../CHANGELOG.md#unreleased).
- Client and worker context managers now preserve Azure Managed subclass types
in type checkers.
- Multiple task completion callback failures now raise the built-in
`ExceptionGroup`; a single failure is still raised directly. Branches on the
previous aggregate type or message can change during replay; see the
[core upgrade warning](../CHANGELOG.md#v1130).

FIXED

- With the corresponding core SDK update, activity and sub-orchestration retry
policies honor `failureDetails.isNonRetriable` and surface the failure immediately
without retrying. Histories that already retried such failures can replay
differently; drain affected instances on the previous SDK before upgrading.
- With the corresponding core SDK update, grouped failures retain nested
exception messages, tracebacks, and notes in failure details. Tracebacks are
limited to 15 children per group and 10 group levels with truncation markers;
group children are not serialized as a complete structured tree.
- With the corresponding core `durabletask` SDK fix, oversized messages fail
large-payload validation before any uploads or payload replacements. Size
errors now take precedence over hypothetical earlier storage failures.
- With the corresponding core `durabletask` SDK fix, orchestrations that exceed
the configured external payload storage limit fail without retrying and release
their entity locks. Transient payload-store failures remain retryable.
- With the corresponding core `durabletask` SDK fix, previously unhandled
non-cancellation `BaseException` exits, including result serialization failures,
now fail with a clear reason. Cancellation and process-control
exits recover dispatch ownership, and uncertain completion or abandonment RPCs
reset the originating work-item stream rather than retaining unfinished work.
- With the corresponding core `durabletask` SDK fix, orchestrator generators
are closed at the end of each dispatch, including while waiting for durable work.
See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks)
for repeated finalizers, discarded teardown actions/status, and yielding restrictions.
- With the corresponding core `durabletask` SDK fix, async cancellation raised by
generator teardown no longer replaces dispatch responses or version-rejection abandonment.
- `EntityInstanceId` is now immutable and hashable, with a diagnostic `repr`.
- Activity and sub-orchestration retry policies now honor
`failureDetails.isNonRetriable`. Histories that already retried such failures
can replay differently; drain affected instances on the previous SDK before
upgrading.
- Grouped failures now retain nested exception messages, tracebacks, and notes.
- Oversized payloads now fail validation before any uploads or replacements.
Orchestrations exceeding the external payload storage limit fail without
retrying and release entity locks; transient payload-store failures remain
retryable.
- Unexpected orchestration failures no longer leave accepted work unfinished.
- Orchestrator generators are now closed at each dispatch boundary, so
`finally` blocks and context-manager exits may run repeatedly before completion.
See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks).
- Async cancellation during generator teardown no longer replaces dispatch
responses or version-rejection abandonment.

## v1.12.0

Expand Down
6 changes: 3 additions & 3 deletions durabletask-azuremanaged/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "durabletask.azuremanaged"
version = "1.12.0"
version = "1.13.0"
description = "Durable Task Python SDK provider implementation for the Azure Durable Task Scheduler"
keywords = [
"durable",
Expand All @@ -31,13 +31,13 @@ requires-python = ">=3.11"
license = {file = "LICENSE"}
readme = "README.md"
dependencies = [
"durabletask>=1.12.0",
"durabletask>=1.13.0",
"azure-identity>=1.19.0"
]

[project.optional-dependencies]
azure-blob-payloads = [
"durabletask[azure-blob-payloads]>=1.12.0"
"durabletask[azure-blob-payloads]>=1.13.0"
]

[project.urls]
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "durabletask"
version = "1.12.0"
version = "1.13.0"
description = "A Durable Task Client SDK for Python"
keywords = [
"durable",
Expand Down
Loading