Skip to content

Feature request: support the new EventBridge Custom Event Bus payloads in data classes and Parser #8657

Closed

Description

@leandrodamascena

Use case

AWS launched a new EventBridge bus, the Custom Event Bus (the one we all know is now "Custom Event Bus - Classic"). It has subscribers instead of rules, retention, FIFO groups, replay, deduplication and PutRawEvents for non-JSON payloads. API is aws eventsv2 / boto3 eventbridgev2 (botocore 1.43.102+).

I deployed a real bus with a Lambda target running our current EventBridgeEvent data class and EventBridgeModel, and the payloads are different enough that both break in every configuration except the one that imitates Classic:

Subscriber config What the function receives Today
Transformer: RAW, default batching, PutEvents a JSON array of classic envelopes, even for a single event TypeError: list indices must be integers
RAW, MaxBatchSize: 1, PutEvents one classic envelope works
RAW, MaxBatchSize: 1, PutRawEvents the producer payload alone, no envelope KeyError: 'source'
Transformer: WITH_METADATA (single or array) {Data, Metadata, SystemMetadata} fails
FIFO subscriber array, events of several groups interleaved in order fails

The default for a Lambda target is the array. Anyone following the AWS migration guide will see @event_source(data_class=EventBridgeEvent) blow up on the first delivery.

Real payloads captured from the bus:

// WITH_METADATA, published with PutRawEvents (content type application/json)
{
  "Data": {"orderId": "s3-0", "total": 5},
  "Metadata": {"sub": "s3", "tenant": "acme"},
  "SystemMetadata": {
    "EventGroupId": "43e6837b-3d96-44db-863d-e8abd80cd1e8",
    "DeduplicationId": "43e6837b-3d96-44db-863d-e8abd80cd1e8",
    "ContentType": "application/json",
    "aws:IngestionTime": "2026-10-25T11:48:54.064Z",
    "aws:EventId": "e3474ffa-0283-4fe7-af27-803e30cd92f4",
    "aws:SequenceNumber": "10000000000000007000",
    "aws:DeliveryType": "LIVE"
  }
}
// WITH_METADATA, published with PutEvents: Data holds the classic envelope, SystemMetadata adds aws:Source / aws:DetailType
[
  {
    "Data": {"version": "0", "id": "1cd1f343-9859-4e28-a539-89352846ce44", "detail-type": "s6", "source": "com.example.orders", "account": "123456789012", "time": "2026-10-25T11:48:53Z", "region": "us-east-1", "resources": [], "detail": {"orderId": "s6-1", "total": 10}},
    "Metadata": {},
    "SystemMetadata": {"EventGroupId": "d8c85953-b07f-4bc1-b2ff-044a1c2b03bc", "DeduplicationId": "d8c85953-b07f-4bc1-b2ff-044a1c2b03bc", "ContentType": "application/eventbridge+json", "aws:IngestionTime": "2026-10-25T11:48:53.977Z", "aws:Source": "com.example.orders", "aws:DetailType": "s6", "aws:EventId": "b9c941f6-ccae-473e-95d5-1982a0b1e8a5", "aws:SequenceNumber": "10000000000000006001", "aws:DeliveryType": "LIVE"}
  }
]

Notes from the test that matter for the design:

  • aws:EventId is not the envelope id. The publisher gets aws:EventId back from PutEvents.
  • aws:DeliveryType is LIVE or REPLAY, which is what a consumer needs to tell replays apart.
  • For application/octet-stream, Data is a base64 string.
  • Partial batch failure is not usable yet: returning {"batchItemFailures": [...]} is recorded as PARTIAL_BATCH_FAILURE but every event in the batch is marked failed, whichever identifier you use (aws:EventId, envelope id, aws:SequenceNumber). An exception retries the whole batch. So Batch integration is out of scope until the service documents an identifier.

Solution/User Experience

A data class and a Parser model for the new shape, accepting both a single object and an array, and exposing the metadata as attributes.

from aws_lambda_powertools.utilities.data_classes import event_source, EventBridgeV2Event

@event_source(data_class=EventBridgeV2Event)
def handler(event: EventBridgeV2Event, context):
    for record in event.records:                     # always iterable, even when the bus sent one object
        record.data                                   # dict for JSON, bytes for octet-stream
        record.detail                                 # shortcut into data["detail"] when published with PutEvents
        record.metadata                               # producer keys from PutRawEvents
        record.system_metadata.event_id               # aws:EventId
        record.system_metadata.sequence_number
        record.system_metadata.delivery_type          # "LIVE" | "REPLAY"
        record.system_metadata.event_group_id
        record.system_metadata.source, .detail_type, .content_type, .ingestion_time

Parser side: EventBridgeV2Model (list or single) and an EventBridgeV2Envelope that yields the Data of each record (or Data.detail for PutEvents traffic) into the user model, like EventBridgeEnvelope does today.

RAW deliveries carry no marker, so the class should also accept the classic envelope and the bare PutEvents array without complaint, and we document that WITH_METADATA is what you want when you need event ids, groups or replay detection.

Alternative solutions

Tell users to set MaxBatchSize=1 and Transformer=RAW so the payload matches Classic. It works, but it throws away batching, ordering metadata and replay detection, which are the reasons to move to the new bus.

Acknowledgment

Activity

  1. Asymtode712 commented on Oct 25, 2026

    @Asymtode712
    Contributor

    @leandrodamascena I would like to take this up, could you please assign me this issue

  2. leandrodamascena commented on Oct 25, 2026

    @leandrodamascena
    ContributorAuthor

    Go ahead @Asymtode712! Thanks 😃

  3. added this to the 3.38.0 milestone on Oct 25, 2026
  4. powertools-for-aws-oss-automation commented on Oct 25, 2026

    @powertools-for-aws-oss-automation

    Warning

    This issue is now closed. Please be mindful that future comments are hard for our team to see.
    If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
    If you wish to keep having a conversation with other community members under this issue feel free to do so.

  5. github-actions commented on Oct 28, 2026

    @github-actions
    Contributor

    This is now released under 3.38.0 version!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions