Repository navigation
Bug: CsvTransform loses its encoding and newline options after the first use #8563
Description
Expected Behaviour
A CsvTransform keeps the options it was created with, so the same instance gives the same result every time it is applied.
Current Behaviour
CsvTransform.transform() calls .pop() on self.transform_options for encoding and newline. After the first call those options are gone from the instance, and later calls silently fall back to utf-8 and the default newline handling.
The same thing happens inside S3Object when its transformed stream is re-created (for example after another transform(..., in_place=True)), because every transformation is applied again.
Code snippet
import io
import boto3
from botocore import stub
from aws_lambda_powertools.utilities.streaming import S3Object
from aws_lambda_powertools.utilities.streaming.compat import PowertoolsStreamingBody
from aws_lambda_powertools.utilities.streaming.transformations import CsvTransform
transform = CsvTransform(encoding="latin-1")
payload = "name\ncafé\n".encode("latin-1")
client = boto3.client("s3", region_name="us-east-1", aws_access_key_id="x", aws_secret_access_key="x")
with stub.Stubber(client) as s3_stub:
for _ in range(2):
body = PowertoolsStreamingBody(raw_stream=io.BytesIO(payload), content_length=len(payload))
s3_stub.add_response("get_object", {"Body": body})
print(list(S3Object("bucket", "key", boto3_client=client).transform(transform)))Possible Solution
Copy the options inside transform() before popping encoding and newline. I have a small fix with a regression test ready.
Steps to Reproduce
Run the snippet above. The first iteration prints [{'name': 'café'}]; the second raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9.
Powertools for AWS Lambda (Python) version
develop (1d4f145, 3.36.0)
AWS Lambda function runtime
3.12
Packaging format used
PyPi
Disclosure: I found this while reviewing the streaming utility with the help of an AI assistant (Claude Code). I ran the snippet above myself against develop.
Activity
boring-cyborg commentedon Oct 16, 2026
Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link
powertools-for-aws-oss-automation commentedon Oct 16, 2026
Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out our GitHub Discussions: Join the conversation
powertools-for-aws-oss-automation commentedon Oct 17, 2026
Warning
This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either reopen the issue, or open a new issue referencing this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.
github-actions commentedon Oct 22, 2026
This is now released under 3.37.0 version!