Skip to content

GH-46130: [Python] Remove use_legacy_format in favour of setting IpcWriteOptions - #46131

Merged
raulcd merged 4 commits into
apache:mainfrom
AlenkaF:remove-use-legacy-format-in-ipc
Apr 16, 2025
Merged

raulcd merged 4 commits into
apache:mainfrom
AlenkaF:remove-use-legacy-format-in-ipc

Conversation

@AlenkaF

@AlenkaF AlenkaF commented Apr 14, 2025 •

Copy link
Copy Markdown
Member

Rationale for this change

use_legacy_format in the ipc writer has been deprecated and can be removed.

What changes are included in this PR?

use_legacy_format keyword is removed in favour of setting IpcWriteOptions.

Are these changes tested?

Existing tests should pass.

Are there any user-facing changes?

Deprecated functionality is removed.

Comment thread python/pyarrow/ipc.py


def new_stream(sink, schema, *, use_legacy_format=None, options=None):
def new_stream(sink, schema, *, options=None):

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There hasn’t been a clean way to deprecate these so far. One approach could be to leave this PR open for now and introduce deprecation warnings in a separate PR first.

@raulcd raulcd Apr 15, 2025 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been the name of the field for more than 5 years (use_legacy_format and the documentation said it was deprecated). Maybe we do not require to wait for a DeprecationWarning for a field that has been documented as deprecated for 5 years and has the name legacy on it? This is the commit that introduced the field (Arrow 1.0 already had it as legacy)
1a7519f
and it seems we can still use an environment variable to force using it :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit on the fence here, but I noticed that David has approved, and it looks like you, @raulcd, are on board as well. Unless there are any objections, I'll go ahead and merge this later today.

@AlenkaF
AlenkaF marked this pull request as ready for review April 15, 2025 07:38
@AlenkaF
AlenkaF requested a review from lidavidm as a code owner April 15, 2025 07:38
@AlenkaF
AlenkaF requested a review from raulcd April 15, 2025 07:38

@raulcd raulcd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AlenkaF !

@raulcd
raulcd merged commit 586ed92 into apache:main Apr 16, 2025
@AlenkaF
AlenkaF deleted the remove-use-legacy-format-in-ipc branch April 16, 2025 10:53
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 586ed92.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants