Skip to content

Assert for invariants in tests and debug builds #594

Closed

Description

@houqp

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Our invariants are formally documented in https://github.com/apache/arrow-datafusion/blob/master/docs/specification/invariants.md, but it's not being fully enforced across the board. During the implementation of #55, I have found couple bugs in our existing code base that could have been prevented if check for invariants more aggressively during development and in CI.

Describe the solution you'd like

One thing we could do is to insert conditional compilation asserts for invariants in places that could break the invariants. For example:

  • add assert to compare logical plan schema before and after each optimizer run
  • add assert to compare fields between logical plan and physical plan after physical planing
  • add assert to make sure physical plan only deals with unqualified fields, for example: ParquetExec::try_from_path

These asserts should be turned of in production builds.

Activity

  1. alamb commented on Jan 6, 2025

    @alamb
    Contributor

    It seems as have re-discovered this idea 10,000 tickets later in

    FYI @wiedld

    Let's close this issue and use the newer one for further discussion

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions