Skip to content

[Python] [FlightRPC] Index with value of 0 is out-of-bounds for array of length 0 #44160

Closed

Description

@heiseish

Describe the bug, including details regarding any error messages, version, and platform.

Context

Description

  • When a table built by concat-ing dictionary arrays of mismatched "schema"/dictionary, the transmitted table appears to be malformed

Reproducible code

import pyarrow.flight as fl
import pyarrow as pa
import enum

class MyEnum(enum.Enum):
    Foo = 0
    Bar = 1
    Baz = 2

schema = pa.schema({
    'col': pa.dictionary(pa.int8(), pa.string())
})

def build_data() -> pa.Table:
    non_empty = pa.table({
        'col': pa.DictionaryArray.from_arrays(pa.array([0, 2], pa.int8()), [x.name for x in MyEnum])
    }, schema=schema)
    empty = pa.table({
        'col': pa.DictionaryArray.from_arrays(pa.array([], pa.int8()), [])
    }, schema=schema)
    # If unify_dictionaries get called here, it works
    return pa.concat_tables([empty, non_empty]) # .unify_dictionaries()

class Server(fl.FlightServerBase):
    def do_get(self, context, ticket):
        table = build_data()
        _ = table['col'].to_pylist()
        print('build table ', table)
        # This doesn't work
        return fl.RecordBatchStream(table, options=pa.ipc.IpcWriteOptions(unify_dictionaries=True))

if __name__ == '__main__':
    server = Server()
    client = fl.FlightClient(f'grpc://localhost:{server.port}')
    client.wait_for_available()
    table = client.do_get(fl.Ticket(bytes())).read_all()
    try:
        _ = table['col'].to_pylist()
        print('got table ', table)
    except Exception as e:
        print(e)
    server.shutdown()

Expectation

  • to_pylist succeeds

Actual

  • to_pylist fails with index with value of 0 is out-of-bounds for array of length 0

Observation

Table before IPC

----
col: [  -- dictionary:
[]  -- indices:
[],  -- dictionary:
["Foo","Bar","Baz"]  -- indices:
[0,2]]

Table after IPC

col: [  -- dictionary:
[]  -- indices:
[],  -- dictionary:
[]  -- indices:
[0,2]]

I'm happy to open a PR if someone can point me to the relevant code. Thanks!

Component(s)

FlightRPC, Python

Activity

  1. changed the title [-]Index with value of 0 is out-of-bounds for array of length 0[/-] [+][Python] [FlightRPC] Index with value of 0 is out-of-bounds for array of length 0[/+] on Sep 18, 2024
  2. heiseish commented on Sep 18, 2024

    @heiseish
    Author

    take

  3. thisisnic commented on Nov 19, 2026

    @thisisnic
    Member

    Thanks for reporting this! I'm just going through older bug issues to see if we can close them out or we need further action.

    Could not reproduce on 26.0.0. Appears to have been fixed in #47115

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions