You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
pyarrow latest parquet map column type isn't supported #2262
Hello
When I trying read parquet file with column of type map, pyarrow.lib.ArrowNotImplementedError: lists with structs are not supported. exception is throws.
Seems like this was fixed here #1530, and only release is required?
Assistance with this would be much appreciated. Unfortunately we haven't been able to get this done in time for 0.10, so it will have to be later this year
Most of the data files in our data lake has map columns. Not being able to read parquet files with map columns using pyarrow creates dependency on Spark. Is there a plan to support map columns?
Activity
xhochy commentedon Jul 14, 2018
No, this would also require apache/parquet-cpp#462
wesm commentedon Jul 19, 2018
Assistance with this would be much appreciated. Unfortunately we haven't been able to get this done in time for 0.10, so it will have to be later this year
damache commentedon Dec 7, 2018
was this fixed? I installed the following
then tried this code
but I get this error
wesm commentedon Dec 8, 2018
No It has not yet been implemented
wesm commentedon Dec 9, 2018
@damache would anyone from IBM like to get involved with Parquet development? We could really use the help.
sujayramaiah commentedon Jan 24, 2020
Most of the data files in our data lake has map columns. Not being able to read parquet files with map columns using pyarrow creates dependency on Spark. Is there a plan to support map columns?
wesm commentedon Jan 28, 2020
Yes, but someone has to do the implementation work. See ARROW-1644 and related issues