You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Ruby] Segmentation Error Returned when Executing Get S3 File Method from Arrow Dataset Library #38156
Describe the bug, including details regarding any error messages, version, and platform.
The following method is executed as part of a step definition to retrieve an uploaded parquet file and transform it into a ruby hash data structure, it works at runtime...
however the following output is appended to the end of the execution:
./cpp/src/arrow/filesystem/s3fs.cc:2829: arrow::fs::FinalizeS3 was not called even though S3 was initialized. This could lead to a segmentation fault at exit
/usr/local/bundle/bin/cucumber: [BUG] Segmentation fault at 0x0000000000000000
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
-- Machine register context ------------------------------------------------
RIP: 0x0000000000000000 RBP: 0x000055e9474b4600 RSP: 0x00007fff79686828
RAX: 0x000055e94763db50 RBX: 0x00007fcacfa4ac00 RCX: 0x00007fcacfa35bb8
RDX: 0x000055e9478d8c70 RDI: 0x000055e9474b4600 RSI: 0x0000000000000007
R8: 0x0000000000000007 R9: 0x000000000000000d R10: 0x0000000000000001
R11: 0x0000000000246678 R12: 0x0000000000000000 R13: 0x0000000000000334
R14: 0x00007fcacfa36e08 R15: 0x000055e946a90dd0 EFL: 0x0000000000010246
-- C level backtrace information -------------------------------------------
corrupted size vs. prev_size in fastbins
I am using the following arrow packages (taken from GemFile):
I have also downgraded to 12.0.1 and 11.0 version for all three above but get the same issues... The error above seems to be catered for in Python, however I couldn't find a fix for this issue within a Ruby context...
changed the title [-]Ruby Segmentation Error Returned when Executing Get S3 File Method from Arrow Dataset Library[/-][+][Ruby] Segmentation Error Returned when Executing Get S3 File Method from Arrow Dataset Library[/+]on Oct 9, 2023
We started using Arrow::Table.load(URI("s3://...")) in a method in our Rails application and started seeing the warning, though not a full segmentation fault.
Activity
[-]Ruby Segmentation Error Returned when Executing Get S3 File Method from Arrow Dataset Library[/-][+][Ruby] Segmentation Error Returned when Executing Get S3 File Method from Arrow Dataset Library[/+]kou commentedon Oct 9, 2023
Could you call
Arrow.s3_finalizeexplicitly after you finished using all S3 related features and before you finish your program?andrewhampton commentedon Sep 4, 2024
FWIW, I dropped the dataset gem and manually retrieved the files from S3 to avoid this issue.
kou commentedon Sep 5, 2024
Arrow.s3_finalizedidn't solve your case?MasterOdin commentedon Apr 11, 2025
We started using
Arrow::Table.load(URI("s3://..."))in a method in our Rails application and started seeing the warning, though not a full segmentation fault.Adding:
in a rails initializer fixed the warning for us.
It might be nice to call out that need to call this function somewhere on https://github.com/apache/arrow/blob/main/ruby/README.md?
kou commentedon Apr 13, 2025
C++ has
arrow::filesystem::EnsureFinalized(). So we can automate it by implement the bindings for it:garrow_file_system_ensure_finalized().at_exist {Arrow::FileSystem.ensure_finalized}inrequire "arrow".