Repository navigation
Java - Flight SQL - StackOverflowException caused by org.apache.arrow.flight.FlightStream.next #172
Description
Describe the bug, including details regarding any error messages, version, and platform.
We're experimenting with Arrow Flight SQL and wrote an initial basic prototype server.
I've tried hitting it with the Java library, but it's a little buggy so far.
The data set I'm querying is a few GBs of data across 10'000'000 rows and 21 columns.
There are 10 double columns and 10 symbol (Dictionary<Int32, Utf8>) columns.
See: https://github.com/timescale/tsbs to get an idea of the type of data being queried.
I think the Java Flight SQL client is struggling with the response sent back.
java.lang.StackOverflowError
at java.base/java.util.Spliterator.getExactSizeIfKnown(Spliterator.java:414)
at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:526)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.IntPipeline.findFirst(IntPipeline.java:552)
at java.base/java.text.DecimalFormatSymbols.findNonFormatChar(DecimalFormatSymbols.java:844)
at java.base/java.text.DecimalFormatSymbols.initialize(DecimalFormatSymbols.java:815)
at java.base/java.text.DecimalFormatSymbols.<init>(DecimalFormatSymbols.java:115)
at java.base/sun.util.locale.provider.DecimalFormatSymbolsProviderImpl.getInstance(DecimalFormatSymbolsProviderImpl.java:85)
at java.base/java.text.DecimalFormatSymbols.getInstance(DecimalFormatSymbols.java:182)
at java.base/java.util.Formatter.zero(Formatter.java:2450)
at java.base/java.util.Formatter$FormatSpecifier.getZero(Formatter.java:4450)
at java.base/java.util.Formatter$FormatSpecifier.localizedMagnitude(Formatter.java:4466)
at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:3276)
at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:3261)
at java.base/java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2957)
at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:2918)
at java.base/java.util.Formatter.format(Formatter.java:2689)
at java.base/java.util.Formatter.format(Formatter.java:2625)
at java.base/java.lang.String.format(String.java:4141)
at org.apache.arrow.memory.util.HistoricalLog.recordEvent(HistoricalLog.java:82)
at org.apache.arrow.memory.BufferLedger.retain(BufferLedger.java:182)
at org.apache.arrow.memory.BufferLedger.retain(BufferLedger.java:169)
at org.apache.arrow.vector.ipc.message.ArrowRecordBatch.<init>(ArrowRecordBatch.java:92)
at org.apache.arrow.vector.ipc.message.ArrowRecordBatch.<init>(ArrowRecordBatch.java:69)
at org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeRecordBatch(MessageSerializer.java:438)
at org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeDictionaryBatch(MessageSerializer.java:514)
at org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeDictionaryBatch(MessageSerializer.java:529)
at org.apache.arrow.flight.ArrowMessage.asDictionaryBatch(ArrowMessage.java:273)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:264)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
at org.apache.arrow.flight.FlightStream.next(FlightStream.java:280)
....
Looking at FlightStream.java:280, there's a recursive call to next().
public boolean next() {
...
} else if (msg.getMessageType() == HeaderType.DICTIONARY_BATCH) {
...
return next(); // <------------- culprit
...
}This should be fixed to allow querying large datasets.
Before I forget, here's the version I'm using:
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-sql</artifactId>
<version>11.0.0</version>
</dependency>
Component(s)
Java
Metadata
Metadata
Assignees
Labels
No labels
Activity
lidavidm commentedon Aug 9, 2023
Interesting, I guess there are enough dictionary batches in the stream that we stack overflow and we should instead iteratively consume them
lidavidm commentedon Aug 11, 2023
CC @davisusanibar
vibhatha commentedon Sep 21, 2023
@amunra
Is there a helpful script or setup to reproduce this error?
amunra commentedon Sep 21, 2023
Not at the moment, it's tied up in a heap of proprietary code: I'd need to spend some time extracting a standalone repro.
I'm afraid it will be weeks before I can spare the cycles to do so. I assure you that this is not due to lack of willingness 😊.
I can tell you in the meantime that I was performing a query that returned 29GB of data over a schema that contained 10 Dict(int, utf8) columns.
These sorts of queries are not unusual for some of our customers.
vibhatha commentedon May 28, 2024
@amunra just following up, any updates that could help us to narrow down this issue?