Repository navigation
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces the Anser runtime instrumentation subsystem for Cloudberry/Greenplum-style MPP execution, delivering an MVP runtime Bloom filter that is produced on segments, unioned on the coordinator, and consumed on segments to prune probe-side rows before hash joins—without changing the existing planning flow (post-plan tree injection).
Changes:
- Adds coordinator-resident Anser shared-memory channel map plus gather/send background workers, and a libpq-based segment→QD transport with token authentication.
- Implements plan-tree injection and executor support via CustomScan “Anser Bloom Producer/Consumer” nodes plus Bloom payload serialization/union helpers.
- Adds a comprehensive regression test module (
src/test/modules/anser) and wires it into Meson/Make and CI.
Reviewed changes
Copilot reviewed 44 out of 44 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/modules/meson.build | Adds anser test module to Meson build. |
| src/test/modules/Makefile | Adds anser test module to make-based test build. |
| src/test/modules/anser/test_anser.control | Defines test_anser extension for SQL-callable test helpers. |
| src/test/modules/anser/test_anser--1.0.sql | Registers SQL functions implemented by anser_test.c. |
| src/test/modules/anser/sql/test_anser.sql | Regression tests for channel lifecycle, payload correctness, libpq transport, and maintenance behavior. |
| src/test/modules/anser/sql/anser_runtime_filter.sql | Plan-shape and correctness regression for runtime filter injection and results stability. |
| src/test/modules/anser/meson.build | Builds/installs the test_anser shared module and regression schedule (Meson). |
| src/test/modules/anser/Makefile | Builds/installs the test_anser shared module and regression schedule (Make). |
| src/test/modules/anser/expected/test_anser.out | Expected output for test_anser regression. |
| src/test/modules/anser/expected/anser_runtime_filter.out | Expected output for runtime-filter plan/correctness regression. |
| src/test/modules/anser/anser_test.c | SQL-callable C helpers driving Anser APIs and libpq loopback tests. |
| src/include/utils/unsync_guc_name.h | Marks Anser GUCs as unsynchronized. |
| src/include/postmaster/postmaster.h | Increases auxiliary background worker count to accommodate Anser workers. |
| src/include/lib/bloomfilter.h | Adds bitset accessors and constructor-from-bitset API for deserialization. |
| src/include/executor/nodeAnserBloomFilter.h | Declares executor helper APIs for Bloom producer/consumer. |
| src/include/cdb/anserplan.h | Declares post-plan runtime-filter injection and CustomScan builders. |
| src/include/cdb/anserfilter.h | Declares Bloom part framing, serialization, deserialization, and fold API. |
| src/include/cdb/anserclient.h | Declares libpq client helpers for segment↔QD Anser transport. |
| src/include/cdb/anser.h | Introduces Anser shared-memory channel map API, GUCs, and service hooks. |
| src/include/catalog/pg_proc.dat | Adds built-in gp_anser_* functions for producer/publish/consume_wait transport. |
| src/include/catalog/catversion.h | Bumps catalog version for new built-ins. |
| src/backend/utils/misc/guc_gp.c | Adds Anser GUC definitions (enable/runtime_filter/limits/timeout/marker). |
| src/backend/utils/init/postinit.c | Registers CustomScan providers on backend init so dispatched plans resolve methods. |
| src/backend/storage/lmgr/lwlocknames.txt | Adds Anser LWLock names. |
| src/backend/storage/ipc/ipci.c | Accounts for and initializes Anser shared memory at postmaster start. |
| src/backend/postmaster/postmaster.c | Adds Anser gather/send background workers. |
| src/backend/postmaster/bgworker.c | Registers Anser worker entrypoints. |
| src/backend/optimizer/plan/planner.c | Calls AnserApplyRuntimeFilters() as a post-plan hook (ORCA + PG planner). |
| src/backend/libpq/auth.c | Adds gp_anser_conn startup marker parsing and token-based auth branch. |
| src/backend/lib/bloomfilter.c | Implements bitset accessors and bloom_create_from_bitset. |
| src/backend/executor/nodeAnserBloomFilterProduce.c | Implements executor helper for building/publishing per-producer Bloom parts. |
| src/backend/executor/nodeAnserBloomFilterConsume.c | Implements executor helper for consuming and reconstructing the merged Bloom filter. |
| src/backend/executor/Makefile | Links new executor helper objects. |
| src/backend/cdb/Makefile | Adds anser backend subdir to cdb build. |
| src/backend/cdb/anser/README.md | Documents architecture, transport/auth, GUCs, and channel state machine. |
| src/backend/cdb/anser/Makefile | Builds Anser subsystem objects. |
| src/backend/cdb/anser/anserservice.c | Implements coordinator-local gather/send worker main loops and error recovery. |
| src/backend/cdb/anser/anserplanexec.c | CustomScan execution providers for producer/consumer plan nodes + EXPLAIN stats. |
| src/backend/cdb/anser/anserplan.c | Plan-tree recognition and injection logic (hash join shape) + sizing and token registration. |
| src/backend/cdb/anser/anserfuncs.c | Implements built-in SQL functions backing the transport (producer/publish/consume_wait). |
| src/backend/cdb/anser/anserfilter.c | Implements Bloom payload framing, serialization/deserialization, and fold-in-place union. |
| src/backend/cdb/anser/anserclient.c | Implements libpq client transport used by segments to reach coordinator services. |
| .github/workflows/build-cloudberry.yml | Adds CI job to run src/test/modules/anser installcheck with gp_anser_enable=on. |
Suppressed comments (1)
src/test/modules/anser/anser_test.c:1076
- anser_test_dsm_free_on_cancel() disables the global Anser sweep (AnserSetSweepEnabled(false)) but never re-enables it in PG_FINALLY. Because sweep_enabled is global shared memory state, this can leak into later tests/sessions.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
what's difference with 6c41d27 impl? |
Runtime filter pushdown is intra-slice only. The RuntimeFilter executor node reaches the HashJoin's in-memory hash table via a plain executor pointer (node->hjstate->hj_HashTable, nodeRuntimeFilter.c:83-86); the Hash variant hands bloom scankeys to a registered SeqScan/DynamicSeqScanState in the same process. Nothing ever crosses a Motion or the network. Use the same hash function. Anser is cross-slice, cross-segment. Per-segment blooms are unioned on the coordinator into a global filter; any consumer anywhere can use it. That's the general MPP case from the paper — non-colocated joins, producer and consumer in different slices, even different joins sharing an equivalence-class condition_key. Could use different hash function. The built-in structurally cannot do any of that. But let's return to the example. For the proposed example you are right, existing approach is better. Explain with Explain with The interesting detail is that Another idea is why use separate step for filter out rows? We could push down all filters close to AM-level. And use it, for example in PAX or in the future iceberg approach. I want to implement it in the future but since we are talking about it here, add push-down to seq scan. The true meaning this PR is to add Anser, bloomfilters here just the tool for check how whole system works. I'm going to address all issues in other PR's, where I could just use working system. Here we have for about 8500 lines of the new code ... |
|
Pushed down filters to seq scan. Now execution plan looks like: I know right now it's better not use Anser, but it's the subject for future improvements. |
|
can we use |
Yes, thank you, really I could move it to the extension. The only tricky moment is with the authentication but I think I just could add hook. Converted PR to draft to rewrite code and tests. |
586d05d to
f0682a8
Compare
|
Could be reviewed, thank you. |
|
Alternative: reuse the dispatch connection instead of a backward libpq connection The QD↔QE dispatch connection is already bidirectional mid-execution. nextval() on a QE does exactly this: it sends a NOTIFY to the QD over its own frontend connection and waits for the reply on the same socket (src/backend/commands/sequence.c:2117-2176). The QD handles it inside the dispatcher's receive loop and writes the answer back with a custom message type (cdbdisp_async.c:1160-1195, send_sequence_response at :972). Bloom filter exchange is the same pattern with N producers and M consumers. Sketch:
What this removes: the segment→QD connection, the pg_hba bypass hooks, the token hash, QD shared memory and background workers, the three catalog functions and the catversion bump. Cancellation and error propagation come for free: any QE error makes the QD cancel all gangs, and the consumer's wait loop exits via CHECK_FOR_INTERRUPTS. That also resolves open question 3 about the consumer having no timeout. |
|
Good to know we have a MVP for runtime filter of intra-slice. |
It's brilliant! I converted PR to a draft and try to implement it. This should solve a lot of problems - we won't need to use DSM (and its expensive mmap calls!), everything will be in the query memory context (and will be released after the query ends), there will be no need for background workers (and we will not be limited to 1 CPU; each query will process its own data using its own process). And, of course, there is no need for a cumbersome token-based authentication mechanism. |
|
Great, another questions concerned:
|
|
Thank you, working on it (expect show here some results within the coming week) |
|
Added support of parallel servers Now performing benchmarks (TPC-DS) in order to measure the overall effect. |
|
I performed TPC-DS tests with SF=10 on a 3-hosts cluster (not so big but contains different hosts). Here the raw results: The negative difference shows that query is slowe when using anser. Positive - query is faster when using anser. The difference indeed in using bloom filterts, I described two the most interesting results in #1959 The overall conclusion - for now there are no significant differences. Some queries quite faster, some quite slower, but the overall sum is only 1,5% better - 1023 seconds without anser and 1008 with anser. |
6901d20 to
5dcea32
Compare
61caff7 to
9d00485
Compare
|
In essential, fixed issues:
|
4bc1436 to
14b02c3
Compare
|
Fixed all the issues and checked them manually on my test clusters - they are not reproducible anymore. At least I couldn't do it. ) |
|
A review by |
Anser needs to move a small piece of information -- today a bloom filter over a join key -- from the segments to the coordinator and back, over the dispatch connection that is already open. Four things in the backend stand in the way, and none of them mentions Anser. A QE can already send a NOTIFY to the coordinator, but the dispatcher dropped any it did not recognise. Add cdbdisp_notify_hook so an extension can claim one, and keep the ones that arrive in the same read as a command completion instead of losing them. The other direction had no message type at all. Add GP_SIDEBAND_MESSAGE, which the QE command loop accepts and discards when nobody is waiting -- a waiter can be cancelled after the coordinator has written, and an unknown type would otherwise be fatal. A CustomScan travelled through the binary plan writer as the text " :methods " plus a token, and the reader took the first four bytes of that as a length and asked for 1.7 GB. Any plan with a CustomScan failed to deserialize on the segments; write the name as a counted string instead. catversion is bumped accordingly. Finally, a bloom filter cannot be shipped without reaching its bitset, so expose it along with the sizing its holder must agree on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Anser is a publish/subscribe facility for MPP execution. Producers on the segments publish something about a running query, the coordinator combines the per-segment parts into one, and consumers on the segments receive it and prune work with it. Today that is a bloom filter over a hash join's build key, used to skip probe rows that cannot join. A channel is the rendezvous for one such piece of information in one query, keyed by session, command, and the condition it describes. It lives only in the coordinator backend running the query, for exactly as long as that query runs: no shared memory, no background worker, no second connection to authenticate. Parts travel up as a NOTIFY and the merged payload comes back as a sideband message. Both nodes wrap a base scan rather than an intermediate node, which makes the filter a superset of the keys the joins below would have left -- less selective, never wrong. The direction is fixed to build -> probe, because a consumer on the inner side would wait on a producer the join is not yet reading, which is a deadlock and not a slow path. Every failure ends in unfiltered execution: a filter too small to reject anything, a squelched producer, a malformed part, a missed deadline. The planner gate is the one that costs nothing, and the later ones exist because a row estimate can be wrong. Built only with --with-anser and off by default; the subsystem creates no catalog objects and is configured entirely by GUCs. Includes the regression suites and the CI job that runs them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Anser builds as a MODULE_big with no SHLIB_EXPORTS, so the module is compiled -fvisibility=hidden and every extern in its headers lands in .symtab but not .dynsym. A second extension cannot reach any of it: RTLD_GLOBAL does not help, because a hidden symbol never reaches the dynamic symbol table, and load_external_function() fails for the same reason. Mark the public declarations PGDLLEXPORT, which is the convention the tree already uses for a cross-module C API -- see contrib/hstore and src/pl/plpython. Two plan-resolution helpers are promoted into that API rather than left for a caller to reimplement. Both encode which plan shapes are safe to inject into, so a second copy would be a second place to get that wrong. The pushdown itself is split so the decision can be tested on its own. A scan key is numbered by the table's attributes while the consumer is numbered by the scan's output targetlist, and the two coincide only when the scan emits every column in order -- so the filter was handed a position where an attribute number was wanted, and a projecting scan tested the wrong column and discarded every row. The decision now maps one to the other, and refuses the shapes that have no mapping: a computed column, a whole-row or system Var, a Var of another relation. A second function walks from the consumer to the deepest node that accepts the key, so stacked consumers over one relation all push into the same scan instead of probing tuple by tuple. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A filter that is a subset of the build side's keys rejects rows that join, so every one of these is a wrong answer rather than a slow one. A build side under a nested loop yields different keys per outer tuple, but neither node rebuilds on rescan: the producer stays published and the consumer keeps the filter it has, so the second iteration is probed against the first iteration's keys. Refuse such a join at injection time, by intersecting the loop's nestParams with the build subtree's extParam -- the same test UpdateChangedParamSet makes at runtime. Cleanup was scoped by counting ExecutorEnd, which only ever measures recursion into ExecutorEnd itself. A statement run through SPI from a function body goes through its whole Start/Run/End inside the outer query's Run, so its End looked outermost and took the running query's channels with it. Count over execution instead, as auto_explain does. Two readers share the QD->QE connection, and the nextval one died on a sideband message: it took the type byte, then errored without the body, leaving the stream desynchronised and the session unusable. It now skips one whole and waits on, and Anser no longer subscribes when it has no deadline to wait out. The coordinator could also leave a QE's notification unread for the whole lifetime of the slice that sent it, because PQisBusy() reports the command rather than the queued message, and the bytes may already have been absorbed by a flush. Drain before deciding to wait. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A blocking push stranded the dispatcher uncancellably when a consumer had timed out: deliver without blocking, and let a consumer withdraw. AnserSliceProducers counted a singleton reader and direct dispatch as the whole cluster, so the channel never completed. A producer whose slice ran but whose node never did stayed silent, leaving consumers to wait out the deadline. A consumer ignored QueryFinishPending, so a query the coordinator had already finished -- a LIMIT satisfied early -- still sat out the full deadline. CHECK_FOR_INTERRUPTS does not cover that flag. Injection is now refused when the build and probe sides share a slice. Slices are divided by Motions only, so a co-located join offers no synchronisation point to hang the exchange on and no network cost to save against the filter's own. probe_slice already carried the answer but was never read, and was left uninitialized on one path. Channel cleanup is wholesale, and inside SPI the executor nesting level never returns to zero, so a PL/pgSQL body held one merged filter per statement until the outermost one ended -- about 1 MB a query, and up to the full anser.max_info_size. A query now takes its own channels with it, keyed by the command count the channel already carries. Only finished ones: gp_command_count names the newest query descriptor rather than the one that is ending, and the two differ whenever a portal has outlived the statements that ran after it. Planning joins that nesting level too. Constant folding can run a query through SPI, and that query reached ExecutorEnd believing it was the outermost one -- the split counter PostgreSQL merged in 76db9cb. The anser.timeout_ms default drops from 100 s to 10 s. Its rationale had outlived itself: delivery no longer blocks, so a wide slice no longer needs the headroom it was sized for. The help text said otherwise, in four places. Hash keys are now checked on the base type and against the type's own equality operator. A domain inherits typlen and typbyval, so a domain over float8 passed both the by-value test and a float exclusion that names only the built-in OIDs -- and -0.0 would have stopped matching 0.0, dropping rows that join. The anser_test harness moves to a library of its own. It was linked into anser.so, which every user preloads, and its extension shipped with the subsystem; it now builds only for installcheck. It resolves the C API against the preloaded library at load time, which is what the PGDLLEXPORT markers on the public headers were for. Tests move to redistributed joins to match the Motion rule. The parallel section had also been running on the 1 ms deadline left over from the preceding block, which let it pass without a filter ever arriving; that block has since moved out of the file altogether. It keeps its coverage under GPORCA only, since the Postgres planner marks Motion paths parallel-unsafe. The catversion bump is withdrawn. The CustomScan serialization fix reaches only outfast/readfast, the binary format the dispatcher uses; the text branch that nodeToString writes into catalogs is untouched, and a CustomScan is a Plan node, which catalogs never store anyway. configure-cloudberry.sh no longer passes --with-anser unconditionally, which had put the extension into every CI and release build despite the configure default of no. The ic-anser job does not need it: the gpcontrib/anser target builds, installs and preloads the extension itself, so the matrix entry no longer arms the cluster either -- and the postgres_conf_addons hook added for that is gone with it. anser_test carries its own PG_MODULE_MAGIC. It had been relying on the one in anserinit.c, which stopped covering it when it became a library of its own. anser.max_info_size is synced to the segments. Both ends read it, and a segment hands it to pq_getmessage() as the cap on the message it is about to read, so a disagreement does not merely size things differently -- it frames the wire differently. Without the flag a gang created after the SET fell back to the compiled-in default. Syncing narrows that window rather than closing it, since a gang busy with an open cursor is never sent the SET, so a read that overruns the cap now terminates the connection instead of going on: pq_getmessage takes the length word and leaves the body behind, and a message boundary cannot be recovered from there. The README said POSTMASTER where the code said USERSET, and now says what the trade-off is. SHLIB_PREREQS is dropped. Anser links nothing and only reads libpq's headers, and under PGXS the prerequisite expands to "make -C $(libdir) all", for which there is no such target. preload-anser no longer mistakes tr's exit status for psql's: a psql that failed left the list empty, and the recipe then wrote shared_preload_libraries back as just anser, dropping every other library. It also leaves the cluster alone when it is already armed instead of restarting on every installcheck. The cases that need gp_inject_fault move to anser_faults. The regression driver already excludes a file that calls one when the debug extensions are off -- faultinjector_prep in src/Makefile.global.in -- so sharing a file with them took the rest of anser_runtime_filter down as well. The TAP helper uses pg_hton32 rather than htonl and so no longer needs <arpa/inet.h>, which MSVC does not have. The fold no longer touches the part header. It incremented total_parts once per fold, but a producer already writes the channel's producer count there, so a merged header carried 2N-1 -- which is N only when N is 1, and that is the shape the one test covering it pinned. The number is what a consumer reports and what the log prints when a filter goes missing, so it was wrong exactly where someone would be reading it. A unit test now folds three parts, where N and 2N-1 differ; it needs no cluster and runs from installcheck. anser.max_info_size stops at what the protocol can carry. The bound was INT_MAX, so a large setting overflowed the pq_getmessage() cap computed from it into a negative int and every delivery was then refused -- and anything past PQ_LARGE_MESSAGE_LIMIT could never have been read in the first place. The frame around the payload has a name now rather than being spelled out at the single place that needed it. The outer-join case in anser_multijoin produces NULL keys at last. Its right side held every key the left could offer, and the join above it is strict in that key, so reduce_outer_joins() made the LEFT JOIN an inner one before any path was considered. It now joins a table with gaps, through a subquery that keeps the reduction from applying. The tables the chains rest on are left alone: thinning one of those would have cost eighteen assertions to fix one. No filter reaches that join, and none can. Keeping the outer join costs a SubqueryScan, which the key walk does not descend through, and there is no way around it: a NULL-keyed row survives only under an outer join, an outer join survives only behind a subquery boundary. The block says so now, and checks what it can -- that such a plan returns the same rows with the filter on and off. The pushdown cases say whether anything was injected. They compared a filtered count against an unfiltered one, which also agree when no filter was ever built -- and the computed-key case turns out to be one of those: its subquery is pulled up, the hash clause becomes an expression, and anser_hashjoin_keys takes plain Vars only. The comment claiming otherwise is corrected rather than the behaviour. Every GUC a test pins is reset at the end of it. Three files out of four left one behind, and anser_faults all five. Smaller things: send_fake_sideband_message() checks what pqPutnchar() returns; the nextval reply no longer serializes uninitialized locals on its error path, where a QE reads overflow before error and would report reaching a sequence's maximum rather than the real failure; the condition key's zero-padding is written down beside the struct that requires it; and the comments naming AnserBloomShapeFor, AnserChannelLock and anser_rf_size now name things that exist. configure is regenerated for --with-anser. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thank you, reviewed and fixed all issues 1 - query with |
|
See an interesting issue with delivering large filters from QD to QE, investigating it |
A QE connection whose output buffer still holds bytes at cleanup time carries a partially delivered asynchronous payload (an Anser sideband message; normal command traffic is fully flushed before cleanup). The peer may already have read a prefix of it, and nobody flushes the rest once the command is over -- checkDispatchResult only flushes connections whose command is still running. Reusing such a connection makes the next dispatch overwrite the remainder of the buffer (PQsendGpQuery_shared), and the peer then misreads its stream. Have cleanupQE decline to recycle any connection with pending output, so the next dispatch builds a fresh one.
03eb81c to
bdadf41
Compare
|
I added sending "Ready to read" messages from QE to QD to read large massages (more then 1 Mb). Before the anser, in the middle of the request, only the nextval response is written to the QD->QE connection, which is a number of bytes. They are always completely placed in the socket buffer, with the output after them being zero. Therefore, interconnect calmly waits only for the readable event: there is nothing to add to QD. Anser data cannot be placed in a buffer and should be split to parts. That leads to the deadlock. The detailed explanation how it happens:
We successfully send ~10mb and then wait for the other data to be sent. The rest data is in conn->outBuffer.
That's the problem. Socket will become readable when we got something from segment. We also check every 250ms if there are something interesting in libpq connection. But it does not work for push data further to QE, the code is We could fix the condition here and try to sent data. But it means 250ms wait for each chunk and what if segment really not ready.
Before the fix pqPutMsgEndNoAutoFlush we sent all data in a blocking fashion. Just sent all data to the segment. Issue with that old approach is could leave partially delivered messages and slow segments will block QD in non-interraptable state. We've discussed it earlier, еhere is no point in going back to the previous state. Now we store delivery status in the structure Performance of the whole solution not differ from the previous one. But we could make performance better - sent chunks in parallel. That's the matter for further researches if we decide it's really important. |
Squashes the review-round commits: the delivery-protocol rework, the debug tracing used to diagnose it, and the single-slot QE assembly. A whole-filter push left tens of MB queued in the QD's output buffer, and on the UDP interconnect it was flushed only while a dispatch socket was readable. Once parts and subscribes were consumed, inbound traffic stopped, flushing stopped with it, and the consumer sat in pq_getmessage() with no deadline of its own -- an uncancellable deadlock, seen as a cursor's FETCH 1 never returning. Deliver the merged payload in chunks instead, one GP_SIDEBAND_MESSAGE each, sending the next only when the consumer asks with a NEXT (kind 'N', its part field carrying the offset wanted). At most one chunk -- sized to fit the connection's send buffer -- is ever in flight, so a consumer walking away leaves nothing torn behind, and every NEXT keeps waking the coordinator: the delivery drives its own servicing. Chunk coordinates (total_len, offset) join the CRC, the QE assembles into one slot and drops chunks for keys it has unsubscribed from, and the wait deadline runs from the last progress, with a hard cap from the subscribe. All waits use the monotonic clock. Wire tag is anser4. Also zero-initialises the dispatcher's buffered[] array (defensive).
That's the MVP for the Anser https://vldb.org/pvldb/vol16/p3636-wu.pdf
Here we covered only scenario with adding bloomfilters to the query execution plan.
The main idea is as a follow
The overall execution plan should looks like (see
Custom Scannodes and their stat)The overall timing to gather data:
34.0 ms first-init to last-received; 25.3 ms publish to received.
The main architecture overview
See detailed description in [src/backend/cdb/anser/README.md]
Implementation consists of parts:
Right now working with bloomfilters is part of Anser, but maybe afterwards I will take it out to the new extension (depend of Anser).
Why MVP
For some queries using bloom filters leads to performance degradation. Usually it happens when there are no significant row dataset reduction after bloom filtering. We will address these issue in future researches/PRs.
Also I haven't checked all the cases where bloomfilters could be used.
So for now we support only one simple type of hashjoin, see details in
anser_hashjoin_keysandanser_resolve_build_scanfunctions.Open questions
I saved not implemented ideas as open questions:
Add instrumentation and latency metrics to the Anser subsystem
Implement Robust Predicate Transfer (RPT+) on top of Anser
Adaptive join: correct a wrong Broadcast decision at runtime
Anser: Folding a segment's parallel workers stat before the coordinator sees them
Bloom filter performance: four experiments
TPC-DS
There are no significant differences. Some queries quite faster, some quite slower, but the overall sum is only 1,5% better - 1023 seconds without anser and 1008 with anser. General usage should be carefully studied.
See detailed description queries with increased and decreased performance in #1959
How to enable && test it
Set
test