[Varun Bhandari (varunbhandarii)]

While integrating ALEX into a transactional database
(learned-index-mako), a randomized
differential test run under AddressSanitizer, UndefinedBehaviorSanitizer and
ThreadSanitizer found the problems below. Each fix comes with a regression test that fails
under the sanitizers without it, and all existing tests still pass.

Problem	Report before the fix	Fix
find_lower() / find_upper() can pass pos == data_capacity_ to get_next_filled_position(), which then reads bitmap_[pos >> 6]: one word past the bitmap when the capacity is a multiple of 64	ASan: heap-buffer-overflow	Return data_capacity_ early
extract_rightmost_one() negates the word as int64_t, which overflows when only the top bit is set	UBSan: negation of -9223372036854775808 (47 reports in the existing unit tests)	Negate as uint64_t; same bits
find_best_fanout_existing_node() casts nodes to AlexDataNode<T, P> with default template arguments, so an Alex with allow_duplicates = false (or another allocator) uses its nodes through the wrong type when splitting	UBSan: downcast of address which does not point to an object of type 'AlexDataNode'	Template the function on the instance's data node type
delete_node() downcasts each node again after destroying it	Undefined behavior	Cast once, before destroying
Unit tests that build data nodes by hand allocate arrays with new[], but nodes free them through std::allocator	ASan: alloc-dealloc-mismatch (12 reports)	Allocate with std::allocator; set bitmap_size_

Testing

• With -fsanitize=address,undefined, the unit tests (45 test cases, 192,499 assertions)
  pass with no sanitizer reports. Before this change, the existing tests produced 47 UBSan
  and 12 ASan reports.

• With the changes to src/core reverted, each new test fails under the sanitizers:
  TestNextFilledPositionAtCapacity (heap-buffer-overflow), TestExtractRightmostOne
  (signed overflow) and TestInsertWithoutDuplicates (invalid downcast).

• The normal CMake build (-O3 -march=native) and ctest pass.

• GCC 12, Ubuntu 22.04. Sanitizer build:

  g++ -std=c++14 -O1 -g -march=native -fsanitize=address,undefined -fno-omit-frame-pointer \
      -Isrc/core -I<doctest dir> test/unittest_main.cpp -o test_alex && ./test_alex

Separately: lookups update per-node statistics (num_lookups_,
num_exp_search_iterations_), so concurrent readers race even under a shared lock. I can
send a small opt-out macro for that as a separate PR if it would be useful.

[马泽明 (ZEMINGMA)]

已收到，感谢来信。祝好！

[Varun Bhandari (varunbhandarii)]

@microsoft-github-policy-service agree