Skip to content

Extend cython to cover inserts as well - #207

Open
TheDistributor wants to merge 2 commits into
masterfrom
mgallwey/use-cython-for-writes
Open

TheDistributor wants to merge 2 commits into
masterfrom
mgallwey/use-cython-for-writes

Conversation

@TheDistributor

@TheDistributor TheDistributor commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Extend cython acceleration to cover the write path as well.

Add new 'encode_batch_rows' and 'decode_batch_results' native methods to accelerate inserts and decoding of batch insert results.

Mark 'cryptography' as a require dependency as performance is just too slow without it and it led me down the garden path at least twice with this changeset.

Extend tests with many more correctness tests.

Extend benchmark tests with more tests for scaled and binary types.

Impact on performance according to the benchmarks run against the branch by CircleCI:

Test                                      master min (ms)  branch min (ms)     delta (ms)    delta %
test_insert_binary                                 12.877           11.039         -1.839    -14.28%
test_insert_large                                 205.736          156.603        -49.134    -23.88%
test_insert_scaled_types                           31.610           24.227         -7.382    -23.35%
test_insert_small                                  14.021           11.068         -2.954    -21.06%

@TheDistributor
TheDistributor marked this pull request as ready for review October 7, 2026 14:30
Comment thread pynuodb/_cutil.h
NUODB_INLINE int _pynuodb_clz64(unsigned long long x)
{
unsigned long idx;
_BitScanReverse64(&idx, x);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to work on 32-bit systems?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's a 32 bit system??

Comment thread setup.py
Comment on lines +14 to +18
`cryptography` is a required dependency: sessions are always encrypted, and
without it the driver falls back to a pure-Python RC4 implementation that is
dramatically slower (an O(n) Python byte loop over every wire message) --
slow enough to dominate the cost of any batch insert/select, masking
unrelated performance work. Installing pynuodb always pulls it in.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to run this change by our Release team.

@pflynnnuodb

Copy link
Copy Markdown

Are the benchmark results a bit underwhelming? I seem to remember the other ones being a lot better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants