Repository navigation
Add Py_TuplePack2 and Py_TuplePack1 #118222
Description
Activity
How large is the difference between
PyTuple_Pack(),_PyTuple_FromArray(),PyTuple_Pack1()andPyTuple_Pack2()?I afraid that it is a preliminary optimization. We need new API if:
- It allows to express what was not possible with the old API. It is obvious.
- It allows to avoid the overhead of more general API for common cases.
- It is much more convenient in common cases. Usually it means less lines of code.
PyTuple_Pack()is already a convenient API in comparison withPyTuple_New()+Py_INCREF()+PyTuple_SET_ITEM(). And it is fast enough, faster than more generalPy_BuildValue(), perhaps even faster thanPyTuple_New()+Py_IncRef()+PyTuple_SetItem().@serhiy-storchaka Thanks for the response. I agree the need for a new API seems slim. The only reason would be performance.
Here are some results: with the benchmark script from https://discuss.python.org/t/nwise-itertools/51718/17 we can compare the performance of the cpython
pairwise(implemented in C) and a simple recipe (python only). With current main the python recipe is faster than the cpython C version.Main:
11.1 ± 0.1 μs consume(recipe(iterable)) 13.1 ± 0.2 μs consume(pairwise(iterable)) Python: 3.13.0a6+ (heads/putuple_pack2:8f25cc9920, Apr 30 2024, 11:26:49) [MSC v.1939 64 bit (AMD64)]With
PyTuple_Pack2(implementation main...eendebakpt:cpython:putuple_pack2) the two versions are almost as fast:11.1 ± 0.1 μs consume(recipe(iterable)) 11.5 ± 0.0 μs consume(pairwise(iterable)) Python: 3.13.0a6+ (heads/putuple_pack2-dirty:f2431b38e7, Apr 30 2024, 11:23:04) [MSC v.1939 64 bit (AMD64)]So using
PyTuple_Pack2gives a performance improvement of 10-15%.I will keep this issue open a few more days to see if there are different opinions or more use cases where the performance difference is important. If not I will close the issue and (perhaps) make a PR to update just the
pairwiseimplementation with a directPyTuple_New() + Py_INCREF() + PyTuple_SET_ITEM()
Feature or enhancement
Proposal:
In the cpython codebase
PyTuple_Packis used at various places (https://github.com/search?q=repo%3Apython%2Fcpython+PyTuple_Pack&type=code). The execution is not very fast as the implementation usesva_arginternally. For the 1- and 2 argument case we can improve performance by providing a direct implementation.Using
PyTuple_Pack2(example implementation main...eendebakpt:cpython:putuple_pack2) performance of components likepairwisecan be improved. See https://discuss.python.org/t/nwise-itertools/51718/22Py_TuplePack1andPy_TuplePack2to the cpython interface? If so, should it be in the public or private API.Py_TuplePack2should we use it at all places applicable, or only the few performance critical ones?Py_TuplePack1that already exists is the internal_PyTuple_FromArraywith second argument1.Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response