Activity

1. finnagin

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 17, 2023
2. eryksun

   added

   OS-windows

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   on Jul 18, 2023
3. sobolevn

   added 2 commits that reference this issue on Jul 18, 2023

   pythongh-106844: Fix null-bytes handling in `LCMapStringEx` in `_winapi`

   b4bdc0a

   pythongh-106844: Fix null-bytes handling in `LCMapStringEx` in `_winapi`

   dff8b2f
4. serhiy-storchaka

   added a commit that references this issue on Aug 10, 2023

   pythongh-106844: Fix null-bytes handling in `LCMapStringEx` in `_winapi`

   eb7fa9d
5. serhiy-storchaka commented on Aug 11, 2023

   

   serhiy-storchaka

   on Aug 11, 2023

   Member

   More actions

   There is a problem not only with the null character handling.

   I tested on 3.12rc1. Note that -1 passed as the input size, so LCMapStringEx() determines the size of the string by the terminating null character, so it is not a limitation of the API.

   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32+5))
   >>> len(x)
   5
   >>> del x
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31+5))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   MemoryError
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**30+5))
   >>> len(x)
   1073741829
   >>> len(x) - 2**30
   5
   >>> del x
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-1))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   MemoryError
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-2))
   >>> len(x) - 2**31
   -2
   >>> del x
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-2))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   MemoryError
   >>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-1))
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   OSError: [WinError 6] The handle is invalid

   • Size up to 2**31-2 including works correctly.
   • Size between 2**31-1 and 2**32-2 raises MemoryError. I think that LCMapStringEx() return a correct size, but converted to int it becomes negative, than it is promoted to Py_ssize_t, and interpreted as a very large size_t value, which causes a MemoryError.
   • Size 2**32-1 raises OSError. LCMapStringEx() returns the output size including the terminating null character. (int)(2**32-1 + 1) == 0, and it is interpreted as a signal of error.
   • Size larger than 2**32-1 loses highest bits.

   Rejecting size larger than INT_MAX is safe, in any case LCMapStringEx() does not support it now (and in worst case it silently returns a truncated result). Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows. And with different content we can get an OSError for strings smaller than INT_MAX characters.

   Perhaps we can make it supporting strings between 2**31-1 and 2**32-2 characters long if cast sizes to unsigned int. But I am not sure that it is guaranteed and that we need this for normcase().
6. serhiy-storchaka commented on Aug 11, 2023

   

   serhiy-storchaka

   on Aug 11, 2023

   Member

   More actions

   Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows.

   No, it always less than INT_MAX on 32-bit Windows.
7. serhiy-storchaka

   added 4 commits that reference this issue on Aug 11, 2023

   gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832)

   ...

   04cc014

   [3.12] pythongh-106844: Fix issues in _winapi.LCMapStringEx (pythonGH…

   ...

   21a0b8d

   [3.11] pythongh-106844: Fix issues in _winapi.LCMapStringEx (pythonGH…

   ...

   a1a91e7

   [3.11] gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832) (GH…

   ...

   ec254c5
8. zooba

   added a commit that references this issue on Aug 15, 2023

   pythongh-106844: Fix issues in _winapi.LCMapStringEx (pythonGH-107832)

   ...

   5d83c72
9. Yhg1s

   added a commit that references this issue on Aug 16, 2023

   [3.12] gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832) (#1…

   ...

   b5176a8
10. serhiy-storchaka

    closed this as completedon Aug 16, 2023