Repository navigation
extend sys module on windows #102536
Description
Activity
Yeah, I don't think we want to bother with making them required attributes, so I'd add:
sys.implementation._architectureand make it the value of$(ArchName)from python.props (this approx. matches platform.architecture)sys.implementation._api_partitionsand make it a tuple of all the ones we test for. No need to add names, simpleinchecks will be the most common use
Is
sys.implementationthe right place? It should have things related only to the Python implementation, that other implementations also have. The architecture is a build characteristic and the api partitions seem like it as well. Where do we normally record build or platform (or OS) information?Reacted by Erlend E. AaslandThe only real alternative is at the top level of the
sysmodule. Maybe you could make a case for it going intosysconfig, but we still need an attribute on a native module somewhere, as it's the only way we're getting the field it at build time.I assume this hasn't come up before because Windows is the only OS with transparent CPU emulation? So querying the current OS architecture isn't sufficient to determine the current process architecture.
sys._build.*?Really it should be
sys.platform_but_for_real_this_timeas we can't just go updatingsys.platformany more.What if we added
sys.architecturefor all platforms?platform.architecture(sys.executable)is too limited in its return values to work here, but it ought to be known at compile time, right? And something like Jython or IronPython can useclrorjvmif they prefer. Maybe they've already got an additional field for that?Reacted by Eric Snowthe platform as 32-bit vs 64-bit vs ARM64.
There are multiple existing APIs to do that. Examples:
>>> platform.architecture() ('64bit', 'ELF') >>> os.uname().machine 'x86_64'I don't think that we should add information retrieved at runtime in sys. Usually, sys is more information computed during the build.
Reacted by Erlend E. Aaslandplatform.architecture()doesn't distinguish between ARM64 and x64, andos.uname().machinedoesn't reflect how CPython was compiled. So we need to store the target architecture at build time.But you're correct that we don't need to retrieve it at runtime. (Though worth noting that a lot of
sysis dynamic - path, modules, various hooks, etc.)Hmmm. Actually, the discussion in the PR failed because no one decided whether to merge it or not (it seems that the only supporter is @zooba).
The newly added arch is used to get the current build architecture in Windows (it comes from python.props). What do you think?link:
#124582
https://discuss.python.org/t/regarding-whether-we-should-add-py-currentarch-or-py-archname-function/65191/17My main concern was/is that this information doesn't belong on
sys.implementation. That's meant for information about the Python implementation (e.g. cpython, pypy, jython, micropython), not about the build. See PEP 421.Something like
sys._architecture,sys.build.architecture, orplatform.???or onsysconfig(like @zooba suggested) would be more appropriate. (FWIW, I like the idea ofsys.build.*, PEP-421-style, but its hypothetical contents might already be covered elsewhere.)All that said, I haven't looked closely at what we're trying to actually accomplish here, so I don't have much feedback on that.
Reacted by Erlend E. AaslandI'm often the only supporter of things involving Windows :-) Usually aggregated from a lot of discussions with a range of users, but not easily cited (many Windows users are scared off by the culture here, so they won't report stuff themselves).
In this case, we've been looking at adding support to project's build scripts to allow building on Windows ARM64, which requires the builder to know whether it should be targeted ARM64 or not. That's usually done by running in the ARM64 interpreter, and then to cross-compile you'd run in the x64 or x86 interpreter, but we don't currently have a reliable way to detect that. Historically we haven't needed it, because
sys.maxsize(orsys.maxint) was sufficient to detect 64-bit vs 32-bit. But that can't tell the difference between ARM64 and AMD64.More importantly, and relevant for this case, is that
platform.architecture()can't return the difference, even if it could detect it, because it's defined as only returning "64" or "32". And PEP 421 explicitly suggestssys.implementationis the place to store information that would be returned fromplatformabout the Python implementation. The architecture the current runtime was compiled for sure seems to be related to the implementation, and sinceplatformwould be the obvious place to return such information (as it already is, albeit insufficiently), adding the field tosys.implementationseems fine.Having now re-read that PEP though,1 I think we should drop the underscore. Make it
sys.implementation.architectureand if it exists thenplatform.architecture(sys.executable)can return it rather than querying the actual binary (if it's within the range of allowable return values, and perhaps we can later addplatform.architecture_but_for_real_this_time())Footnotes
-
Specifically Non-required Attributes and Use Cases, the platform and Jython use cases being most relevant. ↩
-
platform.machine()is another possible place for such information. It already has Windows specific code to detect the architecture.Reacted by Erlend E. AaslandReacted by Steve Dowerplatform.processor()also seems like a good candidate. There's also a handful ofplatform.win32_*()APIs. Perhaps a similar one could be added. I agree with Eric and others thatsys.implementationseems unfit for this API.Reacted by Steve Dower and RUANG (James Roy)platform.machine()andplatform.processor()are for different purposes. We want to know the compile-time target of the runtime, which is unrelated to the current machine type (Windows has emulation on most architectures, so the machine could be ARM64 but the runtime is x86 and there's currently no way to determine this).Reacted by RUANG (James Roy)4 remaining items
As far as the discussion is concerned, it seems that only the
_architectureattribute is what we need, because other platforms are not actually as complicated as windows.My main hesitation with defining it as public
sys.architectureis that now we have to figure out sensible values for all the other platforms as well... what do we put for WASI? How do we under-specify it enough that we can expand to new platforms later on and not run intoplatform.architecturetype issues (orsys.platformtype issues)?Reacted by RUANG (James Roy)Platforms that use our
configurebuild system (that is, most platforms) can retrieve compile time stuff viasysconfigAPIs.The
sysconfigmodule sounds like a good place to add the build-time architecture on Windows.Reacted by Peter Bierma and Erlend E. AaslandThe
sysconfigmodule doesn't have a native component, and we don't have a compile-time generated file that ships with the runtime (yet).sysconfigretrieves compile-time stuff on Linux because we include a copy ofMakefile, and it scans it.Storing a single value somewhere in a native module is much easier on Windows than adding an entirely new static file that is generated at compile time. The question is about where we should store the value, more than where it gets retrieved from.
The
sysconfigmodule doesn't have a native component, and we don't have a compile-time generated file that ships with the runtime (yet).sysconfigretrieves compile-time stuff on Linux because we include a copy ofMakefile, and it scans it.Actually, I was just surprised to learn that it looks like a
_sysconfigmodule has snuck in via gh-88402/GH-1100491.Footnotes
-
And to be clear, I consider that a good thing :). More of
sysconfig/_sysconfigdatashould probably make its way there eventually. ↩
Reacted by Erlend E. Aasland-
IMO, this should be implemented by adding
MULTIARCH— which represents a target triple — insys.get_config_vars()on windows, while we don't have a better API. I don't thinksys.implementationis the right place, this is a build detail, not an implementation-specific detail.Reacted by Erlend E. AaslandI was just surprised to learn that it looks like a
_sysconfigmodule has snuck inAh, so it has.
this should be implemented by adding
MULTIARCH— which represents a target tripleGiven that we can embed it at compile time, I'm fine with this. Do we have official target triples for Windows? Or do we just borrow them from someone else (I'm pretty sure Rust uses them, yeah?)
would that be better than adding it to the
implementation? For example, in terms of declarative?It would be added to the
Modules\_sysconfig.cfile (which I'd forgotten about), so it's easy enough to add. It's a bit rough on users, but at least it'll be possible to get at the information.We need a spec for
MULTIARCHthough, to define exactly what the values should look like.That is, it should look like this
>>> import _sysconfig >>> _sysconfig.config_vars() {'EXT_SUFFIX': '.cp314-win_amd64.pyd', 'SOABI': 'cp314-win_amd64', 'Py_GIL_DISABLED': 0, 'ARCH': 'win32'}Edit
(or maybe I didn't output the value correctly).Well, if
EXT_SUFFIXcontainswin_amd64then I'd expectARCHto beamd64, but the spec has to list all possible values and what they mean, across all platforms (or we make it unsupported on some, and it might already exist on others). And it should go in the documentation somewhere.If you want to access build information,
sysconfigis the right module. Theplatformmodule is for information about the platform Python is currently running on.At the time, the platform module was conceived, there was no ARM64 architecture and only 32bit vs. 64bit was important.
It's certainly possible to add another argument
family=''and have this return the architecture family, if it can be detected. The return value ofplatform.architecture()would then have to be turned into a tuple subclass to maintain backwards compatibility and still return the new field as an attribute (similar to howcodecs.CodecInfoworks). family could then return "ARM64", "ARM32", "x86_64", "x86", etc.If you want to access build information, sysconfig is the right module. The platform module is for information about the platform Python is currently running on.
Maybe add an
ARCHfield to sysconfig? This is my understanding of this :(
It would be useful to be able to detect:
@zooba you mentioned adding them to
sys.implementation. Since they are only available on windows, I assume this would mean adding them with an underscore prefix? The alternative would be a separate API similar tosys.winver.This is a follow up to #102256.
Linked PRs
_architecturefield tosys.implementation#124582