Support python -Xcpu_count=<n> feature for container environment.

@vstinner

Feature or enhancement

As #80235, there are requests for isolating CPU count in k8s or container environment, and this is a very important feature these days. (Practically my corp, a lot of workloads are running under container environments, and controlling CPU count is very important to resolve busy neighborhood issues)

There were a lot of discussions, and following the cgroup spec requires a lot of complexity and performance issues (due to fallback).
JDK 21 chooses not to depend on CPU Shares to compute active processor count and they choose to use -XX:ActiveProcessorCount=<n>.
see: https://bugs.openjdk.org/browse/JDK-8281571

I think that this strategy will be worth using from the CPython side too.
So if the user executes the python with -Xcpu_count=3option os.cpu_count will return 3, instead of the actual CPU count that was calculated from os.cpu_count.

cc @vstinner @indygreg

Linked PRs

I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account. But we should provide the two values: taking affinity in account, and "total" number of CPUs. It might be tricky to change the default :-(

It might be tricky to change the default :-(

Providing -Xcpu_count will not touch any default behavior until the user explicitly set the value. So it will be the expected side-effect from the user side.

I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account

We need to understand the actual user's usecase. AFAIK k8s users or container users never use something like taskset to isolate CPU resources. They just write a Docker file and set the resources to k8s pods.
Even if we update os.cpu_count to reflect sched_getaffinity, it will not be directly helpful to actual container people.

I think that that's why JDK still provides -XX:ActiveProcessorCount=<n>.
Until we do not provide a practical way to limit CPU resources, there is no way to reflect os.cpu_count to follow their limitation.
And as you know, container application is veeeery important use case in modern applications.

os.cpu_count() is used in other use cases than containers, where CPU affinity is used.

If we modify cpu_count() to take affinity in account and/or if your -X option is implemented, we should add an option to get "total number of CPUs".

AFAIK k8s users or container users never use something like taskset to isolate CPU resources.

I have used taskset in Kubernetes. It is useful to pin processes to specific CPU cores without using something more advanced/modern like the CPU Manager feature.

One of the things I used it for was testing the behavior of a JVM when passing -XX:ActiveProcessorCount= to a different value than the number of CPUs being allowed to schedule on.

There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)

I created issue #109649: os.cpu_count(): add "affinity" parameter to get the number of "usable" CPUs.

UPDATE: I renamed the usable parameter to affinity.

There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)

Yeah, this is exactly what I want to say :) Thank you for emphasizing!

pythongh-109595: Add -Xcpu_count=<n> cmdline for container users

The reason I don't support cgroup directly is the following reasons.

cgroups 've existed even before docker or podman borned: https://docs.kernel.org/admin-guide/cgroup-v1/cgroups.html
so there are few variation of representing CPU resources and it makes hard to decide which value that the user want to use

docker run ... --cpu-shares=512 java .... ==> os::active_processor_count() = 1
docker run ... --cpu-shares=1024 java ... ==> os::active_processor_count() = 32 (total CPUs on this system)
docker run ... --cpu-shares=2048 java ... ==> os::active_processor_count() = 2

It will increase overhead itself to os.cpu_count itself, if we decide to parse cgroup, we need to parse cgroup anyway by default, AFAIK there is no way to avoid this path if we decide to support cgroup as seamless approach to what people really want.
Supporting cgroupv2 is much harder than cgroupv1, it will increase the maintenance cost for cpython codebase, and we need to follow up new specifications if cgroupv3 is released.
Let's not follow what JDK already tried and failed: https://bugs.openjdk.org/browse/JDK-8281571
Adding a new flag to os.cpu_count() will not propagate to all libraries that the program depends on but this solution will do.

Is there an appetite for allowing non-integer / special values to -Xcpu_count= that define the mechanism to derive the CPU count? e.g.

-Xcpu_count=system-processor-count would use the equivalent of os.cpu_count() today.
-Xcpu_count=cgroups-shares-1024 would read from cgroups cpu.shares and divide by 1024.
-Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.

Obviously we would need to bikeshed the names and behavior bit. But I see this as a potential elegant solution that not only allows forceful count overrides but also allows dynamic derivation using well-defined semantics. It also gives CPython more flexibility to introduce new variants/behavior without breaking backwards compatibility. It seemingly placates all parties.

-Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.

As the author of PR #109649, I'm interested by this mode 😁

It seemingly placates all parties.

It seems like there is no good default behavior fitting all use cases. Giving more choices should please more people.

Since you added PYTHONCPUCOUNT env var, the effect of the env var is wider than a single process, child processes are affected as well.

Can you please add a -X cpu_count value to ignore PYTHONCPUCOUNT: in short, get Python 3.12 behavior (total number of CPUs) when it is really what I want? Maybe it can just be:-X cpu_count=default.

See my example for a concrete use case: #109652 (comment)

In the UTF-8 mode, I did something similar: PYTHONUTF8=1 python -X utf8=0 ignores PYTHONUTF8 env var and ensures that the UTF-8 Mode is not enabled to this command.

@mdboom

I propose to:

Add os.process_cpu_count(): PR gh-109649: Add os.process_cpu_count() function #109907

Add PYTHONCPUCOUNT=value env var and -X cpu_count=value which overrides os.cpu_count() and os.process_cpu_count() with a number.

For use case (B), display the number of system CPUS, I would like to support -X cpu_count=default (affect current process) which overrides PYTHONCPUCOUNT (system wide).

I like these proposals, they are along the lines of what we can actually accomplish and provide clear concrete results. We're exposing raw os information and allowing the user to override that when they deem appropriate for practical purposes.

Answering deeper questions such as things pertaining to efficiency vs performance, throughput, latency, Linux cgroup shares, other container configs, cpu hotplug, or what VM cores actually are vs what an underlying hypervisor may be configured to actually schedule "core" processes on... are IMNSHO better left to continually evolving PyPI libraries. Because those are pretty fluid concepts.

For example, for practical reasons, software may choose to adopt higher level concept libraries such as https://pypi.org/project/threadpoolctl/ which @mdboom mentioned on Discord. Because application processes and libraries are often not isolated and work best if they coordinate which fractions of available compute resources they each use with the other cross-language libraries and processes all being used at once for their common goal. A raw numbers of available logical cores alone can't accomplish that.

I like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API.

I like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API.

I have mixed feeling about reading cgroups in Python stdlib. What I care the most here is to make sure that with the proposed designed, tomorrow, we can still change our mind, and read cgroups.

I'm saying that because my first proposition was to add an affinity parameter to os.cpu_count(). It's simple. It's easy. Why not? Well. If tomorrow, we read cgroups, what does it mean? We have to add a second cgroups parameters? Now what? The correct would look like:

kwargs = {}
if sys.version_info >= (3, 13):
    kwargs['affinity'] = True
if sys.version_info >= (3, 14):
    kwargs['cgroups'] = True
cpu_count = os.cpu_count(**kwargs)

Hu! That's not convenient.

I prefer a separated function which has no parameter: process_cpu_count(). It gets CPU affinity if available, or not. If tomorrow we consider reading cgroups, again, it would still fit into "return the number of CPU usable by the current process".

Yeah, I like the proposal #109595 (comment), it will makes better situation for exceptional environment of Python users :)

Merge remote-tracking branch 'upstream/main' into pythongh-109595

gh-109595: Add -Xcpu_count=<n> cmdline for container users (#109667)

FYI, I am preparing -Xcpu_count=process option with the separate PR.

Would you mind to create a separated PR for -Xcpu_count=process?

pythongh-109595: Update -Xcpu_count=<n> cmdline to support process mode

Follow-up issue: #110649: Add -Xcpu_count=process cmdline mode to redirect os.cpu_count as os.process_cpu_count.

did the PYTHONCPUCOUNT env var survive to the main branch? If not, how can I control this behavior from the environment, without touching existing code? (Hoping os.cpu_count() will return the reduced number of cores)

did the PYTHONCPUCOUNT env var survive to the main branch?

It was added to Python 3.13 (you can test beta1) and it's documented at: https://docs.python.org/dev/using/cmdline.html#envvar-PYTHON_CPU_COUNT

pythongh-109595: Add -Xcpu_count=<n> cmdline for container users (pyt…

Navigation Menu

Repository navigation

Support python -Xcpu_count=<n> feature for container environment. #109595

Feature or enhancement

Linked PRs

7 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Support python -Xcpu_count=<n> feature for container environment. #109595

Description

Feature or enhancement

Linked PRs

Activity

vstinner commented on Sep 20, 2023

corona10 commented on Sep 20, 2023

vstinner commented on Sep 21, 2023

indygreg commented on Sep 21, 2023

vstinner commented on Sep 21, 2023

corona10 commented on Sep 21, 2023

corona10 commented on Sep 22, 2023

indygreg commented on Sep 22, 2023

vstinner commented on Sep 22, 2023

vstinner commented on Sep 22, 2023

vstinner commented on Sep 22, 2023

7 remaining items

gpshead commented on Sep 29, 2023

vstinner commented on Sep 29, 2023

corona10 commented on Sep 30, 2023

corona10 commented on Oct 10, 2023

vstinner commented on Oct 10, 2023

vstinner commented on Oct 10, 2023

blackliner commented on May 23, 2024

vstinner commented on May 23, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions