-
Notifications
You must be signed in to change notification settings - Fork 15.1k
Add prefer-align-cpus-by-uncorecache feature gate #48340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -189,11 +189,15 @@ The following policy options exist for the static CPU management policy: | |
| `align-by-socket` (alpha, hidden by default) | ||
| : Align CPUs by physical package / socket boundary, rather than logical NUMA boundaries (available since Kubernetes v1.25) | ||
| `distribute-cpus-across-cores` (alpha, hidden by default) | ||
| : allocate virtual cores, sometimes called hardware threads, across different physical cores (available since Kubernetes v1.31) | ||
| : Allocate virtual cores, sometimes called hardware threads, across different physical cores (available since Kubernetes v1.31) | ||
| `distribute-cpus-across-numa` (alpha, hidden by default) | ||
| : spread CPUs across different NUMA domains, aiming for an even balance between the selected domains (available since Kubernetes v1.23) | ||
| : Spread CPUs across different NUMA domains, aiming for an even balance between the selected domains (available since Kubernetes v1.23) | ||
| `full-pcpus-only` (beta, visible by default) | ||
| : Always allocate full physical cores (available since Kubernetes v1.22) | ||
| `strict-cpu-reservation` (alpha, hidden by default) | ||
| : Prevent all the pods regardless of their Quality of Service class to run on reserved CPUs (available since Kubernetes v1.32) | ||
|
Comment on lines
+197
to
+198
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same, fixing a rebase artifact of my previous #48797 |
||
| `prefer-align-cpus-by-uncorecache` (alpha, hidden by default) | ||
| : Align CPUs by uncore (Last-Level) cache boundary on a best-effort way (available since Kubernetes v1.32) | ||
|
|
||
| You can toggle groups of options on and off based upon their maturity level | ||
| using the following feature gates: | ||
|
|
@@ -273,6 +277,24 @@ of `reservedSystemCPUs` and cause host OS services to starve in real life deploy | |
| If the `strict-cpu-reservation` policy option is enabled, the static policy will not allow | ||
| any workload to use the CPU cores specified in `reservedSystemCPUs`. | ||
|
|
||
| ##### `prefer-align-cpus-by-uncorecache` | ||
|
|
||
| If the `prefer-align-cpus-by-uncorecache` policy is specified, the static policy | ||
| will allocate CPU resources for individual containers such that all CPUs assigned | ||
| to a container share the same uncore cache block (also known as the Last-Level Cache | ||
| or LLC). By default, the `CPUManager` will tightly pack CPU assignments which can | ||
| result in containers being assigned CPUs from multiple uncore caches. This option | ||
| enables the `CPUManager` to allocate CPUs in a way that maximizes the efficient use | ||
| of the uncore cache. Allocation is performed on a best-effort basis, aiming to | ||
| affine as many CPUs as possible within the same uncore cache. If the container's | ||
| CPU requirement exceeds the CPU capacity of a single uncore cache, the `CPUManager` | ||
| minimizes the number of uncore caches used in order to maintain optimal uncore | ||
| cache alignment. Specific workloads can benefit in performance from the reduction | ||
| of inter-cache latency and noisy neighbors at the cache level. If the `CPUManager` | ||
| cannot align optimally while the node has sufficient resources, the container will | ||
| still be admitted using the default packed behavior. | ||
|
|
||
|
|
||
| ## Memory Management Policies | ||
|
|
||
| {{< feature-state feature_gate_name="MemoryManager" >}} | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,13 @@ | ||||||||||
| --- | ||||||||||
| title: PreferAlignCpusByUncoreCache | ||||||||||
| content_type: feature_gate | ||||||||||
| _build: | ||||||||||
| list: never | ||||||||||
| render: false | ||||||||||
|
|
||||||||||
| stages: | ||||||||||
| - stage: alpha | ||||||||||
| defaultValue: false | ||||||||||
| fromVersion: "1.32" | ||||||||||
| --- | ||||||||||
| When `PreferAlignCpusByUncoreCache` is enabled while the CPU Manager Policy is set to `static`, containers within a `Guaranteed` pod will individually be aligned to an uncore cache group at a best-effort policy. This feature can optimize performance for certain cache-sensitive workloads by minimizing the cpu allocation across uncore caches. | ||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Try this
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should ideally link to a concept page instead; see #48340 (review)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This feature is a CPU Manager Policy (under Node Resource Managers). Just want to confirm my understanding that you are asking for a standalone concept page for
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Not that, no. I'm asking for this feature gate description to hyperlink to a concept page about CPU manager policies. The target of the hyperlink doesn't have to be specifically about
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The https://kubernetes.io/docs/concepts/policy/node-resource-managers/ concept page is too basic and SIG Node should put some help towards making it more reader friendly. SIG Docs will help.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed in the last upload after rebase on top of #48797 |
||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we can get #48797 merged and merged into v1.32 docs, then the changes here can go there instead (which is what I recommend doing). See #48340 (review)
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Something else we can do: merge this as-is, but track some work to do after the release to make the docs more readable. Adding this extra detail harms the quality and readability of the existing task page, so we should plan that work, but we can accept that harm and not block adding this alpha feature on having it appropriately documented. Ideally we do block beta graduation on getting the docs redone, though. Beta features should have docs that we think are GA quality (and we ship them to find out).
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can and will work on getting #48797 ready anyway (https://kubernetes.slack.com/archives/C0BP8PW9G/p1732284959539839). If we decide to not merge for 1.32 due to the too strict timeline, I'll rebase once the other linked content landed, and once cpumanager content is settled I'll move on to memory manager and others:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is unrelated fix I asked @wongchar to pull in