Tracker for cgroups v2

[vbatts]

Cgroups unified hierarchy has been on the horizon for a while, and is now reaching critical mass to begin switching over. Most all the controllers or equivalent are there, but there are mappings needed.

This issue should serve as the tracker for remaining issues to close this gap.

• ensure ebpf for devices maps to existing configs (device cgroup had a minor incompatibility (blacklisting isn't really possible))
• freezer controller (expected in linux 5.2)
• cgroupfs and/or systemd API
• There's no longer separate limits for kmem and kmem.tcp
• There's no longer a file to set "swappiness" for a single cgroup hierarchy
• There's no longer a way to disable OOMs through the memcg

Related #948

[vbatts]

cc @filbranden for several of these are his findings

[AkihiroSuda]

@giuseppe are you willing to propose the crun conversion spec to OCI Runtime Spec?
https://github.com/giuseppe/crun/blob/master/crun.1.md#cgroup-v2

[giuseppe]

@giuseppe are you willing to propose the crun conversion spec to OCI Runtime Spec?
https://github.com/giuseppe/crun/blob/master/crun.1.md#cgroup-v2

I can. I was already discussing them with @filbranden, @vbatts and @mrunalp.

I wonder if someone else beside me played with it though :-)

[h-vetinari]

@giuseppe, do you mind if I send a PR with the cgroups conversion specs from crun? Might still have a chance to squeeze in such non-invasive changes before the 1.0.2 release...

[giuseppe]

@giuseppe, do you mind if I send a PR with the cgroups conversion specs from crun? Might still have a chance to squeeze in such non-invasive changes before the 1.0.2 release...

sure, I've no problems with that

[h-vetinari]

@h-vetinari: @giuseppe, do you mind if I send a PR with the cgroups conversion specs from crun? Might still have a chance to squeeze in such non-invasive changes before the 1.0.2 release...

For the record, I had less time than expected in February, but I tried once, and found the undertaking more complex than expected (hence no PR).

For example: should the conversion functions be already defined in the runtime-spec? Currently lots of projects have to redundantly define their conversions (e.g. opencontainers/runc#2212, opencontainers/runc#2213, kubernetes/enhancements#1370, containerd/cgroups#111, containerd/cgroups#143, kubernetes/kubernetes#85218, etc.), but I'm not sure if the runtime-spec is intended to host any functional code like conversion functions?

[giuseppe]

For example: should the conversion functions be already defined in the runtime-spec? Currently lots of projects have to redundantly define their conversions (e.g. opencontainers/runc#2212, opencontainers/runc#2213, kubernetes/enhancements#1370, containerd/cgroups#111, containerd/cgroups#143, kubernetes/kubernetes#85218, etc.), but I'm not sure if the runtime-spec is intended to host any functional code like conversion functions?

I think it should not. It will be better to define a way to set cgroup v2 values without any conversion. It will be a breaking change for the entire stack, but it is cleaner than the conversion done now.

I thought of the cgroup v1 -> cgroup v2 conversion as a temporary step and to allow an incremental implementation of cgroup v2 in the other components without requiring changes in the OCI specs.

[vbatts]

woof. i'm not keen on a breaking change. How can we introduce v2 values, but allow a failback to conversion?

[giuseppe]

we had some discussions in the past with @filbranden and @mrunalp about having a separate config section for cgroup v2, but at this point I think we can just add cgroup v2 configuration as part of the resources block as it is now. There should not be conflicts in the names: https://github.com/containers/crun/blob/master/crun.1.md#cgroup-v2

So we could have something:

      "devices": [
      "cpu": {
        "shares": 2
      },
      "pids": {
        "limit": 4096
      }
    },

and:

    "resources": {
      "devices": [
      "cpu": {
        "weight": 1
      },
      "pids": {
        "limit": 4096
      }
    },

to be equivalent