Multiple containers in the same cgroup

[kolyshkin]

Currently runc allows multiple containers to share the same cgroup. While such shared configuration might be OK, there are some issues.

1. When each container has its own resource limits, the order of start determines which limits will be effectively applied.
2. When one of containers is paused, all others are paused, too.
3. When a container is paused, any attempt to do runc create/run/exec end up with runc init stuck inside a frozen cgroup.
4. When a systemd cgroup manager is used, this is not working at all -- such as, stop (or even failed start) of any container results in stop unit being sent to systemd, and so (depending on unit properties) other containers can receive SIGTERM, be killed after a timeout etc.

All this may lead to various hard-to-debug situations in production (runc init stuck, cgroup removal error, wrong resource limits etc).

I originally found issue 3 from the list above and tried to solve it in #3131, and this is how I found issue 4.

What can be done to avoid these bad situations?

1. Require that the cgroup (systemd unit) of a container being created is not present.
2. Require that the cgroup (systemd unit) of a container being created is either not present or empty (no processes).
3. Require that the cgroup is not frozen before running runc init (this is what runc run/create: refuse non-empty cgroup; runc exec: refuse frozen cgroup #3131 does).

Surely, these measures might break some existing usage scenarios. This is why:

• runc 1.1 will warn about non-empty cgroup for a new container (runc run/create);
• runc 1.2 will make this warning an error.

[kolyshkin]

I am going to implement

• solution 2 for runc create/run (i.e. require that the cgroup of a container to be created is empty (no processes), and
• solution 3 for runc exec (i.e. require that the cgroup is not frozen).
  as the least intrusive way to handle this issue.

[kolyshkin]

It seems that crun is equally affected (filed containers/crun#716).