nerdctl stats on a container without a memory limit reports exbibyte level memory limits

[ningziwen]

Description

When running nerdctl stats without a memory limit on a container, nerdctl is reporting the limit as exbibyte level.

There is one issue related to this in finch repo.

I successfully reproduced this with nerdctl in ubuntu by building from source.

My ubuntu host has 8GB memory.

Steps to reproduce the issue

1. git clone https://github.com/containerd/nerdctl
2. make
3. nerdctl run --detach ubuntu sleep 120
4. nerdctl stats

Describe the results you received and expected

In my 8GB memory ubuntu host,
Received:

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl run --detach ubuntu sleep 120
0ab2b121f1f0aa91ffae45f527716505d98ee3ab14b438ae77c8c8be0f1817df

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl stats
CONTAINER ID   NAME           CPU %     MEM USAGE / LIMIT   MEM %     NET I/O         BLOCK I/O   PIDS
0ab2b121f1f0   ubuntu-0ab2b   0.00%     728KiB / 8EiB       0.00%     1.31kB / 682B   0B / 0B     1

Expected:

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl run --detach ubuntu sleep 120
0ab2b121f1f0aa91ffae45f527716505d98ee3ab14b438ae77c8c8be0f1817df

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl stats
CONTAINER ID   NAME           CPU %     MEM USAGE / LIMIT   MEM %     NET I/O         BLOCK I/O   PIDS
0ab2b121f1f0   ubuntu-0ab2b   0.00%     728KiB / 8GiB       0.00%     1.31kB / 682B   0B / 0B     1

What version of nerdctl are you using?

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl  version
WARN[0000] unable to determine buildctl version: exec: "buildctl": executable file not found in $PATH 
Client:
 Version:       v1.0.0-139-g330a916
 OS/Arch:       linux/amd64
 Git commit:    330a9166052246e98235b088e911cfde9c8e343e
 buildctl:
  Version:

Server:
 containerd:
  Version:      1.6.9
  GitCommit:    1c90a442489720eec95342e1789ee8a5e1b9536f
 runc:
  Version:      1.1.4
  GitCommit:    v1.1.4-0-g5fd4c4d

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

Admin:~/nerdctl (main) $ sudo ./_output/nerdctl  info
Client:
 Namespace:     default
 Debug Mode:    false

Server:
 Server Version: 1.6.9
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Log: fluentd journald json-file syslog
  Storage: aufs native overlayfs
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-1089-aws
 Operating System: Ubuntu 18.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.483GiB
 Name: ip-172-31-45-188
 ID: 355b9769-ae6b-40e2-90c8-e07c646904ee

WARNING: No swap limit support

[djdongjin]

In docker, when memory limit is not set, the daemon (maintains and) returns the total memory of the machine (by reading /proc/meminfo). So the client don't need extra logic or interact with system info like /proc/meminfo.

I remember the limit is only shown in unix/linux, not windows.

Seems containerd only set the limit when it's set by a user. If we want the sam behavior, should we follow the same pattern and let containerd matains and returns the system memory as limit, or nerdctl should take care of this?

[fahedouch]

In docker, when memory limit is not set, the daemon (maintains and) returns the total memory of the machine (by reading /proc/meminfo). So the client don't need extra logic or interact with system info like /proc/meminfo.

I remember the limit is only shown in unix/linux, not windows.

Seems containerd only set the limit when it's set by a user. If we want the sam behavior, should we follow the same pattern and let containerd matains and returns the system memory as limit, or nerdctl should take care of this?

This should be fixed on the API side (containerd) to keep considtency with other consumer (e.g ctr..). Here we are just consuming metrics returned by API

[ningziwen]

/assign
🙋‍♂️

[ningziwen]

The line to change in cgroup v2: https://github.com/containerd/cgroups/blob/fc3b020f386da06f1facfddef993420644875926/cgroup2/utils.go#L253

[kzys]

Seems containerd only set the limit when it's set by a user. If we want the sam behavior, should we follow the same pattern and let containerd matains and returns the system memory as limit, or nerdctl should take care of this?

Do we want the same behavior? How about showing NA since there are technically no limits?

[AkihiroSuda]

Seems containerd only set the limit when it's set by a user. If we want the sam behavior, should we follow the same pattern and let containerd matains and returns the system memory as limit, or nerdctl should take care of this?

Do we want the same behavior? How about showing NA since there are technically no limits?

The output should be same as Docker unless there is a specific reason

[ningziwen]

Decision discussion in cgroup draft PR. containerd/cgroups#265 (comment)

[AkihiroSuda]

This was resolved recently