ensure secret pulled images

[mikebrow]

As discussed over a somewhat recent SIG-NODE meeting

We desire to add support for ensuring images pulled with pod imagePullSecrets are
always authenticated even if cached. With this feature always pull would not be required to ensure authentication for images pulled with secrets based auth. Instead kublet will check if the image was pulled with an image pull secret and if so would force a pull of the image to ensure the image pulled with the secret is not used by another pod unless that pod also has the proper auth.

Addresses kubernetes/kubernetes#18787

Signed-off-by: Mike Brown [email protected]

[mattjmcnaughton]

@mikebrow thanks for your KEP!

I jotted down some quick questions that I'm hopeful will help drive some more discussion. Thanks :)

[Random-Liu]

I had an offline discussion with @mikebrow. This is a bit different from what he discussed a while back (something similar to #1608 (comment)). However, after he explained the ideas to me, I think it makes even more sense now!

When the kubelet flag ensureSecretPulledImages turns on, the default behavior of kubelet will be changed to always reauth/repull if a pod tries to use an image-secret pair that hasn't been authenticated recently, regardless of what the ImagePullPolicy is.

This seems to be a good first step, and even be a good default behavior in the future (no need to change the pod spec). Based on my understanding, usually people use cached images with PullNever or PullIfNotPresent for 2 reasons:

1. To avoid re-pulling the image as a performance optimization.
2. To use preloaded images because the node doesn't have internet access or the image is not pushed to any registry.

The "new default behavior" seems safe and won't break the 2 cases above.
For case 1), in the worst case, we'll only do an extra re-auth, so it shouldn't affect performance much. And if users use the image with the previously authenticated secrets, they can even avoid the re-auth.
For case 2), you usually don't need secrets to access those preloaded image anyway.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[mikebrow]

/remove-lifecycle stale

[mikebrow]

Ready to go forward with this on a 1.20 schedule now that the 1.19 log jamb is behind us :-)

[mikebrow]

Updated to address comments and reflect detail around the current implementation kubernetes/kubernetes#94899

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[mikebrow]

/remove-lifecycle stale

[mikebrow]

updated for 1.23 and added test plan detail..

[johnbelamaric]

PRR is OK, but needs SIG approval

[ehashman]

/approve

for PRR

[derekwaynecarr]

can you update the kep to reflect agreement that the feature should maintain security across kubelet restart and node reboot events?

the alpha could be restricted to an in-memory map, but the beta criteria should meet the above goals?

thanks!

[derekwaynecarr]

thanks for the prompt updates.

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, ehashman, mikebrow

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/prod-readiness/OWNERS [ehashman]
• keps/sig-node/OWNERS [derekwaynecarr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment