Investigate unresponsive EKS control plane at scale

The API server seems to be unresponsive when the cluster scales up too quickly. This issues renders the cluster to be unresponsive to kubectl and cortex commands. It also might be related to `cortex cluster down` not cleaning up all of the resources.