Questions about Kubernetes upgrades when K8S is running on AKS #1978
Replies: 2 comments 5 replies
-
|
@sebastianlung you have filed this as an issue in the documentation repo. This issue as filed does not meet our team's criteria, specifically "Deploy to AKS" is not a reproduction step, and neither are steps 2 and 3. Pretty much all the key K8S deployment details that an Operator maintainer would like to know are missing. We do not guess in this community. So please provide those details, or at least evidence and details on why you think that something in this specific Operator is directly responsible for the state of the pods. Or feel free to investigate this behavior on your own. This is an open source project after all. |
Beta Was this translation helpful? Give feedback.
-
|
There are a number of things which can cause a node to fail to shut down. The most common tends to be that there is a quorum queue with only the minimum number of online replicas (e.g. in a 3 node cluster, having only 2 replicas), meaning shutting down a node is prevented because it is quorum critical. Either way, we will need more details if we are to understand and investigate this issue. How many nodes are in your cluster? What type of workload are you running (quorum queues? streams? purely classic queues?)? What version of RabbitMQ are you running? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the bug
We are running RabbitMQ within an Azure Kubernetes service.
Since we deployed RMQ, we are having issues, while we do an update of the k8s version.
New nodes are deployed but the old nodes can't be drained completly because the rabbitmq-cluster-server-0/rabbitmq-cluster-server-1 pod is in "Terminating" state but never completly terminated. The only solution I have so far is to force the pod to terminate with:
kubectl delete pod --force --grace-period=0 rabbitmq-cluster-server-0Then we have to manually delete the pvc so they can be attatched to a new node.
Reproduction steps
Expected behavior
All nodes can be drained and the RMQ pods are stopping correctly and no manual interference regarding pvc.
Additional context
No response
Beta Was this translation helpful? Give feedback.
All reactions