[Zen2] Introduce vote withdrawal #35446
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If shutting down half or more of the master-eligible nodes, their votes must first be explicitly withdrawn to ensure that the cluster doesn't lose its quorum. This works via _voting tombstones_, stored in the cluster state, which tell the reconfigurator to remove nodes from the voting configuration. This change introduces voting tombstones to the cluster state, together with transport APIs for adding and removing them, and makes use of these APIs in `InternalTestCluster` to support tests which remove at least half of the master-eligible nodes at once (e.g. shrinking from two master-eligible nodes to one).
DaveCTurner
added
>enhancement
v7.0.0
:Distributed Coordination/Cluster Coordination
|
Pinging @elastic/es-distributed |
DaveCTurner
commented
Nov 12, 2018
server/src/test/java/org/elasticsearch/cluster/routing/AllocationIdIT.java
Show resolved
Hide resolved
ywelsch
reviewed
Nov 12, 2018
...n/java/org/elasticsearch/action/admin/cluster/configuration/AddVotingTombstonesResponse.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/action/admin/cluster/configuration/ClearVotingTombstonesRequest.java
Outdated
Show resolved
Hide resolved
...org/elasticsearch/action/admin/cluster/configuration/TransportAddVotingTombstonesAction.java
Outdated
Show resolved
Hide resolved
...org/elasticsearch/action/admin/cluster/configuration/TransportAddVotingTombstonesAction.java
Outdated
Show resolved
Hide resolved
...org/elasticsearch/action/admin/cluster/configuration/TransportAddVotingTombstonesAction.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/support/ActionFilters.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/cluster/ClusterState.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/cluster/ClusterState.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/elasticsearch/cluster/routing/AllocationIdIT.java
Show resolved
Hide resolved
|
@elasticmachine test this please |
61 tasks
ywelsch
reviewed
Nov 13, 2018
...org/elasticsearch/action/admin/cluster/configuration/TransportAddVotingTombstonesAction.java
Outdated
Show resolved
Hide resolved
test/framework/src/main/java/org/elasticsearch/test/InternalTestCluster.java
Outdated
Show resolved
Hide resolved
ywelsch
approved these changes
Nov 13, 2018
| final int nodesToRemain = internalCluster().size() - 1; | ||
| logger.info("--> reducing to [{}] nodes", nodesToRemain); | ||
| internalCluster().ensureAtMostNumDataNodes(nodesToRemain); | ||
| assertThat(internalCluster().size(), lessThanOrEqualTo(nodesToRemain)); |
There was a problem hiding this comment.
I'm not sure what you're testing here with this assertion. why not just repeatedly call stopRandomNode(), and maybe check that the cluster is alive and healthy before shutting down the last node.
|
CI failure was due to being a Centos worker (fixed by #35453) so I merged master to zen2 and thence to here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If shutting down half or more of the master-eligible nodes, their votes must
first be explicitly withdrawn to ensure that the cluster doesn't lose its
quorum. This works via voting tombstones, stored in the cluster state, which
tell the reconfigurator to remove nodes from the voting configuration.
This change introduces voting tombstones to the cluster state, together with
transport APIs for adding and removing them, and makes use of these APIs in
InternalTestClusterto support tests which remove at least half of themaster-eligible nodes at once (e.g. shrinking from two master-eligible nodes to
one).