Added a dynamic cluster setting to control merge segment warming #18942

kh3ra · 2025-08-06T11:55:32Z

Description

Added a dynamic cluster setting - indices.replication.merged_segment_warmer_enabled to enable the merged segment warmer feature. This can only be enabled if FeatureFlags.MERGED_SEGMENT_WARMER_EXPERIMENTAL_FLAG is enabled.
The default value for the setting is false.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-08-06T12:04:34Z

❌ Gradle check result for 8dd3cea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

kh3ra · 2025-08-07T05:52:48Z

@ashking94 Do we want to have a validation on the experimental flag for this new setting?

github-actions · 2025-08-07T06:22:01Z

❌ Gradle check result for 426741a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-09T23:31:37Z

✅ Gradle check result for 7ecd349: SUCCESS

codecov · 2025-08-09T23:31:56Z

Codecov Report

❌ Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.81%. Comparing base (3e747cb) to head (cd8b02d).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...g/opensearch/index/engine/MergedSegmentWarmer.java	66.66%	0 Missing and 1 partial ⚠️
...earch/index/engine/MergedSegmentWarmerFactory.java	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18942      +/-   ##
============================================
+ Coverage     72.79%   72.81%   +0.02%     
- Complexity    69605    69689      +84     
============================================
  Files          5658     5658              
  Lines        320079   320102      +23     
  Branches      46345    46350       +5     
============================================
+ Hits         232996   233097     +101     
+ Misses        68230    68128     -102     
- Partials      18853    18877      +24

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-08-11T14:26:55Z

❌ Gradle check result for d0937b4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-12T06:44:13Z

❌ Gradle check result for 00ea9d4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

kh3ra · 2025-08-12T08:19:10Z

❌ Gradle check result for 00ea9d4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #18956

StreamStringTermsAggregatorTests > testBuildAggregationsBatchWithSize FAILED
    java.lang.AssertionError: 
    Expected: <10>
         but: was <6>
        at __randomizedtesting.SeedInfo.seed([ED993AC4F489716F:B58FB725ED4BC5E1]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.junit.Assert.assertThat(Assert.java:930)
        at org.opensearch.search.aggregations.bucket.terms.StreamStringTermsAggregatorTests.testBuildAggregationsBatchWithSize(StreamStringTermsAggregatorTests.java:260)

ashking94 · 2025-08-25T09:34:22Z

Lets add a proper PR title and description

...nalClusterTest/java/org/opensearch/indices/replication/RemoteStoreMergedSegmentWarmerIT.java

server/src/main/java/org/opensearch/index/engine/MergedSegmentWarmer.java

server/src/main/java/org/opensearch/indices/recovery/RecoverySettings.java

server/src/test/java/org/opensearch/indices/recovery/RecoverySettingsDynamicUpdateTests.java

github-actions · 2025-08-29T12:34:02Z

❌ Gradle check result for cfbd8dc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-29T17:38:44Z

❌ Gradle check result for 29b2c68: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-29T19:55:51Z

❌ Gradle check result for ca8cf97: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-09-01T06:54:18Z

❌ Gradle check result for 153f343: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ashking94

Left a comment, overall lgtm

…re. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

Signed-off-by: Aditya Khera <[email protected]>

github-actions · 2025-09-07T16:55:42Z

❌ Gradle check result for 58dcce9: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Aditya Khera <[email protected]>

github-actions · 2025-09-08T07:10:55Z

✅ Gradle check result for cd8b02d: SUCCESS

server/src/main/java/org/opensearch/indices/recovery/RecoverySettings.java

…nsearch-project#18942) * Adding dynamic cluster setting to control merge segment warming feature. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

…nsearch-project#18942) * Adding dynamic cluster setting to control merge segment warming feature. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]> Signed-off-by: Ankit Jain <[email protected]>

…nsearch-project#18942) * Adding dynamic cluster setting to control merge segment warming feature. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

…nsearch-project#18942) * Adding dynamic cluster setting to control merge segment warming feature. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]> Signed-off-by: Pranit Kumar <[email protected]>

…nsearch-project#18942) * Adding dynamic cluster setting to control merge segment warming feature. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

kh3ra force-pushed the dev/remote-merged-segment-warmer-dynamic-setting branch from 84f8746 to 00ea9d4 Compare August 12, 2025 05:55

opensearch-ci-bot mentioned this pull request Aug 12, 2025

[AUTOCUT] Gradle Check Flaky Test Report for StreamStringTermsAggregatorTests #18956

Closed

ashking94 requested changes Aug 25, 2025

View reviewed changes

kh3ra changed the title ~~Dynamic cluster setting to control merge segment warming feature~~ Added a dynamic cluster setting to control merge segment warming Aug 29, 2025

ashking94 approved these changes Sep 4, 2025

View reviewed changes

ashking94 marked this pull request as ready for review September 4, 2025 10:34

ashking94 requested review from Bukhtawar, CEHENKLE, anasalkouz, andrross, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2 and msfroh as code owners September 4, 2025 10:34

kh3ra and others added 13 commits September 7, 2025 20:22

Adding dynamic cluster setting to control merge segment warming featu…

d0842a4

…re. The feature would be disabled by default Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

spotlessApply

3e13293

Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

Adding validations for the setting + adding tests

24dce49

Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

Adding tests for dynamic settings update

9c6b551

Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

SpotlessApply

311a686

Signed-off-by: kh3ra <[email protected]> Signed-off-by: Aditya Khera <[email protected]>

Addressing review comments

71441af

Signed-off-by: Aditya Khera <[email protected]>

SpotlessApply

4175b38

Signed-off-by: Aditya Khera <[email protected]>

Empty

38ca8d8

Signed-off-by: Aditya Khera <[email protected]>

Empty

e2b46f5

Signed-off-by: Aditya Khera <[email protected]>

Empty

93726d0

Signed-off-by: Aditya Khera <[email protected]>

Empty

8334dcd

Signed-off-by: Aditya Khera <[email protected]>

Adding changelog

cb5657d

Signed-off-by: Aditya Khera <[email protected]>

Empty commit to trigger build

58dcce9

Signed-off-by: Aditya Khera <[email protected]>

kh3ra force-pushed the dev/remote-merged-segment-warmer-dynamic-setting branch from 5867ae7 to 58dcce9 Compare September 7, 2025 14:52

Empty commit

cd8b02d

Signed-off-by: Aditya Khera <[email protected]>

shourya035 approved these changes Sep 8, 2025

View reviewed changes

linuxpi reviewed Sep 8, 2025

View reviewed changes

server/src/main/java/org/opensearch/indices/recovery/RecoverySettings.java Show resolved Hide resolved

linuxpi approved these changes Sep 8, 2025

View reviewed changes

linuxpi merged commit 8d07f79 into opensearch-project:main Sep 8, 2025
31 checks passed

kh3ra mentioned this pull request Oct 22, 2025

Remove FeatureFlag.MERGED_SEGMENT_WARMER_EXPERIMENTAL_FLAG #19715

Merged

3 tasks

Added a dynamic cluster setting to control merge segment warming #18942

Added a dynamic cluster setting to control merge segment warming #18942

Uh oh!

Conversation

kh3ra commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

kh3ra commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 9, 2025

Uh oh!

codecov bot commented Aug 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

github-actions bot commented Aug 12, 2025

Uh oh!

kh3ra commented Aug 12, 2025

Uh oh!

ashking94 commented Aug 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

github-actions bot commented Aug 29, 2025

Uh oh!

github-actions bot commented Sep 1, 2025

Uh oh!

ashking94 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 7, 2025

Uh oh!

github-actions bot commented Sep 8, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kh3ra commented Aug 6, 2025 •

edited

Loading

codecov bot commented Aug 9, 2025 •

edited

Loading