Skip to content

Conversation

@hrithikshukla
Copy link
Contributor

@hrithikshukla hrithikshukla commented Mar 15, 2025

Description

Byte fields now display correct values on the _cat/recovery page and now work with the the bytes parameter

http://localhost:9200/_cat/recovery?v

index    shard time  type        stage source_host source_node target_host target_node repository snapshot files files_recovered files_percent files_total bytes  bytes_recovered bytes_percent bytes_total translog_ops translog_ops_recovered translog_ops_percent
index123 0     200ms empty_store done  n/a         n/a         127.0.0.1   runTask-0   n/a        n/a      0     0               0.0%          0           0b     0b              0.0%          0b          0            0                      100.0%
index123 0     423ms peer        done  127.0.0.1   runTask-0   127.0.0.1   runTask-1   n/a        n/a      40    40              100.0%        40          49.4kb 49.4kb          100.0%        49.4kb      0            0                      100.0%

http://localhost:9200/_cat/recovery?v&bytes=b

index    shard time  type        stage source_host source_node target_host target_node repository snapshot files files_recovered files_percent files_total bytes bytes_recovered bytes_percent bytes_total translog_ops translog_ops_recovered translog_ops_percent
index123 0     200ms empty_store done  n/a         n/a         127.0.0.1   runTask-0   n/a        n/a      0     0               0.0%          0           0     0               0.0%          0           0            0                      100.0%
index123 0     423ms peer        done  127.0.0.1   runTask-0   127.0.0.1   runTask-1   n/a        n/a      40    40              100.0%        40          50676 50676           100.0%        50676       0            0                      100.0%

Related Issues

Resolves #17596

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

❌ Gradle check result for 94e8fac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Rishikesh1159
Copy link
Member

Rishikesh1159 commented Mar 17, 2025

❌ Gradle check result for 94e8fac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@hrithikshukla looks like mixed cluster tests are failing:

[org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.recovery/10_basic/Test cat recovery output for closed index}](https://build.ci.opensearch.org/job/gradle-check/54592/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_cat_recovery_10_basic_Test_cat_recovery_output_for_closed_index_/)
[org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.recovery/10_basic/Test cat recovery output}](https://build.ci.opensearch.org/job/gradle-check/54592/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_cat_recovery_10_basic_Test_cat_recovery_output_/)
[org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.recovery/10_basic/Test cat recovery output for closed index}](https://build.ci.opensearch.org/job/gradle-check/54592/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_cat_recovery_10_basic_Test_cat_recovery_output_for_closed_index__2/)
[org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.recovery/10_basic/Test cat recovery output for closed index}](https://build.ci.opensearch.org/job/gradle-check/54592/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_cat_recovery_10_basic_Test_cat_recovery_output_for_closed_index__3/)
[org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=cat.recovery/10_basic/Test cat recovery output for closed index}](https://build.ci.opensearch.org/job/gradle-check/54592/testReport/junit/org.opensearch.backwards/MixedClusterClientYamlTestSuiteIT/test__p0_cat_recovery_10_basic_Test_cat_recovery_output_for_closed_index__4/)

with assertion failure:

java.lang.AssertionError: Failure at [cat.recovery/10_basic:108]: field [$body] was expected to match the provided regex but didn't
Expected: ^
(
  index2      \s+
  \d          \s+                                 # shard
  (?:\d+ms|\d+(?:\.\d+)?s)    \s+                 # time in ms or seconds
  existing_store \s+                              # source type (always existing_store for closed indices)
  done        \s+                                 # stage
  [-\w./]+    \s+                                 # source_host
  [-\w./]+    \s+                                 # target_host
  [-\w./]+    \s+                                 # repository
  [-\w./]+    \s+                                 # snapshot
  \d+         \s+                                 # files
  \d+         \s+                                 # files_recovered
  \d+\.\d+%   \s+                                 # files_percent
  \d+         \s+                                 # files_total
  (\d+|\d+[.]\d+)(kb|b)         \s+               # bytes
  (\d+|\d+[.]\d+)(kb|b)         \s+               # bytes_recovered
  \d+\.\d+%   \s+                                 # bytes_percent
  (\d+|\d+[.]\d+)(kb|b)         \s+               # bytes_total
  0          \s+                                  # translog_ops (always 0 for closed indices)
  0           \s+                                 # translog_ops_recovered (always 0 for closed indices)
  100\.0%                                         # translog_ops_percent (always 100.0% for closed indices)
  \n
)+
$
     but: was "index2 0 72ms existing_store done n/a 127.0.0.1 n/a n/a 0 0 100.0% 1 0 0 100.0% 230 0 0 100.0%\n"

could you please address these failures

@github-actions
Copy link
Contributor

❕ Gradle check result for f927d61: UNSTABLE

  • TEST FAILURES:
      3 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testSnapshotWithStuckNode
      2 org.opensearch.repositories.s3.S3BlobStoreRepositoryTests.classMethod

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@codecov
Copy link

codecov bot commented Mar 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.45%. Comparing base (dcad6b8) to head (be0db61).
Report is 6 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #17598      +/-   ##
============================================
- Coverage     72.46%   72.45%   -0.02%     
+ Complexity    65757    65705      -52     
============================================
  Files          5311     5311              
  Lines        305001   305011      +10     
  Branches      44230    44232       +2     
============================================
- Hits         221022   220984      -38     
+ Misses        65932    65855      -77     
- Partials      18047    18172     +125     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hrithikshukla
Copy link
Contributor Author

Thanks for taking a look @Rishikesh1159. I've updated the tests with a skip for versions < 3.0 since the output is now different between versions, let me know if that's ok or if there's another preferred approach.

@Rishikesh1159
Copy link
Member

Nice, Thanks @hrithikshukla for raising PR. LGTM.

could you please rebase the PR again seems like a merge conflict in changelog file

@github-actions
Copy link
Contributor

❌ Gradle check result for bb1f7db: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

✅ Gradle check result for bb1f7db: SUCCESS

Hrithik Shukla added 2 commits March 19, 2025 12:48
Signed-off-by: Hrithik Shukla <[email protected]>
@Rishikesh1159
Copy link
Member

looks like again there is a merge conflict. sorry @hrithikshukla for the back and forth could you please rebase again.

Signed-off-by: Hrithik Shukla <[email protected]>
@github-actions github-actions bot added Indexing:Replication Issues and PRs related to core replication framework eg segrep Storage Issues and PRs relating to data and metadata storage labels Mar 19, 2025
@hrithikshukla
Copy link
Contributor Author

yeah, just finished rebasing @Rishikesh1159

@github-actions
Copy link
Contributor

❌ Gradle check result for be0db61: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for be0db61: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

✅ Gradle check result for be0db61: SUCCESS

@Rishikesh1159 Rishikesh1159 self-requested a review March 19, 2025 21:22
@github-project-automation github-project-automation bot moved this to 👀 In review in Storage Project Board Mar 19, 2025
@Rishikesh1159 Rishikesh1159 merged commit af5835f into opensearch-project:main Mar 19, 2025
32 checks passed
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Storage Project Board Mar 19, 2025
bzhangam pushed a commit to bzhangam/OpenSearch that referenced this pull request Mar 25, 2025
* Fix how bytes are displayed on _cat/recovery

Signed-off-by: Hrithik Shukla <[email protected]>

* Fix tests

Signed-off-by: Hrithik Shukla <[email protected]>

* Update changelog

Signed-off-by: Hrithik Shukla <[email protected]>

---------

Signed-off-by: Hrithik Shukla <[email protected]>
Co-authored-by: Hrithik Shukla <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Indexing:Replication Issues and PRs related to core replication framework eg segrep _No response_ Storage Issues and PRs relating to data and metadata storage

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

[BUG] Bytes parameter doesn't work on _cat/recovery

2 participants