Skip to content

Conversation

@varunbharadwaj
Copy link
Contributor

@varunbharadwaj varunbharadwaj commented Sep 17, 2025

Description

This PR includes following small fixes and optimizations.

  1. Include the ingestion status field in IndexMetadata in to/fromXContent.
  2. Fail fast for mapper/parsing errors avoiding retry.
  3. Add index name in error logs for easy debugging.
  4. Add validations for all-active ingestion to validate replication type.

Related Issues

Resolves #19286

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing labels Sep 17, 2025
@github-actions
Copy link
Contributor

❌ Gradle check result for 46357ed: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@varunbharadwaj varunbharadwaj changed the title [Pull-based Ingestion] Update ingestion status in index metadata xcontent and avoid retry on parsing errors [Pull-based Ingestion] Update ingestion status in index metadata xcontent and fail fast on parsing errors Sep 22, 2025
@github-actions
Copy link
Contributor

❌ Gradle check result for 667f8fa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 883b5a8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 68ff8d4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

✅ Gradle check result for 68ff8d4: SUCCESS

@codecov
Copy link

codecov bot commented Sep 23, 2025

Codecov Report

❌ Patch coverage is 82.22222% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.97%. Comparing base (e5d01b5) to head (6a68698).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...g/opensearch/cluster/metadata/IngestionStatus.java 72.22% 1 Missing and 4 partials ⚠️
...org/opensearch/cluster/metadata/IndexMetadata.java 92.30% 0 Missing and 1 partial ⚠️
...rch/indices/pollingingest/DefaultStreamPoller.java 75.00% 1 Missing ⚠️
...ndices/pollingingest/MessageProcessorRunnable.java 90.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #19320      +/-   ##
============================================
- Coverage     73.03%   72.97%   -0.07%     
+ Complexity    69997    69935      -62     
============================================
  Files          5676     5676              
  Lines        320923   320959      +36     
  Branches      46392    46401       +9     
============================================
- Hits         234396   234211     -185     
- Misses        67610    67801     +191     
- Partials      18917    18947      +30     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Contributor

✅ Gradle check result for 8bd696a: SUCCESS

@varunbharadwaj varunbharadwaj changed the title [Pull-based Ingestion] Update ingestion status in index metadata xcontent and fail fast on parsing errors [Pull-based Ingestion] Include ingestion status in index metadata xcontent serialization and fail fast on parsing errors Sep 24, 2025
@varunbharadwaj varunbharadwaj changed the title [Pull-based Ingestion] Include ingestion status in index metadata xcontent serialization and fail fast on parsing errors [Pull-based Ingestion] Include ingestionStatus in index metadata xcontent serialization and fail fast on parsing errors Sep 24, 2025
@github-actions
Copy link
Contributor

❌ Gradle check result for 9880beb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 9880beb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 9880beb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 6a68698: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

✅ Gradle check result for 6a68698: SUCCESS

@andrross andrross merged commit e0ee3b4 into opensearch-project:main Sep 25, 2025
61 of 63 checks passed
vinaykpud pushed a commit to vinaykpud/OpenSearch that referenced this pull request Sep 26, 2025
karenyrx pushed a commit to karenyrx/OpenSearch that referenced this pull request Sep 29, 2025
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Pull-based ingestion fails with remote cluster state due to missing XContent serialization

5 participants