Onboarding new maven snapshots publishing to s3 (OpenSearch Core)

[peterzhuamazon]

Once ready we can merge and then backport to all 2.19+3.3.

I already manually pushed 3.0/3.1/3.2/3.3/3.4 to s3.

Description

Onboarding new maven snapshots publishing to s3

Related Issues

opensearch-project/opensearch-build#5360

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[peterzhuamazon]

Will manually backport 2.19 branch now:

• [Backport 2.19] Onboarding new maven snapshots publishing to s3 (OpenSearch Core) #19632

[github-actions]

❕ Gradle check result for 2d1d629: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.45%. Comparing base (252cff8) to head (b936b47).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19619      +/-   ##
============================================
+ Coverage     73.11%   73.45%   +0.34%     
- Complexity    70661    71055     +394     
============================================
  Files          5724     5725       +1     
  Lines        323498   323737     +239     
  Branches      46852    46881      +29     
============================================
+ Hits         236518   237817    +1299     
+ Misses        67846    67099     -747     
+ Partials      19134    18821     -313     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gaiksaya]

Hi @peterzhuamazon

Was discussing the same with @mch2. Do we have any deletion/retention policy on the S3 for the snapshots? Else it would go on increasing overtime. I believe 90 days is a good start. Adding a cron to publish once a month on each branch is good way to store just the latest and archive the old ones.

[github-actions]

❌ Gradle check result for b936b47: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[peterzhuamazon]

Hi @peterzhuamazon

Was discussing the same with @mch2. Do we have any deletion/retention policy on the S3 for the snapshots? Else it would go on increasing overtime. I believe 90 days is a good start. Adding a cron to publish once a month on each branch is good way to store just the latest and archive the old ones.

That is fine for the time being, you can raise the concern in the issue, and that can be a new follow up task.
But with migration to the s3 itself this PR can be merged as is.

Thanks.

[github-actions]

✅ Gradle check result for b936b47: SUCCESS

[sandeshkr419]

@gaiksaya Do we need this change bakcported in 3.3 as well for the patch release?

Looks like there are errors fetching maven repos in #19667

* What went wrong:
Could not determine the dependencies of task ':modules:transport-grpc:javadoc'.
> Could not resolve all dependencies for configuration ':modules:transport-grpc:compileClasspath'.
8 actionable tasks: 8 executed
   > Could not resolve com.google.errorprone:error_prone_annotations:2.24.1.
     Required by:
         project :modules:transport-grpc
      > Could not resolve com.google.errorprone:error_prone_annotations:2.24.1.
         > Could not get resource 'https://repo.maven.apache.org/maven2/com/google/errorprone/error_prone_annotations/2.24.1/error_prone_annotations-2.24.1.pom'.
            > Could not GET 'https://repo.maven.apache.org/maven2/com/google/errorprone/error_prone_annotations/2.24.1/error_prone_annotations-2.24.1.pom'.
               > The server may not support the client's requested TLS protocol versions: (TLSv1.2, TLSv1.3). You may need to configure the client to allow other protocols to be used. For more on this, please refer to https://docs.gradle.org/8.14.3/userguide/build_environment.html#sec:gradle_system_properties in the Gradle documentation.
                  > Remote host terminated the handshake

[gaiksaya]

@gaiksaya Do we need this change bakcported in 3.3 as well for the patch release?

Yes cc: @peterzhuamazon

[peterzhuamazon]

• Backport here: [Backport 3.3] Onboarding new maven snapshots publishing to s3 (OpenSearch Core) #19668