Add build-tooling to run in the FIPS environment

[beanuwave]

Description

Provides additional build tooling to support builds in FIPS env, including a CLI trust-store installer to override $JAVA_HOME/lib/security/cacerts

• add demo/test CLI configurator with the ability to:
  -- migrate JVM's default SSL trust store to a BCFKS-formatted one
  -- use an existing PKCS#11 trust store
  -- display installed 'KeyStore' providers
  -- show help
  -- execute above commands interactively or in script mode
• add BC libs to standalone REST tests.
• print out 'java.security.properties' for reproducibility information

Related Issues

Resolves RFC

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for f656bd4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 20a5611: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2241009: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 1829731: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

❌ Patch coverage is 61.70732% with 157 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.22%. Comparing base (753c135) to head (e4e4710).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
...ools/cli/fips/truststore/CreateFipsTrustStore.java	22.22%	53 Missing and 3 partials ⚠️
.../opensearch/bootstrap/FipsTrustStoreValidator.java	50.81%	28 Missing and 2 partials ⚠️
...h/tools/cli/fips/truststore/TrustStoreService.java	61.40%	20 Missing and 2 partials ⚠️
...ols/cli/fips/truststore/FipsTrustStoreCommand.java	36.00%	16 Missing ⚠️
.../org/opensearch/gradle/test/rest/RestTestUtil.java	0.00%	9 Missing ⚠️
...ls/cli/fips/truststore/UserInteractionService.java	87.03%	7 Missing ⚠️
.../cli/fips/truststore/ProviderSelectionService.java	84.61%	4 Missing and 2 partials ⚠️
...search/gradle/test/StandaloneRestTestPlugin.groovy	0.00%	3 Missing ⚠️
...li/fips/truststore/GeneratedTrustStoreCommand.java	25.00%	3 Missing ⚠️
...pensearch/gradle/test/ClusterFormationTasks.groovy	0.00%	1 Missing ⚠️
... and 4 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18921      +/-   ##
============================================
+ Coverage     73.10%   73.22%   +0.11%     
- Complexity    70959    71124     +165     
============================================
  Files          5737     5753      +16     
  Lines        324766   325184     +418     
  Branches      46981    47032      +51     
============================================
+ Hits         237425   238106     +681     
+ Misses        68226    67911     -315     
- Partials      19115    19167      +52     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[cwperks]

Patch coverage is showing as 0% because the new tests are guarded with

@BeforeClass
public static void beforeClass() throws Exception {
   assumeTrue("Test should run in FIPS JVM", FipsMode.CHECK.isFipsEnabled());
}

The actual patch coverage is much higher, but the gradle check of this repo does not run with FIPS enabled.

[github-actions]

❌ Gradle check result for 974cec3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 974cec3: SUCCESS

[beanuwave]

@andrross @reta @cwperks Just pushed an update to introduce the new cluster-settings - do you think it's evolve into the right direction?

[github-actions]

❌ Gradle check result for 69d46e3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 1839524: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[reta]

@reta @cwperks All comments have been addressed or resolved. Is there anything else that needs to be done on my end?

Thank you @beanuwave , I have few really minor things, @andrross I would love to hear your opinion regarding #18921 (comment) before getting it in, thank you

[github-actions]

❌ Gradle check result for ecff1dc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 3f9f021: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 8f24562: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for a1c2c96: SUCCESS

[github-actions]

❌ Gradle check result for e4e4710: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[reta]

Thank you @beanuwave , great work. Just an ask if you plan to push some more changes, there are quite a few places in tests where we still use manual lifecycle management for sharedTempDir, would be great to unify those with:

@ClassRule
    public static TemporaryFolder sharedTempDir= new TemporaryFolder();

[github-actions]

❌ Gradle check result for e4e4710: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for e4e4710: SUCCESS

[beanuwave]

@cwperks @reta Thank you for the comprehensive reviews and your effort to push it over the line. Of course the next PR is already just around the corner 😄