[TEST] Fix Unit and Integration Tests as part of Removal of XPack code

[adnapibar]

This meta issue tracks fixing all the tests which are failing as part of the removal of references of Elastic licensed code (CR #2)

• @adnapibar Compile Issues (Fix compilation issues for tests. #29)
• Build Tools Integration Tests (Gradle Task :build-tools:integTest)

REST High Level Client Tests (Gradle Task :client:rest-high-level:test)
Tests with failures:

• org.elasticsearch.client.RequestConvertersTests.testExists (Revert back refresh policy in RequestConverters. #55)
• org.elasticsearch.client.RequestConvertersTests.testGetSource (Revert back refresh policy in RequestConverters. #55)
• org.elasticsearch.client.RequestConvertersTests.testMultiGet (Revert back refresh policy in RequestConverters. #55)
• org.elasticsearch.client.RequestConvertersTests.testSourceExistsWithType (Revert back refresh policy in RequestConverters. #55)
• org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent

REST High Level Client Integ Tests (Gradle Task :client:rest-high-level:asyncIntegTest)
Tests with failures:

• org.elasticsearch.client.IndicesClientIT.testFreezeAndUnfreeze(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
• org.elasticsearch.client.IndicesClientIT.testReloadAnalyzer(Removing ReloadSearchAnalyzers related code since its x-pack counterpart is removed #48)
• org.elasticsearch.client.ReindexIT.testDeleteByQuery
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsDoubleMetric(Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testStringStats (Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsSizeTwo (Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsLongMetric (Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsDateMetric (Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.analytics.AnalyticsAggsIT.testTopMetricsManyMetrics (Remove x-pack aggregations hlrc. #59)
• org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery
• org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testFreezeIndex(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
• org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testUnfreezeIndex(Removing FreezeIndex related code since its x-pack counterpart is removed #85)
• org.elasticsearch.client.documentation.IndicesClientDocumentationIT.testReloadSearchAnalyzers(Removing ReloadSearchAnalyzers related code since its x-pack counterpart is removed #48)
• org.elasticsearch.client.documentation.CRUDDocumentationIT.testGetSource
• org.elasticsearch.client.SearchIT.testSearchWithBasicLicensedQuery (IT Failure "testSearchWithBasicLicensedQuery" #60)

[tlfeng]

verification tasks (gradle check) pass in the below sub-projects:

• benchmarks (./gradlew :benchmarks:check)
• example-plugins (./gradlew :example-plugins:xxx:check)
• libs (./gradlew :libs:xxx:check)
• modules (./gradlew :modules:xxx:check)

[tlfeng]

There is one test failed in "Build Tools Integration Tests" [Update: It was fixed by PR #33]

> Task :build-tools:integTest

org.elasticsearch.gradle.internal.InternalDistributionArchiveCheckPluginFuncTest > fails on unexpected notice content FAILED
    org.spockframework.runtime.ConditionNotSatisfiedError at InternalDistributionArchiveCheckPluginFuncTest.groovy:100

67 tests completed, 1 failed

To fix the test "fails on unexpected notice content" in InternalDistributionArchiveCheckPluginFuncTest.groovy,
The product name in this line, which is "OSS Search" now:
https://github.com/opendistro-for-elasticsearch/search/blob/d496925a32385f30d0e4e6a546a2848bf1b0b83f/buildSrc/src/integTest/groovy/org/elasticsearch/gradle/internal/InternalDistributionArchiveCheckPluginFuncTest.groovy#L82
needs to be same with here, which is "Elasticsearch" now:
https://github.com/opendistro-for-elasticsearch/search/blob/d496925a32385f30d0e4e6a546a2848bf1b0b83f/buildSrc/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionArchiveCheckPlugin.java#L93

Note:
Setting JAVA14_HOME environment variable is a prerequisite of the tests, according to:
https://github.com/elastic/elasticsearch/issues/ 63303 , and the contribution guide

Without setting JAVA14_HOME, there will be 3 other tests failure:

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > builds distribution from branches via archives assemble FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:51

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > bwc distribution archives can be resolved as bwc project artifact FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:87

org.elasticsearch.gradle.internal.InternalDistributionBwcSetupPluginFuncTest > bwc expanded distribution folder can be resolved as bwc project artifact FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at InternalDistributionBwcSetupPluginFuncTest.groovy:125

[tlfeng]

There are two permanent tests failure in "REST Hight Level Client Tests", which are not failed in Elasticsearch 7.10.2

> Task :client:rest-high-level:test

Tests with failures:
 - org.elasticsearch.client.RestHighLevelClientTests.testProvidedNamedXContents
 - org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent
 - 
265 tests completed, 2 failed

1. The test testProvidedNamedXContents is fixed by PR [TEST] Fix unit test failure in RestHighLevelClientTests #36
2. The test testFromXContent:
   There is an error mesage [data_stream] unknown field [ilm_policy], it is due to ILM policy field definition missing in DataStream.java, the original PR of adding the definition is: https://github.com/elastic/elasticsearch/pull/ 59128
   If adding the field definition back, like the commit dcf00b6, the failure will be disappear.
   But currently whether to leave of remove the whole DataStreams codes needs to be determined.

Besides, there are several flaky tests:

Tests with failures:
 - org.elasticsearch.client.RequestConvertersTests.testSourceExistsWithType
 - org.elasticsearch.client.RequestConvertersTests.testGetSource
 - org.elasticsearch.client.RequestConvertersTests.testMultiGet
 - org.elasticsearch.client.RequestConvertersTests.testSourceExists

These are caused by the removal of the "RefreshPolicy" in this commit, and are addressed by PR #42

[tlfeng]

15 test failures in Gradle Task :client:rest-high-level:asyncIntegTest, and the tests pass in Elasticsearch v7.10.2

Example:

REPRODUCE WITH: ./gradlew ':client:rest-high-level:asyncIntegTest' --tests "org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery" -Dtests.seed=36DBCD1668F2A29 -Dtests.security.manager=true -Dtests.locale=no -Dtests.timezone=EAT -Druntime.java=15

org.elasticsearch.client.UpdateByQueryIT > testUpdateByQuery FAILED
    java.lang.AssertionError: expected:<2> but was:<1>
        at __randomizedtesting.SeedInfo.seed([36DBCD1668F2A29:F93DCA21C2AB256F]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:645)
        at org.junit.Assert.assertEquals(Assert.java:631)
        at org.elasticsearch.client.UpdateByQueryIT.testUpdateByQuery(UpdateByQueryIT.java:105)

Full errors list are omitted.

[saratvemulapalli]

For 2. Build Tools Integration Tests (Gradle Task :build-tools:integTest)
I have see all the tests passing.

vemsarat@3c22fb88f11c odfe-search % ./gradlew :build-tools:integTest
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 6.6.1
  OS Info               : Mac OS X 10.15.7 (x86_64)
  JDK Version           : 14 (Oracle JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/jdk-14.0.2.jdk/Contents/Home
  Random Testing Seed   : AA1DDF5D1C3EC860
  In FIPS 140 mode      : false

BUILD SUCCESSFUL in 3m 54s
18 actionable tasks: 4 executed, 14 up-to-date

[saratvemulapalli]

For 3. REST Hight Level Client Tests (Gradle Task :client:rest-high-level:test)
There is a single test which is failing org.elasticsearch.client.indices.GetDataStreamResponseTests.testFromXContent which is probably related to DataStreams issue. #40