Skip to content

Commit ec4ab87

Browse files
committed
Merge remote-tracking branch 'elastic/master' into allowed-index-store-types
* elastic/master: (199 commits) Watcher: Remove unused hipchat render method (elastic#32211) Watcher: Remove extraneous auth classes (elastic#32300) Watcher: migrate PagerDuty v1 events API to v2 API (elastic#32285) [TEST] Select free port for Minio (elastic#32837) MINOR: Remove `IndexTemplateFilter` (elastic#32841) Core: Add java time version of rounding classes (elastic#32641) Aggregations/HL Rest client fix: missing scores (elastic#32774) HLRC: Add Delete License API (elastic#32586) INGEST: Create Index Before Pipeline Execute (elastic#32786) Fix NOOP bulk updates (elastic#32819) Remove client connections from TcpTransport (elastic#31886) Increase logging testRetentionPolicyChangeDuringRecovery AwaitsFix case-functions.sql-spec Mute security-cli tests in FIPS JVM (elastic#32812) SCRIPTING: Support BucketAggScript return null (elastic#32811) Unmute WildFly tests in FIPS JVM (elastic#32814) [TEST] Force a stop to save rollup state before continuing (elastic#32787) [test] disable packaging tests for suse boxes Mute IndicesRequestIT#testBulk [ML][DOCS] Refer to rules feature as custom rules (elastic#32785) ...
2 parents 38e64c2 + 6fde9e5 commit ec4ab87

File tree

1,135 files changed

+36774
-11423
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,135 files changed

+36774
-11423
lines changed

CONTRIBUTING.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,12 @@ JDK 10 and testing on a JDK 8 runtime; to do this, set `RUNTIME_JAVA_HOME`
100100
pointing to the Java home of a JDK 8 installation. Note that this mechanism can
101101
be used to test against other JDKs as well, this is not only limited to JDK 8.
102102

103+
> Note: It is also required to have `JAVA7_HOME`, `JAVA8_HOME` and
104+
`JAVA10_HOME` available so that the tests can pass.
105+
106+
> Warning: do not use `sdkman` for Java installations which do not have proper
107+
`jrunscript` for jdk distributions.
108+
103109
Elasticsearch uses the Gradle wrapper for its build. You can execute Gradle
104110
using the wrapper via the `gradlew` script in the root of the repository.
105111

README.textile

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -209,10 +209,6 @@ The distribution for each project will be created under the @build/distributions
209209

210210
See the "TESTING":TESTING.asciidoc file for more information about running the Elasticsearch test suite.
211211

212-
h3. Upgrading from Elasticsearch 1.x?
212+
h3. Upgrading from older Elasticsearch versions
213213

214-
In order to ensure a smooth upgrade process from earlier versions of
215-
Elasticsearch (1.x), it is required to perform a full cluster restart. Please
216-
see the "setup reference":
217-
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html
218-
for more details on the upgrade process.
214+
In order to ensure a smooth upgrade process from earlier versions of Elasticsearch, please see our "upgrade documentation":https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html for more details on the upgrade process.

Vagrantfile

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -115,11 +115,6 @@ Vagrant.configure(2) do |config|
115115
'opensuse-42'.tap do |box|
116116
config.vm.define box, define_opts do |config|
117117
config.vm.box = 'elastic/opensuse-42-x86_64'
118-
119-
# https://github.com/elastic/elasticsearch/issues/30295
120-
config.vm.provider 'virtualbox' do |vbox|
121-
vbox.customize ['storagectl', :id, '--name', 'SATA Controller', '--hostiocache', 'on']
122-
end
123118
suse_common config, box
124119
end
125120
end

benchmarks/README.md

Lines changed: 20 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -4,36 +4,39 @@ This directory contains the microbenchmark suite of Elasticsearch. It relies on
44

55
## Purpose
66

7-
We do not want to microbenchmark everything but the kitchen sink and should typically rely on our
8-
[macrobenchmarks](https://elasticsearch-benchmarks.elastic.co/app/kibana#/dashboard/Nightly-Benchmark-Overview) with
9-
[Rally](http://github.com/elastic/rally). Microbenchmarks are intended to spot performance regressions in performance-critical components.
7+
We do not want to microbenchmark everything but the kitchen sink and should typically rely on our
8+
[macrobenchmarks](https://elasticsearch-benchmarks.elastic.co/app/kibana#/dashboard/Nightly-Benchmark-Overview) with
9+
[Rally](http://github.com/elastic/rally). Microbenchmarks are intended to spot performance regressions in performance-critical components.
1010
The microbenchmark suite is also handy for ad-hoc microbenchmarks but please remove them again before merging your PR.
1111

1212
## Getting Started
1313

14-
Just run `gradle :benchmarks:jmh` from the project root directory. It will build all microbenchmarks, execute them and print the result.
14+
Just run `gradlew -p benchmarks run` from the project root
15+
directory. It will build all microbenchmarks, execute them and print
16+
the result.
1517

1618
## Running Microbenchmarks
1719

18-
Benchmarks are always run via Gradle with `gradle :benchmarks:jmh`.
19-
20-
Running via an IDE is not supported as the results are meaningless (we have no control over the JVM running the benchmarks).
20+
Running via an IDE is not supported as the results are meaningless
21+
because we have no control over the JVM running the benchmarks.
2122

22-
If you want to run a specific benchmark class, e.g. `org.elasticsearch.benchmark.MySampleBenchmark` or have special requirements
23-
generate the uberjar with `gradle :benchmarks:jmhJar` and run it directly with:
23+
If you want to run a specific benchmark class like, say,
24+
`MemoryStatsBenchmark`, you can use `--args`:
2425

2526
```
26-
java -jar benchmarks/build/distributions/elasticsearch-benchmarks-*.jar
27+
gradlew -p benchmarks run --args ' MemoryStatsBenchmark'
2728
```
2829

29-
JMH supports lots of command line parameters. Add `-h` to the command above to see the available command line options.
30+
Everything in the `'` gets sent on the command line to JMH. The leading ` `
31+
inside the `'`s is important. Without it parameters are sometimes sent to
32+
gradle.
3033

3134
## Adding Microbenchmarks
3235

33-
Before adding a new microbenchmark, make yourself familiar with the JMH API. You can check our existing microbenchmarks and also the
36+
Before adding a new microbenchmark, make yourself familiar with the JMH API. You can check our existing microbenchmarks and also the
3437
[JMH samples](http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/).
3538

36-
In contrast to tests, the actual name of the benchmark class is not relevant to JMH. However, stick to the naming convention and
39+
In contrast to tests, the actual name of the benchmark class is not relevant to JMH. However, stick to the naming convention and
3740
end the class name of a benchmark with `Benchmark`. To have JMH execute a benchmark, annotate the respective methods with `@Benchmark`.
3841

3942
## Tips and Best Practices
@@ -42,15 +45,15 @@ To get realistic results, you should exercise care when running benchmarks. Here
4245

4346
### Do
4447

45-
* Ensure that the system executing your microbenchmarks has as little load as possible. Shutdown every process that can cause unnecessary
48+
* Ensure that the system executing your microbenchmarks has as little load as possible. Shutdown every process that can cause unnecessary
4649
runtime jitter. Watch the `Error` column in the benchmark results to see the run-to-run variance.
4750
* Ensure to run enough warmup iterations to get the benchmark into a stable state. If you are unsure, don't change the defaults.
4851
* Avoid CPU migrations by pinning your benchmarks to specific CPU cores. On Linux you can use `taskset`.
49-
* Fix the CPU frequency to avoid Turbo Boost from kicking in and skewing your results. On Linux you can use `cpufreq-set` and the
52+
* Fix the CPU frequency to avoid Turbo Boost from kicking in and skewing your results. On Linux you can use `cpufreq-set` and the
5053
`performance` CPU governor.
5154
* Vary the problem input size with `@Param`.
5255
* Use the integrated profilers in JMH to dig deeper if benchmark results to not match your hypotheses:
53-
* Run the generated uberjar directly and use `-prof gc` to check whether the garbage collector runs during a microbenchmarks and skews
56+
* Run the generated uberjar directly and use `-prof gc` to check whether the garbage collector runs during a microbenchmarks and skews
5457
your results. If so, try to force a GC between runs (`-gc true`) but watch out for the caveats.
5558
* Use `-prof perf` or `-prof perfasm` (both only available on Linux) to see hotspots.
5659
* Have your benchmarks peer-reviewed.
@@ -59,4 +62,4 @@ To get realistic results, you should exercise care when running benchmarks. Here
5962

6063
* Blindly believe the numbers that your microbenchmark produces but verify them by measuring e.g. with `-prof perfasm`.
6164
* Run more threads than your number of CPU cores (in case you run multi-threaded microbenchmarks).
62-
* Look only at the `Score` column and ignore `Error`. Instead take countermeasures to keep `Error` low / variance explainable.
65+
* Look only at the `Score` column and ignore `Error`. Instead take countermeasures to keep `Error` low / variance explainable.

benchmarks/build.gradle

Lines changed: 3 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,8 @@
1818
*/
1919

2020
apply plugin: 'elasticsearch.build'
21-
22-
// order of this section matters, see: https://github.com/johnrengelman/shadow/issues/336
23-
apply plugin: 'application' // have the shadow plugin provide the runShadow task
21+
apply plugin: 'application'
2422
mainClassName = 'org.openjdk.jmh.Main'
25-
apply plugin: 'com.github.johnrengelman.shadow' // build an uberjar with all benchmarks
2623

2724
// Not published so no need to assemble
2825
tasks.remove(assemble)
@@ -50,10 +47,8 @@ compileJava.options.compilerArgs << "-Xlint:-cast,-deprecation,-rawtypes,-try,-u
5047
// needs to be added separately otherwise Gradle will quote it and javac will fail
5148
compileJava.options.compilerArgs.addAll(["-processor", "org.openjdk.jmh.generators.BenchmarkProcessor"])
5249

53-
forbiddenApis {
54-
// classes generated by JMH can use all sorts of forbidden APIs but we have no influence at all and cannot exclude these classes
55-
ignoreFailures = true
56-
}
50+
// classes generated by JMH can use all sorts of forbidden APIs but we have no influence at all and cannot exclude these classes
51+
forbiddenApisMain.enabled = false
5752

5853
// No licenses for our benchmark deps (we don't ship benchmarks)
5954
dependencyLicenses.enabled = false
@@ -69,20 +64,3 @@ thirdPartyAudit.excludes = [
6964
'org.openjdk.jmh.profile.HotspotRuntimeProfiler',
7065
'org.openjdk.jmh.util.Utils'
7166
]
72-
73-
runShadow {
74-
executable = new File(project.runtimeJavaHome, 'bin/java')
75-
}
76-
77-
// alias the shadowJar and runShadow tasks to abstract from the concrete plugin that we are using and provide a more consistent interface
78-
task jmhJar(
79-
dependsOn: shadowJar,
80-
description: 'Generates an uberjar with the microbenchmarks and all dependencies',
81-
group: 'Benchmark'
82-
)
83-
84-
task jmh(
85-
dependsOn: runShadow,
86-
description: 'Runs all microbenchmarks',
87-
group: 'Benchmark'
88-
)

benchmarks/src/main/resources/log4j2.properties

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
appender.console.type = Console
22
appender.console.name = console
33
appender.console.layout.type = PatternLayout
4-
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%m%n
4+
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] [%node_name]%marker %m%n
55

66
# Do not log at all if it is not really critical - we're in a benchmark
77
rootLogger.level = error

0 commit comments

Comments
 (0)