diff --git a/Makefile b/Makefile index 6035099713c..bd64478f5f9 100644 --- a/Makefile +++ b/Makefile @@ -46,16 +46,14 @@ go-generate: .PHONY: create-docs create-docs: - @mkdir -p docs/data/intake-api/generated/{error,transaction,metricset,sourcemap} - @cp testdata/error/payload.json docs/data/intake-api/generated/error/ - @cp testdata/error/rum.json docs/data/intake-api/generated/error/ - @cp testdata/error/minimal_payload_exception.json docs/data/intake-api/generated/error/ - @cp testdata/error/minimal_payload_log.json docs/data/intake-api/generated/error/ - @cp testdata/metricset/payload.json docs/data/intake-api/generated/metricset/ - @cp testdata/transaction/payload.json docs/data/intake-api/generated/transaction/ - @cp testdata/transaction/minimal_payload.json docs/data/intake-api/generated/transaction/ - @cp testdata/transaction/minimal_span.json docs/data/intake-api/generated/transaction/ + @mkdir -p docs/data/intake-api/generated/sourcemap + @cp testdata/intake-v2/events.ndjson docs/data/intake-api/generated/ @cp testdata/sourcemap/bundle.js.map docs/data/intake-api/generated/sourcemap/ + @mkdir -p docs/data/elasticsearch/generated/ + @cp processor/stream/approved-es-documents/testV2IntakeIntegrationErrors.approved.json docs/data/elasticsearch/generated/errors.json + @cp processor/stream/approved-es-documents/testV2IntakeIntegrationTransactions.approved.json docs/data/elasticsearch/generated/transactions.json + @cp processor/stream/approved-es-documents/testV2IntakeIntegrationSpans.approved.json docs/data/elasticsearch/generated/spans.json + @cp processor/stream/approved-es-documents/testV2IntakeIntegrationMetricsets.approved.json docs/data/elasticsearch/generated/metricsets.json # Start manual testing environment with agents start-env: diff --git a/docs/common-problems.asciidoc b/docs/common-problems.asciidoc index aa0e807980a..d5a9c574a39 100644 --- a/docs/common-problems.asciidoc +++ b/docs/common-problems.asciidoc @@ -36,8 +36,8 @@ Particularly, if you are using Docker, ensure to bind to the right interface (fo -If you see requests coming trough the APM Server but they are not accepted (response code other than `202`), consider -the response code to narrow down the possible causes (see sections bellow). +If you see requests coming through the APM Server but they are not accepted (response code other than `202`), consider +the response code to narrow down the possible causes (see sections below). Another reason for data not showing up is that the agent is not auto-instrumenting something you were expecting, check the {apm-agents-ref}/index.html[agent documentation] for details on what is automatically instrumented. @@ -46,7 +46,7 @@ the {apm-agents-ref}/index.html[agent documentation] for details on what is auto [float] === HTTP 400: Data decoding error / Data validation error -The most likely cause for this is that you are using non compatible versions of agent and APM Server. +The most likely cause for this is that you are using incompatible versions of agent and APM Server. For instance, APM Server 6.2.0 changed the Intake API spec and requires a minimum version of each agent. [[unauthorized]] @@ -103,7 +103,7 @@ You have a few options to solve this problem: === HTTP 503: Request timed out waiting to be processed This happens when APM Server exceeds the maximum number of requests that it can process concurrently. -This limit is determined by the `apm-server.concurrent_requests` configuration parameter. +This limit is determined by the `apm-server.concurrent_requests` configuration parameter deprecated[6.5]. To alleviate this problem, you can try to: @@ -157,20 +157,20 @@ To resolve this problem, try one of these solutions: * Create a DNS entry for the hostname mapping it to the server's IP. * Create an entry in `/etc/hosts` for the hostname. Or on Windows add an entry to `C:\Windows\System32\drivers\etc\hosts`. -* Re-create the server certificate and add a SubjectAltName (SAN) for the IP address of the server. This make the +* Re-create the server certificate and add a SubjectAltName (SAN) for the IP address of the server. This makes the server's certificate valid for both the hostname and the IP address. [float] [[getsockopt-no-route-to-host]] ===== getsockopt: no route to host -This is not a SSL problem. It's a networking problem. Make sure the two hosts can communicate. +This is not an SSL problem. It's a networking problem. Make sure the two hosts can communicate. [float] [[getsockopt-connection-refused]] ===== getsockopt: connection refused -This is not a SSL problem. Make sure that Logstash is running and that there is no firewall blocking the traffic. +This is not an SSL problem. Make sure that Logstash is running and that there is no firewall blocking the traffic. [float] [[target-machine-refused-connection]] diff --git a/docs/configuration-process.asciidoc b/docs/configuration-process.asciidoc index d7d28db0d41..75f7557ea9c 100644 --- a/docs/configuration-process.asciidoc +++ b/docs/configuration-process.asciidoc @@ -17,7 +17,8 @@ max_procs: 4 ---- [float] -=== Configuration options `apm-server.*` +[[configuration-apm-server]] +=== Configuration options: `apm-server.*` [[host]] [float] @@ -26,24 +27,12 @@ Defines the host and port the server is listening on. Use "unix:/path/to.sock" to listen on a unix domain socket. Defaults to 'localhost:8200'. -[[max_unzipped_size]] -[float] -==== `max_unzipped_size` -Maximum permitted size of an unzipped request accepted by the server to be processed (in Bytes). -Defaults to 31457280 Bytes (30 MB). - [[max_header_size]] [float] ==== `max_header_size` Maximum permitted size of a request's header accepted by the server to be processed (in Bytes). Defaults to 1048576 Bytes (1 MB). -[[max_request_queue_time]] -[float] -==== `max_request_queue_time` -Maximum duration a request will be queued before being read. -Defaults to 2 seconds. - [[read_timeout]] [float] ==== `read_timeout` @@ -62,32 +51,64 @@ Defaults to 30 seconds. Maximum duration in seconds before releasing resources when shutting down the server. Defaults to 5 seconds. +[float] +[[configuration-v1-api]] +=== Configuration options: v1 Intake API deprecated[6.5] + +Version 6.5 of the APM Server introduced a new intake API. You can learn more about this change in the <> documentation. + +If you do not upgrade your APM Agent, you'll continue to use the deprecated v1 intake API endpoint, and thus, the following configuration options. + +[[max_unzipped_size]] +[float] +==== `max_unzipped_size` deprecated[6.5] +Maximum permitted size of an unzipped request accepted by the server to be processed (in Bytes). +Defaults to 31457280 Bytes (30 MB). + +[[max_request_queue_time]] +[float] +==== `max_request_queue_time` deprecated[6.5] +Maximum duration a request will be queued before being read. +Defaults to 2 seconds. + [[concurrent_requests]] [float] -==== `concurrent_request` +==== `concurrent_request` deprecated[6.5] Maximum number of requests the server can process concurrently. Read more about how to tune data ingestion by <>. Default value is 5. +[float] +[[configuration-v2-api]] +=== Configuration options: v2 Intake API + +Version 6.5 of the APM Server introduced a new intake API. You can learn more about this change in the <> documentation. + +The following configuration options only affect users that have upgraded their agent to take advantage of the new v2 intake API. + +[[max_event_size]] +[float] +==== `max_event_size` +Maximum permitted size of an event accepted by the server to be processed (in Bytes). +Defaults to 307200 Bytes. + +[float] +[[configuration-other]] +=== Configuration options: general + [[max_connections]] [float] ==== `max_connections` Maximum number of TCP connections to accept simultaneously. Default value is 0, which means _unlimited_. -[[instrumentation.enabled]] -[float] -==== `instrumentation.enabled` -Enables self instrumentation of the APM Server itself. -Disabled by default. - [[config-secret-token]] [float] ==== `secret_token` Authorization token for sending data to the APM server. If a token is set, the agents must send it in the following format: Authorization: Bearer . -The token is not used for RUM endpoints. By default no authorization token is set. +The token is not used for RUM endpoints. By default, no authorization token is set. It is recommended to use an authorization token in combination with SSL enabled. Read more about <> and the <>. @@ -111,6 +132,12 @@ Disabled by default. Configure the url to expose expvar. Defaults to `debug/vars`. +[[instrumentation.enabled]] +[float] +==== `instrumentation.enabled` +Enables self instrumentation of the APM Server itself. +Disabled by default. + [[metrics.enabled]] [float] ==== `metrics` @@ -129,9 +156,9 @@ Defaults to false. Overwrites existing pipeline definitions in Elasticsearch. Defaults to true. [float] -=== Configuration options `queue.mem.*` +=== Configuration options: `queue.mem.*` Data is buffered in a memory queue before it is published to the configured output. -`queue.mem.*` settings modify the queue behaviour. +`queue.mem.*` settings modify the queue behavior. [[mem.events]] [float] @@ -155,7 +182,7 @@ if the number of events stored in the queue is < _min_flush_events_. Default value is 1 second. [float] -=== Configuration options `max_procs` +=== Configuration options: `max_procs` [[max_procs]] [float] diff --git a/docs/configuration-rum.asciidoc b/docs/configuration-rum.asciidoc index c6dd09cc6ed..16c745ea5aa 100644 --- a/docs/configuration-rum.asciidoc +++ b/docs/configuration-rum.asciidoc @@ -17,7 +17,7 @@ apm-server.rum.source_mapping.index_pattern: "apm-*-sourcemap*" ---- [float] -=== Configuration options +=== Configuration [[rum-enable]] [float] @@ -26,35 +26,74 @@ For enabling RUM support, set the `apm-server.rum.enabled` to `true`. By default this is disabled. [float] -==== `rate_limit` +[[configuration-rum-v1]] +=== Configuration options: v1 RUM endpoint deprecated[6.5] + +Version 6.5 of the APM Server introduced a new intake API. You can learn more about this change in the <> documentation. + +If you do not upgrade your APM Agent, you'll continue to use the deprecated v1 RUM intake API endpoint, and thus, the following configuration options. + +[float] +==== `rate_limit` deprecated[6.5] Rate limit per second and IP address for requests sent to the RUM endpoint. If the rate limit is hit, the APM Server will return an HTTP status code `429`. The rate limit cannot be disabled. Ensure to have it set to a number suiting your requirements. Default value is set to 10. +[float] +[[configuration-rum-v2]] +=== Configuration options: v2 RUM endpoint + +Version 6.5 of the APM Server introduced a new intake API. You can learn more about this change in the <> documentation. + +The following configuration options only affect users that have upgraded their agent to take advantage of the new v2 intake API. + +[float] +==== `event_rate.limit` +Defines the maximum amount of events allowed to be sent to the APM Server v2 RUM endpoint per ip per second. +Defaults to 300. + +[float] +==== `event_rate.lru_size` +An LRU cache is used to keep a rate limit per IP for the most recently seen IPs. +This setting defines the number of unique IPs that can be tracked in the cache. +Sites with many concurrent clients should consider increasing this limit. +Defaults to 1000. + +[float] +[[configuration-rum-general]] +=== Configuration options: general + [float] ==== `allow_origins` -Comma separated list of permitted origins for RUM supprt. +Comma separated list of permitted origins for RUM support. User-agents send an Origin header that will be validated against this list. This is done automatically by modern browsers as part of the https://www.w3.org/TR/cors/[CORS specification]. -An origin is made of a protocol scheme, host and port, without the url path. +An origin is made of a protocol scheme, host and port, without the URL path. Default value is set to `['*']`, which allows everything. [float] ==== `library_pattern` -Regexp to be matched against a stacktrace frame's `file_name` and `abs_path` attributes. -If the regexp matches, the stacktrace frame is considered to be a library frame. -When source mapping is applied the `error.culprit` is set to reflect the _function_ and the _filename_ -of the first not library frame. +RegExp to be matched against a stacktrace frame's `file_name` and `abs_path` attributes. +If the RegExp matches, the stacktrace frame is considered to be a library frame. +When source mapping is applied, the `error.culprit` is set to reflect the _function_ and the _filename_ +of the first non library frame. This aims to provide an entry point for identifying issues. Default value is `"node_modules|bower_components|~"`. [float] ==== `exclude_from_grouping` -Regexp to be matched against a stacktrace frame's `file_name`. -If the regexp matches, the stacktrace frame is excluded from being used for calculating error groups. +RegExp to be matched against a stacktrace frame's `file_name`. +If the RegExp matches, the stacktrace frame is excluded from being used for calculating error groups. The default pattern excludes stacktrace frames that have a filename starting with `/webpack`. +[[config-sourcemapping-elasticsearch]] +[float] +==== `source_mapping.elasticsearch` +Configure the Elasticsearch source map retrieval location, taking the same options as <>. +This must be set when using an output other than Elasticsearch, and that output is writing to Elasticsearch. +Otherwise leave this section empty. + [[rum-sourcemap-cache]] [float] ==== `source_mapping.cache.expiration` @@ -64,15 +103,7 @@ Source maps are fetched from Elasticsearch and then kept in an in-memory cache f Values configured without a time unit are treated as seconds. Default value is 5 minutes. -[[config-sourcemapping-elasticsearch]] -[float] -==== `source_mapping.elasticsearch` -Configure the Elasticsearch source map retrieval location, taking the same options as <>. -This must be set when using an output other than Elasticsearch, and that output is writing to Elasticsearch. -Othewise leave this section empty. - [float] ==== `source_mapping.index_pattern` -Source maps are stored in a seperate index `apm-%{[beat.version]}-sourcemap` by default. -If changed, a matching index pattern needs to be specified here. - +Source maps are stored in a separate index `apm-%{[beat.version]}-sourcemap` by default. +If changed, a matching index pattern needs to be specified here. \ No newline at end of file diff --git a/docs/configuring-ingest.asciidoc b/docs/configuring-ingest.asciidoc index 47aaa63ad9b..8dcfeacda99 100644 --- a/docs/configuring-ingest.asciidoc +++ b/docs/configuring-ingest.asciidoc @@ -26,7 +26,7 @@ Automatic pipeline registration requires `output.elasticsearch` to be enabled an as well as having the required Elasticsearch plugins installed. APM Server default pipelines require you to have the Ingest user agent plugin installed. -If pipelines are enabeld but required plugins are missing, +If pipelines are enabled but required plugins are missing, the APM Server will not be able to properly connect to the Elasticsearch output. Find the default pipeline configuration at `ingest/pipeline/definition.json` of your APM Server installation's home directory. diff --git a/docs/configuring-output-after.asciidoc b/docs/configuring-output-after.asciidoc index 04f3788243f..d2934d87aab 100644 --- a/docs/configuring-output-after.asciidoc +++ b/docs/configuring-output-after.asciidoc @@ -24,4 +24,4 @@ fields: {project: "myproject", instance-id: "574734885120952459"} ------------------------------------------------------------------------------ To store the custom fields as top-level fields, set the `fields_under_root` option to true. -This is not recommended as backwards compatibility can not be ensured as new fields are added to APM documents. +This is not recommended as when new fields are added to APM documents backward compatibility cannot be ensured. diff --git a/docs/data-ingestion.asciidoc b/docs/data-ingestion.asciidoc index ce877e75984..b97bbeafac2 100644 --- a/docs/data-ingestion.asciidoc +++ b/docs/data-ingestion.asciidoc @@ -48,7 +48,7 @@ Increasing `queue.mem.events` can significantly affect APM Server memory usage. [[adjust-concurrent-requests]] [float] -=== Adjust concurrent requests +=== Adjust concurrent requests deprecated[6.5] APM Server has a limit to how many requests can be processed concurrently. This limit is determined by the `apm-server.concurrent_requests` setting. Increasing this value will improve throughput, but it can significantly affect APM Server memory usage. diff --git a/docs/data/elasticsearch/generated/errors.json b/docs/data/elasticsearch/generated/errors.json new file mode 100644 index 00000000000..f772c26ed63 --- /dev/null +++ b/docs/data/elasticsearch/generated/errors.json @@ -0,0 +1,434 @@ +{ + "events": [ + { + "@timestamp": "2017-05-09T15:04:05.999999Z", + "context": { + "custom": { + "and_objects": { + "foo": [ + "bar", + "baz" + ] + }, + "my_key": 1, + "some_other_value": "foo bar" + }, + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "request": { + "body": "Hello World", + "cookies": { + "c1": "v1", + "c2": "v2" + }, + "env": { + "GATEWAY_INTERFACE": "CGI/1.1", + "SERVER_SOFTWARE": "nginx" + }, + "headers": { + "array": [ + "foo", + "bar", + "baz" + ], + "content-type": "text/html", + "cookie": "c1=v1; c2=v2", + "some-other-header": "foo", + "user-agent": "Mozilla Chrome Edge" + }, + "http_version": "1.1", + "method": "POST", + "socket": { + "encrypted": true, + "remote_address": "12.53.12.1" + }, + "url": { + "full": "https://www.example.com/p/a/t/h?query=string#hash", + "hash": "#hash", + "hostname": "www.example.com", + "pathname": "/p/a/t/h", + "port": "8080", + "protocol": "https:", + "raw": "/p/a/t/h?query=string#hash", + "search": "?query=string" + } + }, + "response": { + "finished": true, + "headers": { + "content-type": "application/json" + }, + "headers_sent": true, + "status_code": 200 + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + }, + "tags": { + "organization_uuid": "9f0e9d64-c185-4d21-a6f4-4673ed561ec8" + }, + "user": { + "email": "foo@example.com", + "id": 99, + "username": "foo" + } + }, + "error": { + "culprit": "my.module.function_name", + "exception": { + "attributes": { + "foo": "bar" + }, + "code": "42", + "handled": false, + "message": "The username root is unknown", + "module": "__builtins__", + "stacktrace": [ + { + "abs_path": "/real/file/name.py", + "context": { + "post": [ + "line4", + "line5" + ], + "pre": [ + "line1", + "line2" + ] + }, + "exclude_from_grouping": false, + "filename": "file/name.py", + "function": "foo", + "library_frame": true, + "line": { + "column": 4, + "context": "line3", + "number": 3 + }, + "module": "App::MyModule", + "vars": { + "key": "value" + } + }, + { + "abs_path": "/Users/watson/code/node_modules/elastic/lib/instrumentation/index.js", + "context": { + "post": [ + " ins.currentTransaction = prev", + " return result", + "}", + "}", + "", + "Instrumentation.prototype._recoverTransaction = function (trans) {", + " if (this.currentTransaction === trans) return" + ], + "pre": [ + " var trans = this.currentTransaction", + "", + " return instrumented", + "", + " function instrumented () {", + " var prev = ins.currentTransaction", + " ins.currentTransaction = trans" + ] + }, + "exclude_from_grouping": false, + "filename": "lib/instrumentation/index.js", + "function": "instrumented", + "line": { + "context": " var result = original.apply(this, arguments)", + "number": 102 + }, + "vars": { + "key": "value" + } + } + ], + "type": "DbError" + }, + "grouping_key": "50f62f37edffc4630c6655ba3ecfcf46", + "id": "0123456789012345", + "log": { + "level": "warning", + "logger_name": "my.logger.name", + "message": "My service could not talk to the database named foobar", + "param_message": "My service could not talk to the database named %s", + "stacktrace": [ + { + "abs_path": "/real/file/name.py", + "context": { + "post": [ + "line4", + "line5" + ], + "pre": [ + "line1", + "line2" + ] + }, + "exclude_from_grouping": false, + "filename": "/webpack/file/name.py", + "function": "foo", + "library_frame": false, + "line": { + "column": 4, + "context": "line3", + "number": 3 + }, + "module": "App::MyModule", + "vars": { + "key": "value" + } + }, + { + "abs_path": "/Users/watson/code/node_modules/elastic/lib/instrumentation/index.js", + "context": { + "post": [ + " ins.currentTransaction = prev", + " return result", + "}", + "}", + "", + "Instrumentation.prototype._recoverTransaction = function (trans) {", + " if (this.currentTransaction === trans) return" + ], + "pre": [ + " var trans = this.currentTransaction", + "", + " return instrumented", + "", + " function instrumented () {", + " var prev = ins.currentTransaction", + " ins.currentTransaction = trans" + ] + }, + "exclude_from_grouping": false, + "filename": "lib/instrumentation/index.js", + "function": "instrumented", + "line": { + "context": " var result = original.apply(this, arguments)", + "number": 102 + }, + "vars": { + "key": "value" + } + } + ] + } + }, + "processor": { + "event": "error", + "name": "error" + }, + "timestamp": { + "us": 1494342245999999 + } + }, + { + "@timestamp": "2018-08-09T14:59:05.999Z", + "context": { + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + } + }, + "error": { + "exception": { + "message": "Cannot read property 'baz' no defined" + }, + "grouping_key": "ae0232fed4cb40e7ebc62a585a421d60", + "id": "cdefab0123456789" + }, + "processor": { + "event": "error", + "name": "error" + }, + "timestamp": { + "us": 1533826745999000 + } + }, + { + "@timestamp": "2018-08-01T10:00:00Z", + "context": { + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + } + }, + "error": { + "exception": { + "type": "DbError" + }, + "grouping_key": "c3868d6704b923014eaffea034e70a3d", + "id": "cdefab0123456789" + }, + "processor": { + "event": "error", + "name": "error" + }, + "timestamp": { + "us": 1533117600000000 + } + }, + { + "@timestamp": "2018-08-09T15:04:05.999Z", + "context": { + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + } + }, + "error": { + "grouping_key": "d6b3f958dfea98dc9ed2b57d5f0c48bb", + "id": "abcdef0123456789", + "log": { + "level": "custom log level", + "message": "Cannot read property 'baz' of undefined" + } + }, + "parent": { + "id": "9632587410abcdef" + }, + "processor": { + "event": "error", + "name": "error" + }, + "timestamp": { + "us": 1533827045999000 + }, + "trace": { + "id": "0123456789abcdeffedcba0123456789" + }, + "transaction": { + "id": "1234567890987654" + } + } + ] +} diff --git a/docs/data/elasticsearch/generated/metricsets.json b/docs/data/elasticsearch/generated/metricsets.json new file mode 100644 index 00000000000..da6753b3599 --- /dev/null +++ b/docs/data/elasticsearch/generated/metricsets.json @@ -0,0 +1,85 @@ +{ + "events": [ + { + "@timestamp": "2017-05-30T18:53:42.281Z", + "byte_counter": 1, + "context": { + "process": { + "pid": 1234 + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "language": { + "name": "ecmascript" + }, + "name": "1234_service-12a3" + }, + "tags": { + "code": "200", + "some.other.code": "abc" + } + }, + "dotted": { + "float": { + "gauge": 6.12 + } + }, + "double_gauge": 3.141592653589793, + "float_gauge": 9.16, + "integer_gauge": 42767, + "long_gauge": 3147483648, + "negative": { + "d": { + "o": { + "t": { + "t": { + "e": { + "d": -1022 + } + } + } + } + } + }, + "processor": { + "event": "metric", + "name": "metric" + }, + "short_counter": 227 + }, + { + "@timestamp": "2017-05-30T18:53:42.281Z", + "context": { + "process": { + "pid": 1234 + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "language": { + "name": "ecmascript" + }, + "name": "1234_service-12a3" + } + }, + "go": { + "memstats": { + "heap": { + "sys": { + "bytes": 6520832 + } + } + } + }, + "processor": { + "event": "metric", + "name": "metric" + } + } + ] +} diff --git a/docs/data/elasticsearch/generated/spans.json b/docs/data/elasticsearch/generated/spans.json new file mode 100644 index 00000000000..4f3b3d1bec5 --- /dev/null +++ b/docs/data/elasticsearch/generated/spans.json @@ -0,0 +1,268 @@ +{ + "events": [ + { + "@timestamp": "2018-07-30T18:53:42.281Z", + "context": { + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "name": "backendspans" + } + }, + "parent": { + "id": "abcdef0123456789" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "span": { + "duration": { + "us": 141581 + }, + "hex_id": "abcdef01234567", + "id": -9175013389437287000, + "name": "GET /api/types", + "parent": 3156441702022342700, + "type": "request" + }, + "timestamp": { + "us": 1532976822281000 + }, + "trace": { + "id": "fdedef0123456789abcdef9876543210" + }, + "transaction": { + "id": "01af25874dec69dd" + } + }, + { + "@timestamp": "2018-07-30T18:53:42.281Z", + "context": { + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "name": "backendspans" + } + }, + "parent": { + "id": "0000000011111111" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "span": { + "duration": { + "us": 32592 + }, + "hex_id": "1234abcdef567895", + "id": -7911509744411052000, + "name": "GET /api/types", + "parent": -9223372036568445000, + "start": { + "us": 22000 + }, + "type": "request" + }, + "timestamp": { + "us": 1532976822281000 + }, + "trace": { + "id": "abcdef0123456789abcdef9876543210" + }, + "transaction": { + "id": "ab45781d265894fe" + } + }, + { + "@timestamp": "2018-08-01T10:00:00.001845Z", + "context": { + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "name": "backendspans" + }, + "tags": { + "tag1": "value1" + } + }, + "parent": { + "id": "abcdefabcdef7890" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "span": { + "duration": { + "us": 3564 + }, + "hex_id": "0123456a89012345", + "id": -9141386494764572000, + "name": "GET /api/types", + "parent": 3156442435030055000, + "start": { + "us": 1845 + }, + "type": "request" + }, + "timestamp": { + "us": 1533117600001845 + }, + "trace": { + "id": "abcdef0123456789abcdef9876543210" + }, + "transaction": { + "id": "ab23456a89012345" + } + }, + { + "@timestamp": "2018-08-01T10:00:00Z", + "context": { + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "name": "backendspans" + } + }, + "parent": { + "id": "ababcdcdefefabde" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "span": { + "duration": { + "us": 13980 + }, + "hex_id": "abcde56a89012345", + "id": 3156431159584433000, + "name": "get /api/types", + "parent": 3146835049025875000, + "start": { + "us": 0 + }, + "type": "request" + }, + "timestamp": { + "us": 1533117600000000 + }, + "trace": { + "id": "abcdef0123456789abcdef9876543210" + }, + "transaction": { + "id": "bed3456a89012345" + } + }, + { + "@timestamp": "2018-08-01T10:00:00.00283092Z", + "context": { + "db": { + "instance": "customers", + "statement": "SELECT * FROM product_types WHERE user_id=?", + "type": "sql", + "user": "readonly_user" + }, + "http": { + "method": "GET", + "status_code": 200, + "url": "http://localhost:8000" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "name": "backendspans" + } + }, + "parent": { + "id": "abcdef0123456789" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "span": { + "duration": { + "us": 3781 + }, + "hex_id": "1234567890aaaade", + "id": -7911603569559950000, + "name": "SELECT FROM product_types", + "parent": 3156441702022342700, + "stacktrace": [ + { + "exclude_from_grouping": false, + "filename": "net.js", + "line": { + "number": 547 + } + }, + { + "context": { + "post": [ + " ins.currentTransaction = prev", + "}" + ] + }, + "exclude_from_grouping": false, + "filename": "file2.js", + "line": { + "number": 12 + } + }, + { + "abs_path": "net.js", + "context": { + "post": [ + " ins.currentTransaction = prev", + " return result" + ], + "pre": [ + " var trans = this.currentTransaction", + "" + ] + }, + "exclude_from_grouping": false, + "filename": "net.js", + "function": "onread", + "library_frame": true, + "line": { + "column": 4, + "context": "line3", + "number": 547 + }, + "module": "some module", + "vars": { + "key": "value" + } + } + ], + "start": { + "us": 2830 + }, + "type": "db.postgresql.query" + }, + "timestamp": { + "us": 1533117600002830 + }, + "trace": { + "id": "abcdef0123456789abcdef9876543210" + }, + "transaction": { + "id": "aff4567890aaaade" + } + } + ] +} diff --git a/docs/data/elasticsearch/generated/transactions.json b/docs/data/elasticsearch/generated/transactions.json new file mode 100644 index 00000000000..df1c5712a14 --- /dev/null +++ b/docs/data/elasticsearch/generated/transactions.json @@ -0,0 +1,282 @@ +{ + "events": [ + { + "@timestamp": "2018-08-01T10:00:00Z", + "context": { + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + } + }, + "parent": { + "id": "abcdefabcdef01234567" + }, + "processor": { + "event": "transaction", + "name": "transaction" + }, + "timestamp": { + "us": 1533117600000000 + }, + "trace": { + "id": "0123456789abcdef0123456789abcdef" + }, + "transaction": { + "duration": { + "us": 32592 + }, + "id": "945254c567a5417e", + "sampled": true, + "span_count": { + "started": 43 + }, + "type": "request" + } + }, + { + "@timestamp": "2017-05-30T18:53:27.154Z", + "context": { + "custom": { + "(": "not a valid regex and that is fine", + "and_objects": { + "foo": [ + "bar", + "baz" + ] + }, + "my_key": 1, + "some_other_value": "foo bar" + }, + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "request": { + "body": { + "additional": { + "bar": 123, + "req": "additional information" + }, + "str": "hello world" + }, + "cookies": { + "c1": "v1", + "c2": "v2" + }, + "env": { + "GATEWAY_INTERFACE": "CGI/1.1", + "SERVER_SOFTWARE": "nginx" + }, + "headers": { + "array": [ + "foo", + "bar", + "baz" + ], + "content-type": "text/html", + "cookie": "c1=v1; c2=v2", + "some-other-header": "foo", + "user-agent": "Mozilla Chrome Edge" + }, + "http_version": "1.1", + "method": "POST", + "socket": { + "encrypted": true, + "remote_address": "12.53.12.1" + }, + "url": { + "full": "https://www.example.com/p/a/t/h?query=string#hash", + "hash": "#hash", + "hostname": "www.example.com", + "pathname": "/p/a/t/h", + "port": "8080", + "protocol": "https:", + "raw": "/p/a/t/h?query=string#hash", + "search": "?query=string" + } + }, + "response": { + "finished": true, + "headers": { + "content-type": "application/json" + }, + "headers_sent": true, + "status_code": 200 + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + }, + "tags": { + "organization_uuid": "9f0e9d64-c185-4d21-a6f4-4673ed561ec8" + }, + "user": { + "email": "foo@example.com", + "id": "99", + "username": "foo" + } + }, + "processor": { + "event": "transaction", + "name": "transaction" + }, + "timestamp": { + "us": 1496170407154000 + }, + "trace": { + "id": "0acd456789abcdef0123456789abcdef" + }, + "transaction": { + "duration": { + "us": 32592 + }, + "id": "4340a8e0df1906ecbfa9", + "name": "GET /api/types", + "result": "success", + "sampled": true, + "span_count": { + "started": 17 + }, + "type": "request" + } + }, + { + "@timestamp": "2018-07-30T18:53:42.281Z", + "context": { + "process": { + "argv": [ + "node", + "server.js" + ], + "pid": 1234, + "ppid": 6789, + "title": "node" + }, + "request": { + "method": "POST" + }, + "service": { + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "environment": "staging", + "framework": { + "name": "Express", + "version": "1.2.3" + }, + "language": { + "name": "ecmascript", + "version": "8" + }, + "name": "1234_service-12a3", + "runtime": { + "name": "node", + "version": "8.0.0" + }, + "version": "5.1.3" + }, + "system": { + "architecture": "x64", + "hostname": "prod1.example.com", + "ip": "192.0.0.1", + "platform": "darwin" + } + }, + "processor": { + "event": "transaction", + "name": "transaction" + }, + "timestamp": { + "us": 1532976822281000 + }, + "trace": { + "id": "0acd456789abcdef0123456789abcdef" + }, + "transaction": { + "duration": { + "us": 13980 + }, + "id": "cdef4340a8e0df19", + "marks": { + "another_mark": { + "some_float": 10, + "some_long": 10 + }, + "navigationTiming": { + "appBeforeBootstrap": 608.93, + "navigationStart": -21 + } + }, + "sampled": true, + "span_count": { + "dropped": { + "total": 55 + }, + "started": 436 + }, + "type": "request" + } + } + ] +} diff --git a/docs/data/intake-api/generated/error/minimal_payload_exception.json b/docs/data/intake-api/generated/error/minimal_payload_exception.json deleted file mode 100644 index f616c4bc552..00000000000 --- a/docs/data/intake-api/generated/error/minimal_payload_exception.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "service": { - "name": "1234_service-12a3", - "agent": { - "name": "python", - "version": "1.0" - } - }, - "errors": [ - { - "exception": { - "message": "" - } - } - ] -} diff --git a/docs/data/intake-api/generated/error/minimal_payload_log.json b/docs/data/intake-api/generated/error/minimal_payload_log.json deleted file mode 100644 index 79f51e395ec..00000000000 --- a/docs/data/intake-api/generated/error/minimal_payload_log.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "service": { - "name": "1234_service-12a3", - "agent": { - "name": "python", - "version": "1.0" - } - }, - "errors": [ - { - "log": { - "message": "" - } - } - ] -} diff --git a/docs/data/intake-api/generated/error/payload.json b/docs/data/intake-api/generated/error/payload.json deleted file mode 100644 index e0c6fa0e763..00000000000 --- a/docs/data/intake-api/generated/error/payload.json +++ /dev/null @@ -1,254 +0,0 @@ -{ - "service": { - "name": "1234_service-12a3", - "version": "5.1.3", - "environment": "staging", - "language": { - "name": "ecmascript", - "version": "8" - }, - "runtime": { - "name": "node", - "version": "8.0.0" - }, - "framework": { - "name": "Express", - "version": "1.2.3" - }, - "agent": { - "name": "elastic-node", - "version": "3.14.0" - } - }, - "process": { - "pid": 1234, - "ppid": 7788, - "title": "node", - "argv": [ - "node", - "server.js" - ] - }, - "system": { - "hostname": "prod1.example.com", - "architecture": "x64", - "platform": "darwin" - }, - "errors": [ - { - "id": "5f0e9d64-c185-4d21-a6f4-4673ed561ec8", - "timestamp": "2017-05-09T15:04:05.999999Z", - "culprit": "my.module.function_name", - "log": { - "message": "My service could not talk to the database named foobar", - "param_message": "My service could not talk to the database named %s", - "logger_name": "my.logger.name", - "level": "warning", - "stacktrace": [ - { - "abs_path": "/real/file/name.py", - "filename": "/webpack/file/name.py", - "function": "foo", - "vars": { - "key": "value" - }, - "pre_context": [ - "line1", - "line2" - ], - "context_line": "line3", - "library_frame": false, - "lineno": 3, - "module": "App::MyModule", - "colno": 4, - "post_context": [ - "line4", - "line5" - ] - }, - { - "filename": "lib/instrumentation/index.js", - "lineno": 102, - "function": "instrumented", - "abs_path": "/Users/watson/code/node_modules/elastic/lib/instrumentation/index.js", - "vars": { - "key": "value" - }, - "pre_context": [ - " var trans = this.currentTransaction", - "", - " return instrumented", - "", - " function instrumented () {", - " var prev = ins.currentTransaction", - " ins.currentTransaction = trans" - ], - "context_line": " var result = original.apply(this, arguments)", - "post_context": [ - " ins.currentTransaction = prev", - " return result", - "}", - "}", - "", - "Instrumentation.prototype._recoverTransaction = function (trans) {", - " if (this.currentTransaction === trans) return" - ] - } - ] - }, - "exception": { - "message": "The username root is unknown", - "type": "DbError", - "module": "__builtins__", - "code": 42, - "handled": false, - "attributes": { - "foo": "bar" - }, - "stacktrace": [ - { - "abs_path": "/real/file/name.py", - "filename": "file/name.py", - "function": "foo", - "vars": { - "key": "value" - }, - "pre_context": [ - "line1", - "line2" - ], - "context_line": "line3", - "library_frame": true, - "lineno": 3, - "module": "App::MyModule", - "colno": 4, - "post_context": [ - "line4", - "line5" - ] - }, - { - "filename": "lib/instrumentation/index.js", - "lineno": 102, - "function": "instrumented", - "abs_path": "/Users/watson/code/node_modules/elastic/lib/instrumentation/index.js", - "vars": { - "key": "value" - }, - "pre_context": [ - " var trans = this.currentTransaction", - "", - " return instrumented", - "", - " function instrumented () {", - " var prev = ins.currentTransaction", - " ins.currentTransaction = trans" - ], - "context_line": " var result = original.apply(this, arguments)", - "post_context": [ - " ins.currentTransaction = prev", - " return result", - "}", - "}", - "", - "Instrumentation.prototype._recoverTransaction = function (trans) {", - " if (this.currentTransaction === trans) return" - ] - } - ] - }, - "context": { - "request": { - "socket": { - "remote_address": "12.53.12.1", - "encrypted": true - }, - "http_version": "1.1", - "method": "POST", - "url": { - "protocol": "https:", - "full": "https://www.example.com/p/a/t/h?query=string#hash", - "hostname": "www.example.com", - "port": "8080", - "pathname": "/p/a/t/h", - "search": "?query=string", - "hash": "#hash", - "raw": "/p/a/t/h?query=string#hash" - }, - "headers": { - "user-agent": "Mozilla Chrome Edge", - "content-type": "text/html", - "cookie": "c1=v1; c2=v2", - "some-other-header": "foo", - "array": [ - "foo", - "bar", - "baz" - ] - }, - "cookies": { - "c1": "v1", - "c2": "v2" - }, - "env": { - "SERVER_SOFTWARE": "nginx", - "GATEWAY_INTERFACE": "CGI/1.1" - }, - "body": "Hello World" - }, - "response": { - "status_code": 200, - "headers": { - "content-type": "application/json" - }, - "headers_sent": true, - "finished": true - }, - "user": { - "id": 99, - "username": "foo", - "email": "foo@example.com" - }, - "tags": { - "organization_uuid": "9f0e9d64-c185-4d21-a6f4-4673ed561ec8" - }, - "custom": { - "my_key": 1, - "some_other_value": "foo bar", - "and_objects": { - "foo": [ - "bar", - "baz" - ] - } - } - }, - "transaction": { - "id": "945254c5-67a5-417e-8a4e-aa29efcbfb79" - } - }, - { - "id": "8f0e9d68-c185-4d21-a6f4-4673ed561ec8", - "timestamp": "2017-05-09T15:04:05.1Z", - "exception": { - "message": "foo is not defined", - "code": "35" - } - }, - { - "id": "7f0e9d68-c185-4d21-a6f4-4673ed561ec8", - "timestamp": "2017-05-09T15:04:05Z", - "exception": { - "type": "connection error" - } - }, - { - "id": "0f0e9d67-c185-4d21-a6f4-4673ed561ec8", - "timestamp": "2017-05-09T15:04:05.999Z", - "log": { - "level": "custom log level", - "message": "Cannot read property 'baz' of undefined" - } - } - ] -} diff --git a/docs/data/intake-api/generated/error/rum.json b/docs/data/intake-api/generated/error/rum.json deleted file mode 100644 index 38bdd58dd32..00000000000 --- a/docs/data/intake-api/generated/error/rum.json +++ /dev/null @@ -1,88 +0,0 @@ - { - "service": { - "name": "apm-agent-js", - "version": "1.0.1", - "agent": { - "name": "apm-js", - "version": "0.0.0" - } - }, - "errors": [ - { - "id": "aba2688e-0338-48ce-9c4e-4005f1caa534", - "timestamp": "2017-12-08T12:18:50.291Z", - "culprit": "test/e2e/general-usecase/bundle.js.map", - "log": { - "message": "Uncaught Error: log timeout test error", - "stacktrace": [ - { - "abs_path": "http://localhost:8000/test/e2e/general-usecase/bundle.js.map", - "filename": "~/test/e2e/general-usecase/bundle.js.map", - "function": "", - "lineno": 1, - "colno": 18 - } - ] - }, - "exception": { - "message": "Uncaught Error: timeout test error", - "stacktrace": [ - { - "abs_path": "http://localhost:8000/test/../test/e2e/general-usecase/bundle.js.map", - "filename": "test/e2e/general-usecase/bundle.js.map", - "function": "", - "library_frame": true, - "lineno": 1, - "colno": 18 - }, - { - "abs_path": "http://localhost:8000/test/./e2e/general-usecase/bundle.js.map", - "filename": "~/test/e2e/general-usecase/bundle.js.map", - "function": "invokeTask", - "library_frame": false, - "lineno": 1, - "colno": 181 - }, - { - "abs_path": "http://localhost:8000/test/e2e/general-usecase/bundle.js.map", - "filename": "~/test/e2e/general-usecase/bundle.js.map", - "function": "runTask", - "lineno": 1, - "colno": 15 - }, - { - "abs_path": "http://localhost:8000/test/e2e/general-usecase/bundle.js.map", - "filename": "~/test/e2e/general-usecase/bundle.js.map", - "function": "invoke", - "lineno": 1, - "colno": 199 - }, - { - "abs_path": "http://localhost:8000/test/e2e/general-usecase/bundle.js.map", - "filename": "~/test/e2e/general-usecase/bundle.js.map", - "function": "timer", - "lineno": 1, - "colno": 33 - } - ], - "type": "Error" - }, - "context": { - "environment": { - "browserWidth": 150, - "browserHeight": 726, - "screenWidth": 1280, - "screenHeight": 800, - "language": "en-US", - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36", - "platform": "MacIntel" - }, - "page": { - "referer": "http://localhost:8000/test/e2e/", - "host": "localhost", - "location": "http://localhost:8000/test/e2e/general-usecase/" - } - } - } - ] -} diff --git a/docs/data/intake-api/generated/events.ndjson b/docs/data/intake-api/generated/events.ndjson new file mode 100644 index 00000000000..53c41863d61 --- /dev/null +++ b/docs/data/intake-api/generated/events.ndjson @@ -0,0 +1,5 @@ +{"metadata": { "process": {"pid": 1234 }, "service": {"name": "1234_service-12a3", "language": {"name": "ecmascript"}, "agent": {"version": "3.14.0", "name": "elastic-node"}}}} +{ "error": {"id": "abcdef0123456789", "timestamp": 1533827045999000,"log": {"level": "custom log level","message": "Cannot read property 'baz' of undefined"}}} +{ "span": { "id": "0123456a89012345", "trace_id": "0123456789abcdef0123456789abcdef", "parent_id": "ab23456a89012345", "transaction_id": "ab23456a89012345", "parent": 1, "name": "GET /api/types", "type": "request", "start": 1.845, "duration": 3.5642981, "stacktrace": [], "context": {} }} +{ "transaction": { "trace_id": "01234567890123456789abcdefabcdef", "id": "abcdef1478523690", "type": "request", "duration": 32.592981, "timestamp": 1535655207154000, "result": "200", "context": null, "spans": null, "sampled": null, "span_count": { "started": 0 }}} +{ "metricset": { "samples": { "go.memstats.heap.sys.bytes": { "value": 61235 } }, "timestamp": 1496170422281000 }} diff --git a/docs/data/intake-api/generated/metricset/payload.json b/docs/data/intake-api/generated/metricset/payload.json deleted file mode 100644 index 25ec0583ac6..00000000000 --- a/docs/data/intake-api/generated/metricset/payload.json +++ /dev/null @@ -1,62 +0,0 @@ -{ - "service": { - "name": "apm-server", - "version": "6.x", - "environment": "staging", - "language": { - "name": "go", - "version": "go1.10.1" - }, - "runtime": { - "name": "gc", - "version": "go1.10.1" - }, - "agent": { - "name": "elastic-go", - "version": "0.1.0" - } - }, - "process": { - "pid": 32472, - "ppid": 26110, - "argv": ["./apm-server"] - }, - "system": { - "ip":"127.0.0.1", - "architecture":"amd64", - "platform": "darwin" - }, - "metrics": [{ - "samples": { - "byte_counter": { - "value": 1 - }, - "short_counter": { - "value": 227 - }, - "integer_gauge": { - "value": 42767 - }, - "long_gauge": { - "value": 3147483648 - }, - "float_gauge": { - "value": 9.16 - }, - "double_gauge": { - "value": 3.141592653589793 - }, - "dotted.float.gauge": { - "value": 6.12 - }, - "negative.d.o.t.t.e.d": { - "value": -1022 - } - }, - "tags": { - "code": "200", - "some.other.code": "abc" - }, - "timestamp": "2017-05-30T18:53:42.281Z" - }] -} diff --git a/docs/data/intake-api/generated/transaction/minimal_payload.json b/docs/data/intake-api/generated/transaction/minimal_payload.json deleted file mode 100644 index bfdc2f54e87..00000000000 --- a/docs/data/intake-api/generated/transaction/minimal_payload.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "service": { - "name": "service1", - "agent": { - "name": "python", - "version": "1.0" - } - }, - "transactions": [ - { - "id": "945254c5-67a5-417e-8a4e-aa29efcbfb79", - "type": "request", - "duration": 32.592981 - } - ] -} diff --git a/docs/data/intake-api/generated/transaction/minimal_span.json b/docs/data/intake-api/generated/transaction/minimal_span.json deleted file mode 100644 index 9bf32c2e2bc..00000000000 --- a/docs/data/intake-api/generated/transaction/minimal_span.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "service": { - "name": "service1", - "agent": { - "name": "python", - "version": "1.0" - } - }, - "transactions": [ - { - "id": "945254c5-67a5-417e-8a4e-aa29efcbfb79", - "type": "request", - "duration": 32.592981, - "timestamp": "2017-05-30T18:53:27.154Z", - "spans": [ - { - "id": 100000000000, - "name": "GET /api/types", - "type": "request", - "start": 0, - "duration": 32.592981 - } - ] - } - ] -} diff --git a/docs/data/intake-api/generated/transaction/payload.json b/docs/data/intake-api/generated/transaction/payload.json deleted file mode 100644 index 6c87193cdd8..00000000000 --- a/docs/data/intake-api/generated/transaction/payload.json +++ /dev/null @@ -1,272 +0,0 @@ -{ - "service": { - "name": "1234_service-12a3", - "version": "5.1.3", - "environment": "staging", - "language": { - "name": "ecmascript", - "version": "8" - }, - "runtime": { - "name": "node", - "version": "8.0.0" - }, - "framework": { - "name": "Express", - "version": "1.2.3" - }, - "agent": { - "name": "elastic-node", - "version": "3.14.0" - } - }, - "process": { - "pid": 1234, - "ppid": 6789, - "title": "node", - "argv": [ - "node", - "server.js" - ] - }, - "system": { - "hostname": "prod1.example.com", - "architecture": "x64", - "platform": "darwin" - }, - "transactions": [ - { - "id": "945254c5-67a5-417e-8a4e-aa29efcbfb79", - "name": "GET /api/types", - "type": "request", - "duration": 32.592981, - "result": "success", - "timestamp": "2017-05-30T18:53:27.154Z", - "sampled": true, - "span_count": { - "dropped": { - "total": 2 - } - }, - "context": { - "request": { - "socket": { - "remote_address": "12.53.12.1", - "encrypted": true - }, - "http_version": "1.1", - "method": "POST", - "url": { - "protocol": "https:", - "full": "https://www.example.com/p/a/t/h?query=string#hash", - "hostname": "www.example.com", - "port": "8080", - "pathname": "/p/a/t/h", - "search": "?query=string", - "hash": "#hash", - "raw": "/p/a/t/h?query=string#hash" - }, - "headers": { - "user-agent": "Mozilla Chrome Edge", - "content-type": "text/html", - "cookie": "c1=v1; c2=v2", - "some-other-header": "foo", - "array": [ - "foo", - "bar", - "baz" - ] - }, - "cookies": { - "c1": "v1", - "c2": "v2" - }, - "env": { - "SERVER_SOFTWARE": "nginx", - "GATEWAY_INTERFACE": "CGI/1.1" - }, - "body": { - "str": "hello world", - "additional": { - "foo": {}, - "bar": 123, - "req": "additional information" - } - } - }, - "response": { - "status_code": 200, - "headers": { - "content-type": "application/json" - }, - "headers_sent": true, - "finished": true - }, - "user": { - "id": "99", - "username": "foo", - "email": "foo@example.com" - }, - "tags": { - "organization_uuid": "9f0e9d64-c185-4d21-a6f4-4673ed561ec8" - }, - "custom": { - "my_key": 1, - "some_other_value": "foo bar", - "and_objects": { - "foo": [ - "bar", - "baz" - ] - }, - "(": "not a valid regex and that is fine" - } - }, - "spans": [ - { - "id": 0, - "parent": null, - "name": "SELECT FROM product_types", - "type": "db.postgresql.query", - "start": 2.83092, - "duration": 3.781912, - "stacktrace": [ - { - "function": "onread", - "abs_path": "net.js", - "filename": "net.js", - "lineno": 547, - "library_frame": true, - "vars": { - "key": "value" - }, - "module": "some module", - "colno": 4, - "context_line": "line3", - "pre_context": [ - " var trans = this.currentTransaction", - "" - ], - "post_context": [ - " ins.currentTransaction = prev", - " return result", - "}" - ] - }, - { - "filename": "my2file.js", - "lineno": 10 - } - ], - "context": { - "db": { - "instance": "customers", - "statement": "SELECT * FROM product_types WHERE user_id=?", - "type": "sql", - "user": "readonly_user" - }, - "http": { - "url": "http://localhost:8000", - "status_code": 200, - "method": "GET" - }, - "tags": { - "span_tag": "something" - } - } - }, - { - "id": 1, - "parent": 0, - "name": "GET /api/types", - "type": "request", - "start": 0, - "duration": 32.592981 - }, - { - "id": 2, - "parent": 1, - "name": "GET /api/types", - "type": "request", - "start": 1.845, - "duration": 3.5642981, - "stacktrace": [], - "context": {} - }, - { - "id": 3, - "parent": 2, - "name": "GET /api/types", - "type": "request", - "start": 0, - "duration": 13.9802981, - "stacktrace": null, - "context": null - } - ], - "marks": { - "navigationTiming": { - "appBeforeBootstrap": 608.9300000000001, - "navigationStart": -21 - - }, - "another_mark": { - "some_long": 10, - "some_float": 10.0 - }, - "performance": {} - } - }, - { - "id": "85925e55-b43f-4340-a8e0-df1906ecbf7a", - "name": "GET /api/types", - "type": "request", - "duration": 13.980558, - "result": "failure", - "timestamp": "2017-05-30T18:53:42.281Z", - "spans": [], - "sampled": true - }, - { - "id": "85925e55-b43f-4340-a8e0-df1906ecbf78", - "name": "GET /api/types", - "type": "request", - "duration": 13.980558, - "result": "200", - "timestamp": "2017-05-30T18:53:42Z", - "sampled": false - }, - { - "id": "85925e55-b43f-4340-a8e0-df1906ecbfa9", - "name": "GET /api/types", - "type": "request", - "duration": 13.980558, - "result": "200", - "timestamp": "2017-05-30T18:53:42.281999Z", - "sampled": true, - "span_count": { - "dropped": { - "total": 258 - } - }, - "spans": [ - { - "id": 15, - "name": "SELECT FROM product_types", - "type": "db.postgresql.query", - "start": 2.83092, - "duration": 3.781912, - "stacktrace": [], - "context": { - "db": { - "instance": "customers", - "statement": "SELECT * FROM product_types WHERE user_id=?", - "type": "sql", - "user": "readonly_user" - } - } - } - ] - } - ] -} diff --git a/docs/error-api.asciidoc b/docs/error-api.asciidoc index 432b2b1c4d5..02b337f0675 100644 --- a/docs/error-api.asciidoc +++ b/docs/error-api.asciidoc @@ -1,148 +1,20 @@ [[error-api]] -== Error API +=== Errors -The following section contains information about: +An error or a logged error message captured by an agent occurring in a monitored service. -* <> -* <> -* <> - -[[error-endpoint]] -[float] -=== Endpoint - -Send a `HTTP POST` request to the APM Server `errors` endpoint: -[source,bash] ------------------------------------------------------------- -http(s)://{hostname}:{port}/v1/errors ------------------------------------------------------------- - -For <> send a `HTTP POST` request to the `rum errors` endpoint instead: - -[source,bash] ------------------------------------------------------------- -http(s)://{hostname}:{port}/v1/rum/errors ------------------------------------------------------------- - -[[error-schema-definition]] -[float] -=== Schema Definition - -The APM Server uses JSON Schema for validating requests. The specification for errors is defined bellow: -* <> -* <> -* <> -* <> -* <> -* <> -* <> - - -[[error-payload-schema]] -[float] -==== Payload - -[source,json] ----- -include::./spec/errors/v1_error.json[] ----- - -[[error-service-schema]] -[float] -==== Service - -[source,json] ----- -include::./spec/service.json[] ----- - -[[error-system-schema]] -[float] -==== System - -[source,json] ----- -include::./spec/system.json[] ----- - -[[error-context-schema]] -[float] -==== Context - -[source,json] ----- -include::./spec/context.json[] ----- - -[[error-stacktraceframe-schema]] [float] -==== Stacktrace Frame +[[error-schema]] +==== Error Schema -[source,json] ----- -include::./spec/stacktrace_frame.json[] ----- - -[[error-request-schema]] -[float] -==== Request +The APM Server uses JSON Schema for validating requests. The specification for errors is defined below: [source,json] ---- -include::./spec/request.json[] +include::./spec/errors/v2_error.json[] ---- -[[error-user-schema]] -[float] -==== User - [source,json] ---- -include::./spec/user.json[] ----- - -[[error-api-examples]] -[float] -=== Examples - -Request example: - -["source","sh",subs="attributes"] ------------------------------------------------------------- -curl http://localhost:8200/v1/errors \ - --header "Content-Type: application/json" \ - --data @docs/data/intake-api/generated/error/payload.json ------------------------------------------------------------- - -Example error requests: - -* <> -* <> -* <> - -[[payload-with-error]] -[float] -==== Payload with an Error - -[source,json] ----- -include::./data/intake-api/generated/error/payload.json[] ----- - -[[payload-with-minimal-exception]] -[float] -==== Payload with an Error with minimal Exception Information - -[source,json] ----- -include::./data/intake-api/generated/error/minimal_payload_exception.json[] ----- - -[[payload-with-minimal-log]] -[float] -==== Payload with an Error with minimal Log Information - -[source,json] ----- -include::./data/intake-api/generated/error/minimal_payload_log.json[] ----- +include::./spec/errors/common_error.json[] +---- \ No newline at end of file diff --git a/docs/error-indices.asciidoc b/docs/error-indices.asciidoc index c513cf98777..ce296b9df75 100644 --- a/docs/error-indices.asciidoc +++ b/docs/error-indices.asciidoc @@ -5,11 +5,11 @@ Errors are stored in separate indices of the format `apm-[version]-error-[date]` [[error-example]] [float] -=== Example document +=== Example documents -See an example error indexed in Elasticsearch: +See how error documents can look like when indexed in Elasticsearch: [source,json] ---- -include::./data/elasticsearch/error.json[] +include::./data/elasticsearch/generated/errors.json[] ---- diff --git a/docs/errors.asciidoc b/docs/errors.asciidoc deleted file mode 100644 index 2c046199294..00000000000 --- a/docs/errors.asciidoc +++ /dev/null @@ -1,21 +0,0 @@ -[[errors]] -== Errors - -Errors are identified by a unique ID. -An error event contains at least -information about the original `exception` that occured -or about a `log` created when the exception occured. - -Both the captured `exception` and the captured `log` of an error can contain a `stack trace`, -helpful for debugging. - -The `culprit` of an error indicates where is originated. - -An error might relate to the <> during which it happened, -via the `transaction.id`. - -Errors also have some contextual data. - -include::./context.asciidoc[] - -Errors are stored in <>. diff --git a/docs/event-types.asciidoc b/docs/event-types.asciidoc deleted file mode 100644 index 070d92df1f6..00000000000 --- a/docs/event-types.asciidoc +++ /dev/null @@ -1,13 +0,0 @@ -[[event-types]] -= Event Types - -[partintro] --- -Agents capture different types of information, namely `transactions` and `errors`. - -* <> -* <> --- - -include::./transactions.asciidoc[] -include::./errors.asciidoc[] diff --git a/docs/events-api.asciidoc b/docs/events-api.asciidoc new file mode 100644 index 00000000000..9ab53f9b229 --- /dev/null +++ b/docs/events-api.asciidoc @@ -0,0 +1,113 @@ +[[events-api]] +== Events API + +Agents capture different types of information, known as events. +These events are sent to a single endpoint which then sorts and processes the events. +Events can be: + +* Transactions +* Spans +* Errors +* Metricsets + +You can learn more about events in the {apm-get-started-ref}/apm-data-model.html[APM Data Model] documentation. + +[[events-api-endpoint]] +[float] +=== Endpoint + +Send an `HTTP POST` request to the APM Server `intake/v2/events` endpoint: + +[source,bash] +------------------------------------------------------------ +http(s)://{hostname}:{port}/intake/v2/events +------------------------------------------------------------ + +For <> send an `HTTP POST` request to the APM Server `intake/v2/rum/events` endpoint instead: + +[source,bash] +------------------------------------------------------------ +http(s)://{hostname}:{port}/intake/v2/rum/events +------------------------------------------------------------ + +[[events-api-response]] +[float] +=== Response + +On success, the server will respond with a 202 Accepted status code and no body. + +Keep in mind that events can succeed and fail independently of each other. Only if all events succeed does the server respond with a 202. + +[[events-api-errors]] +[float] +==== Errors + +There are two types of errors that the APM Server may return to an agent: + +* Event related errors (typically validation errors) +* Non-event related errors + +The APM Server processes events one after the other. +If an error is encountered while processing an event, +the error encountered as well as the document causing the error are added to an internal array. +The APM Server will only save 5 event related errors. +If it encounters more than 5 event related errors, +the additional errors will not be returned to agent. +Once all events have been processed, +the error response is sent. + +Some errors, not relating to specific events, +may terminate the request immediately. +For example: queue is full, IP rate limit reached, wrong metadata, etc. +If at any point one of these errors is encountered, +it is added to the internal array and immediately returned. + +An example error response might look something like this: + +[source,json] +------------------------------------------------------------ +{ + "errors": [ + { + "message": "", <1> + "document": "" <2> + },{ + "message": "", + "document": "" + },{ + "message": "", + "document": "" + },{ + "message": "queue is full" <3> + }, + ], + "accepted": 2320 <4> +} +------------------------------------------------------------ + +<1> An event related error +<2> The document causing the error +<3> An immediately returning non-event related error +<4> The number of accepted events + +If you're developing an agent, these errors can be useful for debugging your agent while building it. + +[[events-api-schema-definition]] +[float] +=== Event API Schemas + +The APM Server uses a collection of JSON Schemas for validating requests to the intake API: + +* <> +* <> +* <> +* <> +* <> +* <> + +include::./metadata-api.asciidoc[] +include::./transaction-api.asciidoc[] +include::./span-api.asciidoc[] +include::./error-api.asciidoc[] +include::./metricset-api.asciidoc[] +include::./example-intakev2-events.asciidoc[] diff --git a/docs/example-intakev2-events.asciidoc b/docs/example-intakev2-events.asciidoc new file mode 100644 index 00000000000..0d4be93dcc5 --- /dev/null +++ b/docs/example-intakev2-events.asciidoc @@ -0,0 +1,9 @@ +[[example-intakev2-events]] +=== Example Request Body + +See a request body example containing one event for all currently supported event types. + +[source,json] +---- +include::./data/intake-api/generated/events.ndjson[] +---- diff --git a/docs/exploring-es-data.asciidoc b/docs/exploring-es-data.asciidoc index 65969df3510..a6bb79bdf03 100644 --- a/docs/exploring-es-data.asciidoc +++ b/docs/exploring-es-data.asciidoc @@ -3,10 +3,22 @@ [partintro] -- -By default Elastic APM data is stored in separated indices following the format: -`apm-%{[version]}-{type}-%{+yyyy.MM.dd}`. -Data types are described <>. +NOTE: Templates describing how the APM Server stores data in Elasticsearch have changed in v6.5. +Read the <> for more information. + +By default, Elastic APM data is stored in separated indices in the following formats: +------------------------------------------------------------ +apm-%{[version]}-sourcemap +apm-%{[version]}-error-%{+yyyy.MM.dd} +apm-%{[version]}-transaction-%{+yyyy.MM.dd} +apm-%{[version]}-span-%{+yyyy.MM.dd} +apm-%{[version]}-metric-%{+yyyy.MM.dd} +------------------------------------------------------------ + +If you're unfamiliar with the data types shown above, they are described in the {apm-get-started-ref}/apm-data-model.html[APM data model]. + +TIP: If your APM data is being stored in a different format, you may be using an outdated `apm-server.yml` file. You must update your `apm-server.yml` file in order to take advantage of the new format of indices. To get an overview of existing indices you can run: ["source","sh"] @@ -39,7 +51,7 @@ GET apm-*transactions-*/_search // CONSOLE If you are interested in the _settings_ and _mappings_ of the Elastic APM indices, -first run a query to find template names: +first, run a query to find template names: ["source","sh"] ------------------------------------------------------------ @@ -65,4 +77,5 @@ When clicking on a specific index you can view the _settings_ and _mapping_ for include::./transaction-indices.asciidoc[] include::./span-indices.asciidoc[] include::./error-indices.asciidoc[] +include::./metricset-indices.asciidoc[] include::./sourcemap-indices.asciidoc[] diff --git a/docs/guide/agent-server-compatibility.asciidoc b/docs/guide/agent-server-compatibility.asciidoc new file mode 100644 index 00000000000..7cd49f41a83 --- /dev/null +++ b/docs/guide/agent-server-compatibility.asciidoc @@ -0,0 +1,73 @@ +[[agent-server-compatibility]] +== Agent/Server Compatibility + +Below is a chart that outlines the compatibility between different versions of the APM agents and the APM Server. + +NOTE: APM Server version 6.5 introduced a new way for APM agents to communicate with the APM Server. +To take advantage of new features, +it is recommended that you update to the newest version of the applicable agent. + +[float] +[[nodejs-compatibility]] +==== Node.js Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|1.x |6.2-6.x +|2.x |>= 6.5 +|======================================================================= + +[float] +[[python-compatibility]] +==== Python Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|2.x/3.x |6.2-6.x +|4.x |>= 6.5 +|======================================================================= + +[float] +[[ruby-compatibility]] +==== Ruby Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|1.x |6.4-6.x +|2.x |>= 6.5 +|======================================================================= + +[float] +[[java-compatibility]] +==== Java Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|1.x |>= 6.5 +|======================================================================= + +[float] +[[go-compatibility]] +==== Go Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|1.x |>= 6.5 +|======================================================================= + +[float] +[[rum-compatibility]] +==== RUM Agent Compatibility + +[options="header"] +|======================================================================= +|Agent Version |APM Server Version +|0.x |6.3-6.4 +|1.x |6.4 +|2.x |>= 6.5 +|======================================================================= diff --git a/docs/guide/apm-data-model.asciidoc b/docs/guide/apm-data-model.asciidoc new file mode 100644 index 00000000000..50fcc249dc3 --- /dev/null +++ b/docs/guide/apm-data-model.asciidoc @@ -0,0 +1,107 @@ +[[apm-data-model]] +== Data Model + +Elastic APM agents capture different types of information from within their instrumented applications - namely `transactions`, `spans`, and `errors`. + +* <> +* <> +* <> + +[[transactions]] +=== Transactions + +A transaction describes an event captured by an Elastic APM agent instrumenting a service. Some examples of a transaction might be: + +* An HTTP request +* A background job + +A transaction contains: + +* The timestamp and duration of the event +* A unique id, type, and name +* A result (e.g. a response code) +* Some contextual data (see below for details) +* Other relevant information depending on the agent. Example: The JavaScript RUM captures transaction marks, +which are points in time relative to the start of the transaction with some label. + +[[transactions-context]] +include::../context.asciidoc[] + +Agents decide whether to sample transactions or not, +and provide settings to control sampling behavior. +If sampled, +the <> of a transaction are sent and stored as separate documents. Within one transaction there can be several, or no spans captured. + +Transactions are stored in {apm-server-ref}/transaction-indices.html[transaction indices]. + +[[transaction-spans]] +=== Spans +Transactions can have 0, 1, or many spans. Spans have a `transaction.id` attribute that refers to their parent transaction. They also have a `parent.id` attribute that refers to the parent span, or their transaction. + +Agents typically automatically instrument a variety of of libraries, +but also provide an API for ad hoc instrumentation of specific code paths. +A span contains information about a specific code path that has been executed as part of a transaction. +This information includes: + +* start time +* duration +* name +* type +* `stack trace` (optional) + +As an example, if a database query happens within a sampled transaction, +a span describing the database query will be created. +The name of this span will contain information about the query itself, +and the type of this span will contain information about the database. + +Spans are stored in {apm-server-ref}/span-indices.html[span indices]. +Note that these indices are separate from {apm-server-ref}/transaction-indices.html[transaction indices] by default. + +[float] +[[dropped-spans]] +==== Dropped Spans + +For performance reasons, some APM agents can choose to purposefully sample or omit spans. +One example of this might be for long running transactions with over 100 spans. +These edge cases can overload both the agent and the APM Server. +To avoid this, agents will drop spans. When they do this, +they notify the server of exactly how many spans were dropped. +This note is then passed on to the user in the UI. + +Settings affecting dropped spans, and more details on why they might occur, +are available in the relevant agent documentation: + +* {apm-node-ref}/agent-api.html#transaction-max-spans[Node.js Agent max spans] +* {apm-py-ref}/configuration.html[Python Agent max spans] + +[float] +[[missing-spans]] +==== Missing Spans + +Similarly to dropped spans, transactions may have missing spans. +This can happen because spans are streamed from the APM Agent to the APM Server separately from their transaction. +Unforseen errors may cause spans to go missing. +Because the agent notifies the server about how many spans there should be, +the number of missing spans is able to be calculated and shown in the UI. + +[[errors]] +=== Errors + +Errors are represented by a unique ID. +An error event contains at least +information about the original `exception` that occurred +or about a `log` created when the exception occurred. + +Both the captured `exception` and the captured `log` of an error can contain a `stack trace`, +helpful for debugging. + +The `culprit` of an error indicates where it originated. + +An error might relate to the <> during which it happened, +via the `transaction.id`. + +Errors also have some contextual data. + +include::../context.asciidoc[] + +Errors are stored in {apm-server-ref}/error-indices.html[error indices]. diff --git a/docs/guide/apm-release-notes.asciidoc b/docs/guide/apm-release-notes.asciidoc new file mode 100644 index 00000000000..87d6f1684fb --- /dev/null +++ b/docs/guide/apm-release-notes.asciidoc @@ -0,0 +1,84 @@ +[[kibana]] +[[apm-release-notes]] +== Release notes + +[float] +==== APM version 6.4.1 + +[float] +===== Bug Fixes +Changes introduced in 6.4.0 potentially caused an empty APM Kibana UI. +This happened in case the APM Server was using an outdated configuration file, not configured to index events into separate indices. +To fix this, the APM Kibana UI now falls back to use `apm-*` as default indices to query. +Users can still leverage separate indices for queries by overriding the default values described in {kibana-ref}/apm-settings-kb.html[Kibana APM settings]. + + +[float] +==== APM version 6.4.0 + +[float] +===== Breaking changes + +We previously split APM data into separate indices (transaction, span, error, etc.). +In 6.4 APM Kibana UI starts to leverage those separate indices for queries. + +In case you only update Kibana but run an older version of APM Server you will not be able to see any APM data by default. +To fix this, use the {kibana-ref}/apm-settings-kb.html[Kibana APM settings] to specify the location of the APM index: +["source","sh"] +------------------------------------------------------------ +apm_oss.errorIndices: apm-* +apm_oss.spanIndices: apm-* +apm_oss.transactionIndices: apm-* +apm_oss.onboardingIndices: apm-* +------------------------------------------------------------ + +In case you are upgrading APM Server from an older version, you might need to refresh your APM index pattern for certain APM UI features to work. +Also ensure to add the new config options in `apm-server.yml` in case you keep your existing configuration file: +["source","sh"] +------------------------------------------------------------ +output.elasticsearch: + indices: + - index: "apm-%{[beat.version]}-sourcemap" + when.contains: + processor.event: "sourcemap" + - index: "apm-%{[beat.version]}-error-%{+yyyy.MM.dd}" + when.contains: + processor.event: "error" + - index: "apm-%{[beat.version]}-transaction-%{+yyyy.MM.dd}" + when.contains: + processor.event: "transaction" + - index: "apm-%{[beat.version]}-span-%{+yyyy.MM.dd}" + when.contains: + processor.event: "span" + - index: "apm-%{[beat.version]}-metric-%{+yyyy.MM.dd}" + when.contains: + processor.event: "metric" + - index: "apm-%{[beat.version]}-onboarding-%{+yyyy.MM.dd}" + when.contains: + processor.event: "onboarding" +------------------------------------------------------------ + +[float] +===== New features + +*APM Server* + +* Logstash output +* Kafka output + + +*APM UI* + +* Query bar +* Machine Learning integration: Anomaly detection on service response times +* Kibana objects (index pattern, dashboards, etc.) can now be imported via the Kibana setup instuctions + + +*APM agents* + +* RUM is now GA +* Ruby is now GA +* Java is now Beta +* Go is now Beta +* Python added instrumentation for Cassandra, PyODBC and PyMSSQL +* Node.js added instrumentation for Cassandra and broader MySQL support diff --git a/docs/guide/distributed-tracing.asciidoc b/docs/guide/distributed-tracing.asciidoc new file mode 100644 index 00000000000..4bc69f5ed01 --- /dev/null +++ b/docs/guide/distributed-tracing.asciidoc @@ -0,0 +1,22 @@ +[[distributed-tracing]] +== Distributed tracing + +Elastic APM supports distributed tracing. +Distributed tracing enables you to analyze performance throughout your microservices architecture all in one view. +This is accomplished by tracing all of the requests - from the initial web request to your front-end service - to queries made to your back-end services. +This makes finding possible bottlenecks throughout your application much easier and faster. +Best of all, there's no additional configuration needed for distributed tracing, just ensure you're using the latest version of the applicable {apm-agents-ref}/index.html[agent]. + +The APM UI in Kibana also supports distributed tracing. +The Timeline visualization has been redesigned to show all of the transactions from individual services that are connected in a trace: + +[role="screenshot"] +image::distributed-tracing.jpg[distribute tracing in the UI] + +NOTE: As we're still working on enhancing the distributed tracing experience in the UI, +we're flagging the feature as beta in the UI. + +Should you still encounter any bug or issues, +please reach out in the https://github.com/elastic/apm[APM GitHub repository]. +For questions and feature requests, +visit our https://discuss.elastic.co/c/apm[discussion forum]. \ No newline at end of file diff --git a/docs/guide/distributed-tracing.jpg b/docs/guide/distributed-tracing.jpg new file mode 100644 index 00000000000..99f641043cb Binary files /dev/null and b/docs/guide/distributed-tracing.jpg differ diff --git a/docs/guide/index.asciidoc b/docs/guide/index.asciidoc index bd3274e529e..0c145ab5755 100644 --- a/docs/guide/index.asciidoc +++ b/docs/guide/index.asciidoc @@ -1,380 +1,22 @@ include::../version.asciidoc[] include::{asciidoc-dir}/../../shared/attributes.asciidoc[] -[[gettting-started]] -= Getting Started with APM - -[[overview]] -== Overview - -Elastic APM is an application performance monitoring system built on the Elastic Stack. -It allows you to monitor software services and applications in real time, -collecting detailed performance information on response time for incoming requests, -database queries, -calls to caches, -external HTTP requests, -etc. -This makes it easier to pinpoint and fix performance problems quickly. - -Elastic APM also collects automatically unhandled errors and exceptions. -Errors are grouped based primarily on the stacktrace, -so you can identify new errors as they appear and keep an eye on how many times specific errors happen. - -NOTE: this guide will indiscriminately use the word service for both services and applications. - -[[components]] -[float] -=== APM Components - -Elastic APM consists of four components: - -* {ref}/index.html[Elasticsearch] -* {apm-agents-ref}/index.html[APM agents] -* {apm-server-ref}/index.html[APM Server] -* {kibana-ref}/xpack-apm.html[Kibana APM UI] - -image::apm-architecture.png[Architecture of Elastic APM] - -*APM agents* are open source libraries written in the same language as your service. -You install them into your service as you would install any other library. -They instrument your code and collect performance data and errors at runtime. -This data is buffered for a short period and sent on to APM Server. - -Out of the box, -APM automatically instruments web frameworks, -database drivers, -calls to caching servers, -and HTTP libraries for requests to external services. -Additionally, -agents provide an API to manually instrument anything else you find interesting. - -*APM Server* is an open source application written in Go which typically runs on dedicated servers. -It listens on port 8200 by default and receives data from agents through a JSON HTTP API. -Then it creates documents from that data and stores them in Elasticsearch. - -To visualize the data after it's sent to Elasticsearch, -you can use the the dedicated APM UI bundled in X-Pack, -or the pre-built open source Kibana dashboards that can be loaded directly via the {kibana-ref}/apm-getting-started.html[APM Kibana UI]. - - -[[install-and-run]] -== Install and run Elastic APM - -To get started using Elastic APM, -you need to have: - -* an Elasticsearch cluster and Kibana (version 5.6 or above) -* APM Server -* APM agents installed in your services - - -For information about setting up an Elasticsearch cluster, -see the Elasticsearch {ref}/getting-started.html[Getting Started]. - -The following sections show how to get started quickly with Elastic APM on a local machine. - -[[apm-server]] -[float] -=== Install and run APM Server - -First, https://www.elastic.co/downloads/apm/apm-server[download APM Server] for your operating system and extract the package. - -In a production environment you would put APM Server on its own machines, -similar to how you run Elasticsearch. -You _can_ run it on the same machines as Elasticsearch, -but this is not recommended, -as the processes will be competing for resources. - -To start APM Server, run: - -[source,bash] ----------------------------------- -./apm-server -e ----------------------------------- - -It will try to connect to Elasticsearch on localhost port 9200 and expose an API to agents on port 8200. -You can change the defaults by supplying different addresses on the command line: - -[source,bash] ----------------------------------- -./apm-server -e -E output.elasticsearch.hosts=ElasticsearchAddress:9200 -E apm-server.host=localhost:8200 ----------------------------------- - -Or you can update the `apm-server.yml` configuration file: - -[source,yaml] ----------------------------------- -apm-server: - host: localhost:8200 - -output: - elasticsearch: - hosts: ElasticsearchAddress:9200 ----------------------------------- - -NOTE: If you are using an X-Pack secured version of Elastic Stack, -you need to specify credentials in the config file: - -[source,yaml] ----- -output.elasticsearch: - hosts: ["ElasticsearchAddress:9200"] - username: "elastic" - password: "elastic" ----- - - - -[[secure-api-access]] -[float] -==== Secure access to the API -If you change the listen address from `localhost` to something that is accessible from outside of the machine, -we recommend setting up firewall rules to ensure that only your own systems can access the API. -Alternatively, -you can use a {apm-server-ref}/securing-apm-server.html[secret token and TLS]. - -If you have APM Server running on the same host as your service, you can configure it to listen on a Unix domain socket. - -[[kibana-dashboards]] -[float] -==== Install the Kibana dashboards - -From APM Server and Kibana 6.4 on you can load dashboards directly via the {kibana-ref}/apm-getting-started.html[APM -Kibana UI]. - -See an example screenshot of a Kibana dashboard: - -image::kibana-dashboard.png[Screenshot of a Kibana Dashboard] - -[[more-information]] -[float] -=== More information -For detailed instructions on how to install and secure APM Server in your server environment, -including details on how to run APM Server in a highly available environment, -please see {apm-server-ref}/index.html[APM Server documentation]. - -Once APM Server is up and running, -you need to install an agent in your service. - -[[agents]] -[float] -=== Install and configure APM agents - -Agents are written in the same language as your service. -Currently Elastic APM has agents for Node.js, Python, Ruby, and JavaScript. - -Setting up a new service to be monitored requires installing the agent, -and configuring it with the address of your APM Server and the service name. - -[[choose-service-name]] -[float] -==== Choose a service name - -The service name is used by Elastic APM to differentiate between data coming from different services. - -Elastic APM includes the service name field on every document that it saves in Elasticsearch. -If you change the service name after using Elastic APM, -you will see the old service name and the new service name as two separate services. -Make sure you choose a good service name before you get started. - -The service name can only contain alphanumeric characters, -spaces, -underscores, -and dashes (must match `^[a-zA-Z0-9 _-]+$`). - -[[nodejs-agent]] -[float] -==== Install the Node.js agent - -Install the elastic-apm-node module from npm in your service: - -[source,bash] ----------------------------------- -npm install elastic-apm-node --save ----------------------------------- - -Then configure the elastic-apm-node module by adding the following lines to the top of your code: - -[source,javascript] ----------------------------------- -// Add this to the VERY top of the first file loaded in your app -var apm = require('elastic-apm-node').start({ - // Override service name from package.json - // Allowed characters: a-z, A-Z, 0-9, -, _, and space - serviceName: '', - - // Use if APM Server requires a token - secretToken: '', - - // Set custom APM Server URL (default: http://localhost:8200) - serverUrl: '' -}) ----------------------------------- - -The Node.js agent automatically instruments Express, -hapi, -Koa, -and Restify out of the box. -See the {apm-node-ref}/index.html[APM Node.js Agent documentation] for more details. - -[[python-agent]] -[float] -==== Install the Python agent - -Install the Elastic APM module from pypi: - -[source,bash] ----------------------------------- -pip install elastic-apm ----------------------------------- - -The Python agent automatically instruments Django and Flask out of the box. -See the {apm-py-ref}/index.html[APM Python Agent documentation] for more details. - -[[ruby-agent]] -[float] -==== Install the Ruby agent - -Add the gem to your Gemfile: - -[source,bash] ----------------------------------- -gem 'elastic-apm' ----------------------------------- - -Create a config file config/elastic_apm.yml: - -[source,bash] ----------------------------------- -server_url: http://localhost:8200 -secret_token: '' ----------------------------------- - -The Ruby agent automatically instruments Rails out of the box. -See the {apm-ruby-ref}/index.html[APM Ruby Agent documentation] for more details. - - -[[rum-agent]] -[float] -==== Install the RUM JavaScript agent - -Install the agent as a dependency to your application: - -[source,bash] ----------------------------------- -npm install elastic-apm-js-base --save ----------------------------------- - -Configure the agent: - -[source,bash] ----------------------------------- -import { init as initApm } from 'elastic-apm-js-base' -var apm = initApm({ - - // Set required service name (allowed characters: a-z, A-Z, 0-9, -, _, and space) - serviceName: '', - - // Set custom APM Server URL (default: http://localhost:8200) - serverUrl: 'http://localhost:8200', - - // Set service version (required for sourcemap feature) - serviceVersion: '' -}) ----------------------------------- - -See the {apm-rum-ref}/index.html[APM RUM JavaScript Agent documentation] for more details. - -[[distributed-tracing]] -== Distributed tracing - -Elastic APM supports distributed tracing to enable you to analyze performance through your microservices architecture by tracing all the requests from the initial web request to your front-end service, to queries made to your back-end services all in one view. This makes finding possible bottlenecks through-out your application much easier and faster. There's no additional configuration needed for distributed tracing, but please ensure you are using the latest version of the agent. - -The APM UI in Kibana supports the displaying of traces and an updated Timeline visualization has been implemented to show all the transactions from individual services that are connected in a trace. As we're still working on enhancing the distributed tracing experience in the UI, we're flagging the feature beta in the UI. Should you still encounter any bug or issues, please reach out in our https://discuss.elastic.co/c/apm[discussion forum]. - - -[[kibana]] -[[apm-release-notes]] -== Release notes - -[float] -==== APM version 6.4.1 - -[float] -===== Bug Fixes -Changes introduced in 6.4.0 potentially caused an empty APM Kibana UI. -This happened in case the APM Server was using an outdated configuration file, not configured to index events into separate indices. -To fix this, the APM Kibana UI now falls back to use `apm-*` as default indices to query. -Users can still leverage separate indices for queries by overriding the default values described in {kibana-ref}/apm-settings-kb.html[Kibana APM settings]. - - -[float] -==== APM version 6.4.0 - -[float] -===== Breaking changes - -We previously split APM data into separate indices (transaction, span, error, etc.). -In 6.4 APM Kibana UI starts to leverage those separate indices for queries. - -In case you only update Kibana but run an older version of APM Server you will not be able to see any APM data by default. -To fix this, use the {kibana-ref}/apm-settings-kb.html[Kibana APM settings] to specify the location of the APM index: -["source","sh"] ------------------------------------------------------------- -apm_oss.errorIndices: apm-* -apm_oss.spanIndices: apm-* -apm_oss.transactionIndices: apm-* -apm_oss.onboardingIndices: apm-* ------------------------------------------------------------- - -In case you are upgrading APM Server from an older version, you might need to refresh your APM index pattern for certain APM UI features to work. -Also ensure to add the new config options in `apm-server.yml` in case you keep your existing configuration file: -["source","sh"] ------------------------------------------------------------- -output.elasticsearch: - indices: - - index: "apm-%{[beat.version]}-sourcemap" - when.contains: - processor.event: "sourcemap" - - index: "apm-%{[beat.version]}-error-%{+yyyy.MM.dd}" - when.contains: - processor.event: "error" - - index: "apm-%{[beat.version]}-transaction-%{+yyyy.MM.dd}" - when.contains: - processor.event: "transaction" - - index: "apm-%{[beat.version]}-span-%{+yyyy.MM.dd}" - when.contains: - processor.event: "span" - - index: "apm-%{[beat.version]}-metric-%{+yyyy.MM.dd}" - when.contains: - processor.event: "metric" - - index: "apm-%{[beat.version]}-onboarding-%{+yyyy.MM.dd}" - when.contains: - processor.event: "onboarding" ------------------------------------------------------------- - -[float] -===== New features - -*APM Server* - -* Logstash output -* Kafka output +ifdef::env-github[] +NOTE: For the best reading experience, +please view this documentation at https://www.elastic.co/guide/en/apm/get-started[elastic.co] +endif::[] +[[gettting-started]] += APM Overview -*APM UI* +include::./overview.asciidoc[] -* Query bar -* Machine Learning integration: Anomaly detection on service response times -* Kibana objects (index pattern, dashboards, etc.) can now be imported via the Kibana setup instuctions +include::./apm-data-model.asciidoc[] +include::./distributed-tracing.asciidoc[] -*APM agents* +include::./install-and-run.asciidoc[] -* RUM is now GA -* Ruby is now GA -* Java is now Beta -* Go is now Beta -* Python added instrumentation for Cassandra, PyODBC and PyMSSQL -* Node.js added instrumentation for Cassandra and broader MySQL support +include::./agent-server-compatibility.asciidoc[] +include::./apm-release-notes.asciidoc[] diff --git a/docs/guide/install-and-run.asciidoc b/docs/guide/install-and-run.asciidoc new file mode 100644 index 00000000000..707bae50e2b --- /dev/null +++ b/docs/guide/install-and-run.asciidoc @@ -0,0 +1,169 @@ +[[install-and-run]] +== Install and run Elastic APM + +To get started using Elastic APM, +you need to have: + +* an Elasticsearch cluster and Kibana (version 5.6 or above) +* APM Server +* APM agents installed in your services + +TIP: For best results, ensure you're using the same version of Elasticsearch, Kibana, and APM Server. + +For information about setting up an Elasticsearch cluster, +see the Elasticsearch {ref}/getting-started.html[Getting Started]. + +The following sections show how to get started quickly with Elastic APM on a local machine. + +[[apm-server]] +[float] +=== Install and run APM Server + +First, https://www.elastic.co/downloads/apm/apm-server[download APM Server] for your operating system and extract the package. + +In a production environment you would put APM Server on its own machines, +similar to how you run Elasticsearch. +You _can_ run it on the same machines as Elasticsearch, +but this is not recommended, +as the processes will be competing for resources. + +To start APM Server, run: + +[source,bash] +---------------------------------- +./apm-server -e +---------------------------------- + +It will try to connect to Elasticsearch on localhost port 9200 and expose an API to agents on port 8200. +You can change the defaults by supplying different addresses on the command line: + +[source,bash] +---------------------------------- +./apm-server -e -E output.elasticsearch.hosts=ElasticsearchAddress:9200 -E apm-server.host=localhost:8200 +---------------------------------- + +Or you can update the `apm-server.yml` configuration file: + +[source,yaml] +---------------------------------- +apm-server: + host: localhost:8200 + +output: + elasticsearch: + hosts: ElasticsearchAddress:9200 +---------------------------------- + +NOTE: If you are using an X-Pack secured version of Elastic Stack, +you need to specify credentials in the config file: + +[source,yaml] +---- +output.elasticsearch: + hosts: ["ElasticsearchAddress:9200"] + username: "elastic" + password: "elastic" +---- + + + +[[secure-api-access]] +[float] +==== Secure access to the API +If you change the listen address from `localhost` to something that is accessible from outside of the machine, +we recommend setting up firewall rules to ensure that only your own systems can access the API. +Alternatively, +you can use a {apm-server-ref}/securing-apm-server.html[secret token and TLS]. + +If you have APM Server running on the same host as your service, you can configure it to listen on a Unix domain socket. + +[[kibana-dashboards]] +[float] +==== Install the Kibana dashboards + +From APM Server and Kibana 6.4 on you can load dashboards directly via the {kibana-ref}/apm-getting-started.html[APM +Kibana UI]. + +See an example screenshot of a Kibana dashboard: + +image::kibana-dashboard.png[Screenshot of a Kibana Dashboard] + +[[more-information]] +[float] +=== More information +For detailed instructions on how to install and secure APM Server in your server environment, +including details on how to run APM Server in a highly available environment, +please see {apm-server-ref}/index.html[APM Server documentation]. + +Once APM Server is up and running, +you need to install an agent in your service. + +[[agents]] +[float] +=== Install and configure APM agents + +Agents are written in the same language as your service. +Currently Elastic APM has agents for Node.js, Python, Ruby, Java, Go, and JavaScript RUM. + +Setting up a new service to be monitored requires installing the agent, +and configuring it with the address of your APM Server and the service name. + +[[choose-service-name]] +[float] +==== Choose a service name + +The service name is used by Elastic APM to differentiate between data coming from different services. + +Elastic APM includes the service name field on every document that it saves in Elasticsearch. +If you change the service name after using Elastic APM, +you will see the old service name and the new service name as two separate services. +Make sure you choose a good service name before you get started. + +The service name can only contain alphanumeric characters, +spaces, underscores, and dashes (must match `^[a-zA-Z0-9 _-]+$`). + +[[nodejs-agent]] +[float] +==== Install the Node.js agent + +The Node.js agent automatically instruments Express, +hapi, +Koa, +and Restify out of the box. +See the {apm-node-ref}/index.html[APM Node.js Agent documentation] for more information. + +[[python-agent]] +[float] +==== Install the Python agent + +The Python agent automatically instruments Django and Flask out of the box. +See the {apm-py-ref}/index.html[APM Python Agent documentation] for more information. + +[[ruby-agent]] +[float] +==== Install the Ruby agent + +The Ruby agent automatically instruments Rails out of the box. +See the {apm-ruby-ref}/index.html[APM Ruby Agent documentation] for more information. + +[[java-agent]] +[float] +==== Install the Java Agent + +The Java agent automatically instruments Servlet API, Spring MVC, and Spring Boot out of the box. +See the {apm-java-ref}/index.html[APM Java Agent documentation] for more information. + +[[go-agent]] +[float] +==== Install the Go Agent + +The Go agent automatically instruments Gorilla and Gin, +as well as support for Go's built-in net/http and database/sql drivers. +See the {apm-go-ref}/index.html[APM Go Agent documentation] for more information. + +[[rum-agent]] +[float] +==== Install the RUM JavaScript agent + +Real User Monitoring (RUM) captures user interactions with clients such as web browsers. +See the {apm-rum-ref}/index.html[APM RUM JavaScript Agent documentation] for more information. diff --git a/docs/guide/overview.asciidoc b/docs/guide/overview.asciidoc new file mode 100644 index 00000000000..87c4903765f --- /dev/null +++ b/docs/guide/overview.asciidoc @@ -0,0 +1,43 @@ +[[overview]] +== Overview + +Elastic APM is an application performance monitoring system built on the Elastic Stack. +It allows you to monitor software services and applications in real time, +collecting detailed performance information on response time for incoming requests, +database queries, +calls to caches, +external HTTP requests, +etc. +This makes it easier to pinpoint and fix performance problems quickly. + +Elastic APM also collects automatically unhandled errors and exceptions. +Errors are grouped based primarily on the stacktrace, +so you can identify new errors as they appear and keep an eye on how many times specific errors happen. + +NOTE: This guide will indiscriminately use the word service for both services and applications. + +[[components]] +[float] +=== APM Components + +Elastic APM consists of four components: + +* {ref}/index.html[Elasticsearch] +* {apm-agents-ref}/index.html[APM agents] +* {apm-server-ref}/index.html[APM Server] +* {kibana-ref}/xpack-apm.html[Kibana APM UI] + +image::apm-architecture.png[Architecture of Elastic APM] + +*APM agents* are open source libraries written in the same language as your service. +You install them into your service as you would install any other library. +They instrument your code and collect performance data and errors at runtime. +This data is buffered for a short period and sent on to APM Server. + +*APM Server* is an open source application written in Go which typically runs on dedicated servers. +It listens on port 8200 by default and receives data from agents through a JSON HTTP API. +Then it creates documents from that data and stores them in Elasticsearch. + +To visualize the data after it's sent to Elasticsearch, +you can use the the dedicated APM UI bundled in X-Pack, +or the pre-built open source Kibana dashboards that can be loaded directly via the {kibana-ref}/apm-getting-started.html[APM Kibana UI]. \ No newline at end of file diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 2f8ac88c7e5..50ce8792602 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -46,8 +46,6 @@ include::./copied-from-beats/shared-securing-beat.asciidoc[] include::./copied-from-beats/monitoring/monitoring-beats.asciidoc[] -include::./event-types.asciidoc[] - include::./rum.asciidoc[] include::./data-ingestion.asciidoc[] @@ -63,3 +61,5 @@ include::./fields.asciidoc[] include::./troubleshooting.asciidoc[] include::./release-notes.asciidoc[] + +include::./redirects.asciidoc[] diff --git a/docs/intake-api.asciidoc b/docs/intake-api.asciidoc index f2783a51b67..a4ac879514a 100644 --- a/docs/intake-api.asciidoc +++ b/docs/intake-api.asciidoc @@ -2,17 +2,37 @@ = Intake API [partintro] -- -The Intake API is HTTP JSON based. The APM Server exposes endpoints for +NOTE: Most users will not need to interact directly with the intake API unless they are implementing an agent. -* <> -* <> +The Intake API is what we call the internal protocol that APM agents use to talk to the APM Server. +This API has been redesigned in 6.5. +Read the <> for more information. + +APM Agents communicate with the APM server by sending events in an HTTP request. +Each event is sent as its own line in the HTTP request body. +This is known as http://ndjson.org[newline delimited JSON (NDJSON)]. +The request body looks roughly like this: + +[source,bash] +------------------------------------------------------------ +{"metadata": {"service": {"name": "ecommerce-front"}}} +{"span": {"name": "SELECT FROM products", "duration": 323, "transaction_id": "A"}} +{"span": {"name": "SELECT FROM users", "duration": 202, "transaction_id": "A"}} +{"transaction": {"name": "GET /index", "id": "A"}} +------------------------------------------------------------ + +With NDJSON, agents can open an HTTP POST request and use chunked encoding to stream events to the APM Server as soon as they are recorded in the agent. +This makes it simple for agents to serialize each event to a stream of newline delimited JSON. + +The APM Server also treats the HTTP body as a compressed stream and thus reads and handles each event independently. + +The APM Server exposes endpoints for + +* <> * <> * <> -Unless you are implementing an agent, you don't need to know about the specifics of this API. - -- -include::./transaction-api.asciidoc[] -include::./error-api.asciidoc[] +include::./events-api.asciidoc[] include::./sourcemap-api.asciidoc[] -include::./server-info.asciidoc[] +include::./server-info.asciidoc[] \ No newline at end of file diff --git a/docs/metadata-api.asciidoc b/docs/metadata-api.asciidoc new file mode 100644 index 00000000000..54ae02103ba --- /dev/null +++ b/docs/metadata-api.asciidoc @@ -0,0 +1,63 @@ +[[metadata-api]] +=== Metadata + +Every new connection to the APM Server starts with a `metadata` stanza. +This provides general metadata concerning the other objects in the stream. + +Rather than send this metadata information from the agent multiple times, +the APM Server hangs on to this information and applies it to other objects in the stream as necessary. + +TIP: Metadata is stored under `context` when viewing documents in Elasticsearch. + +* <> +* <> +* <> +* <> +* <> + +[[metadata-schema]] +[float] +==== Metadata Schema + +The APM Server uses JSON Schema for validating requests. The specification for metadata is defined below: + +[source,json] +---- +include::./spec/metadata.json[] +---- + +[[metadata-service-schema]] +[float] +===== Service Schema + +[source,json] +---- +include::./spec/service.json[] +---- + +[[metadata-process-schema]] +[float] +===== Process Schema + +[source,json] +---- +include::./spec/process.json[] +---- + +[[metadata-system-schema]] +[float] +===== System Schema + +[source,json] +---- +include::./spec/system.json[] +---- + +[[metadata-user-schema]] +[float] +===== User Schema + +[source,json] +---- +include::./spec/user.json[] +---- \ No newline at end of file diff --git a/docs/metricset-api.asciidoc b/docs/metricset-api.asciidoc new file mode 100644 index 00000000000..84edd1fae5b --- /dev/null +++ b/docs/metricset-api.asciidoc @@ -0,0 +1,20 @@ +[[metricset-api]] +=== Metricsets + +Metricsets contain application metric data captured by an APM agent. + +[[metricset-schema]] +[float] +==== Metricset Schema + +The APM Server uses JSON Schema for validating requests. The specification for metricsets is defined below: + +[source,json] +---- +include::./spec/metricsets/common_metricset.json[] +---- + +[source,json] +---- +include::./spec/metricsets/v2_metricset.json[] +---- \ No newline at end of file diff --git a/docs/metricset-indices.asciidoc b/docs/metricset-indices.asciidoc new file mode 100644 index 00000000000..51d06a55bc6 --- /dev/null +++ b/docs/metricset-indices.asciidoc @@ -0,0 +1,15 @@ +[[metricset-indices]] +== Metricset Indices + +Metricsets are by default stored to indices of the format `apm-[version]-metric-[date]`. + +[[metricset-example]] +[float] +=== Example Documents + +See how metricset documents can look like when indexed in Elasticsearch: + +[source,json] +---- +include::./data/elasticsearch/generated/metricsets.json[] +---- diff --git a/docs/overview.asciidoc b/docs/overview.asciidoc index 056de8606cc..097639f810b 100644 --- a/docs/overview.asciidoc +++ b/docs/overview.asciidoc @@ -14,6 +14,8 @@ To get an overview of the whole Elastic APM system, * {apm-node-ref}/index.html[APM Node.js Agent] * {apm-py-ref}/index.html[APM Python Agent] * {apm-ruby-ref}/index.html[APM Ruby Agent] +* {apm-go-ref}/index.html[APM Go Agent] +* {apm-java-ref}/index.html[APM Java Agent] * {apm-rum-ref}/index.html[APM RUM JavaScript Agent] * {ref}/index.html[Elasticsearch] @@ -26,8 +28,7 @@ The APM Server is a separate component for the following reasons: * It helps to keep the agents as light as possible and since the APM Server is a stateless separate component, it can be scaled independently. * For Real User monitoring data is collected in browsers. - APM Server prevents browsers from interacting directly with Elasticsearch (which poses a security risk), - and controls the amount of data flowing into Elasticsearch. + APM Server prevents browsers from interacting directly with Elasticsearch (which poses a security risk) and controls the amount of data flowing into Elasticsearch. * In cases where Elasticsearch becomes unresponsive, APM Server can buffer data temporarily without adding overhead to the agents. * Acts as a middleware for source mapping for javascript in the browser. diff --git a/docs/redirects.asciidoc b/docs/redirects.asciidoc new file mode 100644 index 00000000000..b7bf5aa3be9 --- /dev/null +++ b/docs/redirects.asciidoc @@ -0,0 +1,155 @@ +["appendix",role="exclude",id="redirects"] += Deleted pages + +The following pages have moved or been deleted. + +// Event Types + +[role="exclude",id="event-types"] +=== Event types + +This page has moved. Please see {apm-get-started-ref}/apm-data-model.html[APM data model]. + +[role="exclude",id="errors"] +=== Errors + +This page has moved. Please see {apm-get-started-ref}/errors.html[Errors]. + +[role="exclude",id="transactions"] +=== Transactions + +This page has moved. Please see {apm-get-started-ref}/transactions.html[Transactions]. + +[role="exclude",id="transactions-spans"] +=== Spans + +This page has moved. Please see {apm-get-started-ref}/transaction-spans.html[Spans]. + +// Error API + +[role="exclude",id="error-endpoint"] +=== Error endpoint + +The error endpoint has been deprecated. Instead, see <>. + +[role="exclude",id="error-schema-definition"] +=== Error schema definition + +The error schema has moved. Please see <>. + +[role="exclude",id="error-api-examples"] +=== Error API examples + +The error API examples have moved. Please see <>. + +[role="exclude",id="error-payload-schema"] +=== Error payload schema + +This schema has changed. Please see <>. + +[role="exclude",id="error-service-schema"] +=== Error service schema + +This schema has changed. Please see <>. + +[role="exclude",id="error-system-schema"] +=== Error system schema + +This schema has changed. Please see <>. + +[role="exclude",id="error-context-schema"] +=== Error context schema + +This schema has changed. Please see <>. + +[role="exclude",id="error-stacktraceframe-schema"] +=== Error stacktrace frame schema + +This schema has changed. Please see <>. + +[role="exclude",id="payload-with-error"] +=== Payload with error + +This is no longer helpful. Please see <>. + +[role="exclude",id="payload-with-minimal-exception"] +=== Payload with minimal exception + +This is no longer helpful. Please see <>. + +[role="exclude",id="payload-with-minimal-log"] +=== Payload with minimal log + +This is no longer helpful. Please see <>. + +// Transaction API + +[role="exclude",id="transaction-endpoint"] +=== Transaction endpoint + +The transaction endpoint has been deprecated. Instead, see <>. + +[role="exclude",id="transaction-schema-definition"] +=== Transaction schema definition + +The transaction schema has moved. Please see <>. + +[role="exclude",id="transaction-api-examples"] +=== Transaction API examples + +The transaction API examples have moved. Please see <>. + +[role="exclude",id="transaction-span-schema"] +=== Transaction span schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-payload-schema"] +=== Transaction payload schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-service-schema"] +=== Transaction service schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-system-schema"] +=== Transaction system schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-context-schema"] +=== Transaction context schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-stacktraceframe-schema"] +=== Transaction stacktrace frame schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-request-schema"] +=== Transaction request schema + +This schema has changed. Please see <>. + +[role="exclude",id="transaction-user-schema"] +=== Transaction user schema + +This schema has changed. Please see <>. + +[role="exclude",id="payload-with-transactions"] +=== Payload with transactions + +This is no longer helpful. Please see <>. + +[role="exclude",id="payload-with-minimal-transaction"] +=== Payload with minimal transaction + +This is no longer helpful. Please see <>. + +[role="exclude",id="payload-with-minimal-span"] +=== Payload with minimal span + +This is no longer helpful. Please see <>. \ No newline at end of file diff --git a/docs/rum.asciidoc b/docs/rum.asciidoc index da2a81e259f..8d183e97ecf 100644 --- a/docs/rum.asciidoc +++ b/docs/rum.asciidoc @@ -9,9 +9,9 @@ To use it you need to <> in the APM Server. [[sourcemaps]] == Source Maps -It is common practice to minify JavaScript code, for example to reduce network latency. -While improving performance, minified code is also hard to debug. -A source map library helps by mapping the minified files back the the original source code. +For a number of reasons, it is common practice to minify JavaScript code, for example to reduce network latency. +While improving performance, minified code can be hard to debug. +A source map library helps by mapping the minified files back to the the original source code. APM Server provides a <> for uploading source maps. diff --git a/docs/security.asciidoc b/docs/security.asciidoc index b839e3f89ee..15bf075d3e3 100644 --- a/docs/security.asciidoc +++ b/docs/security.asciidoc @@ -22,11 +22,9 @@ Secret tokens are not applicable for RUM, as they would be publicly exposed. ==== SSL/TLS setup To enable SSL/TLS you need a private key and a certificate issued by a certification authority (CA). -Then you can specify the path to those files in the configuration properties -`apm-server.ssl.key` and -`apm-server.ssl.certificate` -respectively. -This will make the APM Server to serve HTTPS requests instead of HTTP. -Hence, you also need to enable SSL in the agent. -For agent specific details, -please check the {apm-agents-ref}/index.html[agent documentation] for how to do it. +You can then specify the path to those files in your configuration properties: +`apm-server.ssl.key` and `apm-server.ssl.certificate` respectively. +This will make the APM Server serve HTTPS requests instead of HTTP. +Don't forget, you also need to enable SSL in the agent. +For agent specific details on enabling SSL/TLS, +please see the {apm-agents-ref}/index.html[agent documentation]. \ No newline at end of file diff --git a/docs/server-info.asciidoc b/docs/server-info.asciidoc index 11de3259dc3..a20751ca9af 100644 --- a/docs/server-info.asciidoc +++ b/docs/server-info.asciidoc @@ -1,8 +1,7 @@ [[server-info]] == Server Information -The APM Server exposes an API endpoint to query some server information. - +The APM Server exposes an API endpoint to query general server information. This lightweight endpoint is useful as a server up/down healthcheck. [[server-info-endpoint]] @@ -17,7 +16,7 @@ http(s)://{hostname}:{port}/ This endpoint always returns an HTTP 200. -If a <> is set, only requests including <> will include server details. +If a <> is set, only requests including <> will receive server details. Set the `Accept` header set to `text/plain` to move the server information to the root level of the response, removing `ok`. diff --git a/docs/span-api.asciidoc b/docs/span-api.asciidoc new file mode 100644 index 00000000000..31c3236681a --- /dev/null +++ b/docs/span-api.asciidoc @@ -0,0 +1,20 @@ +[[span-api]] +=== Spans + +Spans are events captured by an agent occurring in a monitored service. + +[[span-schema]] +[float] +==== Span Schema + +The APM Server uses JSON Schema for validating requests. The specification for spans is defined below: + +[source,json] +---- +include::./spec/spans/v2_span.json[] +---- + +[source,json] +---- +include::./spec/spans/common_span.json[] +---- \ No newline at end of file diff --git a/docs/span-indices.asciidoc b/docs/span-indices.asciidoc index 3d13729dd41..76320294669 100644 --- a/docs/span-indices.asciidoc +++ b/docs/span-indices.asciidoc @@ -6,10 +6,12 @@ Conceptually, spans are part of transactions. [[span-example]] [float] -=== Example document +=== Example documents + +See how span documents can look like when indexed in Elasticsearch: [source,json] ---- -include::./data/elasticsearch/span.json[] +include::./data/elasticsearch/generated/spans.json[] ---- diff --git a/docs/storage-management.asciidoc b/docs/storage-management.asciidoc index 96f2adf30db..91aa81863c4 100644 --- a/docs/storage-management.asciidoc +++ b/docs/storage-management.asciidoc @@ -3,11 +3,11 @@ [partintro] -- -In the following section you can learn how to +In the following section you can learn how to: -* <> -* <> -* <> +* <> +* <> +* <> -- [[manage-indices-kibana]] @@ -15,7 +15,7 @@ In the following section you can learn how to The Kibana UI for managing indices allows you to view indices, index settings, mappings, docs count, used storage per index and much more. You can perform management operations, -for example deleting indices directly via the Kibana UI. +for example, deleting indices directly via the Kibana UI. The UI also supports applying bulk operations on several indices at once. Check out the {kibana-ref}/managing-indices.html[Kibana Managing Indices] documentation for more details. @@ -140,7 +140,7 @@ Follow the {kibana-ref}/managing-indices.html[Kibana Index Management] documenta You might want to update documents already indexed, for example if you your service name was set incorrectly. -For this you can use the {ref}/docs-update-by-query.html[Update By Query API]. +For this, you can use the {ref}/docs-update-by-query.html[Update By Query API]. [[update-data-rename-a-service]] [float] diff --git a/docs/transaction-api.asciidoc b/docs/transaction-api.asciidoc index 9718f5c6c6b..769ed41e71c 100644 --- a/docs/transaction-api.asciidoc +++ b/docs/transaction-api.asciidoc @@ -1,158 +1,20 @@ [[transaction-api]] -== Transaction API +=== Transactions -The following section contains information about: +Transactions are events corresponding to an incoming request or similar task occurring in a monitored service. -* <> -* <> -* <> - -[[transaction-endpoint]] -[float] -=== Endpoint -Send a `HTTP POST` request to the APM Server `transactions` endpoint: - -[source,bash] ------------------------------------------------------------- -http(s)://{hostname}:{port}/v1/transactions ------------------------------------------------------------- - -For <> send a `HTTP POST` request to the APM Server `rum transactions` endpoint instead: - -[source,bash] ------------------------------------------------------------- -http(s)://{hostname}:{port}/v1/rum/transactions ------------------------------------------------------------- - -[[transaction-schema-definition]] -[float] -=== Schema Definition - -The APM Server uses JSON Schema for validating requests. The specification for transactions is defined bellow: - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -[[transaction-payload-schema]] -[float] -==== Payload - -[source,json] ----- -include::./spec/transactions/v1_transaction.json[] ----- - -[[transaction-span-schema]] -[float] -==== Span - -[source,json] ----- -include::./spec/spans/v1_span.json[] ----- - -[[transaction-service-schema]] -[float] -==== Service - -[source,json] ----- -include::./spec/service.json[] ----- - -[[transaction-system-schema]] -[float] -==== System - -[source,json] ----- -include::./spec/system.json[] ----- - -[[transaction-context-schema]] -[float] -==== Context - -[source,json] ----- -include::./spec/context.json[] ----- - -[[transaction-stacktraceframe-schema]] -[float] -==== Stacktrace Frame - -[source,json] ----- -include::./spec/stacktrace_frame.json[] ----- - -[[transaction-request-schema]] -[float] -==== Request - -[source,json] ----- -include::./spec/request.json[] ----- - -[[transaction-user-schema]] -[float] -==== User - -[source,json] ----- -include::./spec/user.json[] ----- - -[[transaction-api-examples]] -[float] -=== Examples - -Request example: - -["source","sh",subs="attributes"] ------------------------------------------------------------- -curl http://localhost:8200/v1/transactions \ - --header "Content-Type: application/json" \ - --data @docs/data/intake-api/generated/transaction/payload.json ------------------------------------------------------------- - -Example transaction requests: - -* <> -* <> -* <> - -[[payload-with-transactions]] +[[transaction-schema]] [float] -==== Payload with several Transactions +==== Transaction Schema -[source,json] ----- -include::./data/intake-api/generated/transaction/payload.json[] ----- - -[[payload-with-minimal-transaction]] -[float] -==== Payload with a minimal Transaction +The APM Server uses JSON Schema for validating requests. The specification for transactions is defined below: [source,json] ---- -include::./data/intake-api/generated/transaction/minimal_payload.json[] +include::./spec/transactions/common_transaction.json[] ---- -[[payload-with-minimal-span]] -[float] -==== Payload with a Transaction with a minimal Span - [source,json] ---- -include::./data/intake-api/generated/transaction/minimal_span.json[] ----- +include::./spec/transactions/v2_transaction.json[] +---- \ No newline at end of file diff --git a/docs/transaction-indices.asciidoc b/docs/transaction-indices.asciidoc index e63910fab1e..29e77a0bf81 100644 --- a/docs/transaction-indices.asciidoc +++ b/docs/transaction-indices.asciidoc @@ -5,10 +5,12 @@ Transactions are by default stored to indices of the format `apm-[version]-trans [[transaction-example]] [float] -=== Example Document -See an example transaction indexed in Elasticsearch: +=== Example Documents + +See how transaction documents can look like when indexed in Elasticsearch: + [source,json] ---- -include::./data/elasticsearch/transaction.json[] +include::./data/elasticsearch/generated/transactions.json[] ---- diff --git a/docs/transactions.asciidoc b/docs/transactions.asciidoc deleted file mode 100644 index b73d74d7e3c..00000000000 --- a/docs/transactions.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -[[transactions]] -== Transactions - -A transaction describes an event captured by an Elastic APM agent instrumeting a service. -A transaction can for example be an HTTP request or an asynchrounous background job. - -It contains the timestamp and duration of the event, -a unique id, type, name, -a result (for example a response code), some contextual data, and other relevant information depending on the agent. -For instance, the JavaScript RUM captures transaction marks, -which are points in time relative to the start of the transaction with some label. - -[[transactions-context]] -include::./context.asciidoc[] - -Agents decide whether to sample transactions or not, and provide settings to control sampling behaviour. -If sampled, -the <> of a transaction are sent and stored seperate documents. - -Transactions are stored in <>. - -[[transaction-spans]] -[float] -=== Spans -Transactions can have 0, 1, or many spans. Spans have a `transaction.id` attribute that refer to their transaction. - -A span contains information about a specific code path, -executed as part of a transaction. Such information includes start time, duration, name, type, and optionally a `stack trace`. - -If for example a database query happens within a sampled transaction, -a span describing the database query will be created. -In such a case the name of the span will contain information about the query itself, -and the type about the database. - -Typically, agents instrument automatically a variety of libraries, -but also provide an API for ad hoc instrumentation of specific code paths. - -Transactions are stored in <> and spans are stored in separated indices by default. - -Spans are stored in <>. diff --git a/docs/upgrading-to-65.asciidoc b/docs/upgrading-to-65.asciidoc new file mode 100644 index 00000000000..4a640f7c68b --- /dev/null +++ b/docs/upgrading-to-65.asciidoc @@ -0,0 +1,195 @@ +[[upgrading-to-65]] +=== Upgrading to APM Server v6.5 + +If you're an advanced user, are implementing a custom agent, +or provide custom data to Elasticsearch, +there are some changes in APM Server version 6.5 that you should be aware of. + +None of the changes in APM Server v6.5 are breaking changes, +and the APM Server is still compatible with your currently implemented agents. +However, to take advantage of new features, +it's recommended you read about the changes below and update accordingly. +If you'd like to continue using the deprecated Intake API and Elasticsearch Schema, +you can view https://www.elastic.co/guide/en/apm/server/6.4/overview.html[previous documentation]. + +[[upgrade-steps-65]] +==== Upgrade Steps + +. Upgrade the Elastic Stack: {ref}/setup-upgrade.html[Elasticsearch], +{kibana-ref}/upgrade.html[Kibana], +and APM Server. +. Upgrade your {apm-agents-ref}/index.html[APM Agent]. + +Check the {apm-get-started-ref}/agent-server-compatibility.html[agent/server compatibility matrix] for more details on which APM agents are compatible with APM Server version 6.5. + +IMPORTANT: You must upgrade the APM Server before upgrading your APM Agent. +New agents do not support APM Server <6.5 and will not work if you upgrade in the wrong order. + +[float] +[[intake-api-changes-65]] +==== Intake API Changes + +The Intake API is what we call the internal protocol that APM agents use to talk to the APM Server. +This API has been redesigned in 6.5 to increase the memory efficiency and predictability of the APM Server and APM agents. + +Notably, most of the endpoints associated with this API have been deprecated, +and replaced with a new endpoint: `intake/v2/events`. + +Similarly, +the <> endpoint has been deprecated and replaced with a new endpoint: `intake/v2/rum/events`. + +Details about these changes, +and the information you can send to the new endpoint, +is available in the <> documentation. + +[float] +[[metadata-api-changes-65]] +===== Metadata + +Metadata is now sent to the APM Server only once per request. +APM Server will retain this information and apply it to the applicable documents when it writes to Elasticsearch. + +Metadata should be sent to the new `intake/v2/events` endpoint. +See the <> for more information. + +[float] +[[error-api-changes-65]] +===== Error + +The error schema adds three new properties, and deprecates two others. + +* _deprecate_ pattern of `error.id`, change format to hex encoded 64 random bits +* _deprecate_ `transaction.id` +* _add_ `transaction_id` (required, hex encoded 64 random bits) +* _add_ `parent_id` (optional, hex encoded 64 random bits) +* _add_ `trace_id` (optional, hex encoded 128 random bits) + +Error events should be sent to the new `intake/v2/events` endpoint. +See the <> for more information. + +[float] +[[transaction-api-changes-65]] +===== Transaction + +The transaction schema adds two new properties, and deprecates two others. + +* _deprecate_ pattern of `transaction.id`, change format to hex encoded 64 random bits +* _deprecate_ spans (don't forbid them, just don't process them) +* _add_ `parent_id` (optional, hex encoded 64 random bits string) +* _add_ `trace_id` (required, hex encoded 128 random bits) + +Transaction events should be sent to the new `intake/v2/events` endpoint. +See the <> for more information. + +[float] +[[span-api-changes-65]] +===== Span + +The span schema adds three new properties, removes two, and changes one. + +* _change_ pattern of `id` to hex encoded 64 random bits +* _deprecate_ pattern of `transaction_id` +* _deprecate_ use of `parent` +* _add_ `parent_id` (required, hex encoded 64 random bits) +* _add_ `trace_id` (required, hex encoded 128 random bits) +* _add_ `timestamp` + +Span events should be sent to the new `intake/v2/events` endpoint. +See the <> for more information. + +[float] +[[healthcheck-api-changes-65]] +===== Healthcheck Endpoint + +The Healthcheck endpoint, previously `/healthcheck`, has been deprecated. +The <> api has replaced it. + +[float] +[[assets-api-changes-65]] +===== Assets Endpoint + +The assets endpoint, `/v1/rum/sourcemaps` has been deprecated, +and replaced with the endpoint: `/assets/v1/sourcemaps`. +The data format remains unchanged. + +[float] +[[metrics-api-changes-65]] +===== Metricsets Endpoint + +The Metricsets endpoint, previously `/v1/metrics`, +has also integrated with the new intake endpoint at `/intake/v2/events`. +In terms of functionality, the only change relates to timestamps. +See the <> for more information. + +[float] +[[server-config-changes-65]] +==== Server Configuration Changes + +There are some configuration changes that are worth mentioning. + +Deprecated configuration options: + +* <> +* <> +* <> + +New configuration options: + +* <> with a default value: `307200` bytes. You may notice this default value is much smaller than the deprecated <> default value. This is because `max_unzipped_size` referred to the whole request, while `max_event_size` only refers to a single event within the request. + +Notable options: + +* <> and <> remain at 30 seconds, +but if you have a load balancer in front of the APM server you may notice connections are a little bit longer lived than they used to be. + +TIP: If you are updating APM Server from version 6.3 or earlier, +you may not be using an updated `apm-server.yml` configuration file. Update your `apm-server.yml` configuration file to take advantage of new configuration options. + + +[float] +[[es-schema-changes-65]] +==== Elasticsearch Schema Changes + +The Elasticsearch schema defines how APM data is stored in Elasticsearch. +There have been a number of changes to the Elasticsearch schema for 6.5. + +An important change to note is the addition of the `trace` and `parent` keys, +which have been added to errors, transactions, and spans. +Both only hold a field `id`. +These new keys are essential to taking advantage of APM's new {apm-get-started-ref}/distributed-tracing.html[distributed tracing] feature. + +[float] +[[es-error-changes-65]] +===== Error + +The Elasticsearch error schema adds two new keys: + +* _add_ `trace.id` +* _add_ `parent.id` + +View the sample Elasticsearch <> for more information. + +[float] +[[es-transaction-changes-65]] +===== Transaction + +The Elasticsearch transaction schema adds two new keys: + +* _add_ `trace.id` +* _add_ `parent.id` + +View the sample Elasticsearch <> for more information. + +[float] +[[es-span-changes-65]] +===== Span + +The Elasticsearch span schema adds three new keys, and deprecates two: + +* _add_ `trace.id` +* _add_ `parent.id` +* _add_ `hex_id` +* _deprecate_ `parent` long +* _deprecate_ `id` long + +View the sample Elasticsearch <> for more information. \ No newline at end of file diff --git a/docs/upgrading.asciidoc b/docs/upgrading.asciidoc index 84ce51d1624..04187d761e6 100644 --- a/docs/upgrading.asciidoc +++ b/docs/upgrading.asciidoc @@ -1,7 +1,7 @@ [[upgrading]] == Upgrading APM Server -We do our best to keep APM Server backwards compatible between minor releases. +We do our best to keep APM Server backward compatible between minor releases. However, check out the <> section for exceptions. Before upgrading: @@ -31,25 +31,25 @@ and applies the correct template automatically. [[breaking-changes]] === Breaking Changes APM Server is built on top of {beats-ref}/index.html[libbeat]. -As such any breaking change in libbeat is also considered to be a breaking change in APM Server. +As such, any breaking change in libbeat is also considered to be a breaking change in APM Server. [float] ==== HEAD -no breaking changes in APM Server - -https://www.elastic.co/guide/en/beats/libbeat/master/breaking-changes.html[breaking changes in libbeat] +* There are no breaking changes in APM Server. +* Advanced users may find the <> useful. +* https://www.elastic.co/guide/en/beats/libbeat/master/breaking-changes.html[Breaking changes in libbeat] [float] ==== 6.4 -Indexing the `onboarding` document in it's own index by default. - +* Indexing the `onboarding` document in it's own index by default. [float] ==== 6.3 -Indexing events in separate indices by default. - -https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-6.3.html[breaking changes in libbeat] +* Indexing events in separate indices by default. +* https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-6.3.html[Breaking changes in libbeat] [float] ==== 6.2 -APM Server GA +* APM Server GA + +include::./upgrading-to-65.asciidoc[] \ No newline at end of file