Docs: WIP V2/6.5 additions and changes

docs/common-problems.asciidoc

@@ -36,8 +36,8 @@ Particularly, if you are using Docker, ensure to bind to the right interface (fo
 
 
 
-If you see requests coming trough the APM Server but they are not accepted (response code other than `202`), consider
-the response code to narrow down the possible causes (see sections bellow).
+If you see requests coming through the APM Server but they are not accepted (response code other than `202`), consider
+the response code to narrow down the possible causes (see sections below).
 
 Another reason for data not showing up is that the agent is not auto-instrumenting something you were expecting, check
 the {apm-agents-ref}/index.html[agent documentation] for details on what is automatically instrumented.
@@ -46,7 +46,7 @@ the {apm-agents-ref}/index.html[agent documentation] for details on what is auto
 [float]
 === HTTP 400: Data decoding error / Data validation error
 
-The most likely cause for this is that you are using non compatible versions of agent and APM Server.
+The most likely cause for this is that you are using incompatible versions of agent and APM Server.
 For instance, APM Server 6.2.0 changed the Intake API spec and requires a minimum version of each agent.
 
 [[unauthorized]]
@@ -157,20 +157,20 @@ To resolve this problem, try one of these solutions:
 * Create a DNS entry for the hostname mapping it to the server's IP.
 * Create an entry in `/etc/hosts` for the hostname. Or on Windows add an entry to
 `C:\Windows\System32\drivers\etc\hosts`.
-* Re-create the server certificate and add a SubjectAltName (SAN) for the IP address of the server. This make the
+* Re-create the server certificate and add a SubjectAltName (SAN) for the IP address of the server. This makes the
 server's certificate valid for both the hostname and the IP address.
 
 [float]
 [[getsockopt-no-route-to-host]]
 ===== getsockopt: no route to host
 
-This is not a SSL problem. It's a networking problem. Make sure the two hosts can communicate.
+This is not an SSL problem. It's a networking problem. Make sure the two hosts can communicate.
 
 [float]
 [[getsockopt-connection-refused]]
 ===== getsockopt: connection refused
 
-This is not a SSL problem. Make sure that Logstash is running and that there is no firewall blocking the traffic.
+This is not an SSL problem. Make sure that Logstash is running and that there is no firewall blocking the traffic.
 
 [float]
 [[target-machine-refused-connection]]

docs/configuration-process.asciidoc

@@ -87,7 +87,7 @@ Disabled by default.
 Authorization token for sending data to the APM server.
 If a token is set, the agents must send it in the following format:
 Authorization: Bearer <secret-token>.
-The token is not used for RUM endpoints. By default no authorization token is set.
+The token is not used for RUM endpoints. By default, no authorization token is set.
 
 It is recommended to use an authorization token in combination with SSL enabled.
 Read more about <<securing-apm-server, Securing APM Server>> and the <<secret-token, secret token>>.

docs/configuration-rum.asciidoc

@@ -34,25 +34,25 @@ Default value is set to 10.
 
 [float]
 ==== `allow_origins`
-Comma separated list of permitted origins for RUM supprt.
+Comma separated list of permitted origins for RUM support.
 User-agents send an Origin header that will be validated against this list.
 This is done automatically by modern browsers as part of the https://www.w3.org/TR/cors/[CORS specification].
-An origin is made of a protocol scheme, host and port, without the url path.
+An origin is made of a protocol scheme, host and port, without the URL path.
 Default value is set to `['*']`, which allows everything.
 
 [float]
 ==== `library_pattern`
-Regexp to be matched against a stacktrace frame's `file_name` and `abs_path` attributes.
-If the regexp matches, the stacktrace frame is considered to be a library frame.
-When source mapping is applied the `error.culprit` is set to reflect the _function_ and the _filename_
-of the first not library frame.
+RegExp to be matched against a stacktrace frame's `file_name` and `abs_path` attributes.
+If the RegExp matches, the stacktrace frame is considered to be a library frame.
+When source mapping is applied, the `error.culprit` is set to reflect the _function_ and the _filename_
+of the first non library frame.
 This aims to provide an entry point for identifying issues.
 Default value is `"node_modules|bower_components|~"`.
 
 [float]
 ==== `exclude_from_grouping`
-Regexp to be matched against a stacktrace frame's `file_name`.
-If the regexp matches, the stacktrace frame is excluded from being used for calculating error groups.
+RegExp to be matched against a stacktrace frame's `file_name`.
+If the RegExp matches, the stacktrace frame is excluded from being used for calculating error groups.
 The default pattern excludes stacktrace frames that have a filename starting with `/webpack`.
 
 [[rum-sourcemap-cache]]
@@ -69,10 +69,10 @@ Default value is 5 minutes.
 ==== `source_mapping.elasticsearch`
 Configure the Elasticsearch source map retrieval location, taking the same options as <<elasticsearch-output,output.elasticsearch>>.
 This must be set when using an output other than Elasticsearch, and that output is writing to Elasticsearch.
-Othewise leave this section empty.
+Otherwise leave this section empty.
 
 [float]
 ==== `source_mapping.index_pattern`
-Source maps are stored in a seperate index `apm-%{[beat.version]}-sourcemap` by default.
+Source maps are stored in a separate index `apm-%{[beat.version]}-sourcemap` by default.
 If changed, a matching index pattern needs to be specified here.
 

docs/configuring-ingest.asciidoc

@@ -26,7 +26,7 @@ Automatic pipeline registration requires `output.elasticsearch` to be enabled an
 as well as having the required Elasticsearch plugins installed.
 
 APM Server default pipelines require you to have the Ingest user agent plugin installed.
-If pipelines are enabeld but required plugins are missing, 
+If pipelines are enabled but required plugins are missing, 
 the APM Server will not be able to properly connect to the Elasticsearch output.
 Find the default pipeline configuration at `ingest/pipeline/definition.json` of your APM Server
 installation's home directory.

docs/configuring-output-after.asciidoc

@@ -24,4 +24,4 @@ fields: {project: "myproject", instance-id: "574734885120952459"}
 ------------------------------------------------------------------------------
 
 To store the custom fields as top-level fields, set the `fields_under_root` option to true.
-This is not recommended as backwards compatibility can not be ensured as new fields are added to APM documents.
+This is not recommended as when new fields are added to APM documents backward compatibility cannot be ensured.

docs/error-api.asciidoc

@@ -1,115 +1,41 @@
 [[error-api]]
-== Error API
+=== Error API
 
 The following section contains information about:
 
-* <<error-endpoint>>
 * <<error-schema-definition>>
+* <<error-schema>>
 * <<error-api-examples>>
 
-[[error-endpoint]]
-[float]
-=== Endpoint
-
-Send a `HTTP POST` request to the APM Server `errors` endpoint:
-[source,bash]
-------------------------------------------------------------
-http(s)://{hostname}:{port}/v1/errors
-------------------------------------------------------------
-
-For <<rum, RUM>> send a `HTTP POST` request to the `rum errors` endpoint instead:
-
-[source,bash]
-------------------------------------------------------------
-http(s)://{hostname}:{port}/v1/rum/errors
-------------------------------------------------------------
-
 [[error-schema-definition]]
 [float]
-=== Schema Definition
-
-The APM Server uses JSON Schema for validating requests. The specification for errors is defined bellow:
-* <<error-payload-schema>>
-* <<error-service-schema>>
-* <<error-system-schema>>
-* <<error-context-schema>>
-* <<error-stacktraceframe-schema>>
-* <<error-request-schema>>
-* <<error-user-schema>>
-
-
-[[error-payload-schema]]
-[float]
-==== Payload
-
-[source,json]
-----
-include::./spec/errors/v1_error.json[]
-----
-
-[[error-service-schema]]
-[float]
-==== Service
-
-[source,json]
-----
-include::./spec/service.json[]
-----
-
-[[error-system-schema]]
-[float]
-==== System
+==== Schema Definition
 
-[source,json]
-----
-include::./spec/system.json[]
-----
+The APM Server uses JSON Schema for validating requests. The specification for errors is defined below:
 
-[[error-context-schema]]
 [float]
-==== Context
+[[error-schema]]
+===== Error
 
 [source,json]
 ----
-include::./spec/context.json[]
+include::./spec/errors/common_error.json[]
 ----
 
-[[error-stacktraceframe-schema]]
-[float]
-==== Stacktrace Frame
-
-[source,json]
-----
-include::./spec/stacktrace_frame.json[]
-----
-
-[[error-request-schema]]
-[float]
-==== Request
-
-[source,json]
-----
-include::./spec/request.json[]
-----
-
-[[error-user-schema]]
-[float]
-==== User
-
 [source,json]
 ----
-include::./spec/user.json[]
+include::./spec/errors/v2_error.json[]
 ----
 
 [[error-api-examples]]
 [float]
-=== Examples
+==== Examples
 
 Request example:
 
 ["source","sh",subs="attributes"]
 ------------------------------------------------------------
-curl http://localhost:8200/v1/errors \
+curl http://localhost:8200/intake/v2/events \
   --header "Content-Type: application/json" \
   --data @docs/data/intake-api/generated/error/payload.json
 ------------------------------------------------------------
@@ -122,7 +48,7 @@ Example error requests:
 
 [[payload-with-error]]
 [float]
-==== Payload with an Error
+===== Payload with an Error
 
 [source,json]
 ----
@@ -131,7 +57,7 @@ include::./data/intake-api/generated/error/payload.json[]
 
 [[payload-with-minimal-exception]]
 [float]
-==== Payload with an Error with minimal Exception Information
+===== Payload with an Error with minimal Exception Information
 
 [source,json]
 ----
@@ -140,7 +66,7 @@ include::./data/intake-api/generated/error/minimal_payload_exception.json[]
 
 [[payload-with-minimal-log]]
 [float]
-==== Payload with an Error with minimal Log Information
+===== Payload with an Error with minimal Log Information
 
 [source,json]
 ----

docs/events-api.asciidoc

@@ -0,0 +1,94 @@
+[[events-api]]
+== Events API
+
+Agents capture different types of information, known as events.
+These events are sent to a single endpoint which then sorts and processes the events.
+Events can be:
+
+* Transactions
+* Spans
+* Errors
+* Metricsets
+
+You can learn more about events in the {apm-get-started-ref}/apm-data-model.html[APM Data Model] documentation. 
+
+[[events-api-endpoint]]
+[float]
+=== Endpoint
+
+Send an `HTTP POST` request to the APM Server `intake/v2/events` endpoint:
+
+[source,bash]
+------------------------------------------------------------
+http(s)://{hostname}:{port}/intake/v2/events
+------------------------------------------------------------
+
+For <<rum, RUM>> send an `HTTP POST` request to the APM Server `intake/v2/rum/events` endpoint instead:
+
+[source,bash]
+------------------------------------------------------------
+http(s)://{hostname}:{port}/intake/v2/rum/events
+------------------------------------------------------------
+
+[[events-api-response]]
+[float]
+=== Response
+
+On success, the server will respond with a 202 Accepted status code and no body.
+
+[[events-api-errors]]
+[float]
+=== Errors
+
+Events are processed one after the either.
+If an error is encountered while processing an event,
+the error encountered as well as the document causing the error are added to an internal array.
+The server will wait to encounter up to five errors before sending a response to the agent.
+If there are multiple errors, the highest status code is returned. 
+
+Here's an example response:
+
+[source,json]
+------------------------------------------------------------
+{
+  "errors": [
+    {
+      "message": "queue is full"
+    },{
+      "message": "timeout while waiting to process"
+    },{
+      "message": "<json-schema-err>",
+      "document": "<ndjson-obj>"
+    },{
+      "message": "<json-decoding-err>",
+      "document": "<ndjson-obj>"
+    },{
+      "message": "invalid content-type: 'html/text'"
+    }
+  ],
+  "accepted": 2320
+}
+------------------------------------------------------------
+
+Some errors, not relating to specific events, may be immediately returnable.
+For example: Request limit reached, bad header, or internal queue full.
+If at any point one of these errors is encountered,
+it is added to the internal array and immediately returned. 
+
+If you're developing an agent, these errors can be useful for debugging your agent while building it. 
+
+[[events-api-schema-definition]]
+[float]
+=== Event API Schemas
+
+The APM Server uses a collection of JSON Schemas for validating requests to the intake API:
+
+* <<metadata-api, Metadata>>
+* <<transaction-api, Transactions and Spans>>
+* <<error-api, Errors>>
+* <<metricset-api, Metricsets>>
+
+include::./metadata-api.asciidoc[]
+include::./transaction-api.asciidoc[]
+include::./error-api.asciidoc[]
+include::./metricset-api.asciidoc[]