elastic · simitt · Jul 9, 2018 · Jul 5, 2018 · Jul 6, 2018 · Jul 9, 2018
diff --git a/docs/configuration-rum.asciidoc b/docs/configuration-rum.asciidoc
@@ -0,0 +1,14 @@
+[[configuration-rum]]
+== Configuring Real User Monitoring (RUM) Support
+
+This section describes configuration settings related to real user monitoring.
+
+[[rum-enable]]
+[float]
+=== Enable RUM Support
+To try out real user montioring, set the `apm-server.frontend.enabled` to `true`.
+See https://github.com/elastic/apm-server/blob/{doc-branch}/apm-server.yml[`apm-server.yml`] for configuration options.
+
+Read more about specific features related to <<rum, real user monitoring>>
+and how to set up the
+{apm-rum-ref}/index.html[JavaScript RUM Agent].
diff --git a/docs/configuring.asciidoc b/docs/configuring.asciidoc
@@ -13,7 +13,7 @@ The following topics describe how to configure APM Server:
 * <<configuration-logging>>
 * <<setup-kibana-endpoint>>
 * <<configuration-dashboards>>
-* <<frontend>>
+* <<configuration-rum>>
 * <<using-environ-vars>>
 * <<configuration-path>>
 --
@@ -32,7 +32,7 @@ include::./copied-from-beats/shared-kibana-config.asciidoc[]
 
 include::./copied-from-beats/dashboardsconfig.asciidoc[]
 
-include::./frontend.asciidoc[]
+include::./configuration-rum.asciidoc[]
 
 :standalone:
 include::./copied-from-beats/shared-env-vars.asciidoc[]

diff --git a/docs/context.asciidoc b/docs/context.asciidoc
@@ -1,5 +1,3 @@
-[float]
-=== Context
 An event's context bundles information regarding the environment in which it is recorded.
 It describes the `service` in which the event is captured, 
 the `system` in which the monitored service is running and the event's `process` information.
@@ -12,5 +10,3 @@ In case a http request is captured, it contains information about the `url`, `co
 The agents provide some configuration options with which the users can also capture customized information.
 The non-indexed information is captured within a `custom` object,
 while the searchable information is stored within `tags`.
-
-The context information is defined for <<transactions,transaction>> and <<errors,error>> events. 
diff --git a/docs/error-api.asciidoc b/docs/error-api.asciidoc
@@ -20,8 +20,9 @@ To send an error record you need to send a `HTTP POST` request to the APM Server
 http(s)://{hostname}:{port}/v1/errors
 ------------------------------------------------------------
 
-To send a record for a frontend error (see <<frontend,frontend support>>), 
-you need to send a `HTTP POST` request to the APM Server `frontend errors` endpoint:
+To send a record for an error monitored by the
+{apm-rum-ref}/index.html[JavaScript RUM Agent]
+you need to send a `HTTP POST` request to the APM Server `client-side errors` endpoint:
 
 [source,bash]
 ------------------------------------------------------------

diff --git a/docs/error-indices.asciidoc b/docs/error-indices.asciidoc
@@ -0,0 +1,17 @@
+[[error-indices]]
+== Error Indices
+
+Errors are stored in separate indices of the format `apm-[version]-error-[date]`.
+Read more about the general purpose of <<errors, error>> documents.
+
+
+[[error-example]]
+[float]
+=== Example document
+
+Example of an error document indexed in Elasticsearch:
+
+[source,json]
+----
+include::./data/elasticsearch/error.json[]
+----
diff --git a/docs/errors.asciidoc b/docs/errors.asciidoc
@@ -1,17 +1,9 @@
 [[errors]]
 == Errors
 
-An error record represents one error event, captured by Elastic APM agents within one service. 
+An error record represents one error event, 
+captured by Elastic APM agents within one service. 
 It is identified by a unique ID.
-
-Some of the key attributes of an error are described in more detail:
-
-[[errors-context]]
-include::./context.asciidoc[]
-
-[[errors-error]]
-[float]
-=== Error
 An error event contains at least
 information about the original `exception` that occured 
 or information about a `log` that was created when the exception occured. 
@@ -21,17 +13,9 @@ helpful for debugging an error.
 
 The `culprit` of an error gives some information about the origin of the error. 
 
-Errors also contain a `grouping_key` which can be used to group together identical or similar errors. 
-
 An error can be mapped to the <<transactions,transaction>> during which it happened, 
-via the `transaction.id`, indexed together with the error event.  
-
+via the `transaction.id`.
 
-[[error-example]]
-[float]
-=== Example document
+include::./context.asciidoc[]
 
-[source,json]
-----
-include::./data/elasticsearch/error.json[]
-----
+Read more about how an <<error-indices, error document>> is indexed in Elasticsearch.
diff --git a/docs/event-types.asciidoc b/docs/event-types.asciidoc
@@ -1,17 +1,14 @@
 [[event-types]]
 = Event Types
+
 [partintro]
 --
-
-Elastic APM agents capture information about `transaction`, `span` and `error` events and send the information to the APM Server.
+Elastic APM agents capture information about `transaction` and `error` events and send the information to the APM Server.
 The actual available information might be optimized per agent. 
 
 * <<transactions,Transactions>>
-* <<spans,Spans>>
 * <<errors,Errors>>
-
 --
 
 include::./transactions.asciidoc[]
-include::./spans.asciidoc[]
 include::./errors.asciidoc[]
diff --git a/docs/exploring-es-data.asciidoc b/docs/exploring-es-data.asciidoc
@@ -0,0 +1,73 @@
+[[exploring-es-data]]
+= Exploring data in Elasticsearch
+
+[partintro]
+--
+By default Elastic APM data are stored in different indices, 
+in the format of 
+`apm-%{[beat.version]}-{type}-%{+yyyy.MM.dd}`.
+
+For getting an overview of existing indices you can run:
+["source","sh"]
+------------------------------------------------------------
+GET _cat/indices/apm*
+------------------------------------------------------------
+// CONSOLE
+
+Default APM `template` and `indices`:
+
+//* <<fields>>
+* <<transaction-indices>>
+* <<span-indices>>
+* <<error-indices>>
+* <<sourcemap-indices>>
+* <<healthcheck-indices>>
+
+
+For querying all APM data:
+["source","sh"]
+------------------------------------------------------------
+GET apm*/_search
+------------------------------------------------------------
+// CONSOLE
+
+Querying documents that have been collected with a specific APM Server version:
+["source","sh",subs="attributes"]
+------------------------------------------------------------
+GET apm-{version}-*/_search
+------------------------------------------------------------
+// CONSOLE
+
+If you are only interested in specific document types, e.g. error documents you can use the type in your query:
+["source","sh",subs="attributes"]
+------------------------------------------------------------
+GET apm-*error-*/_search
+------------------------------------------------------------
+// CONSOLE
+
+If you are interested in the _settings_ and _mappings_ applied to the Elastic APM indices,
+you can fetch the index templates.
+First run a query for figuring out which templates exist:
+
+["source","sh"]
+------------------------------------------------------------
+GET _cat/templates/apm*
+------------------------------------------------------------
+// CONSOLE
+
+Then you can retrieve the specific template you are interested in by sending:
+["source","sh"]
+------------------------------------------------------------
+GET  /_template/your-template-name
+------------------------------------------------------------
+// CONSOLE
+
+You can read more about {ref}/indices-templates.html[Index Templates] and how they are used.
+--
+
+
+include::./transaction-indices.asciidoc[]
+include::./span-indices.asciidoc[]
+include::./error-indices.asciidoc[]
+include::./sourcemap-indices.asciidoc[]
+include::./healthcheck-indices.asciidoc[]
diff --git a/docs/frontend.asciidoc b/docs/frontend.asciidoc
diff --git a/docs/healthcheck-indices.asciidoc b/docs/healthcheck-indices.asciidoc
@@ -0,0 +1,6 @@
+[[healthcheck-indices]]
+== Healthcheck Indices 
+
+Healthcheck indices by default are in the format of `apm-version-date`.
+When starting an APM Server a healtcheck document is sent to Elasticsearch, 
+ensuring the configuration is set up properly.
diff --git a/docs/index.asciidoc b/docs/index.asciidoc
@@ -34,13 +34,17 @@ include::./setting-up-and-running.asciidoc[]
 
 include::./configuring.asciidoc[]
 
+include::./event-types.asciidoc[]
+
+include::./rum.asciidoc[]
+
 include::./data_ingestion.asciidoc[]
 
 include::./storage_management.asciidoc[]
 
 include::./intake-api.asciidoc[]
 
-include::./event-types.asciidoc[]
+include::./exploring-es-data.asciidoc[]
 
 include::./fields.asciidoc[]
 

diff --git a/docs/intake-api.asciidoc b/docs/intake-api.asciidoc
@@ -6,6 +6,8 @@ The APM Server exposes API Endpoints for
 
 * <<transaction-api,Transaction API>>
 * <<error-api,Error API>>
+* <<sourcemap-api,Source Map API>>
 --
 include::./transaction-api.asciidoc[]
 include::./error-api.asciidoc[]
+include::./sourcemap-api.asciidoc[]
diff --git a/docs/rum.asciidoc b/docs/rum.asciidoc
@@ -0,0 +1,55 @@
+[[rum]]
+= Real User Monitoring (RUM)
+[partintro]
+--
+By supporting real user monitoring (RUM).
+Elastic APM allows to collect performance data about your frontend JavaScript applications 
+by setting up the {apm-rum-ref}/index.html[JavaScript RUM Agent].
+
+You can make use of <<sourcemaps, source maps>> when using Elastic RUM. 
+--
+
+[[sourcemaps]]
+== Source Maps
+It is common practice to minify client-side JavaScript code for several reasons, e.g. performance gain. 
+This can make debugging very difficult, as it is hard to read the minified files.
+Source mapping can help for debugging minfied JavaScript files, 
+by mapping code from the minified files to the original source code. 
+
+APM Server provides a <<sourcemap-api,Source Map API>> 
+which accepts source maps complying to the 
+https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k[Source map revision 3 proposal].
+
+Uploaded source maps are used to map _stack trace_ information from recorded transaction and error documents 
+to the original source code files for easier debugging.  
+
+[[sourcemap-apply]]
+[float]
+=== How source maps are applied
+
+When source maps have been <<sourcemap-endpoint,uploaded>> and <<rum-enable,RUM support>> is enabled, 
+source mapping is automatically applied to the _stack trace frames_ of all errors and transactions 
+recorded with the
+{apm-rum-ref}/index.html[JavaScript RUM Agent].
+
+The server tries to find an uploaded source map for every `stack trace frame` of the record.
+The following information is used to find the previously uploaded source map entry:
+
+* the record's `service.name` is matched against the source map's `service_name`
+* the record's `service.version` is matched against the source map's `service_version`
+* the stack trace frame's `abs_path` is matched against the source map's `bundle_filepath`
+
+If multiple source maps with the same meta information are found, 
+the source map with the latest upload timestamp is used. 
+
+In case a matching source map is found and the source map can be applied to the `stack trace frame`, 
+the frame's information is updated with the mapped information before the record is indexed.
+The following information is changed to reflect the original source code file, when source mapping is applied:
+
+* `filename`
+* `function`
+* `line number`
+* `column number`
+* `abs path`: is https://golang.org/pkg/path/#Clean[cleaned] to be the shortest path name equivalent to the given path name 
+
+See how an <<sourcemap-example, example source map>> looks like in Elasticsearch.
diff --git a/docs/sourcemap-api.asciidoc b/docs/sourcemap-api.asciidoc
@@ -0,0 +1,49 @@
+[[sourcemap-api]]
+== Sourcemap Upload API
+
+The APM Server exposes an API Endpoint to upload source maps,
+helpful if you are using <<rum, real user monitoring>>.
+
+[[sourcemap-endpoint]]
+[float]
+=== Upload Endpoint 
+To upload a source map you need to send a `HTTP POST` request
+with `Content-Type` set to `multipart/form-data` to the APM Server source maps endpoint:
+
+[source,bash]
+------------------------------------------------------------
+http(s)://{hostname}:{port}/v1/client-side/sourcemaps
+------------------------------------------------------------
+
+[[sourcemap-request-fields]]
+[float]
+==== Request Fields
+The request consists of some meta information and the actual source map.
+The meta information must contain the following fields:
+
+* `service_name`
+* `service_version`
+* `bundle_filepath`: needs to be the absolute path of the final bundle as it is used in the web application
+
+The meta information is used to identify a `source map` when source mapping is applied.
+
+The actual source map must be attached to the request as a `file upload`
+and it must match the specification for 
+https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k[Source map revision 3 proposal].
+
+
+[[sourcemap-api-examples]]
+[float]
+==== Example
+
+Send an example source map to the APM Server:
+
+["source","sh",subs="attributes"]
+---------------------------------------------------------------------------
+curl -X POST http://127.0.0.1:8200/v1/client-side/sourcemaps \
+  -F service_name="test-service" \
+  -F service_version="1.0" \
+  -F bundle_filepath="http://localhost/static/js/bundle.js" \
+  -F [email protected]
+---------------------------------------------------------------------------
+
diff --git a/docs/sourcemap-indices.asciidoc b/docs/sourcemap-indices.asciidoc
@@ -0,0 +1,16 @@
+[[sourcemap-indices]]
+== Source Map Indices
+Source maps are also stored in a separate index, 
+defined by the Elastic APM Server version and the date. 
+
+
+[[sourcemap-example]]
+[float]
+==== Example document
+
+Example of an acceptable source map file:
+
+[source,json]
+----
+include::./data/intake-api/generated/sourcemap/bundle.js.map[]
+----