[DOCS] Refactor script processor docs #72691
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,99 +4,158 @@ | |
| <titleabbrev>Script</titleabbrev> | ||
| ++++ | ||
|
|
||
| Runs an inline or stored <<modules-scripting,script>> on incoming documents. The | ||
| script runs in the {painless}/painless-ingest-processor-context.html[`ingest`] | ||
| context. | ||
|
|
||
| The script processor uses the <<scripts-and-search-speed,script cache>> to avoid | ||
| recompiling the script for each incoming document. To improve performance, | ||
| ensure the script cache is properly sized before using a script processor in | ||
| production. | ||
|
|
||
| [[script-options]] | ||
| .Script options | ||
| [options="header"] | ||
| |====== | ||
| | Name | Required | Default | Description | ||
| | `lang` | no | "painless" | <<scripting-available-languages,Script language>>. | ||
| | `id` | no | - | ID of a <<create-stored-script-api,stored script>>. | ||
| If no `source` is specified, this parameter is required. | ||
| | `source` | no | - | Inline script. | ||
| If no `id` is specified, this parameter is required. | ||
| | `params` | no | - | Object containing parameters for the script. | ||
| include::common-options.asciidoc[] | ||
| |====== | ||
|
|
||
| [discrete] | ||
| [[script-processor-access-source-fields]] | ||
| ==== Access source fields | ||
|
|
||
| The script processor parses each incoming document's JSON source fields into a | ||
| set of maps, lists, and primitives. To access these fields with a Painless | ||
| script, use the | ||
| {painless}/painless-operators-reference.html#map-access-operator[map access | ||
| operator]: `ctx['my-field']`. You can also use the shorthand `ctx.<my-field>` | ||
| syntax. | ||
|
|
||
| NOTE: The script processor does not support the `ctx['_source']['my-field']` or | ||
| `ctx._source.<my-field>` syntaxes. | ||
|
|
||
| The following processor uses a Painless script to extract the `tags` field from | ||
| the `env` source field. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| POST _ingest/pipeline/_simulate | ||
|
Contributor
There was a problem hiding this comment. Do we explain what the simulate query is anywhere?
Contributor
Author
There was a problem hiding this comment. That's covered in the ingest pipeline docs. I don't feel we need to cover that here. |
||
| { | ||
| "pipeline": { | ||
| "processors": [ | ||
| { | ||
| "script": { | ||
| "description": "Extract 'tags' from 'env' field", | ||
| "lang": "painless", | ||
| "source": """ | ||
| String[] envSplit = ctx['env'].splitOnToken(params['delimiter']); | ||
| ArrayList tags = new ArrayList(); | ||
| tags.add(envSplit[params['position']].trim()); | ||
| ctx['tags'] = tags; | ||
| """, | ||
| "params": { | ||
| "delimiter": "-", | ||
| "position": 1 | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| "docs": [ | ||
| { | ||
| "_source": { | ||
| "env": "es01-prod" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
|
|
||
| The processor produces: | ||
|
|
||
| [source,console-result] | ||
| ---- | ||
| { | ||
| "docs": [ | ||
| { | ||
| "doc": { | ||
| ... | ||
| "_source": { | ||
| "env": "es01-prod", | ||
| "tags": [ | ||
| "prod" | ||
| ] | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
| // TESTRESPONSE[s/\.\.\./"_index":"_index","_id":"_id","_ingest":{"timestamp":$body.docs.0.doc._ingest.timestamp},/] | ||
|
|
||
|
|
||
| [discrete] | ||
| [[script-processor-access-metadata-fields]] | ||
| ==== Access metadata fields | ||
|
|
||
| You can also use a script processor to access metadata fields. The following | ||
| processor uses a Painless script to set an incoming document's `_index`. | ||
|
|
||
| [source,console] | ||
| ---- | ||
| POST _ingest/pipeline/_simulate | ||
| { | ||
| "pipeline": { | ||
| "processors": [ | ||
| { | ||
| "script": { | ||
| "description": "Set index based on `lang` field and `dataset` param", | ||
| "lang": "painless", | ||
| "source": """ | ||
| ctx['_index'] = ctx['lang'] + '-' + params['dataset']; | ||
| """, | ||
| "params": { | ||
| "dataset": "catalog" | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| }, | ||
| "docs": [ | ||
| { | ||
| "_index": "generic-index", | ||
| "_source": { | ||
| "lang": "fr" | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
|
|
||
| The processor changes the document's `_index` to `fr-catalog` from | ||
| `generic-index`. | ||
|
|
||
| [source,console-result] | ||
| ---- | ||
| { | ||
| "docs": [ | ||
| { | ||
| "doc": { | ||
| ... | ||
| "_index": "fr-catalog", | ||
| "_source": { | ||
| "lang": "fr" | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
| // TESTRESPONSE[s/\.\.\./"_id":"_id","_ingest":{"timestamp":$body.docs.0.doc._ingest.timestamp},/] | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's confusing to have this table say 'no' to all parameters in an either/or situation. One of id/source is required, and while this is specified as part of the description, is there a way we could change the no to a '*' or some other indicator that at least one of these is required because it's easy to miss on quick glance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree this isn't ideal, but it's consistent with the other ingest processors. Changing this would break the later
include::common-options.asciidoc[]statement, which is used in all our processor reference docs.I've opened #72717 to address your concerns as a separate effort.