thycotic_ss: Handle missing cef.extensions.deviceReceiptTime

[kcreddy]

Proposed commit message

thycotic_ss: Handle missing cef.extensions.deviceReceiptTime

When "rt" field is empty in raw data, the CEF beats processor doesn't 
populate the field "cef.extensions.deviceReceiptTime". This makes the 
"date" processor to fail because there is no conditional null-check.
This subsequently stops the rest of the pipeline from executing.

This PR adds a conditional "if" and also "on_failure" onto the 
"date" processor so the rest of the pipeline is executed.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Pipeline tests

Added new test event.
Current version receives error:

[0] unexpected pipeline error: [field [deviceReceiptTime] not present as part of path [cef.extensions.deviceReceiptTime]]

After update:

--- Test results for package: thycotic_ss - START ---
╭─────────────┬─────────────┬───────────┬──────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME                                        │ RESULT │ TIME ELAPSED │
├─────────────┼─────────────┼───────────┼──────────────────────────────────────────────────┼────────┼──────────────┤
│ thycotic_ss │ logs        │ pipeline  │ (ingest pipeline warnings test-thycotic-ss.json) │ PASS   │ 512.177958ms │
│ thycotic_ss │ logs        │ pipeline  │ test-thycotic-ss.json                            │ PASS   │  57.793833ms │
╰─────────────┴─────────────┴───────────┴──────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: thycotic_ss - END   ---
Done

System tests

Added new test event.
Current version receives error:

thycotic_ss/logs logfile:
[0] found error.message in event: [field [deviceReceiptTime] not present as part of path [cef.extensions.deviceReceiptTime]]

After update:

--- Test results for package: thycotic_ss - START ---
╭─────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├─────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ thycotic_ss │ logs        │ system    │ logfile   │ PASS   │ 33.647945083s │
│ thycotic_ss │ logs        │ system    │ tcp       │ PASS   │ 42.680618458s │
│ thycotic_ss │ logs        │ system    │ udp       │ PASS   │  36.63450325s │
╰─────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: thycotic_ss - END   ---
Done

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[andrewkroh]

Can you add a test log to the system tests that does not have an rt= (or whatever reproduces the problem we are fixing).

[kcreddy]

@andrewkroh, I forgot to add test files. I added them now (pipeline and system).

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b64e535

History

• 💔 Build #33888 failed 002941c
• 💚 Build #33821 succeeded 4b8ee8a

cc @kcreddy