Skip to content

Commit bb7d0e7

Browse files
mattappersonmattapperson
committed
Beats/update (#21702)
* [ML] Fixing issue with historical job audit messages (#21718) * Add proper aria-label for close inspector (#21719) * [Beats Management] Initial scaffolding for plugin (#18977) * Initial scaffolding for Beats plugin * Removing bits not (yet) necessary in initial scaffolding * [Beats Management] Install Beats index template on plugin init (#19072) * Install Beats index template on plugin init * Adding missing files * [Beats Management] APIs: Create enrollment tokens (#19018) * WIP checkin * Register API routes * Fixing typo in index name * Adding TODOs * Removing commented out license checking code that isn't yet implemented * Remove unnecessary async/await * Don't return until indices have been refreshed * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Adding TODO * Fixing variable name * Using a single index * Adding expiration date field * Adding test for expiration date field * Ignore non-existent index * Fixing logic in test * Creating constant for default enrollment tokens TTL value * Updating test * Fixing name of test file (#19100) * [Beats Management] APIs: Enroll beat (#19056) * WIP checkin * Add API integration test * Converting to Jest test * Create API for enrolling a beat * Handle invalid or expired enrollment tokens * Use create instead of index to prevent same beat from being enrolled twice * Adding unit test for duplicate beat enrollment * Do not persist enrollment token with beat once token has been checked and used * Fix datatype of host_ip field * Make Kibana API guess host IP instead of requiring it in payload * Fixing error introduced in rebase conflict resolution * [Beats Management] APIs: List beats (#19086) * WIP checkin * Add API integration test * Converting to Jest test * WIP checkin * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Updating mapping * [Beats Management] APIs: Verify beats (#19103) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Fleshing out remaining tests * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Moving TODO comment to right file * Rename determine* helper functions to find* * Fixing assertions (#19194) * [Beats Management] APIs: Update beat (#19148) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Add API tests * Update template to allow version field for beat * Implement PUT /api/beats/agent/{beat ID} API * Make enroll beat code consistent with update beat code * Fixing minor typo in TODO comment * Allow version in request payload * Make sure beat is not updated in ES in error scenarios * Adding version as required field in Enroll Beat API payload * Using destructuring * Fixing rename that was accidentally reversed in conflict fixing * [Beats Management] APIs: take auth tokens via headers (#19210) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Fixing minor typo in TODO comment * Make "Enroll Beat" API take enrollment token via header instead of request body * Make "Update Beat" API take access token via header instead of request body * [Beats Management] APIs: Create configuration block (#19270) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Fixing minor typo in TODO comment * Implementing POST /api/beats/configuration_blocks API * Removing unnecessary escaping * Fleshing out types + adding validation for them * Making output singular (was outputs) * Removing metricbeat.inputs * Revert implementation of `POST /api/beats/configuration_blocks` API (#19340) This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks). * [Beats Management] APIs: Create or update tag (#19342) * Updating mappings * Implementing PUT /api/beats/tag/{tag} API * [Beats Management] Prevent timing attacks when checking auth tokens (#19363) * Using crypto.timingSafeEqual() for comparing auth tokens * Prevent subtler timing attack in token comparison function * Introduce random delay after we try to find token in ES to mitigate timing attack * Remove random delay * [Beats Management] APIs: Assign tag(s) to beat(s) (#19431) * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Rename "determine" to "find" * Remove random delay * Starting to implement POST /api/beats/beats_tags API * Changing API * Updating tests for changes to API * Updating ES archive * Renaming * Use destructuring * Moving start of script to own line to increase readability * Using destructuring * [Beats Management] APIs: Remove tag(s) from beat(s) (#19440) * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Remove random delay * Starting to implement POST /api/beats/beats_tags API * Changing API * Updating tests for changes to API * Renaming * Use destructuring * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Implementing `POST /api/beats/agents_tags/removals` API * Updating ES archive * Use destructuring * Moving start of script to own line to increase readability * Nothing to remove if there are no existing tags! * Updating tests to match changes in bulk update painless script * Use destructuring * [Beats Management] Move to Ingest UI arch and initial TS effort (#20039) * [Beats Management] Initial scaffolding for plugin (#18977) * Initial scaffolding for Beats plugin * Removing bits not (yet) necessary in initial scaffolding * [Beats Management] Install Beats index template on plugin init (#19072) * Install Beats index template on plugin init * Adding missing files * [Beats Management] APIs: Create enrollment tokens (#19018) * WIP checkin * Register API routes * Fixing typo in index name * Adding TODOs * Removing commented out license checking code that isn't yet implemented * Remove unnecessary async/await * Don't return until indices have been refreshed * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Adding TODO * Fixing variable name * Using a single index * Adding expiration date field * Adding test for expiration date field * Ignore non-existent index * Fixing logic in test * Creating constant for default enrollment tokens TTL value * Updating test * Fixing name of test file (#19100) * [Beats Management] APIs: Enroll beat (#19056) * WIP checkin * Add API integration test * Converting to Jest test * Create API for enrolling a beat * Handle invalid or expired enrollment tokens * Use create instead of index to prevent same beat from being enrolled twice * Adding unit test for duplicate beat enrollment * Do not persist enrollment token with beat once token has been checked and used * Fix datatype of host_ip field * Make Kibana API guess host IP instead of requiring it in payload * Fixing error introduced in rebase conflict resolution * [Beats Management] APIs: List beats (#19086) * WIP checkin * Add API integration test * Converting to Jest test * WIP checkin * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Updating mapping * [Beats Management] APIs: Verify beats (#19103) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Fleshing out remaining tests * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Moving TODO comment to right file * Rename determine* helper functions to find* * Fixing assertions (#19194) * [Beats Management] APIs: Update beat (#19148) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Add API tests * Update template to allow version field for beat * Implement PUT /api/beats/agent/{beat ID} API * Make enroll beat code consistent with update beat code * Fixing minor typo in TODO comment * Allow version in request payload * Make sure beat is not updated in ES in error scenarios * Adding version as required field in Enroll Beat API payload * Using destructuring * Fixing rename that was accidentally reversed in conflict fixing * [Beats Management] APIs: take auth tokens via headers (#19210) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Fixing minor typo in TODO comment * Make "Enroll Beat" API take enrollment token via header instead of request body * Make "Update Beat" API take access token via header instead of request body * [Beats Management] APIs: Create configuration block (#19270) * WIP checkin * WIP checkin * Add API integration test * Converting to Jest test * Fixing API for default case + adding test for it * Fixing copy pasta typos * Fixing variable name * Using a single index * Implementing GET /api/beats/agents API * Creating POST /api/beats/agents/verify API * Refactoring: extracting out helper functions * Expanding TODO note so I won't forget :) * Fixing file name * Updating mapping * Fixing minor typo in TODO comment * Implementing POST /api/beats/configuration_blocks API * Removing unnecessary escaping * Fleshing out types + adding validation for them * Making output singular (was outputs) * Removing metricbeat.inputs * Revert implementation of `POST /api/beats/configuration_blocks` API (#19340) This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks). * [Beats Management] APIs: Create or update tag (#19342) * Updating mappings * Implementing PUT /api/beats/tag/{tag} API * [Beats Management] Prevent timing attacks when checking auth tokens (#19363) * Using crypto.timingSafeEqual() for comparing auth tokens * Prevent subtler timing attack in token comparison function * Introduce random delay after we try to find token in ES to mitigate timing attack * Remove random delay * [Beats Management] APIs: Assign tag(s) to beat(s) (#19431) * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Rename "determine" to "find" * Remove random delay * Starting to implement POST /api/beats/beats_tags API * Changing API * Updating tests for changes to API * Updating ES archive * Renaming * Use destructuring * Moving start of script to own line to increase readability * Using destructuring * [Beats Management] APIs: Remove tag(s) from beat(s) (#19440) * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Remove random delay * Starting to implement POST /api/beats/beats_tags API * Changing API * Updating tests for changes to API * Renaming * Use destructuring * Using crypto.timingSafeEqual() for comparing auth tokens * Introduce random delay after we try to find token in ES to mitigate timing attack * Implementing `POST /api/beats/agents_tags/removals` API * Updating ES archive * Use destructuring * Moving start of script to own line to increase readability * Nothing to remove if there are no existing tags! * Updating tests to match changes in bulk update painless script * Use destructuring * Ported over base types and arch structure * move management of installIndexTemplate into the framework adapter * ts-lint fix * tslint fixes * more ts tweaks * fix paths * added several working endpoints * add more routes and bug fixes * fix linting * fix type remove CRUFT * remove more cruft * remove more CRUFT * added comments, change plurality * add tsconfig file * add extends path * fixed typo * serveral PR review fixes * fixed lodash type version * “fix” types by applying a lot of any * [Beats Management] Move tokens to use JWT, add more complete test suite (#20317) * inital effort to move to JWT and added jest based tests on libs * assign beats tests all passing * token tests now pass * add more tests * all tests now green * fix broken test, this is beats CM not logstash 😊 * added readme * move enrollment token back to a hash * remove un-needed comment * alias lodash get to avoid confusion * isolated hash creation * [Beats Management] add more tests, update types, break out ES into it's own adapter (#20566) * inital effort to move to JWT and added jest based tests on libs * assign beats tests all passing * token tests now pass * add more tests * all tests now green * move enrollment token back to a hash * remove un-needed comment * alias lodash get to avoid confusion * isolated hash creation * Add initial efforts for backend framework adapter testing * move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES * re-typed * renamed types to match pattern * aditional renames * adapter tests should always just use adapterSetup(); * database now uses InternalRequest * corrected spelling of framework * fix typings * remove CRUFT * RequestOrInternal * Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible * fix tests, add test, removed extra comment * fix auth * updated lock file * [Beats Management] add get beat endpoint (#20603) * [Beats Management] Move tokens to use JWT, add more complete test suite (#20317) * inital effort to move to JWT and added jest based tests on libs * assign beats tests all passing * token tests now pass * add more tests * all tests now green * fix broken test, this is beats CM not logstash 😊 * added readme * move enrollment token back to a hash * remove un-needed comment * alias lodash get to avoid confusion * isolated hash creation * inital effort to move to JWT and added jest based tests on libs * assign beats tests all passing * token tests now pass * add more tests * all tests now green * move enrollment token back to a hash * remove un-needed comment * alias lodash get to avoid confusion * isolated hash creation * Add initial efforts for backend framework adapter testing * move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES * re-typed * renamed types to match pattern * aditional renames * adapter tests should always just use adapterSetup(); * database now uses InternalRequest * corrected spelling of framework * fix typings * remove CRUFT * RequestOrInternal * Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible * fix tests, add test, removed extra comment * Moved critical path code from route, to more easeley tested domain * fix auth * remove beat verification, added get beat endpoint to return configs * fix type * update createGetBeatConfigurationRoute URL * rename method * update to match PR #20566 * updated lock file * fix bad merge * update TSLinting * fix bad rebase * [Beats Management] [WIP] Create public resources for management plugin (#20864) * Init plugin public resources. * rename beats to beats_management * rendering react now * Beats/initial ui (#20994) * initial layout and main nav * modal UI and pattern for UI established * fix path * wire up in-memroy adapters * tweak adapters * add getAll method to tags adapter (#21287) * Beats/real adapters (#21481) * add initial real adapters, and nulled data where we need endpoints * UI adapters and needed endpoints added (though not tested) * prep for route tests and some cleanup * move files * [Beats Management] Add BeatsTable/Bulk Action Search Component (#21182) * Add BeatsTable and control bar components. * Clean yarn.lock. * Move raw numbers/strings to constants. Remove obsolete state/props. * Update/add tests. * Change prop name from "items" to "beats". * Rename some variables. * Move search bar filter definitions to table render. * Update table to support assignment options. * Update action control position. * Refactor split render function into custom components. * Beats/basic use cases (#21660) * tweak adapter responses / types. re-add enroll ui * routes enabled, enroll now pings the server * full enrollment path now working * improved pinging for beat enrollment * fix location of history call * reload beats list on beat enrollment completion * add update on client side, expand update on server to allow for partial data, and user auth * remove double beat lookup * fix tests * only return active beats * disenroll now working * fig getAll query * re-enrolling a beat will now work * fix types * fix types * update deps * update kibana API for version
1 parent 264a566 commit bb7d0e7

File tree

28 files changed

+313
-74
lines changed

28 files changed

+313
-74
lines changed

‎x-pack/plugins/beats_management/common/domain_types.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ export interface ConfigurationBlock {
1313
export interface CMBeat {
1414
id: string;
1515
enrollment_token: string;
16+
active: boolean;
1617
access_token: string;
1718
verified_on?: string;
1819
type: string;

‎x-pack/plugins/beats_management/public/components/table/controls.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ export function ControlBar(props: ControlBarProps) {
3030
selectionCount,
3131
showAssignmentOptions,
3232
} = props;
33+
3334
const filters = controlDefinitions.filters.length === 0 ? null : controlDefinitions.filters;
3435
return selectionCount !== 0 && showAssignmentOptions ? (
3536
<AssignmentOptions

‎x-pack/plugins/beats_management/public/components/table/table.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ export class Table extends React.Component<BeatsTableProps, BeatsTableState> {
4343
}
4444

4545
public render() {
46+
4647
const {
4748
actionHandler,
4849
assignmentOptions,

‎x-pack/plugins/beats_management/public/components/table/table_type_configs.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import { EuiBadge, EuiFlexGroup, EuiIcon, EuiLink } from '@elastic/eui';
99
import { flatten, uniq } from 'lodash';
1010
import moment from 'moment';
1111
import React from 'react';
12+
1213
import { TABLE_CONFIG } from '../../../common/constants';
1314
import { BeatTag, CMPopulatedBeat, ConfigurationBlock } from '../../../common/domain_types';
1415

‎x-pack/plugins/beats_management/public/lib/adapters/beats/adapter_types.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import { CMBeat } from '../../../../common/domain_types';
88

99
export interface CMBeatsAdapter {
1010
get(id: string): Promise<CMBeat | null>;
11+
update(id: string, beatData: Partial<CMBeat>): Promise<boolean>;
1112
getAll(): Promise<CMBeat[]>;
1213
removeTagsFromBeats(removals: BeatsTagAssignment[]): Promise<BeatsRemovalReturn[]>;
1314
assignTagsToBeats(assignments: BeatsTagAssignment[]): Promise<CMAssignmentReturn[]>;

‎x-pack/plugins/beats_management/public/lib/adapters/beats/memory_beats_adapter.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,17 @@ export class MemoryBeatsAdapter implements CMBeatsAdapter {
2525
return this.beatsDB.find(beat => beat.id === id) || null;
2626
}
2727

28+
public async update(id: string, beatData: Partial<CMBeat>): Promise<boolean> {
29+
const index = this.beatsDB.findIndex(beat => beat.id === id);
30+
31+
if (index === -1) {
32+
return false;
33+
}
34+
35+
this.beatsDB[index] = { ...this.beatsDB[index], ...beatData };
36+
return true;
37+
}
38+
2839
public async getAll() {
2940
return this.beatsDB.map<CMBeat>((beat: any) => omit(beat, ['access_token']));
3041
}
@@ -35,6 +46,12 @@ export class MemoryBeatsAdapter implements CMBeatsAdapter {
3546
const beatIds = removals.map(r => r.beatId);
3647

3748
const response = this.beatsDB.filter(beat => beatIds.includes(beat.id)).map(beat => {
49+
const tagData = removals.find(r => r.beatId === beat.id);
50+
if (tagData) {
51+
if (beat.tags) {
52+
beat.tags = beat.tags.filter(tag => tag !== tagData.tag);
53+
}
54+
}
3855
const removalsForBeat = removals.filter(r => r.beatId === beat.id);
3956
if (removalsForBeat.length) {
4057
removalsForBeat.forEach((assignment: BeatsTagAssignment) => {

‎x-pack/plugins/beats_management/public/lib/adapters/beats/rest_beats_adapter.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,11 @@ export class RestBeatsAdapter implements CMBeatsAdapter {
2828
return (await this.REST.get<{ beats: CMBeat[] }>('/api/beats/agents')).beats;
2929
}
3030

31+
public async update(id: string, beatData: Partial<CMBeat>): Promise<boolean> {
32+
await this.REST.put<{ success: true }>(`/api/beats/agent/${id}`, beatData);
33+
return true;
34+
}
35+
3136
public async removeTagsFromBeats(removals: BeatsTagAssignment[]): Promise<BeatsRemovalReturn[]> {
3237
return (await this.REST.post<{ removals: BeatsRemovalReturn[] }>(
3338
`/api/beats/agents_tags/removals`,

‎x-pack/plugins/beats_management/public/lib/adapters/framework/kibana_framework_adapter.ts‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ import {
1717

1818
export class KibanaFrameworkAdapter implements FrameworkAdapter {
1919
public appState: object;
20-
public kbnVersion?: string;
2120

2221
private management: any;
2322
private adapterService: KibanaAdapterServiceProvider;

‎x-pack/plugins/beats_management/public/lib/adapters/rest_api/axios_rest_api_adapter.ts‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,7 @@ import { RestAPIAdapter } from './adapter_types';
99
let globalAPI: AxiosInstance;
1010

1111
export class AxiosRestAPIAdapter implements RestAPIAdapter {
12-
constructor(
13-
private readonly kbnVersion: string,
14-
private readonly xsrfToken: string,
15-
private readonly basePath: string
16-
) {}
12+
constructor(private readonly xsrfToken: string, private readonly basePath: string) {}
1713

1814
public async get<ResponseData>(url: string): Promise<ResponseData> {
1915
return await this.REST.get(url).then(resp => resp.data);
@@ -48,7 +44,7 @@ export class AxiosRestAPIAdapter implements RestAPIAdapter {
4844
Accept: 'application/json',
4945
credentials: 'same-origin',
5046
'Content-Type': 'application/json',
51-
'kbn-version': this.kbnVersion,
47+
'kbn-version': this.xsrfToken,
5248
'kbn-xsrf': this.xsrfToken,
5349
},
5450
});

‎x-pack/plugins/beats_management/public/lib/compose/kibana.ts‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,7 @@ import { RestTokensAdapter } from '../adapters/tokens/rest_tokens_adapter';
2222
import { FrontendDomainLibs, FrontendLibs } from '../lib';
2323

2424
export function compose(): FrontendLibs {
25-
const kbnVersion = (window as any).__KBN__.version;
26-
const api = new AxiosRestAPIAdapter(kbnVersion, chrome.getXsrfToken(), chrome.getBasePath());
25+
const api = new AxiosRestAPIAdapter(chrome.getXsrfToken(), chrome.getBasePath());
2726

2827
const tags = new RestTagsAdapter(api);
2928
const tokens = new RestTokensAdapter(api);

0 commit comments

Comments
 (0)