Skip to content

Commit 8932876

Browse files
ruai0511kaushalmahi12
ruai0511
and
kaushalmahi12
authored
[Rule based autotagging] Add Get Rule API Logic (#17336)
* add get rule api logic Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> * rebase from main after the schema merged Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> * extract common logics to libs Signed-off-by: Ruirui Zhang <[email protected]> * Add javadocs for libs Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> * correct UT Signed-off-by: Ruirui Zhang <[email protected]> * modify based on comments Signed-off-by: Ruirui Zhang <[email protected]> r * refactor code and fix ut Signed-off-by: Kaushal Kumar <[email protected]> * remove commented code Signed-off-by: Kaushal Kumar <[email protected]> * address comments Signed-off-by: Kaushal Kumar <[email protected]> * change method name Signed-off-by: Kaushal Kumar <[email protected]> * fix merge conflicts Signed-off-by: Kaushal Kumar <[email protected]> * rename queryGroup to workloadGroup Signed-off-by: Kaushal Kumar <[email protected]> * add guice binding related changes Signed-off-by: Kaushal Kumar <[email protected]> * refactor code to create a generic rule framework structure Signed-off-by: Kaushal Kumar <[email protected]> * fix javadoc Signed-off-by: Ruirui Zhang <[email protected]> * fix UT Signed-off-by: Ruirui Zhang <[email protected]> * restructure tests Signed-off-by: Kaushal Kumar <[email protected]> * rebase with mainline Signed-off-by: Kaushal Kumar <[email protected]> * fix gradlew file Signed-off-by: Ruirui Zhang <[email protected]> * add UT Signed-off-by: Ruirui Zhang <[email protected]> * add action UTs Signed-off-by: Kaushal Kumar <[email protected]> * correct the comment Signed-off-by: Kaushal Kumar <[email protected]> * add more UT Signed-off-by: Ruirui Zhang <[email protected]> --------- Signed-off-by: Kaushal Kumar <[email protected]> Co-authored-by: Kaushal Kumar <[email protected]>
1 parent eba12fa commit 8932876

File tree

70 files changed

+2452
-59
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+2452
-59
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
55

66
## [Unreleased 3.x]
77
### Added
8+
- [Rule based auto-tagging] Add get rule API ([#17336](https://github.com/opensearch-project/OpenSearch/pull/17336))
89
- Add multi-threaded writer support in pull-based ingestion ([#17912](https://github.com/opensearch-project/OpenSearch/pull/17912))
910
- Unset discovery nodes for every transport node actions request ([#17682](https://github.com/opensearch-project/OpenSearch/pull/17682))
1011
- Implement parallel shard refresh behind cluster settings ([#17782](https://github.com/opensearch-project/OpenSearch/pull/17782))

‎modules/autotagging-commons/build.gradle‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
10+
opensearchplugin {
11+
description = 'OpenSearch Rule Framework plugin'
12+
classname = 'org.opensearch.rule.RuleFrameworkPlugin'
13+
}
14+
15+
dependencies {
16+
api project("spi")
17+
api project("common")
18+
testImplementation(project(":test:framework")) {
19+
exclude group: 'org.opensearch', module: 'opensearch-core'
20+
}
21+
}

‎libs/autotagging-commons/build.gradle‎ renamed to ‎modules/autotagging-commons/common/build.gradle‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,22 @@
66
* compatible open source license.
77
*/
88

9+
apply plugin: 'opensearch.build'
10+
apply plugin: 'opensearch.publish'
11+
12+
description = 'OpenSearch Rule framework common constructs which spi and module shares'
13+
914
dependencies {
1015
api 'org.apache.commons:commons-collections4:4.4'
11-
api project(":server")
16+
implementation project(":libs:opensearch-common")
17+
compileOnly project(":server")
1218

1319
testImplementation(project(":test:framework")) {
1420
exclude group: 'org.opensearch', module: 'opensearch-core'
1521
}
1622
}
1723

24+
1825
tasks.named("dependencyLicenses").configure {
1926
mapping from: /commons-collections.*/, to: 'commons-collections'
2027
}

‎libs/autotagging-commons/licenses/commons-collections-LICENSE.txt‎ renamed to ‎modules/autotagging-commons/common/licenses/commons-collections-LICENSE.txt‎

File renamed without changes.

‎libs/autotagging-commons/licenses/commons-collections-NOTICE.txt‎ renamed to ‎modules/autotagging-commons/common/licenses/commons-collections-NOTICE.txt‎

File renamed without changes.

‎libs/autotagging-commons/licenses/commons-collections4-4.4.jar.sha1‎ renamed to ‎modules/autotagging-commons/common/licenses/commons-collections4-4.4.jar.sha1‎

File renamed without changes.

‎modules/autotagging-commons/common/src/main/java/org/opensearch/rule/GetRuleRequest.java‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.rule;
10+
11+
import org.opensearch.action.ActionRequest;
12+
import org.opensearch.action.ActionRequestValidationException;
13+
import org.opensearch.common.annotation.ExperimentalApi;
14+
import org.opensearch.core.common.io.stream.StreamInput;
15+
import org.opensearch.core.common.io.stream.StreamOutput;
16+
import org.opensearch.rule.autotagging.Attribute;
17+
import org.opensearch.rule.autotagging.FeatureType;
18+
import org.opensearch.rule.autotagging.Rule;
19+
import org.opensearch.rule.autotagging.RuleValidator;
20+
21+
import java.io.IOException;
22+
import java.util.HashSet;
23+
import java.util.Map;
24+
import java.util.Set;
25+
26+
/**
27+
* A request for get Rule
28+
* Example Request:
29+
* The endpoint "localhost:9200/_wlm/rule" is specific to the Workload Management feature to manage rules
30+
* curl -X GET "localhost:9200/_wlm/rule" - get all rules
31+
* curl -X GET "localhost:9200/_wlm/rule/{_id}" - get single rule by id
32+
* curl -X GET "localhost:9200/_wlm/rule?index_pattern=a,b" - get all rules containing attribute index_pattern as a or b
33+
* @opensearch.experimental
34+
*/
35+
@ExperimentalApi
36+
public class GetRuleRequest extends ActionRequest {
37+
private final String id;
38+
private final Map<Attribute, Set<String>> attributeFilters;
39+
private final String searchAfter;
40+
private final FeatureType featureType;
41+
42+
/**
43+
* Constructor for GetRuleRequest
44+
* @param id - Rule id to get
45+
* @param attributeFilters - Rules will be filtered based on the attribute-value mappings.
46+
* @param searchAfter - The sort value used for pagination.
47+
* @param featureType - The feature type related to rule.
48+
*/
49+
public GetRuleRequest(String id, Map<Attribute, Set<String>> attributeFilters, String searchAfter, FeatureType featureType) {
50+
this.id = id;
51+
this.attributeFilters = attributeFilters;
52+
this.searchAfter = searchAfter;
53+
this.featureType = featureType;
54+
}
55+
56+
/**
57+
* Constructs a GetRuleRequest from a StreamInput for deserialization.
58+
* @param in - The {@link StreamInput} instance to read from.
59+
*/
60+
public GetRuleRequest(StreamInput in) throws IOException {
61+
super(in);
62+
id = in.readOptionalString();
63+
featureType = FeatureType.from(in);
64+
attributeFilters = in.readMap(i -> Attribute.from(i, featureType), i -> new HashSet<>(i.readStringList()));
65+
searchAfter = in.readOptionalString();
66+
}
67+
68+
@Override
69+
public ActionRequestValidationException validate() {
70+
if (RuleValidator.isEmpty(id)) {
71+
throw new IllegalArgumentException(Rule._ID_STRING + " cannot be empty.");
72+
}
73+
if (RuleValidator.isEmpty(searchAfter)) {
74+
throw new IllegalArgumentException("search_after cannot be empty.");
75+
}
76+
return null;
77+
}
78+
79+
@Override
80+
public void writeTo(StreamOutput out) throws IOException {
81+
super.writeTo(out);
82+
out.writeOptionalString(id);
83+
featureType.writeTo(out);
84+
out.writeMap(attributeFilters, (output, attribute) -> attribute.writeTo(output), StreamOutput::writeStringCollection);
85+
out.writeOptionalString(searchAfter);
86+
}
87+
88+
/**
89+
* id getter
90+
*/
91+
public String getId() {
92+
return id;
93+
}
94+
95+
/**
96+
* attributeFilters getter
97+
*/
98+
public Map<Attribute, Set<String>> getAttributeFilters() {
99+
return attributeFilters;
100+
}
101+
102+
/**
103+
* searchAfter getter
104+
*/
105+
public String getSearchAfter() {
106+
return searchAfter;
107+
}
108+
109+
/**
110+
* FeatureType getter
111+
* @return
112+
*/
113+
public FeatureType getFeatureType() {
114+
return featureType;
115+
}
116+
}

‎modules/autotagging-commons/common/src/main/java/org/opensearch/rule/GetRuleResponse.java‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.rule;
10+
11+
import org.opensearch.common.annotation.ExperimentalApi;
12+
import org.opensearch.core.action.ActionResponse;
13+
import org.opensearch.core.common.io.stream.StreamInput;
14+
import org.opensearch.core.common.io.stream.StreamOutput;
15+
import org.opensearch.core.xcontent.ToXContent;
16+
import org.opensearch.core.xcontent.ToXContentObject;
17+
import org.opensearch.core.xcontent.XContentBuilder;
18+
import org.opensearch.rule.autotagging.Rule;
19+
20+
import java.io.IOException;
21+
import java.util.Map;
22+
23+
import static org.opensearch.rule.autotagging.Rule._ID_STRING;
24+
25+
/**
26+
* Response for the get API for Rule.
27+
* Example response:
28+
* {
29+
* "rules": [
30+
* {
31+
* "_id": "z1MJApUB0zgMcDmz-UQq",
32+
* "description": "Rule for tagging query_group_id to index123"
33+
* "index_pattern": ["index123"],
34+
* "query_group": "query_group_id",
35+
* "updated_at": "2025-02-14T01:19:22.589Z"
36+
* },
37+
* ...
38+
* ],
39+
* "search_after": ["z1MJApUB0zgMcDmz-UQq"]
40+
* }
41+
* @opensearch.experimental
42+
*/
43+
@ExperimentalApi
44+
public class GetRuleResponse extends ActionResponse implements ToXContent, ToXContentObject {
45+
private final Map<String, Rule> rules;
46+
private final String searchAfter;
47+
48+
/**
49+
* Constructor for GetRuleResponse
50+
* @param rules - Rules get from the request
51+
* @param searchAfter - The sort value used for pagination.
52+
*/
53+
public GetRuleResponse(final Map<String, Rule> rules, String searchAfter) {
54+
this.rules = rules;
55+
this.searchAfter = searchAfter;
56+
}
57+
58+
/**
59+
* Constructs a GetRuleResponse from a StreamInput for deserialization
60+
* @param in - The {@link StreamInput} instance to read from.
61+
*/
62+
public GetRuleResponse(StreamInput in) throws IOException {
63+
this(in.readMap(StreamInput::readString, Rule::new), in.readOptionalString());
64+
}
65+
66+
@Override
67+
public void writeTo(StreamOutput out) throws IOException {
68+
out.writeMap(rules, StreamOutput::writeString, (outStream, rule) -> rule.writeTo(outStream));
69+
out.writeOptionalString(searchAfter);
70+
}
71+
72+
@Override
73+
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
74+
builder.startObject();
75+
builder.startArray("rules");
76+
for (Map.Entry<String, Rule> entry : rules.entrySet()) {
77+
entry.getValue().toXContent(builder, new MapParams(Map.of(_ID_STRING, entry.getKey())));
78+
}
79+
builder.endArray();
80+
if (searchAfter != null && !searchAfter.isEmpty()) {
81+
builder.field("search_after", new Object[] { searchAfter });
82+
}
83+
builder.endObject();
84+
return builder;
85+
}
86+
87+
/**
88+
* rules getter
89+
*/
90+
public Map<String, Rule> getRules() {
91+
return rules;
92+
}
93+
}

‎modules/autotagging-commons/common/src/main/java/org/opensearch/rule/RuleAttribute.java‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.rule;
10+
11+
import org.opensearch.rule.autotagging.Attribute;
12+
13+
/**
14+
* Generic Rule attributes that features can use out of the use by using the lib.
15+
* @opensearch.experimental
16+
*/
17+
public enum RuleAttribute implements Attribute {
18+
/**
19+
* Represents the index_pattern attribute in RuleAttribute
20+
*/
21+
INDEX_PATTERN("index_pattern");
22+
23+
private final String name;
24+
25+
RuleAttribute(String name) {
26+
this.name = name;
27+
validateAttribute();
28+
}
29+
30+
@Override
31+
public String getName() {
32+
return name;
33+
}
34+
35+
/**
36+
* Retrieves the RuleAttribute from a name string
37+
* @param name - attribute name
38+
*/
39+
public static RuleAttribute fromName(String name) {
40+
for (RuleAttribute attr : RuleAttribute.values()) {
41+
if (attr.getName().equals(name)) {
42+
return attr;
43+
}
44+
}
45+
throw new IllegalArgumentException("Unknown RuleAttribute: " + name);
46+
}
47+
}

‎modules/autotagging-commons/common/src/main/java/org/opensearch/rule/RuleEntityParser.java‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.rule;
10+
11+
import org.opensearch.common.annotation.ExperimentalApi;
12+
import org.opensearch.rule.autotagging.Rule;
13+
14+
/**
15+
* Interface to parse various string representation of Rule entity
16+
* clients can use/implement as per their choice of storage for the Rule
17+
*/
18+
@ExperimentalApi
19+
public interface RuleEntityParser {
20+
/**
21+
* Parses the src string into {@link Rule} object
22+
* @param src String representation of Rule, it could be a XContentObject or something else based on
23+
* where and how it is stored
24+
* @return Rule
25+
*/
26+
Rule parse(String src);
27+
}

0 commit comments

Comments
 (0)