Skip to content

Add index tags #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 10, 2025
Merged

Add index tags #169

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2225d61
add index tags for serverless index
rohanshah18 Dec 19, 2024
ea7ed9a
fix list vectors test and add tags for create serverless index
rohanshah18 Jan 3, 2025
54f9890
add index tags for configurePodIndex()
rohanshah18 Jan 8, 2025
5bde9dd
update readme
rohanshah18 Jan 9, 2025
d45381b
clean up createServerlessIndex() and address code review comments
rohanshah18 Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ serverless and regional availability, see [Understanding indexes](https://docs.p
import io.pinecone.clients.Pinecone;
import org.openapitools.db_control.client.model.IndexModel;
import org.openapitools.db_control.client.model.DeletionProtection;
import java.util.HashMap;
...

Pinecone pinecone = new Pinecone.Builder("PINECONE_API_KEY").build();
Expand All @@ -180,8 +181,10 @@ String similarityMetric = "cosine";
int dimension = 1538;
String cloud = "aws";
String region = "us-west-2";
HashMap<String, String> tags = new HashMap<>();
tags.put("env", "test");

IndexModel indexModel = pinecone.createServerlessIndex(indexName, similarityMetric, dimension, cloud, region, DeletionProtection.ENABLED);
IndexModel indexModel = pinecone.createServerlessIndex(indexName, similarityMetric, dimension, cloud, region, DeletionProtection.ENABLED, tags);
```

### Create a pod index
Expand Down Expand Up @@ -309,12 +312,16 @@ The following example enables deletion protection for a serverless index.
```java
import io.pinecone.clients.Pinecone;
import org.openapitools.db_control.client.model.DeletionProtection;
import java.util.HashMap;
...

Pinecone pinecone = new Pinecone.Builder("PINECONE_API_KEY").build();

String indexName = "example-index";
pinecone.configureServerlessIndex(indexName, DeletionProtection.ENABLED);
HashMap<String, String> tags = new HashMap<>();
tags.put("env", "test");

pinecone.configureServerlessIndex(indexName, DeletionProtection.ENABLED, tags);
```

## Describe index statistics
Expand Down
5 changes: 4 additions & 1 deletion src/integration/java/io/pinecone/helpers/TestResourcesManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import org.slf4j.LoggerFactory;

import java.util.Arrays;
import java.util.HashMap;
import java.util.List;

import static io.pinecone.helpers.BuildUpsertRequest.buildRequiredUpsertRequestByDimension;
Expand Down Expand Up @@ -315,9 +316,11 @@ public String getOrCreateServerlessIndex() throws InterruptedException, Pinecone
}

String indexName = RandomStringBuilder.build("serverless-index", 8);
HashMap<String, String> tags = new HashMap<>();
tags.put("env", "testing");

serverlessIndexModel = pineconeClient.createServerlessIndex(indexName, metric, dimension, cloud,
region, DeletionProtection.DISABLED);
region, DeletionProtection.DISABLED, tags);
waitUntilIndexIsReady(pineconeClient, indexName);

// Explicitly wait after ready to avoid the "no healthy upstream" issue
Expand Down
9 changes: 5 additions & 4 deletions ...io/pinecone/integration/controlPlane/serverless/CreateDescribeListAndDeleteIndexTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import org.openapitools.db_control.client.model.*;

import java.util.Arrays;
import java.util.Collections;

import static org.junit.jupiter.api.Assertions.*;

Expand Down Expand Up @@ -48,7 +49,7 @@ public void describeAndListIndex() {
@Test
public void createServerlessIndexWithInvalidName() {
try {
controlPlaneClient.createServerlessIndex("Invalid-name", "cosine", 3, "aws", "us-west-2", DeletionProtection.DISABLED);
controlPlaneClient.createServerlessIndex("Invalid-name", "cosine", 3, "aws", "us-west-2", DeletionProtection.DISABLED, null);

fail("Expected to throw PineconeBadRequestException");
} catch (PineconeBadRequestException expected) {
Expand All @@ -59,7 +60,7 @@ public void createServerlessIndexWithInvalidName() {
@Test
public void createServerlessIndexWithInvalidDimension() {
try {
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", -3, "aws", "us-west-2", DeletionProtection.DISABLED);
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", -3, "aws", "us-west-2", DeletionProtection.DISABLED, null);
fail("Expected to throw PineconeValidationException");
} catch (PineconeValidationException expected) {
assertTrue(expected.getLocalizedMessage().contains("Dimension must be greater than 0"));
Expand All @@ -69,7 +70,7 @@ public void createServerlessIndexWithInvalidDimension() {
@Test
public void createServerlessIndexWithInvalidCloud() {
try {
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", 3, "blah", "us-west-2", DeletionProtection.DISABLED);
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", 3, "blah", "us-west-2", DeletionProtection.DISABLED, null);
fail("Expected to throw PineconeValidationException");
} catch (PineconeValidationException expected) {
assertTrue(expected.getLocalizedMessage().contains("Cloud cannot be null or empty. Must be one of " + Arrays.toString(ServerlessSpec.CloudEnum.values())));
Expand All @@ -79,7 +80,7 @@ public void createServerlessIndexWithInvalidCloud() {
@Test
public void createServerlessIndexWithInvalidRegion() {
try {
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", 3, "aws", "invalid-region", DeletionProtection.DISABLED);
controlPlaneClient.createServerlessIndex("serverless-test-index", "cosine", 3, "aws", "invalid-region", DeletionProtection.DISABLED, null);
fail("Expected to throw PineconeNotFoundException");
} catch (PineconeNotFoundException expected) {
assertTrue(expected.getLocalizedMessage().contains("Resource cloud: aws region: invalid-region not found"));
Expand Down
14 changes: 10 additions & 4 deletions ...egration/java/io/pinecone/integration/controlPlane/serverless/DeletionProtectionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
import org.openapitools.db_control.client.model.DeletionProtection;
import org.openapitools.db_control.client.model.IndexModel;

import java.util.HashMap;

public class DeletionProtectionTest {
private static final Pinecone controlPlaneClient = new Pinecone
.Builder(System.getenv("PINECONE_API_KEY"))
Expand All @@ -16,8 +18,10 @@ public class DeletionProtectionTest {
@Test
public void createIndexWithDeletionProtectionEnabled() {
String indexName = RandomStringBuilder.build("create-serv", 8);
HashMap<String, String> tags = new HashMap<>();
tags.put("test", "deletion-protection-enabled");
// Create serverless index with deletion protection enabled
controlPlaneClient.createServerlessIndex(indexName, "cosine", 3, "aws", "us-west-2", DeletionProtection.ENABLED);
controlPlaneClient.createServerlessIndex(indexName, "cosine", 3, "aws", "us-west-2", DeletionProtection.ENABLED, tags);
Copy link
Contributor

@austin-denoble austin-denoble Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to add an assertion for the tags that are applied to the index since we're doing that for DeletionProtection.

Copy link
Contributor Author

@rohanshah18 rohanshah18 Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, added the additional assertions

// Describe index to verify deletion protection is enabled
IndexModel indexModel = controlPlaneClient.describeIndex(indexName);
DeletionProtection deletionProtection = indexModel.getDeletionProtection();
Expand All @@ -27,18 +31,20 @@ public void createIndexWithDeletionProtectionEnabled() {
@Test
public void createPodIndexWithDeletionProtectionDisabled() {
String indexName = RandomStringBuilder.build("create-pod", 8);
HashMap<String, String> tags = new HashMap<>();
tags.put("test", "deletion-protection-disabled");
// Create serverless index with deletion protection disabled
controlPlaneClient.createServerlessIndex(indexName, "cosine", 3, "aws", "us-west-2", DeletionProtection.DISABLED);
controlPlaneClient.createServerlessIndex(indexName, "cosine", 3, "aws", "us-west-2", DeletionProtection.DISABLED, tags);
IndexModel indexModel = controlPlaneClient.describeIndex(indexName);
DeletionProtection deletionProtection = indexModel.getDeletionProtection();
Assertions.assertEquals(deletionProtection, DeletionProtection.DISABLED);
// Configure index to enable deletionProtection
controlPlaneClient.configureServerlessIndex(indexName, DeletionProtection.ENABLED);
controlPlaneClient.configureServerlessIndex(indexName, DeletionProtection.ENABLED, tags);
indexModel = controlPlaneClient.describeIndex(indexName);
deletionProtection = indexModel.getDeletionProtection();
Assertions.assertEquals(deletionProtection, DeletionProtection.ENABLED);
// Configure index to disable deletionProtection
controlPlaneClient.configureServerlessIndex(indexName, DeletionProtection.DISABLED);
controlPlaneClient.configureServerlessIndex(indexName, DeletionProtection.DISABLED, tags);
// Delete index
controlPlaneClient.deleteIndex(indexName);
}
Expand Down
17 changes: 0 additions & 17 deletions src/integration/java/io/pinecone/integration/dataPlane/ListEndpointTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@

import static org.junit.jupiter.api.Assertions.*;


public class ListEndpointTest {
private static final TestResourcesManager indexManager = TestResourcesManager.getInstance();
private static Index indexConnection;
Expand Down Expand Up @@ -70,13 +69,6 @@ public void testSyncListEndpoint() throws InterruptedException {
listResponseWithPaginationNoPrefix1.getPagination().getNext()
);
assertEquals(listResponseWithPaginationNoPrefix2.getVectorsList().size(), 2);
ListResponse listResponseWithPaginationNoPrefix3 = indexConnection.list(
Copy link
Contributor Author

@rohanshah18 rohanshah18 Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove extra checks that are failing consistently in integration tests

Copy link
Contributor

@austin-denoble austin-denoble Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are they failing?

Copy link
Contributor Author

@rohanshah18 rohanshah18 Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The total number of vectors in the custom namespace ("example-namespace") is 4. When listResponseWithPaginationNoPrefix1 returns 2 vectors and includes a pagination token pointing to the remaining vector IDs, these are then returned as part of listResponseWithPaginationNoPrefix2, which has a limit of 2. Since there are no more vector IDs left, no pagination token will be returned in the response of listResponseWithPaginationNoPrefix2. As a result, the assertion on line #79 will fail with the error message: "Pagination token cannot be null or empty." Similarly, this issue will also occur in the asynchronous test.

customNamespace,
2,
listResponseWithPaginationNoPrefix2.getPagination().getNext()
);
assertEquals(listResponseWithPaginationNoPrefix3.getVectorsList().size(), 0);
assertEquals(listResponseWithPaginationNoPrefix3.getPagination().getNext(), "");
}

@Test
Expand Down Expand Up @@ -122,14 +114,5 @@ public void testAsyncListEndpoint() throws InterruptedException {
);
ListResponse asyncListResponseWithPaginationNoPrefix2 = Futures.getUnchecked(futureResponseWithPaginationNoPrefix2);
assertEquals(asyncListResponseWithPaginationNoPrefix2.getVectorsList().size(), 2);
ListenableFuture<ListResponse> futureResponseWithPaginationNoPrefix3 = asyncIndexConnection.list(
customNamespace,
2,
asyncListResponseWithPaginationNoPrefix2.getPagination().getNext()
);
ListResponse asyncListResponseWithPaginationNoPrefix3 = Futures.getUnchecked(futureResponseWithPaginationNoPrefix3);
assertEquals(asyncListResponseWithPaginationNoPrefix3.getVectorsList().size(), 0);
assertEquals(asyncListResponseWithPaginationNoPrefix3.getPagination().getNext(), "");
}

}
Loading
Loading