Skip to content

Conversation

ThomasVitale
Copy link
Contributor

  • Consolidate usage of “db.collection.name” attribute to track table name, collection name, index name, document name, or whatever concept a vector database uses to store data. Removed “db.index” that was use sometimes instead of “db.collection.name”. This usage is in line with the OpenTelemetry Semantic Conventions.
  • Configure query response content to be included as a “span event” instead of a “span attribute” if the backend system supports that, similar to how we do for the model observations.
  • Structure vector store observation attributes in dedicated enums, including one for the Spring AI Kinds to avoid hard-coding the same value in a lot of places. This follows the OpenTelemetry Semantic Conventions as much as possible. Also, adopt Spring usual non-null-by-default strategy as much as possible.
  • Align vector store conventions to the chat model ones, and follow alphabetical order for values. This is particularly useful for the convention classes, for which the Micrometer performance of exporting telemetry data improves when key values are added already sorted to the context.
  • Fix flaky test in Mistral AI.
  • Improve Qdrant integration tests.

@ThomasVitale ThomasVitale force-pushed the enhance-vector-store-observability branch from 29139ee to a5fd8ed Compare August 22, 2024 06:16
.hasHighCardinalityKeyValue(HighCardinalityKeyNames.REQUEST_TOP_K.asString(), KeyValue.NONE_VALUE)
.hasHighCardinalityKeyValue(HighCardinalityKeyNames.REQUEST_TOP_P.asString(), "1.0")
.hasHighCardinalityKeyValue(HighCardinalityKeyNames.RESPONSE_ID.asString(), responseMetadata.getId())
.hasHighCardinalityKeyValue(HighCardinalityKeyNames.RESPONSE_ID.asString(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was flaky, Mistral AI doesn't always return a Response ID in streaming mode.

chatModelObservationContext
.addHighCardinalityKeyValue(ChatModelObservationDocumentation.HighCardinalityKeyNames.COMPLETION
.withValue(ChatModelObservationContentProcessor.concatenateStrings(completions)));
.withValue(TracingHelper.concatenateStrings(completions)));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted the logic into the TracingHelper to re-use it for the vector store observations.

/**
* The name of the operation or command being executed.
*/
DB_OPERATION_NAME("db.operation.name"),;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to VectorStoreObservationAttributes


// @formatter:off

CHAT_CLIENT("chat_client"),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first two values here are not used yet, I didn't want to make the PR even bigger. We can adopt them in a separate PR.

import io.micrometer.common.KeyValues;

/**
* Default conventions to populate observations for vector store operations.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main change here is a resorting to feed Micrometer already sorted list of key values for a performance improvement, and updating the observation name to be closer to the semantic conventions

* Consolidate usage of “db.collection.name” attribute to track table name, collection name, index name, document name, or whatever concept a vector database uses to store data. Removed “db.index” that was use sometimes instead of “db.collection.name”. This usage is in line with the OpenTelemetry Semantic Conventions.
* Configure query response content to be included as a “span event” instead of a “span attribute” if the backend system supports that, similar to how we do for the model observations.
* Structure vector store observation attributes in dedicated enums, including one for the Spring AI Kinds to avoid hard-coding the same value in a lot of places. This follows the OpenTelemetry Semantic Conventions as much as possible. Also, adopt Spring usual non-null-by-default strategy as much as possible.
* Align vector store conventions to the chat model ones, and follow alphabetical order for values. This is particularly useful for the convention classes, for which the Micrometer performance of exporting telemetry data improves when key values are added already sorted to the context.
* Fix flaky test in Mistral AI.
* Improve Qdrant integration tests.

Signed-off-by: Thomas Vitale <[email protected]>
@ThomasVitale ThomasVitale force-pushed the enhance-vector-store-observability branch from a5fd8ed to 89ffd68 Compare August 22, 2024 06:23
@tzolov
Copy link
Contributor

tzolov commented Aug 22, 2024

I've added .withCollectionName(this.pineconeIndexName) to the PineconeVectorStore

@tzolov
Copy link
Contributor

tzolov commented Aug 22, 2024

Rebased and merged at 036093a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants