KAFKA-14691; Add TopicId to OffsetFetch API #4
Conversation
|
Warning Rate limit exceeded@arvi18 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 17 minutes and 41 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
/codehelper |
1 similar comment
|
/codehelper |
![do-refacto[bot]](https://avatars.githubusercontent.com/in/1142269?s=60&v=4){kind=small}
| @@ -8136,17 +8136,37 @@ class KafkaApisTest extends Logging { | |||
| @ParameterizedTest | |||
| @ApiKeyVersionsSource(apiKey = ApiKeys.OFFSET_FETCH) | |||
| def testHandleOffsetFetchWithMultipleGroups(version: Short): Unit = { | |||
| val foo = "foo" | |||
There was a problem hiding this comment.
Unused variable in test method: The variable bar is declared but not used in this test method. Consider removing it or using it in the test to avoid confusion.
| @@ -1056,14 +1078,53 @@ class KafkaApis(val requestChannel: RequestChannel, | |||
| offsetFetchRequest: OffsetFetchRequestData.OffsetFetchRequestGroup, | |||
There was a problem hiding this comment.
Topic ID Support Version Handling: Please verify that the implementation correctly handles both topic IDs and topic names across different API versions, especially around version 10 which introduces topic ID support. The conditional logic paths based on version checks need careful verification to ensure backward compatibility is maintained.
| @@ -1040,13 +1042,33 @@ class KafkaApis(val requestChannel: RequestChannel, | |||
| offsetFetchResponse | |||
There was a problem hiding this comment.
Authorization Logic with Topic IDs: This section modifies authorization logic to handle topic IDs. Please confirm that authorization checks work properly when using topic IDs instead of topic names, as this is security-critical functionality.
| @@ -8242,21 +8283,169 @@ class KafkaApisTest extends Logging { | |||
| .setGroupId("group-4") | |||
There was a problem hiding this comment.
Test Coverage Suggestion: Consider adding specific tests for version compatibility edge cases. The PR adds version-specific behavior for TopicId support (version >= 10), but test coverage for version boundary transitions could be strengthened.
|
/refacto-test |
|
Refacto is reviewing this PR. Please wait for the review comments to be posted. |
Code Review: Kafka OffsetFetch API Topic ID Support👍 Well Done
📌 Files Processed
📝 Additional Comments
|
![refacto-test[bot]](https://avatars.githubusercontent.com/u/196302493?s=60&v=4){kind=small}
| offsetFetchRequest.topics.forEach { topic => | ||
| if (topic.topicId != Uuid.ZERO_UUID) { | ||
| metadataCache.getTopicName(topic.topicId).ifPresent(name => topic.setName(name)) | ||
| } | ||
| } |
There was a problem hiding this comment.
Null Check Missing
Missing null check on offsetFetchRequest.topics could cause NullPointerException. Attackers could send malformed requests with null topics to trigger server errors.
| offsetFetchRequest.topics.forEach { topic => | |
| if (topic.topicId != Uuid.ZERO_UUID) { | |
| metadataCache.getTopicName(topic.topicId).ifPresent(name => topic.setName(name)) | |
| } | |
| } | |
| if (useTopicIds) { | |
| if (offsetFetchRequest.topics != null) { | |
| offsetFetchRequest.topics.forEach { topic => | |
| if (topic.topicId != Uuid.ZERO_UUID) { | |
| metadataCache.getTopicName(topic.topicId).ifPresent(name => topic.setName(name)) | |
| } | |
| } | |
| } | |
| } |
Standards
- CWE-476
- OWASP-A06
| val authorizedNames = authHelper.filterByAuthorized( | ||
| requestContext, | ||
| DESCRIBE, | ||
| TOPIC, | ||
| offsetFetchResponse.topics.asScala | ||
| )(_.name) |
There was a problem hiding this comment.
Null Check Missing
No null check on offsetFetchResponse.topics before conversion to Scala collection. Could cause NullPointerException if topics is null, enabling denial of service attacks.
| val authorizedNames = authHelper.filterByAuthorized( | |
| requestContext, | |
| DESCRIBE, | |
| TOPIC, | |
| offsetFetchResponse.topics.asScala | |
| )(_.name) | |
| val authorizedNames = if (offsetFetchResponse.topics == null) { | |
| Set.empty[String] | |
| } else { | |
| authHelper.filterByAuthorized( | |
| requestContext, | |
| DESCRIBE, | |
| TOPIC, | |
| offsetFetchResponse.topics.asScala | |
| )(_.name) | |
| } |
Standards
- CWE-476
- OWASP-A06
| if (useTopicIds) { | ||
| offsetFetchRequest.topics.forEach { topic => | ||
| if (topic.topicId != Uuid.ZERO_UUID) { | ||
| metadataCache.getTopicName(topic.topicId).ifPresent(name => topic.setName(name)) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Missing TopicId Validation
Missing validation when topic ID is present but name lookup fails. This creates inconsistency where topicId exists but name remains empty, causing potential reliability issues.
| if (useTopicIds) { | |
| offsetFetchRequest.topics.forEach { topic => | |
| if (topic.topicId != Uuid.ZERO_UUID) { | |
| metadataCache.getTopicName(topic.topicId).ifPresent(name => topic.setName(name)) | |
| } | |
| } | |
| } | |
| if (useTopicIds) { | |
| offsetFetchRequest.topics.forEach { topic => | |
| if (topic.topicId != Uuid.ZERO_UUID) { | |
| val topicNameOpt = metadataCache.getTopicName(topic.topicId) | |
| if (topicNameOpt.isPresent) { | |
| topic.setName(topicNameOpt.get) | |
| } else { | |
| // If we can't resolve the topic ID, mark it with empty name | |
| // It will be handled later with UNKNOWN_TOPIC_ID error | |
| topic.setName("") | |
| } | |
| } | |
| } | |
| } |
Standards
- ISO-IEC-25010-Reliability-Fault-Tolerance
- ISO-IEC-25010-Functional-Correctness-Appropriateness
- DbC-Precondition-Validation
- SRE-Error-Handling
| def buildErrorResponse( | ||
| topic: OffsetFetchRequestData.OffsetFetchRequestTopics, | ||
| error: Errors | ||
| ): OffsetFetchResponseData.OffsetFetchResponseTopics = { | ||
| val topicResponse = new OffsetFetchResponseData.OffsetFetchResponseTopics() | ||
| .setTopicId(topic.topicId) | ||
| .setName(topic.name) | ||
| topic.partitionIndexes.forEach { partitionIndex => | ||
| topicResponse.partitions.add(new OffsetFetchResponseData.OffsetFetchResponsePartitions() | ||
| .setPartitionIndex(partitionIndex) | ||
| .setCommittedOffset(-1) | ||
| .setErrorCode(error.code)) | ||
| } | ||
| topicResponse | ||
| } |
There was a problem hiding this comment.
Incomplete Error Handling
Error response builder doesn't set committed leader epoch. This inconsistency could lead to client-side errors when processing responses with errors, as other code paths set this field.
| def buildErrorResponse( | |
| topic: OffsetFetchRequestData.OffsetFetchRequestTopics, | |
| error: Errors | |
| ): OffsetFetchResponseData.OffsetFetchResponseTopics = { | |
| val topicResponse = new OffsetFetchResponseData.OffsetFetchResponseTopics() | |
| .setTopicId(topic.topicId) | |
| .setName(topic.name) | |
| topic.partitionIndexes.forEach { partitionIndex => | |
| topicResponse.partitions.add(new OffsetFetchResponseData.OffsetFetchResponsePartitions() | |
| .setPartitionIndex(partitionIndex) | |
| .setCommittedOffset(-1) | |
| .setErrorCode(error.code)) | |
| } | |
| topicResponse | |
| } | |
| def buildErrorResponse( | |
| topic: OffsetFetchRequestData.OffsetFetchRequestTopics, | |
| error: Errors | |
| ): OffsetFetchResponseData.OffsetFetchResponseTopics = { | |
| val topicResponse = new OffsetFetchResponseData.OffsetFetchResponseTopics() | |
| .setTopicId(topic.topicId) | |
| .setName(topic.name) | |
| topic.partitionIndexes.forEach { partitionIndex => | |
| topicResponse.partitions.add(new OffsetFetchResponseData.OffsetFetchResponsePartitions() | |
| .setPartitionIndex(partitionIndex) | |
| .setCommittedOffset(-1) | |
| .setCommittedLeaderEpoch(-1) | |
| .setErrorCode(error.code)) | |
| } | |
| topicResponse | |
| } |
Standards
- ISO-IEC-25010-Reliability-Maturity
- ISO-IEC-25010-Functional-Correctness-Completeness
- DbC-Postcondition-Consistency
- SRE-Error-Handling
| new OffsetFetchResponseData.OffsetFetchResponseTopics() | ||
| .setTopicId(topic.topicId()) | ||
| .setName(topic.name()); |
There was a problem hiding this comment.
Inconsistent Name Handling
Topic name is set inconsistently in the response. When using topic IDs, the response must include both ID and name for backward compatibility and proper client-side handling.
| new OffsetFetchResponseData.OffsetFetchResponseTopics() | |
| .setTopicId(topic.topicId()) | |
| .setName(topic.name()); | |
| new OffsetFetchResponseData.OffsetFetchResponseTopics() | |
| .setTopicId(topic.topicId()) | |
| .setName(topic.name()) |
Standards
- ISO-IEC-25010-Reliability-Maturity
- ISO-IEC-25010-Functional-Correctness-Appropriateness
- DbC-Contract-Consistency
- SRE-Backward-Compatibility
| val authorizedNames = authHelper.filterByAuthorized( | ||
| requestContext, | ||
| DESCRIBE, | ||
| TOPIC, | ||
| offsetFetchResponse.topics.asScala | ||
| )(_.name) |
There was a problem hiding this comment.
Unnecessary String Comparisons
Filtering by topic names causes unnecessary string comparisons when topic IDs are available. With many topics, this creates O(n) lookups where O(1) is possible using topic IDs directly.
| val authorizedNames = authHelper.filterByAuthorized( | |
| requestContext, | |
| DESCRIBE, | |
| TOPIC, | |
| offsetFetchResponse.topics.asScala | |
| )(_.name) | |
| val authorizedIdentifiers = if (useTopicIds) { | |
| authHelper.filterByAuthorized(requestContext, DESCRIBE, TOPIC, offsetFetchResponse.topics.asScala)(topic => metadataCache.getTopicName(topic.topicId).orElse(topic.name)) | |
| } else { | |
| authHelper.filterByAuthorized(requestContext, DESCRIBE, TOPIC, offsetFetchResponse.topics.asScala)(_.name) | |
| } |
Standards
- ISO-IEC-25010-Performance-Time-Behaviour
- Algorithm-Opt-Hash-Map
- Netflix-Hot-Path-Optimization
|
Reopening this in a new PR |
|
/refacto-test |
|
Refacto is reviewing this PR. Please wait for the review comments to be posted. |
|
/refacto-test |
WIP