LCORE-383: Fix attachments in /conversations HACK #286
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added a new function `parse_attachments_from_content` to extract attachment information from the content structure. - Updated the `simplify_session_data` function to include parsed attachments in the cleaned messages. - Modified the `retrieve_response` methods in `query.py` and `streaming_query.py` to format and include attachment information in the messages sent to the agent.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
This is the other NON-HACKY PR #285 |
18 tasks
|
I would love feedback from anyone about which direction should be pursued. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is one of 2 that is being proposed to close LCORE-383.
Basically what is happening is that attachment content is being appended to messages when accessing them through the
/conversationsendpoint.This error can be seen below along with what it looks like after the fix.
Before Fix
This is when the attachment is sent before using the
/conversationsendpoint to reload the saved conversation.This is after the
/conversationsendpoint is used to reload the conversation.After Fix
This is when the attachment is sent before using the
/conversationsendpoint to reload the saved conversation.This is after the
/conversationsendpoint is used to reload the conversation.This is what the response of the
/conversationsendpoint looks like:{ "conversation_id": "94ad27d1-674b-48e5-a2cb-b126370fab3e", "chat_history": [ { "messages": [ { "content": "What does this file say?", "type": "user", "attachments": [ { "attachment_type": "log", "content_type": "text/plain", "content": "data:text/plain;base64,SW5ncmVkaWVudHM6Ci0gNiBjdXBzIHRoaW5seSBzbGljZWQgYXBwbGVzCi0gMy80IGN1cCBzdWdhcgotIDIgdGFibGVzcG9vbnMgYWxsLXB1cnBvc2UgZmxvdXIKLSAxLzIgdGVhc3Bvb24gZ3JvdW5kIGNpbm5hbW9uCi0gMS80IHRlYXNwb29uIHNhbHQKLSAxLzggdGVhc3Bvb24gZ3JvdW5kIG51dG1lZwotIDEgdGFibGVzcG9vbiBsZW1vbiBqdWljZQotIDEgcGFja2FnZSAoMiBjcnVzdHMpIHJlZnJpZ2VyYXRlZCBwaWUgY3J1c3RzCgpJbnN0cnVjdGlvbnM6CjEuIFByZWhlYXQgb3ZlbiB0byA0MjUgZGVncmVlcyBGICgyMjAgZGVncmVlcyBDKS4KMi4gSW4gYSBsYXJnZSBib3dsLCBtaXggc2xpY2VkIGFwcGxlcywgc3VnYXIsIGZsb3VyLCBjaW5uYW1vbiwgc2FsdCwgbnV0bWVnLCBhbmQgbGVtb24ganVpY2UuCjMuIFBsYWNlIG9uZSBwaWUgY3J1c3QgaW4gYSBwaWUgZGlzaC4gRmlsbCB3aXRoIGFwcGxlIG1peHR1cmUuIENvdmVyIHdpdGggdGhlIHNlY29uZCBjcnVzdC4KNC4gU2VhbCBhbmQgZmx1dGUgZWRnZXMuIEN1dCBzbGl0cyBpbiB0aGUgdG9wIGNydXN0Lgo1LiBCYWtlIGluIHRoZSBwcmVoZWF0ZWQgb3ZlbiBmb3IgNDUgbWludXRlcyBvciB1bnRpbCBjcnVzdCBpcyBnb2xkZW4gYnJvd24uIEFsbG93IHRvIGNvb2wgYmVmb3JlIHNlcnZpbmcu" } ] }, { "content": "The file contains a recipe for an apple pie. Here are the ingredients and instructions:\n\n**Ingredients:**\n- 6 cups thinly sliced apples\n- 3/4 cup sugar\n- 2 tablespoons all-purpose flour\n- 1/2 teaspoon ground cinnamon\n- 1/4 teaspoon salt\n- 1/8 teaspoon ground nutmeg\n- 1 tablespoon lemon juice\n- 1 package (2 crusts) refrigerated pie crusts\n\n**Instructions:**\n1. Preheat oven to 425 degrees F (220 degrees C).\n2. In a large bowl, mix sliced apples, sugar, flour, cinnamon, salt, nutmeg, and lemon juice.\n3. Place one pie crust in a pie dish. Fill with apple mixture. Cover with the second crust.\n4. Seal and flute edges. Cut slits in the top crust.\n5. Bake in the preheated oven for 45 minutes or until crust is golden brown. Allow to cool before serving.", "type": "assistant" } ], "started_at": "2025-07-24T17:52:44.361756Z", "completed_at": "2025-07-24T17:52:49.855988Z" } ] }I originally wanted to solve this by leveraging the session data in llama-stack to extract the attachment content, however because we pass the message as a string and not a list of
InterleavedContentItem's it directly appends the content to the message content string making it difficult to remove.This PR represents solving the issue in a hacky way that does fully meet the acceptance criteria of LCORE-383. What it does is adds another
TextContentItemafter the original user message (but before theTextContentItems that contain the attachment content) that contains the attachment meta data. This item is denoted by having<attachment_info>tags.The attachment metadata is combined with the attachment content to enhance the conversation endpoint response as shown in the jira ticket.
The issue with this however is that we are introducing more context into the query.
Type of change
Related Tickets & Documents
Checklist before requesting a review
Testing