How to take user uploaded files and manipulate them downstream?

[col30]

I am using 'adk web' command to ask questions to the agent and upload files. I want the agent to take this user uploaded file, remember it and use it in another basic python function. I am getting an error when I try to run my code.

Error: Failed to parse the parameter context: google.adk.agents.callback_context.CallbackContext of function save_artifacts for automatic function calling. Automatic function calling works best with simpler function signature schema,consider manually parse your function declaration for function save_artifacts.

This is the code:
from pathlib import Path
from google.adk.agents import Agent, LlmAgent
from langchain_experimental.agents.agent_toolkits import create_csv_agent
from google.adk.tools import load_artifacts, ToolContext, FunctionTool
from google.adk.runners import Runner
from google.adk.artifacts import InMemoryArtifactService
from google.adk.agents.callback_context import CallbackContext
import google.genai.types as types
from langchain_core.tools.base import BaseTool
from typing import Dict, Any, Optional

def get_agent_response(artifact_name: str, query: str, tool_context: ToolContext) -> dict:
"""Answers users csv related questions.
Args:
artifact_name: name of csv file uploaded by the user
query: this is the users question
"""
try:
file_path = tool_context.load_artifact(artifact_name)
except Exception:
available_files = tool_context.list_artifacts()
if not available_files:
return "You have no saved artifacts."
else:
file_path = available_files[0]

model = create_gemini_model()
agent = create_csv_agent(
    llm=model,
    path=file_path,
    verbose=True,
    allow_dangerous_code=True
)
query = query
output = agent.run(query)
return {'answer': output, 'status': 'success'}

def save_artifacts(context: CallbackContext,
filename: str,
file_bytes: bytes)->Optional[Dict]:
"""Saves the uploaded file by the user as an artifact.
args:
context = context of the session
filename = name to give to the uploaded csv file
file_bytes = csv file converted into bytes"""

csv_artifact = types.Part.from_data(
    data=file_bytes,
    mime_type="text/csv"
)
try:
    version = context.save_artifact(filename=filename, artifact=csv_artifact)
except ValueError:
    {"status": "error", "message": "Artifact service is not configured!"}
except Exception as e:
    {"status": "error", "message": f"Failed to save artifact! {e}"}

return {
    "status": "success",
    "version": version
}

root_agent = Agent(
name="main_agent",
model=model,
instruction=ANSWER_PROMPT,
description="Uses the get_agent_response tool to answer csv related queries.",
tools=[FunctionTool(func=get_agent_response), FunctionTool(func=save_artifacts)]
)

artifact_service = InMemoryArtifactService()

runner = Runner(
agent=root_agent,
app_name="artifact_app",
session_service=InMemorySessionService(),
artifact_service=artifact_service
)

[halfpasttense]

when using functions as tools you must use exactly "tool_context: ToolContext" as the context parameter, with that exact name and type.

[omkarganda]

@col30 , did you get it running. Were you able to upload your csv file and able to interact with it with the adk web. Please let me know.

[joseph-cavard]

From what I have seen, there is an issue with saving user-uploaded files while using the ADK web command. If I understand correctly how it works: the ADK web command creates a default runner with default parameters for your agent. Therefore, all your runner definitions are likely to be ignored by the command. You can't change the parameter save_input_blobs_as_artifacts to True and thus save user-uploaded docs as artifacts.

I tried creating a custom class or monkey-patching the runner class, but they are just ignored by the ADK web command. The only way I found to make it work is to manually remove this condition (save_input_blobs_as_artifacts) at line 306 for the method _append_new_message_to_session of the Runner class (google.adk) in google.adk. After that, it works with the ADK web command.

I opened an issue for this : #2176 (comment)