Better agent logging

[filip-michalsky]

The Current Flow

1. Agent decides: "I should click on Menu" → { method: 'act', parameters: 'click on "Menu"', reasoning: '...', taskComplete: false }

2. Stagehand implements: "Let me find the Menu button" → { elements: [{ elementId: "0-326", description: "button: Menu", method: "click" }] }

You're only seeing #2 in the logs, but you want #1!

The Issue

The logInferenceToFile feature only captures calls made through Stagehand's inference.ts functions (observe() and extract()). But agents make their own LLM calls directly:

OperatorHandler: Uses llmClient.createChatCompletion() with operatorResponseSchema

AnthropicCUAClient: Uses client.beta.messages.create() directly

OpenAICUAClient: Uses client.responses.create() directly

These agent-level calls bypass the logging infrastructure entirely.

Potential Solutions

Option 1: Quick Fix - Add Logging to Operator Handler
Since you're using the operator handler, you could modify getNextStep() to log its LLM calls.

// In operatorHandler.ts getNextStep()
private async getNextStep(currentStep: number): Promise<OperatorResponse> {
  const { data: response } = await this.llmClient.createChatCompletion<OperatorResponse>({
    // ... existing code
  });

  // Add logging here
  if (this.stagehand.logInferenceToFile) {
    writeTimestampedTxtFile(
      "agent_summary", 
      "agent_decision",
      {
        step: currentStep,
        decision: response,
        timestamp: new Date().toISOString()
      }
    );
  }

  return response;
}