Unable to get usage while using streaming function

[talhaanwarch]

Here is my code snippet

client = Client(api_key=config["LANGSMITH_API_KEY"])
chain = client.pull_prompt(langsmith_prompt_name, include_model=True)
callback = UsageMetadataCallbackHandler()
  response_stream = chain.stream(
      input, stream_usage=True, config={"callbacks": [callback]}
  )

  for chunk in response_stream:
      print(f"chunkx: {chunk}")
      if hasattr(chunk, "content"):
          yield chunk.content

This is last chunk

content='' additional_kwargs={} response_metadata={'finish_reason': 'stop', 'model_name': 'gpt-4.1-2025-04-14', 'system_fingerprint': 'fp_51e1070cf2', 'service_tier': 'default'} id='run--c1d9dd50-3ae6-4f87-9a9a-0f62f4246239'

Versions
langchain-openai: 0.3.28
langchain-core: 0.3.70
langchain: 0.3.26
langsmith: 0.4.8
openai: 1.97.0

[onestardao]

You’re not alone — usage metadata often gets lost during streaming if the callback doesn’t capture it post-yield. One workaround I’ve used:

Instead of streaming directly to the terminal, try buffering chunks and triggering the callback after the generator completes. Something like:

chunks = []
for chunk in response_stream:
    if hasattr(chunk, "content"):
        chunks.append(chunk.content)
# force flush callback manually
callback.flush()  # or check what your handler exposes

Also double-check if your UsageMetadataCallbackHandler is async-safe — some streaming pipelines swallow events if the callback returns too early.

Let me know if that helps — ran into this exact hiccup while building a .txt‑based LLM agent framework recently

[talhaanwarch]

I tried but got AttributeError: 'UsageMetadataCallbackHandler' object has no attribute 'flush'

[onestardao]

yep, this actually lines up with one of the structural failures i’ve seen way too often — usage stats getting dropped because the callback returns before the stream context fully flushes.

technically this maps to a variant of No.7: Memory Breaks Across Sessions (context handler didn’t persist), and if the logic around tracking breaks down midstream, you also touch No.8: Debugging is a Black Box — since the failure happens silently unless you know where to look.

you can check both here:
https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md

happy to walk you through a fix if you’ve got a test case. i’m still refining patches around this failure pattern — pretty common when callbacks run too early.