Repository navigation
Conversation
BaseLLM declared 'async def agenerate_streaming(...) -> AsyncGenerator[...]', which type checkers resolve as a coroutine of a generator. Every implementation is an async generator function and every call site (e.g. answer_synthesize) iterates the call result with plain 'async for', so the declaration contradicted the implemented contract and produced the "Expected 'AsyncIterable', got 'Coroutine[...]'" FIXME. - declare agenerate_streaming as 'def ... -> AsyncGenerator[str, None]' (implement as an async generator function; do not await the call) - align num_tokens_from_string with the implementations, which return int - drop the now-resolved FIXME at the call site - add contract tests that fail on the old declarations Co-Authored-By: Claude Code <noreply@anthropic.com>
bitflicker64
approved these changes
Nov 11, 2026
bitflicker64
left a comment
Contributor
There was a problem hiding this comment.
Blocking: no. Score: 9/10. Summary: The BaseLLM declarations for agenerate_streaming and num_tokens_from_string match every provider and every async for caller, ty drops the override and Coroutine diagnostics with none added, and the focused models/llms and llm_op tests pass.
imbajin
approved these changes
Nov 14, 2026
imbajin
left a comment
Member
There was a problem hiding this comment.
Score: 9.0/10 · Confidence: high
Blocking: no
Summary
The abstract signatures match the async generator and token-count contracts used by the providers and caller.
Evidence
Exact-head provider and caller inspection; applicable LLM and quality CI succeeded.
Validation notes
No live provider integration or local test run; the change does not alter provider execution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #107 (Tiny Task: fix TODO/FIXME in LLM code)
What
BaseLLM.agenerate_streamingwas declaredasync def ... -> AsyncGenerator[str, None]witha docstring-only body, so type checkers resolve the call as
Coroutine[..., AsyncGenerator[...]]— exactly the FIXME inanswer_synthesize.py("Expected type 'AsyncIterable', got 'Coroutine[Any, Any, AsyncGenerator[str, None]]'
instead"). Every implementation (hugegraph-llm-sdk openai/ollama/litellm providers) is a
true async generator function, and every call site uses plain
async for. The declarationwas the only thing disagreeing with the runtime contract.
Changes
BaseLLM.agenerate_streaming:async def→def, documented as "implement as an asyncgenerator function; the call result is directly async-iterable and must not be awaited".
BaseLLM.num_tokens_from_string: return annotation-> str→-> int(implementationsreturn token counts).
answer_synthesize.py.src/tests/models/llms/test_base_llm_contract.pypinning both contracts.Validation (actually run; offline only)
SKIP_EXTERNAL_SERVICES=true uv run pytest src/tests/models/ src/tests/operators/llm_op/→ 231 passed, 4 skipped
ruff format --check/ruff check→ cleanty check→ the FIXME's Coroutine-vs-AsyncIterable diagnostic is gone (stash-compared;remaining
answer_synthesizehits are pre-existingstr | Noneprompt noise)Refs #107
🤖 Generated with Claude Code