Skip to content

feat(streaming): add streaming conversation support - #123

Open
tharropoulos wants to merge 16 commits into
typesense:masterfrom
tharropoulos:stream-convo
Open

tharropoulos wants to merge 16 commits into
typesense:masterfrom
tharropoulos:stream-convo

Conversation

@tharropoulos

@tharropoulos tharropoulos commented Feb 6, 2026 •

Copy link
Copy Markdown
Collaborator

Change Summary

Add streaming conversational search (conversation_stream) for both documents.search and multi_search, in the sync and async clients.

  • documents.search_stream() and multi_search.perform_stream() return a context-managed stream. Iterate it for the answer's {conversation_id, message} pieces, then call get_final_response() for the full search response.
  • search()/perform() with conversation_stream: True still accept a stream_config (dict or StreamConfigBuilder) with on_chunk, on_complete and on_error, matching typesense-js. They return the final response.
  • The events are parsed by a small SSE decoder that follows the WHATWG rules. It splits on CR/LF at the byte level, so CRLF and multi-byte characters split across reads, and U+2028 inside JSON, are handled. httpx's iter_lines gets the last one wrong.
  • Streams go through the same node selection and failover as other requests, but only until the response headers arrive. After that, errors are raised from the iterator and never retried, so callers never see an answer piece twice. A stream holds its max_concurrent_requests slot until it is closed.
  • New stream_read_timeout_seconds setting (default 60s, the server's LLM timeout). It is used as the read timeout for streams, since the first piece only arrives once the LLM starts answering.
  • Adds the conversation, conversation_model_id, conversation_id and conversation_stream search parameters, and the top-level conversation on multi-search responses.

Tested with respx (failover, mid-stream errors, early close, httpx2) and against Typesense 30.2 and 31.0.rc18 with an OpenAI model (-m open_ai).

PR Checklist

@tharropoulos
tharropoulos force-pushed the stream-convo branch 2 times, most recently from b868933 to cce50e7 Compare February 6, 2026 15:57
- introduce typed stream callbacks and message chunks for conversation search
- add decorator-based StreamConfigBuilder and wire streaming params into search
- parse conversation stream sse lines into message chunks or search responses
- combine streamed chunks into a final search response for async calls
- add async sse handling with chunk parsing, callbacks, and final response combine
- wire stream_config and conversation_stream through async search api
- provide fake sse stream responses and contexts for unit tests
- add integration fixtures for conversational streaming collections and docs
- test both async and sync version of the client
- add unit tests and tests against real typesense instance

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant