Repository navigation
feat(chat): suggest fresh threads for independent drafts - #55
Merged
Merged
Conversation
|
ScriptedAlchemy
changed the title
Most real user messages carry a context object with zero records, and the suggestion service treated any context object as a reason to abstain, so most threads could never receive a suggestion. Abstain only when a prior message actually references context records. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…raft is edited Dismissal was keyed to the exact draft text, so continuing to type the same draft requested inference again and brought the suggestion back. Dismissal now holds until the draft is sent or cleared, on web and mobile. Mobile also skips requests when the environment has no decision endpoint, matching web. The user guide names the setting as the UI labels it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…unts The composer remounts when switching threads or restoring a persisted draft, which reset dismissal to false and brought the suggestion back for a draft the user already declined. Hold dismissal in a per-thread set so it survives remounts until the draft is sent or cleared, on web and mobile. Co-Authored-By: SWE-2 High <noreply@cognition.ai>
Running the thread-split unittest leaves evals/thread-split/__pycache__ untracked; ignore it alongside eval run output. Co-Authored-By: SWE-2 High <noreply@cognition.ai>
…luation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptedAlchemy
marked this pull request as ready for review
November 4, 2026 10:58
This was referenced Nov 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fresh task can inherit a long thread's context even when it needs none of that work. This draft adds an opt-in, passive suggestion beside a text-only draft on web/desktop and mobile. Choosing it opens a fresh thread with the draft; nothing is automatically split or sent. Dismissal and concurrent edits preserve the current draft.
Settings → Labs enables suggestions per environment. Web/desktop configures a loopback Jev-compatible endpoint serving
pplx-decider-flash-v 1.2; mobile uses that environment's configuration. T 3 does not download models or start a model server. The server checks permissions and eligibility, serializes inference, and abstains on invalid/truncated responses, stale history, errors or timeout. Remote clients call their connected T 3 server.The expanded evaluation does not establish useful recall. It contains 194 app-attributed human follow-ups across 24 threads and five provider configurations. DeepSeek and Opus independently labeled every case; GPT-6.1 Sol adjudicated 13 disagreements. Whole ancestry groups reserve 54 held-out cases, with 140 development/calibration cases. Private transcripts remain outside the repository.
At the shipped 0.94 threshold, the baseline produced zero held-out suggestions. Of 23 eligible cases, it missed the one actionable new task and falsely flagged none of 21 negatives; one label remained uncertain. Two additional new tasks were excluded by production gates. Zero false positives on 21 negatives has a 95% Wilson upper bound of 15.5%, even assuming independent checkpoints. No near-perfect accuracy claim is supported.
Replay through the actual production service, reconstructed projections and real local HTTP inference confirmed zero suggestions across 54 cases:23 HTTP requests,31 exclusions, no state mismatches or observed failures. Sequential service latency was 494 ms median and 885 ms p 95. This does not reproduce historical settings, authenticated RPC, or client typing behavior. Three fixed prompts were scored on calibration; only 23 eligible negatives were available, below the predeclared 100-negative requirement, so no tuned candidate qualified. The prompt and threshold remain unchanged.
Validation:
vp run check:changedpasses on the rebased change, and all six focused Python tests pass. The service replay harness also passes 28 synthetic gate/response/timeout scenarios. The committed evaluator checks session-group separation, frozen labels/sources/model identities, calibration-only selection and explicit coverage denominators.Interactive verification remains incomplete: the T 3 Browser panel opened, but snapshot/evaluate repeatedly timed out after 15 seconds; agent device access was disabled. No UI screenshots or native device pass are claimed. Keep this PR draft: useful recall and integrated client verification remain unproven. Automatic splitting and default-on rollout are outside scope.
Implementation and evaluation: GPT-6 Astra through Codex in T 3 Code; independent labeling/review by DeepSeek V 4.1 Flash via OpenCode, Claude Opus 5.5, and GPT-6.1 Sol medium via Devin.