Skip to content

improvement(search): rank live results across sources by the query terms they hold - #9134

Merged
waleedlatif1 merged 3 commits into
stagingfrom
improvement/search-cross-source-relevance
Oct 16, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
improvement/search-cross-source-relevance

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Live search fused every account's results by 1 / (RRF_K + rank) within that account, so each account's first result tied with every other account's first result and the stable sort kept account order: the merged list was a round-robin of Drive, Gmail, Slack, Jira, Confluence regardless of the query. The topK cut then fell before any cross-source comparison, so a page whose title is the query sat behind a file-name-only Drive hit and an unrelated Jira ticket, and a source's 5th-best match never surfaced even when it beat every other source's best
  • A plain query now orders the merged results by queryRelevance: the distinct (stemmed) query terms the title and preview hold, then the terms the title holds, then the share of the title's words that are query terms; fused similarity breaks the remaining ties
  • Date-sorted, queryless and native-query searches keep fusion order (native queries carry provider operators, and were not measured)
  • Applies to the Search tab and the Search MCP / assistant tools alike; no new provider requests

Type of Change

  • Improvement

Testing

  • application.test.ts: new case where a later result holding the query (and a title named for it) must outrank another account's first result; it fails on the pre-change sort. application.test.ts + term-fallback pass (73)
  • Offline replay on a large production tenant (read-only): each source's own ranked list captured at topK 20 (the Search tab and MCP default; reconstructs the live merged order 19/20 positions), every pooled document labeled blind by two independent judges, product queryRelevance imported directly:
    • 10 keyword queries from the customer's own Search-vs-competitor check (dev): nDCG@10 0.42 → 0.77 (Opus) / 0.38 → 0.73 (Sonnet); top result relevant 1 → 7 of 10; irrelevant top result 3 → 0
    • 66 held-out queries issued by a Claude client during benchmark sessions, judged against each question's expected facts: nDCG@10 0.29 → 0.65 / 0.28 → 0.65, better on 65/66 and 66/66 queries, worse on 1/0; expected-fact documents in the top 10 0.28 → 0.62, in the top 20 0.48 → 0.81; irrelevant top result 89% → 21%
    • Variants rejected on the same data: coverage only, title first, a weighted sum, pool-IDF weighting (each lower on held-out nDCG and recall, some with 5–13 losses)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…rms they hold

Fusion scores rank a result only within its own account, so every account's
first result tied and the merged list followed account order. A plain query now
orders the merged results by the terms each title and preview hold, then the
terms the title holds, then how much of the title they make up; fusion breaks
ties. Date-sorted, queryless and native-query searches keep fusion order.
@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 16, 2026 7:04pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Plain live searches now compare results across accounts by the query terms in their titles and previews before applying topK.

  • Plain searches rank results across accounts by query matches.

No actionable issues were found. This review used code inspection; tests were not run.

Reviews (1) · Last reviewed commit: "improvement(search): rank live results a..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/sim-search/live/text-match.ts

This branch was previously deployed

1 inactive deployment
Preview — d1b04a2e Deployed Oct 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant