Repository navigation
a chain name outranked the thing the question was about - #2770
Merged
Merged
Conversation
"cheapest bridge to arbitrum" returned arbitrum-nova-rpc and arbitrum-rpc above bridge-fee. The scorer weighed "arbitrum" and "bridge" the same, and "arbitrum" hit two slugs while "bridge" hit one, so two RPC benchmarks beat every bridge benchmark on a question about bridges. Found by re-running the five test cases the app review will run, rather than by a test: the fixtures in mcp-tools.test.ts have one bench per category, so a category can never lose to a chain in them. A query token matching the benchmark's category now scores alongside the slug match. The category is what a benchmark is about, as opposed to which chain it covers, and that is the distinction the query was missing. Matched as a prefix both ways so the singular a user types reaches the plural we store: "bridge" against "Bridges". Simulated over the real 229-benchmark catalogue before shipping: the bridge query now returns four bridge benchmarks, and "which solana rpc is fastest", "alchemy or quicknode on base", "bridge fees", "perp fees" and "fastest price feed" are all unchanged.
Flotapponnier
added a commit
that referenced
this pull request
Oct 17, 2026
…3019) Preparing the dev to main merge, the conflict triage turned up one file group where dev is not ahead but behind, and behind on a leak. `src/lib/mcp-tools.ts` and the MCP route exist on both branches and were written independently, so they conflict as add/add. main's copies carry five commits dev never received: a thousandfold error and a 35k-token catalogue (#2764), the search endpoint handing back an API key it had been asked to search for (#2766), two tools that answered nothing (#2765), a third annotation the submission scan reads (#2768), and a chain name outranking the subject of the question (#2770). Measured rather than inferred from the titles: `redact` appears 7 times in main's mcp-tools.ts and 3 times in its route, and 0 times in either of dev's. Resolving those conflicts in dev's favour, which a bulk resolution would do, reintroduces the leak. Checked before overwriting: the dev-to-main diff is additive. main adds category-aware scoring, a minimum score that cuts prefix-only noise, and redactSecrets(); the only lines it removes are the ones it replaces. dev holds nothing in these files that main lacks. The privacy page is the same shape without the security edge: main's is 219 lines against dev's 140 and is alone in documenting retention and the Vercel dependency. dev's predates it, from when the page existed only because the app directory requires a policy URL. Doing this now, in its own change, means the merge no longer has a resolution that can silently restore a credential leak. Gates: tsc clean, eslint clean, 567 tests (up from 561: main's test file carries six more).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by re-running the five test cases the app review will run.
Two RPC benchmarks above every bridge benchmark, on a question about bridges. The scorer weighed "arbitrum" and "bridge" the same, and "arbitrum" hit two slugs while "bridge" hit one.
The unit fixtures could never catch this: they have one benchmark per category, so a category can never lose to a chain in them.
The fix
A query token matching the benchmark's category now scores alongside a slug match. The category is what a benchmark is about, as opposed to which chain it covers, and that distinction is what the query was missing. Prefix-matched both ways so the singular a user types reaches the plural we store ("bridge" → "Bridges").
Simulated over the real catalogue before shipping
One query fixed, five untouched, all 229 benchmarks re-scored to check.
tscclean, 412 tests pass.