Skip to content

Add article: text-to-SQL access control on a benchmark I didn't write - #72

Open
ashishsinha1602 wants to merge 1 commit into
Data-Engineering-Weekly:mainfrom
ashishsinha1602:text-to-sql-access-control
Open

ashishsinha1602 wants to merge 1 commit into
Data-Engineering-Weekly:mainfrom
ashishsinha1602:text-to-sql-access-control

Conversation

@ashishsinha1602

@ashishsinha1602 ashishsinha1602 commented Oct 28, 2026 •

Copy link
Copy Markdown

schemagate in numbers

  • BIRD: Data Engineering Weekly Article submission #1 on the official BIRD Mini-Dev leaderboard in SQLite, PostgreSQL and MySQL (81.0 / 72.6 / 72.4 EX). BIRD hidden test set, scored by the BIRD team: 74.34 EX (simple 81.03 / moderate 72.25 / challenging 56.14; leaderboard row pending), dev 84.03 EX.
  • Tokens: ~91% fewer schema tokens on 50+ table databases (median, Spider 2.0-lite; BENCHMARKS.md).
  • Enterprise: MIT BEAVER warehouses 31-39% EX vs 11.4% for the paper's best setup; LiveSQLBench-Large 34.79% (both self-reported).
  • LangChain / LangGraph: SchemagateRetriever is listed in LangChain's official retriever integrations (langchain-schemagate) and works in LangChain and LangGraph agents; also a LlamaIndex retriever and the official MCP Registry. 34.5k PyPI downloads, Apache-2.0.

Article submission for the weekly curation (weekly/text-to-sql-access-control-benchmark.yml). Disclosure: I wrote the article and maintain the open-source library it describes (Apache-2.0, no commercial offering).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant