Repository navigation
Expand catalog to 2,937 entries and make counts self-checking - #5
Merged
Merged
Conversation
Adds two dated expansions and rebuilds the repository around stable, machine-readable entry points: - 2026-11-10 (82 entries): software engineering foundations and standards (SemVer, Keep a Changelog, Conventional Commits, licensing, ADRs, Diataxis, codes of conduct), mainstream browser/unit test frameworks, HTTP clients and mock servers, editors, package managers, and observability/BI tools. - 2026-11-11 (653 entries): discovered from 160 GitHub topics, gittrend.io hidden gems, and awesomehub.js.org topic indexes. Fills whole families that were missing: AI agent skills, memory, MCP servers and clients, guardrails and red teaming, reverse-engineering tooling, ORMs and database servers, message queues and workflow engines, OpenTelemetry SDKs, i18n, OCR/document extraction, editor ecosystems, .NET application blocks, and Android libraries. Structure for AI consumers: - index/catalog-stats.json is the single generated source of counts, rollups, and sizes; scripts/catalog_stats.py --write regenerates it and rewrites the stats block in README.md, AI_INDEX.md, and llms.txt. validate.py now fails when that block or the JSON drifts. - README rewritten with a reading-path table (how big each entry point is and when to use it), the corrected 2,937/101/13 numbers, and the new expansion catalogs. AI_INDEX.md lists every machine-readable entry point and the 13 domains with real counts. AGENTS.md, index/README.md, CONTRIBUTING.md, and generated navigation are consistent with 13 domains and 101 categories. - CONTRIBUTING.md documents the repeatable topic-sweep discovery recipe and the dedupe rules used for the new batch. Evidence: each new record records source, check date, review scope, and verification level; docs/expansion-2026-11-11-checks.json holds per-entry repository IDs, GitHub-detected licenses, topics, and last-push times. Metadata-only, not adoption audits. scripts/validate.py: 2937 records, 101 categories, CSV/JSON agreement, generated indexes, local links all pass. Unit tests: 7 passing. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Grows the catalog from 2,202 → 2,937 entries (2,931 distinct URLs, 101 categories, 13 domains) and restructures the repo so an AI can be pointed at it and reach the right few files fast — plus so future additions are mechanical rather than manual.
1. Structure for AI consumption
index/catalog-stats.json(new, generated)scripts/catalog_stats.py(new)--writeregenerates the stats file and rewrites the stats block inREADME.md,AI_INDEX.md,llms.txt;--checkfails on drift.validate.pynow gates countscatalog-stats.jsonis stale or a generated stats block drifts. Count errors can no longer ship silently.README.mdrewrittenAI_INDEX.mdrewrittenAGENTS.md,index/README.md,CONTRIBUTING.md, generated navEntry points stay stable and small-first:
AI_INDEX.md(3 KB) →index/by-category/<id>.json(1–30 KB) →index/resources.compact.json(1.2 MB, 35% of canonical) →resources.json(3.3 MB) →llms-full.txt(1.8 MB, one-shot).2. Two dated expansions
expansion-2026-11-10— 82 entries. Gap analysis of project entry points: SemVer, Keep a Changelog, Conventional Commits, Choose a License, SPDX, REUSE, ADRs, Diátaxis, Contributor Covenant, Twelve-Factor, C4 (new category Software engineering foundations and standards), plus Cypress/Selenium/WebdriverIO/Jest/Robot Framework, HTTP clients and mock servers, editors, package managers, and observability/BI tools.expansion-2026-11-11— 653 entries. The discovery channels the request named:gh api search/repositories(ai-agents,llm,llms,ai-prompt,ai-prompts,mcp,mcp-server,reverse-engineering,vibe-coding,ai-skills,skills,copilot,claude,codex,model-context-protocol,agent-skills,llm-evaluation,guardrails,llmops, plus 140 more), ranked by stars with a 100-star floor.gittrend.io/hidden-gems— today/week/month boards.awesomehub.js.org— topic indexes (cli-apps, selfhosted, go, python, rust, github-actions, …).Candidates were deduplicated against the catalog by GitHub
owner/repoidentity, normalized URL, and repository name, then hand-reviewed per category; archived, forked, off-topic, and unlicensed-where-material repos were dropped (the few dual-use or copyleft entries carry the trade-off inline).Families that previously had zero entries and now have coverage: AI agent skills/memory/MCP clients/guardrails, reverse-engineering tooling, ORMs and database servers, message queues and workflow engines, OpenTelemetry SDKs and log pipelines, i18n libraries, OCR/document extraction, rich-text and note-taking engines, editor/plugin ecosystems, .NET application blocks, Android libraries.
3. Evidence and limits
Per-entry evidence in
docs/expansion-2026-11-11-checks.json: GitHub repository ID, detected license (SPDX), description, stars at check time, archived/fork flags, last push, and the catalog topics that surfaced it. Every new record carriessource,checked_on,reviewscope, andverification_level.This is metadata and reachability evidence only — no README review, release/maintenance audit, licensing opinion, security assessment, compatibility test, or adoption recommendation. No package was installed or executed.
Verification
The 2026-11-10 batch had been sitting uncommitted; this PR includes it. Existing records keep their IDs, facets, and related-link symmetry (verified against
git show HEAD:resources.json).Follow-ups (not in this PR)
9f4d7c8a(2,202 entries); it needs a re-import to show the new batch.awesome-*lists are possible with the recipe now documented inCONTRIBUTING.md.