Reorganize and optimize the repo for AI reference

.github/workflows/link-check.yml

@@ -0,0 +1,49 @@
+name: Link check
+
+# Monthly reachability check of every catalog URL. A live link proves
+# reachability only. Dead or moved links are reported on a single standing
+# issue (exact title below); the issue is updated, never duplicated.
+on:
+  schedule:
+    - cron: "17 6 1 * *" # 06:17 UTC on the 1st of each month
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  issues: write
+
+jobs:
+  link-check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.x"
+      - name: Run link check
+        id: check
+        run: |
+          set +e
+          python scripts/link_check.py --report link-check-report.md --json-out link-check-results.json
+          echo "exit_code=$?" >> "$GITHUB_OUTPUT"
+          cat link-check-report.md >> "$GITHUB_STEP_SUMMARY"
+      - name: Upload report
+        uses: actions/upload-artifact@v4
+        with:
+          name: link-check-report
+          path: |
+            link-check-report.md
+            link-check-results.json
+      - name: Open or update the standing link-rot issue
+        if: steps.check.outputs.exit_code == '1'
+        env:
+          GH_TOKEN: ${{ github.token }}
+          ISSUE_TITLE: "Link check: dead or moved links found"
+        run: |
+          existing=$(gh issue list --state open --json number,title \
+            --jq "[.[] | select(.title == \"$ISSUE_TITLE\") | .number] | first // empty")
+          if [ -n "$existing" ]; then
+            gh issue comment "$existing" --body-file link-check-report.md
+          else
+            gh issue create --title "$ISSUE_TITLE" --body-file link-check-report.md
+          fi

AGENTS.md

@@ -1,17 +1,13 @@
-# FOR_AI - Machine-Readable Resource Hub & Agent Index
-> **Target Consumer**: Autonomous AI Agents, LLM Scrapers, Developer Assistants  
-> **Core Purpose**: High-density context ingestion, zero-friction resource lookup, and programmatic tool discovery.
-
 # ForAI — instructions for AI assistants
 
 Use this repository as a **curated pointer library** when answering software-development and design questions. It is not executable project code, an installation manifest, a live service, or a guarantee that a listed resource is current or suitable.
 
-## Start here
+## Start here — smallest first
 
-1. Read `README.md` for the repository's scope, caveats, and the reusable prompt under **Point your AI here**.
-2. Read `categories/README.md` to find relevant topic pages. Consult only the categories useful to the current task, then follow their links to the primary source.
-3. For machine-readable lookup, search `resources.json` or run `python scripts/search.py "your query"`. Search ranks names and descriptive/facet fields; it intentionally ignores boilerplate review notes, link statuses, and collection provenance. Narrow results with `--topic`, `--type`, `--use-case`, `--language`, `--platform`, or `--category`. `resources.csv` is the matching tabular export. `llms.txt` is a compact navigation index.
-4. Read any linked topic-specific catalog for additional provenance and cautions. For design research, see `catalogs/design-inspiration-resources.md` and `categories/ui-design-inspiration-and-reference.md`. The two 2026-10-27 development expansions are `catalogs/development-expansion-2026-10-27.md` and `catalogs/development-expansion-2026-10-27-round-2.md`; each has a linked metadata snapshot. Both snapshots are metadata-only, not adoption audits.
+1. Read `AI_INDEX.md`: it maps 12 broad domains to the categories worth opening and explains the file layout in one page.
+2. Open only what the task needs: one category page in `categories/`, or its machine-readable shard `index/by-category/<category_id>.json`. `index/category-groups.json` lists which categories belong to each domain.
+3. For machine-readable lookup, search `resources.json` or run `python scripts/search.py "your query"`. Search ranks names and descriptive/facet fields; it intentionally ignores boilerplate review notes, link statuses, and collection provenance. Narrow results with `--topic`, `--type`, `--use-case`, `--language`, `--platform`, or `--category`. `index/resources.compact.json` is the full catalog at ~38% of the size (shortlisting fields only); `resources.csv` is the matching tabular export; `llms.txt` is a compact navigation index.
+4. Read a linked topic-specific catalog in `catalogs/` only when you need its extra provenance and cautions. For design research, see `catalogs/design-inspiration-resources.md` and `categories/ui-design-inspiration-and-reference.md`. The two 2026-10-27 development expansions are `catalogs/development-expansion-2026-10-27.md` and `catalogs/development-expansion-2026-10-27-round-2.md`; each has a linked metadata snapshot. Both snapshots are metadata-only, not adoption audits.
 
 ## Selection workflow
 
@@ -27,10 +23,10 @@ Use this repository as a **curated pointer library** when answering software-dev
 
 ## Repository data and contribution conventions
 
-The canonical catalog is `resources.json`; `resources.csv` is a lossless, derived export. The current record contract is documented in `index/resource-schema.json`, with discovery facets in `index/taxonomy.json`. Stable `resource_id` and `category_id` values identify records and categories; topic, language, platform, use-case, audience, and resource-type facets are conservative discovery hints, not guarantees. `verification_level` describes recorded evidence, not present-day quality. Keep the catalog, export, category pages, `index/categories.json`, README counts, and `llms.txt` in sync; regenerate CSV and navigation with `python scripts/export_csv.py` and `python scripts/generate_indexes.py`. Run `python scripts/validate.py` and `python -m unittest discover -s tests -v`. Use `CONTRIBUTING.md` for contribution rules.
+The canonical catalog is `resources.json`; `resources.csv` is a lossless, derived export, and `index/resources.compact.json`, `index/by-category/`, and `index/category-groups.json` are derived AI views. The current record contract is documented in `index/resource-schema.json`, with discovery facets in `index/taxonomy.json`. Stable `resource_id` and `category_id` values identify records and categories; topic, language, platform, use-case, audience, and resource-type facets are conservative discovery hints, not guarantees. `verification_level` describes recorded evidence, not present-day quality. Keep the catalog, export, category pages, `index/categories.json`, README counts, and `llms.txt` in sync; regenerate CSV and navigation with `python scripts/export_csv.py` and `python scripts/generate_indexes.py`, and the AI views with `python scripts/generate_ai_views.py`. Run `python scripts/validate.py` and `python -m unittest discover -s tests -v`. Use `CONTRIBUTING.md` for contribution rules.
 
 Record what was actually checked. A user-supplied URL is not independently verified: mark it as such, leave `http_status` empty, and avoid implying a license, current feature set, or service availability. Preserve source and review caveats. Linked resources retain their own terms and licenses; this repository contains references and original descriptions, not copied third-party documentation or project code.
 
-The historical `AI_INDEX.md`, `index/master-manifest.json`, `index/registry-schema.json`, and `manifests/resources.json` are retained as legacy/research artifacts and are not authoritative for the active catalog. Do not merge their data automatically: some paths are missing, schemas differ, and older manifests contain stale or template-like entries.
+Everything under `archive/legacy-template/` is archived placeholder material: generated template entries whose resources and file paths do not exist, plus an incompatible old schema. It is not authoritative, must never be cited or recommended from, and must not be merged into the active catalog.
 
 Run `python scripts/validate.py` before finishing catalog changes. It checks schema/taxonomy conformity, stable and related IDs, HTTPS syntax, duplicate records, category coverage, generated indexes, CSV/JSON agreement, local Markdown links, and accidental local paths. It does not access the network or verify claims.

AI_INDEX.md

@@ -1,30 +1,49 @@
-# AI Resource Index
-
-## 1. Win32 APIs & Native Windows Systems
-* **Scope**: Low-level Windows integration, hooks, registry manipulation, and window management.
-* **Key Files**: 
-  * `modules/win32-apis/process_monitor.rs` - Real-time process creation/termination tracking.
-  * `modules/win32-apis/window_hooks.cpp` - Low-level keyboard/mouse hooks and subclassing templates.
-
-## 2. Rust Tooling & Portable Utilities
-* **Scope**: Zero-dependency, lightweight, high-performance systems engineering.
-* **Key Files**:
-  * `modules/rust-tooling/installer_tracker/` - Filesystem change capture engine (akin to InstallScope core logic).
-  * `modules/rust-tooling/portable_packager/` - Automated PortableApps structure generator.
-
-## 3. Automation & Scripting
-* **Scope**: Native batch utilities, PowerShell deployment scripts, and zero-telemetry environment tweaks.
-* **Key Files**:
-  * `modules/automation/win11_debloat.ps1` - Safe, modular component removal script.
-  * `modules/automation/env_setup.bat` - Portable developer workstation initialization.
-
-## 4. Agent Skills & Manifests
-* **Scope**: Reusable modular prompts, context tokens, and capability definitions.
-* **Key Files**:
-  * `modules/agent-skills/npx-skills/` - Executable capability integrations.
-  * `modules/agent-skills/manifests/` - Structured capability schemas.
-
-## 5. UI/UX & Design Systems
-* **Scope**: High-contrast interfaces, dark mode templates, and clean aesthetic layouts.
-* **Key Files**:
-  * `modules/ui-ux/red_black_theme.json` - High-contrast visual styling configuration.
+# AI index — fastest path through this repository
+
+This repository is a curated pointer library: 2,202 entries, 2,198 distinct
+URLs, 97 categories. It contains references and original notes, not
+third-party code. Entries are discovery hints, not endorsements — verify
+maintenance, license, and compatibility at the primary source.
+
+## Read in this order (smallest first)
+
+1. **This file** — pick a domain below.
+2. **`index/category-groups.json`** — the 97 categories rolled up into
+   12 domains, with counts and category IDs.
+3. **One category shard** — `index/by-category/<category_id>.json` holds
+   only that category's records (name, URL, purpose, facets). Most shards
+   are a few KB; load one, not the whole catalog.
+4. **Only if you need provenance** — `resources.json` is the canonical
+   full catalog (2.3 MB, adds source, check date, review notes per record).
+   `index/resources.compact.json` is the same catalog at ~38% of the size
+   when you genuinely need every category at once.
+
+Single-file alternative: `llms-full.txt` concatenates every category page and catalog for one-shot ingestion.
+Local alternative: `python scripts/search.py "your query"` with
+`--topic`, `--type`, `--use-case`, `--language`, or `--platform` filters.
+
+## Domains
+
+| Domain | Resources | Start with |
+|---|---|---|
+| ai | 343 | `ai-mcp-servers-and-tool-use`, `ai-machine-learning-and-local-inference`, `ai-agent-frameworks-and-orchestration` |
+| windows | 232 | `windows-native-libraries`, `windows-diagnostics`, `windows-desktop-ui` (47 small Windows categories — most hold under 10 entries) |
+| languages-runtimes | 328 | `python-rust-and-go-building-blocks`, `languages-and-runtimes`, `niche-language-ecosystems` |
+| web-api-architecture | 245 | `api-contracts-clients-and-networking`, `web-interfaces-and-application-frameworks` |
+| cloud-infra-devops | 211 | `infrastructure-as-code-and-cloud-native-operations`, `cloud-infrastructure-and-deployment` |
+| security | 143 | `security-identity-and-supply-chain`, `security-testing-defense-and-supply-chain-assurance` |
+| data | 141 | `databases-data-access-and-analytics`, `data-science-and-engineering-tools` |
+| design-ux | 125 | `design-systems-and-design-engineering`, `ui-design-inspiration-and-reference` |
+| media-games-docs | 125 | `game-creation-assets-and-automation`, `documents-ocr-and-file-formats`, `world-of-warcraft-addon-development` |
+| desktop-mobile | 121 | `android-apk-development-and-analysis`, `desktop-mobile-and-device-sdks` |
+| testing-quality | 109 | `test-automation-and-quality-engineering`, `testing-verification-and-performance` |
+| operations-tools | 79 | `observability-debugging-and-operations`, `backup-storage-transfer-and-resilience` |
+
+Full category list with counts: `categories/README.md` or `llms.txt`.
+Human-readable category pages: `categories/<category_id>.md`.
+Long-form catalogs with provenance and cautions: `catalogs/`.
+
+## Do not use
+
+`archive/legacy-template/` holds generated placeholder manifests and an
+old index describing files that never existed. Never cite or merge it.

CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 Add resources that solve a concrete development need. Prefer canonical upstream links and concise original descriptions: what it is useful for, where it fits, and meaningful constraints. Inclusion is not an endorsement or adoption audit.
 
-Update the appropriate full catalog and `resources.json`, preserving each existing record and its provenance. For new records, provide the original evidence and descriptions; generate stable IDs and conservative derived facets with `python scripts/enrich_resources.py --apply`, then regenerate `resources.csv` with `python scripts/export_csv.py` and navigation with `python scripts/generate_indexes.py`. Use `index/resource-schema.json` and `index/taxonomy.json` as the active metadata contract. Do not assign unsupported language/platform claims; leave uncertain facets empty or flag them for review. `scripts/enrich_resources.py --check` verifies the derived metadata is present; the explicit `--refresh-topics`, `--refresh-types`, and `--refresh-verification` modes recalculate only those derived fields when their rules change. Preserve different interfaces to the same project when they serve different purposes; do not count raw entries as unique projects. Category page names should be lowercase hyphenated names.
+Update the appropriate full catalog and `resources.json`, preserving each existing record and its provenance. For new records, provide the original evidence and descriptions; generate stable IDs and conservative derived facets with `python scripts/enrich_resources.py --apply`, then regenerate `resources.csv` with `python scripts/export_csv.py`, navigation with `python scripts/generate_indexes.py`, and the AI views (compact catalog, category shards, domain rollup) with `python scripts/generate_ai_views.py`. Use `index/resource-schema.json` and `index/taxonomy.json` as the active metadata contract. Do not assign unsupported language/platform claims; leave uncertain facets empty or flag them for review. `scripts/enrich_resources.py --check` verifies the derived metadata is present; the explicit `--refresh-topics`, `--refresh-types`, and `--refresh-verification` modes recalculate only those derived fields when their rules change. Preserve different interfaces to the same project when they serve different purposes; do not count raw entries as unique projects. Category page names should be lowercase hyphenated names.
 
 Record the date, source, and scope of actual verification. A successful HTTP request only verifies reachability. Label archived, dormant, unmaintained, mirrored, or service-bound resources honestly and cite upstream evidence. Describe a potential revival separately from a tested release. Keep existing WR task IDs stable; the included backlog is a dated snapshot, not a promise of ongoing project work.
 

README.md

@@ -1,5 +1,9 @@
 # Useful Development Resources
 
+## TL;DR
+
+A curated library of 2,202 development resources (repositories, SDKs, APIs, libraries, tools) organized into 97 categories, built so an AI assistant can find the right resource for a project without reading the whole library. **AI assistants: start with [AI_INDEX.md](AI_INDEX.md)** — one page that maps your task to the one or two category files worth opening.
+
 A project-aware resource library for developers and their AI assistants, curated by Sean Shaffer. Find useful repositories, SDKs, APIs, libraries, diagnostics, build tools, and deployment resources for the work you are actually doing.
 
 **2202 catalog entries · 2198 distinct URLs · 97 purpose-based categories.** Includes 459 general resources, 232 Windows repositories, 301 AI resources, 200 development resources added 2026-10-23, two 500-resource development expansions added 2026-10-27, and 10 user-requested design-reference links. Entries overlap; distinct URLs do not necessarily mean distinct projects.
@@ -25,6 +29,7 @@ An assistant needs browsing, repository access, or a downloaded copy to read the
 - [Windows development catalog](catalogs/windows-development-repositories.md): native APIs, interop, DLLs, diagnostics, UI, graphics, devices, build and packaging, plus revival candidates.
 - [Windows revival backlog](catalogs/windows-development-todo.md): historical WR-001 through WR-010 snapshot, including completed win-pipe work and Microsoft IPC development status.
 - [JSON catalog](resources.json) / [CSV export](resources.csv): all entries with stable IDs, typed facets, and provenance.
+- [Compact catalog](index/resources.compact.json) / [domain rollup](index/category-groups.json): the AI-optimized views — same catalog at ~38% of the size, plus per-category shards in `index/by-category/` (one category at a time).
 - [Resource schema](index/resource-schema.json) / [facet taxonomy](index/taxonomy.json): the current record and facet contract.
 - [Generated category index](index/categories.json): category IDs, counts, paths, and resource membership.
 - [llms.txt](llms.txt): AI-readable index of the whole library.
@@ -45,7 +50,9 @@ python scripts/search.py "SQLite" --type repository --limit 10
 python scripts/search.py "browser test" --use-case test-and-verify-software --json
 python scripts/export_csv.py       # after editing resources.json
 python scripts/generate_indexes.py # after changing catalog categories/resources
+python scripts/generate_ai_views.py # after changing catalog categories/resources
 python scripts/validate.py
+python scripts/link_check.py --only-unchecked   # reachability only; a monthly workflow checks every URL
 python -m unittest discover -s tests -v
 ```
 

archive/legacy-template/AI_INDEX-original.md

@@ -0,0 +1,30 @@
+# AI Resource Index
+
+## 1. Win32 APIs & Native Windows Systems
+* **Scope**: Low-level Windows integration, hooks, registry manipulation, and window management.
+* **Key Files**: 
+  * `modules/win32-apis/process_monitor.rs` - Real-time process creation/termination tracking.
+  * `modules/win32-apis/window_hooks.cpp` - Low-level keyboard/mouse hooks and subclassing templates.
+
+## 2. Rust Tooling & Portable Utilities
+* **Scope**: Zero-dependency, lightweight, high-performance systems engineering.
+* **Key Files**:
+  * `modules/rust-tooling/installer_tracker/` - Filesystem change capture engine (akin to InstallScope core logic).
+  * `modules/rust-tooling/portable_packager/` - Automated PortableApps structure generator.
+
+## 3. Automation & Scripting
+* **Scope**: Native batch utilities, PowerShell deployment scripts, and zero-telemetry environment tweaks.
+* **Key Files**:
+  * `modules/automation/win11_debloat.ps1` - Safe, modular component removal script.
+  * `modules/automation/env_setup.bat` - Portable developer workstation initialization.
+
+## 4. Agent Skills & Manifests
+* **Scope**: Reusable modular prompts, context tokens, and capability definitions.
+* **Key Files**:
+  * `modules/agent-skills/npx-skills/` - Executable capability integrations.
+  * `modules/agent-skills/manifests/` - Structured capability schemas.
+
+## 5. UI/UX & Design Systems
+* **Scope**: High-contrast interfaces, dark mode templates, and clean aesthetic layouts.
+* **Key Files**:
+  * `modules/ui-ux/red_black_theme.json` - High-contrast visual styling configuration.