Skip to content

Add LLM gateway routing and fallback skill and prompt - #90

Closed
rmkr-dev wants to merge 3 commits into
mainfrom
cursor/add-llm-gateway-routing-fallback-5f2e
Closed

rmkr-dev wants to merge 3 commits into
mainfrom
cursor/add-llm-gateway-routing-fallback-5f2e

Conversation

@rmkr-dev

@rmkr-dev rmkr-dev commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Adds a Skillbook entry and a matching Promptbook entry for production routing and fallback on an LLM gateway.

Skill: llm-gateway-routing-fallback-guardrails

  • Alias to ordered targets (provider, model or deployment, region), with priority or weighted selection and sticky versus per-request
  • Health-aware skip, then fall through on timeout, 429, 5xx, connection errors, and auth or deployment-config failures
  • 400, 413, and 422 stop the walk unless that target sets fallbackOnBadRequest (default off), and those 4xx responses do not open the breaker
  • When every target fails, return the last meaningful upstream status. A caller 4xx stays that 4xx rather than a synthetic 502
  • No provider switch after the first streamed token; cancel the upstream and emit stream_interrupted
  • One adapter per provider family, secrets by reference, per-attempt metrics under one correlation id
  • Two-stub tests for fallback order, no fallback on 400, last 4xx preserved, and no mid-stream switch

This skill owns outbound model routing and fallback. Cost and latency budgets, prompt injection, eval gates, HTTP client pools, circuit-breaker primitives, inbound rate limits, and Idempotency-Key stay with the existing skills.

Prompt: add-llm-gateway-routing-fallback

  • Placeholders for the gateway name, providers, aliases, streaming, SLO, and observability stack
  • Numbered requirements from inventory through routing, failure classes, final status, streaming, adapters, tests, and rollout notes
  • Deliverable is the inventory, the config and code diff, the tests, and rollout notes

Catalogs go to 53 skills / 53 prompts. node scripts/site-check.mjs passes locally.

Open in Web Open in Cursor 

rmkr-dev and others added 3 commits November 7, 2026 05:15
Co-authored-by: Ramkumar Ethiraj <rmkr-dev@users.noreply.github.com>
Co-authored-by: Ramkumar Ethiraj <rmkr-dev@users.noreply.github.com>
Co-authored-by: Ramkumar Ethiraj <rmkr-dev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant