Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/ci-ready.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@ jobs:
run: |
python -m venv "$RUNNER_TEMP/wheel-env"
wheels=("$RUNNER_TEMP"/dist/*.whl)
"$RUNNER_TEMP/wheel-env/bin/python" -m pip install "${wheels[0]}[skills]"
cp tests/check_installed.py "$RUNNER_TEMP/check_installed.py"
"$RUNNER_TEMP/wheel-env/bin/python" -m pip install "${wheels[0]}"
cp tests/check_installed.py tests/check_task_installed.py tests/fixtures/task-v1.yaml "$RUNNER_TEMP/"
cd "$RUNNER_TEMP"
env -u PYTHONPATH "$RUNNER_TEMP/wheel-env/bin/python" -I check_installed.py
env -u PYTHONPATH "$RUNNER_TEMP/wheel-env/bin/python" -I check_task_installed.py

full-check:
strategy:
Expand Down
10 changes: 5 additions & 5 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,11 +221,11 @@ Keep the existing four lanes:
| [Merge](../.github/workflows/ci-merge.yml) | Manifest check and lightweight CPU smoke on main |
| [Nightly](../.github/workflows/ci-nightly.yml) | Scheduled lint/tests, evidence layout and label drift |

Ready CI also builds the source distribution and wheel, installs the wheel with
the optional `skills` parser in a fresh virtual environment, and runs
`tests/check_installed.py` outside the checkout with `PYTHONPATH` unset and Python
isolated mode enabled. This checks packaged knowledge, skill bodies, licenses,
helpers and CLI reads without installing any inference engine.
Ready CI builds the source distribution and wheel and installs the wheel with
its core dependencies in a fresh virtual environment. It runs
`tests/check_installed.py` and `tests/check_task_installed.py` outside the checkout
with `PYTHONPATH` unset and Python isolated mode enabled. The checks cover packaged
knowledge, skills, task schema and template, and read-only CLI operations.
The ready test and installed-package jobs cover the declared minimum Python
3.10 alongside Python 3.11. This matrix does not certify later Python releases
or accelerator runtime compatibility.
Expand Down
15 changes: 12 additions & 3 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ those resources retain their repository owner and source attribution.

### Discover/read packaged or custom skills — `skills list` / `show` / `resource` / `reference`

- **Inputs:** `list` reads frontmatter and sidecar metadata; `show <skill>` selects one instruction body; `resource <skill> <relative-path>` and `reference <skill> <relative-path>` resolve one path. The default is the installed library's registered skills; an explicit `--root` selects a custom flat tree or a categorized library with `skill-index.json`. The optional `skills` extra supplies the YAML parser.
- **Inputs:** `list` reads frontmatter and sidecar metadata; `show <skill>` selects one instruction body; `resource <skill> <relative-path>` and `reference <skill> <relative-path>` resolve one path. The default is the installed library's registered skills; an explicit `--root` selects a custom flat tree or a categorized library with `skill-index.json`.
- **Scope:** complete packaged skills under `data/skill_library/`, or the explicitly selected custom root. The 21 English engine and research-router skills remain accessible through catalog records and library navigation, separately from the registry.
- **Output:** metadata listing, one instruction body, or a resolved path.
- **Side effects:** none. `resource`/`reference` do not load or execute the target.
Expand Down Expand Up @@ -190,9 +190,18 @@ normal scoped PR path.

## Experiments

### Read shared YAML input — `task schema`, `task template`, `task validate`, `task preview`

- **Inputs** The packaged schema/template, or a v1 YAML task path and optional preview overrides.
- **Output** Schema or template, validation result, or normalized values and required-field blockers for plan/run. Use `--json` for Agent handoff.
- **Side effects** Read-only. These commands run no probes or subprocesses and write nothing to the workspace or experiment directory.
- **Failure behavior** Exit 2 with a field or location for invalid input. A valid draft can have missing execution inputs.

See the [shared YAML task guide](task-input.md) for paths, merging and the CLI/Agent workflow.

### Inspect/prepare — `doctor`, `plan`, `--dry-run`

- **Inputs:** repository/backend/device flags; `plan` takes the full experiment specification.
- **Inputs:** repository/backend/device flags; `plan` takes the full experiment specification as flags or `--task` YAML.
- **Scope:** environment inspection and specification preview.
- **Output:** environment facts, or a preview of the RunSpec.
- **Side effects:** `doctor` may invoke bounded read-only probes; `plan`/`--dry-run` persist nothing.
Expand All @@ -205,7 +214,7 @@ compatibility.

### Execute/inspect — `run`, `status`, `report`

- **Inputs:** the full experiment specification (`run`); a run ID and output directory (`status`/`report`).
- **Inputs:** the full experiment specification as flags or `--task` YAML with `--confirm-acceptance` (`run`); a run ID and output directory (`status`/`report`).
- **Scope:** caller-supplied benchmark, candidate and validation commands under the experiment contract.
- **Output:** a run ID and evidence directory; recorded progress; an HTML report.
- **Side effects:** `run` executes the supplied commands and owns child-process cleanup; `report` writes HTML into the run directory.
Expand Down
17 changes: 8 additions & 9 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Quickstart

This release uses CLI args and a **local executor on a prepared machine**. Install it directly on the remote machine where the target engine already works. It does not set up models, CUDA/ROCm/CANN, services, SSH workers or autonomous Agents.
Use CLI flags or a [shared YAML task](task-input.md) with the **local executor
on a prepared machine**. Install InferRSI where the target engine already works.

## Install and smoke

Expand All @@ -14,11 +15,7 @@ inferrsi --help
inferrsi doctor --repo . --repo-type vllm_omni --backend cpu
```

Package-only knowledge commands (`knowledge`, `context`, package `collect`/`read`)
work with this base install. Reading skill libraries — the `skills` command and
any `--skill-root` selection — additionally requires
`python -m pip install '.[skills]'` (PyYAML); without it those operations fail
with that install hint instead of a traceback.
The base install supports packaged knowledge, skill-library reads and YAML tasks.

For the maintained model examples, see [Qwen B300](qwen-image-2.1-turbo-b300.md)
and [Qwen Ascend](qwen-image-2.1-turbo-ascend.md). Continue below for the generic
Expand Down Expand Up @@ -53,15 +50,15 @@ entry point, effect class and boundary:
```bash
inferrsi knowledge search 'Qwen-Image-2.1' --repo-type vllm_omni --backend ascend
inferrsi context --repo-type vllm_omni --backend cuda --output context.md
inferrsi skills list --json # requires python -m pip install '.[skills]'
inferrsi skills list --json
inferrsi read inferrsi.ascend.skills.ascend_graph_mode --material
```

Normalized Collect, exact-revision reads and resource access are available as
CLI commands and as the Python `KnowledgeReader`
([contract](knowledge-operations.md)); authorized local candidate Create/Update
remains a Python `KnowledgeStore` operation ([contract](knowledge-store.md)).
The `--skill-root` examples below require the `skills` extra from above:
To read from a checkout's skill root:

```bash
inferrsi collect 'Ascend OOM' --kind case --limit 3 --material --skill-root src/inferrsi/data/skill_library
Expand All @@ -88,8 +85,10 @@ examples above explicitly adds that root's skill and case IDs.
## The actual configuration interface

```text
task: schema | template | validate TASK.yaml | preview TASK.yaml --json
doctor: --repo --repo-type --backend --device-ids --device-model --python
plan/run: all doctor flags, plus
plan/run: --task TASK.yaml (--confirm-acceptance to execute) or the existing flags
all doctor flags, plus
--candidate-repo --scenario --mode --agent --executor local
--baseline-command --candidate-command --validation-command
--metric-key --metric-unit --comparison-scope --direction
Expand Down
73 changes: 73 additions & 0 deletions docs/task-input.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Shared YAML tasks

Use a v1 task file to give the CLI and an Agent the same input. A normal
InferRSI install includes the parser, [v1 schema](../src/inferrsi/data/task_input/schema-v1.json)
and [YAML template](../src/inferrsi/data/task_input/template-v1.yaml).

```bash
inferrsi task template > task.yaml
inferrsi task schema
inferrsi task validate task.yaml
inferrsi task preview task.yaml --json
```

An Agent can read or fill in the same template. Have it run the same validation
and JSON preview and review the effective values with you before proceeding.
A valid draft can omit any section or leave a field as null. The preview lists
missing fields by dotted path and sets `complete` when all required inputs are
present. Validation and preview read the file without invoking processes,
inspecting the runtime, or writing workspace or experiment files. Invalid input
exits with status 2.

| Section | Required for `plan` and `run` |
| --- | --- |
| `optimization` | Repository, model, framework (the CLI's repository type), workload, objective and SIAS plane |
| `environment` | Backend, baseline and candidate argv, benchmark repetitions and time budget |
| `acceptance` | Issue URL or local plan path, full baseline Git object ID, metric key, unit, direction and agreed minimum improvement percentage |

Use the schema for exact field names, types and CLI mappings. Supply any device
selectors or quality inputs the accepted plan requires. Set
`acceptance.require_quality: true` when the quality command is mandatory.
`acceptance.constraints` records additional plan requirements for review.
The existing runner evaluates its configured metric and validation command.

- Use a single UTF-8 YAML document up to 64 KiB. Use plain mappings and argv
lists of strings. Duplicate and unknown keys, aliases, anchors and explicit
tags are errors. Numeric inputs use decimal notation, booleans use
`true`/`false`, and strings that resemble these values should be quoted.
- YAML filesystem paths are relative to the file. Paths in explicit CLI flags
are relative to the caller's working directory. With a task file, the default
output directory is `runs` next to the YAML. A bare Python executable name is
resolved through the existing runtime lookup. Script paths and other argv
tokens keep the existing per-checkout working-directory semantics. Model
identifiers and HTTP(S) acceptance references stay as given.
- In preview, plan and run, explicit mapped CLI flags override YAML for
optimization and environment inputs. Built-in defaults fill omitted optional
fields. Acceptance settings come from YAML. A conflicting CLI acceptance
flag is rejected. Existing flag-only commands use their usual defaults.
- Keep credentials in the existing runtime channels. Review the YAML,
normalized preview and experiment records before sharing them.

After reviewing the exact commands and the effective values against the
accepted frozen gates, inspect the prepared environment and explicitly run
the experiment:

```bash
inferrsi plan --task task.yaml
inferrsi run --task task.yaml --confirm-acceptance
```

`plan` and `run --dry-run` use the existing environment probes, including
the configured Python interpreter. Task-file operations check the baseline
checkout's HEAD against the pinned object ID. A local acceptance reference
must point to an existing file. Pass `--confirm-acceptance` after checking
the effective values against the referenced frozen gates. InferRSI records
the confirmation and effective task. Review of the issue or plan is performed
by the caller or Agent. The [operation guide](operations.md#experiments) and
[evidence rules](contributing.md#evidence-and-claims) apply to the result.

The DSH draft format is tracked separately under
[#152](https://github.com/InferMatrixLab/InferRSI/issues/152). Check legacy
DSH files against the core schema and migrate incompatible fields before
using them with these commands. DSH integration uses the core contract and
shared fixtures when it advertises core v1 support.
9 changes: 5 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description = "CLI-first reproducible inference experiments and source-grounded
readme = "README.md"
license = "Apache-2.0"
requires-python = ">=3.10"
dependencies = []
dependencies = ["PyYAML>=6.0"]

[project.scripts]
inferrsi = "inferrsi.cli:main"
Expand All @@ -23,7 +23,7 @@ where = ["src"]
namespaces = false

[tool.setuptools.package-data]
inferrsi = ["data/**/*.json", "data/**/*.md", "data/**/LICENSE", "data/**/THIRD_PARTY_NOTICES", "data/**/scripts/*.py", "data/**/references/licenses/*.txt", "data/**/references/upstream-NOTICE", "data/dashboard/*.html"]
inferrsi = ["data/**/*.json", "data/**/*.md", "data/**/LICENSE", "data/**/THIRD_PARTY_NOTICES", "data/**/scripts/*.py", "data/**/references/licenses/*.txt", "data/**/references/upstream-NOTICE", "data/dashboard/*.html", "data/task_input/*.yaml"]

[tool.setuptools_scm]
version_file = "src/inferrsi/_version.py"
Expand All @@ -34,8 +34,9 @@ initial_version = "0.1.0"
prerelease = true

[project.optional-dependencies]
skills = ["PyYAML>=6.0"]
dev = ["pytest>=7.4", "ruff>=0.6", "PyYAML>=6.0"]
# Keep the former extra as an alias for existing installs.
skills = []
dev = ["pytest>=7.4", "ruff>=0.6"]
image = ["numpy>=1.24", "Pillow>=10", "scikit-image>=0.21"]

[tool.pytest.ini_options]
Expand Down
Loading
Loading