Skip to content

Executor: Support numeric list arguments - #877

Draft
gusui-msft wants to merge 1 commit into
mainfrom
fix/executor-numeric-lists-20261014
Draft

gusui-msft wants to merge 1 commit into
mainfrom
fix/executor-numeric-lists-20261014

Conversation

@gusui-msft

Copy link
Copy Markdown
Contributor

Description
The executor supports list-valued parameters but joins their elements as strings without conversion. Valid numeric lists such as msg_size: [8, 16] raise TypeError before the benchmark starts.

Major Revision

  • Serialize supported string/int/float list elements to command-line tokens.
  • Explicitly log and reject nested, null and boolean list elements instead of silently stringifying unsupported values.
  • Test plain Python dictionaries and actual OmegaConf ListConfig values through argparse round trips.

Minor Revision

  • Preserve existing string lists, empty lists, scalar values, scalar boolean flags and omitted null scalars.
  • Document the supported list element types in the conversion method.

Validation

  • Base: 3e8f2b5e66fb65ef08bded80e693d7537728bee4.
  • Before: the numeric-list regression failed; existing and invalid-input controls passed.
  • After: tests/executor/test_executor.py -k get_arguments — 3 passed, covering both containers and integer/float/string/flag/empty/invalid cases.
  • Focused flake8 and git diff --check passed.
  • No benchmark process, GPU or RDMA workload was launched. Draft pending maintainer/CI review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 14, 2026 20:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused implementation matches the requirements, includes regression coverage and has no identified blocking issues.

0 open findings

What changed in this PR

Fixes executor argument conversion so numeric lists can reach benchmark argument parsers without raising TypeError.

Changes:

  • Converts string, integer and float list elements to argument tokens.
  • Explicitly rejects unsupported list elements.
  • Adds dictionary and OmegaConf round-trip tests while preserving existing argument behavior.
File Description
tests/​executor/​test_executor.py Tests supported lists, rejected elements and existing argument behavior.
superbench/​executor/​executor.py Validates, serializes and documents supported list elements.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.32%. Comparing base (3e8f2b5) to head (ea80b65).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #877      +/-   ##
==========================================
+ Coverage   86.31%   86.32%   +0.01%     
==========================================
  Files         107      107              
  Lines        8256     8258       +2     
==========================================
+ Hits         7126     7129       +3     
+ Misses       1130     1129       -1     
Flag Coverage Δ
cpu-python3.12-unit-test 71.91% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants