Skip to content

Provide a command line tool to run jobs #103

Closed

Description

@hategan

Swift/T, and other non-python or shell based projects, in order to use PSI/J, would need a command-line interface to running jobs. This issue is here to discuss the requirements, come up with a design, and track the implementation.

  • an initial task would be to attempt to gather requirements from teams that are potential users of such a feature (@j-woz mentioned one such team).
  • an initial rough design is based on the idea that one needs to communicate a JobSpec (and properties) and executor + config from one process to the other; this would be done through some serialization mechanism, which could take the form of:
    • command line arguments (e.g., --spec.attributes.duration=...)
    • json
  • we talked about a sync vs async interface and concluded that the biggest benefit/cost ratio would be achieved with a sync implementation, since more complex projects would use higher level tooling; for reference:
    • sync here means run-job ... which exits when the job completes
    • async implies a set of tools, such as submit-job, cancel-job, job-status; there is considerably more complexity in implementing these, at least in layer 0 (basically these would roughly be a local version of layer 1)

Activity

  1. self-assigned this
    on Feb 4, 2022
  2. wilke commented on Feb 25, 2022

    @wilke
    Collaborator

    Running into following error:

    Traceback (most recent call last):
    File "scripts/import-export.py", line 36, in
    jex.submit(job)
    File "/home/wilke/miniconda3/envs/psi-j-py3.7/lib/python3.7/site-packages/psi_j_python-0.1-py3.7.egg/psij/executors/batch/batch_scheduler_executor.py", line 195, in submit
    raise SubmitException(ex.output) from None
    psij.exceptions.SubmitException: sbatch: error: invalid partition specified: None
    sbatch: error: Batch job submission failed: Invalid partition name specified

  3. hategan commented on Feb 28, 2022

    @hategan
    CollaboratorAuthor

    @wilke I'm not sure what that error has to do with this issue. If it is not related, can you file a separate issue?

  4. wilke commented on Mar 7, 2022

    @wilke
    Collaborator

    Depends on #161

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions