Skip to content

test_handler_clearml_image cannot finish #6148

Closed

Description

@wyli

Describe the bug
error https://github.com/Project-MONAI/MONAI/actions/runs/4397243843/jobs/7703476756

test_task_init (tests.test_handler_clearml_image.TestHandlerClearMLImageHandler) ... ClearML Task: created new task id=offline-05250354845641a39e84dc2a5f6655e7
ClearML running in offline mode, session stored in /github/home/.clearml/cache/offline/offline-05250354845641a39e84dc2a5f6655e7
2023-03-12 19:54:03,413 - clearml.Task - INFO - No repository found, storing script code instead
2023-03-12 19:54:03,416 - clearml.log - WARNING - Event reporting sub-process lost, switching to thread based reporting
2023-03-12 19:54:03,427 - clearml.Task - INFO - Waiting for repository detection and full package requirement analysis
2023-03-12 19:54:03,505 - clearml.Task - INFO - Finished repository detection and package analysis
Error: The operation was canceled.

this happens randomly (3 out 7 runs) with the github ci runner

https://github.com/Project-MONAI/MONAI/actions/workflows/docker.yml

Activity

  1. wyli commented on Mar 15, 2023

    @wyli
    ContributorAuthor

    cc @skinan we might need to revert #6013 if it's not stable in case this blocks the v1.2rc tagging scheduled 20th Mar.

  2. added
    bugSomething isn't working
    on Mar 15, 2023
  3. skinan commented on Mar 15, 2023

    @skinan
    Contributor

    @wyli, I think we might have got a solution for this issue. We are actively working on it. We expect to fix it within 1-2 days. Please, allow us some time.

  4. added a commit that references this issue on Mar 15, 2023
  5. wyli commented on Mar 17, 2023

    @wyli
    ContributorAuthor

    any updates on this @skinan? for now we mute the error 75cebe4 but the issue is still around I think.

  6. skinan commented on Mar 20, 2023

    @skinan
    Contributor

    @wyli , working on figuring it out. I am going to create another pull request to solve this issue.

  7. skinan commented on Mar 21, 2023

    @skinan
    Contributor

    #6212
    @wyli , please review the pull request for fixing this issue.

  8. added a commit that references this issue on Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions