Sphinx gallery scraper is not parallel-safe

Sphinx Gallery now allows parallel execution of gallery examples. The plotly Orca renderer correctly outputs PNGs with the example filename as root, e.g., plot_my_example.png.

However, the scraper grabs all PNGs from the example directory. So, if other plotly examples execute simultaneously, those PNGs may be "stolen" by the scraper.

To fix the issue, the scraper would have to be modified from:

    examples_dir = os.path.dirname(block_vars["src_file"])
    pngs = sorted(glob(os.path.join(examples_dir, "*.png")))

to

    examples_dir = os.path.dirname(block_vars["src_file"])
    fn_root, _ = os.path.splitext(block_vars["src_file"])
    pngs = sorted(glob(os.path.join(examples_dir, f"{fn_root}_*.png")))

Note the more selective filter on the globbed PNGs.

For example this comes up frequently in scikit-image because we build the gallery in parallel e.g., see https://github.com/scikit-image/scikit-image/actions/runs/12680820918/job/35343334532#step:6:2316:

Details

Extension error:
Here is a summary of the problems encountered when running the examples:

Unexpected failing examples (1):

    ../examples/applications/plot_3d_interaction.py failed leaving traceback:

    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/sphinx_gallery/scrapers.py", line 387, in save_figures
        rst = scraper(block, block_vars, gallery_conf)
      File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/plotly/io/_sg_scraper.py", line 59, in plotly_sg_scraper
        shutil.move(png, this_image_path_png)
        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/shutil.py", line 876, in move
        copy_function(src, real_dst)
        ~~~~~~~~~~~~~^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/shutil.py", line 468, in copy2
        copyfile(src, dst, follow_symlinks=follow_symlinks)
        ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/shutil.py", line 260, in copyfile
        with open(src, 'rb') as fsrc:
             ~~~~^^^^^^^^^^^
    FileNotFoundError: [Errno 2] No such file or directory: '/Users/runner/work/scikit-image/scikit-image/doc/examples/applications/plot_3d_interaction.png'

@stevanv

Thanks for the bug report @stevanv - we're no longer supporting Orca, but I'll see if we can get someone to look at this once the current release goes out the door. Cheers - @gvwilson

@gvwilson

Thanks, @gvwilson.

Unfortunately, we haven't been able to get our CI to work with the new Kaleido, but I'll try the latest version today. Once a new rc of plotly 6 is out, we'll attempt to switch to that, which may also resolve our problem.

Is this related to #4722 ?

Navigation Menu

Repository navigation

Sphinx gallery scraper is not parallel-safe #4959

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Sphinx gallery scraper is not parallel-safe #4959

Description

Activity

lagru commented on Jan 9, 2025

gvwilson commented on Jan 9, 2025

stefanv commented on Jan 9, 2025

gvwilson commented on Jun 17, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions