Build conda nightlies jobs are failing on main for aarch64 #659

Activity

1. raulcd

   added

   bugSomething isn't workingSomething isn't working

   on Apr 29, 2024
2. Michael-J-Ward commented on May 3, 2024

   

   Michael-J-Ward

   on May 3, 2024

   Last edited by Michael-J-Ward

   Contributor

   More actions

   Taking a look at the logs, a few things look off.

   1. conda-forge/linux-64 seems like it's the wrong cache for the linux-aarch64 job.
   2. warning libmamba Cache file was modified by another program makes me think concurrent jobs are modifying the same cache
   3. rust-std-x86_64 also seems like the wrong download, and that's the one that causes the mismatched hash error.

   Attempting to finalize metadata for datafusion
   conda-forge/linux-64                                        Using cache
   conda-forge/noarch                                          Using cache
   Reloading output folder: 
   /home/runner/work/datafusion-python/datafusion-python/packages
   warning  libmamba Cache file "/home/runner/conda_pkgs_dir/cache/09cdf8bf.json" was modified by another program
   
   ...
   
   Conda detected a mismatch between the expected content and downloaded content
   for url 'https://conda.anaconda.org/conda-forge/noarch/rust-std-x86_64-unknown-linux-gnu-1.77.2-h2c6d0dc_0.conda'.
     download saved to: /home/runner/conda_pkgs_dir/rust-std-x86_64-unknown-linux-gnu-1.77.2-h2c6d0dc_0.conda
     expected sha256: a482597672076f47c83d0dd3f204eb437007b99ada4d630d56fa64b4b193c5db
     actual sha256: 73f7537db6bc0471135a85a261798abe77e7e83794f945a0355c4068973f31f6

   The things I'd like to try, in order.

   1. set the concurrency to hard lock of 1 conda job at a time.
   2. upgrade miniconda action (v3 has automatic aarch detection) EDIT: looks like there's a PR for that and it failed build(deps): bump conda-incubator/setup-miniconda from 2.2.0 to 3.0.4 #658
   3. blow out the conda and start fresh
3. Michael-J-Ward

   added 4 commits that reference this issue on May 3, 2024

   ci: reduce concurrency to 1 globally for conda

   ...

   a3cc710

   ci: clean conda cache before building the packages

   ...

   7813897

   ci: reduce concurrency to 1 globally for conda

   ...

   6fbeea7

   ci: clean conda cache before building the packages

   ...

   51d5ad0
4. Michael-J-Ward commented on May 6, 2024

   

   Michael-J-Ward

   on May 6, 2024

   Last edited by Michael-J-Ward

   Contributor

   More actions

   Investigating further. The actual sha256sum that the CI report matches both what I calculate when downloading the files and what conda-forge lists.

   file: rust-std-aarch64-unknown-linux-gnu-1.77.2-hbe8e118_0.conda
   sha256sum: 9d583f04bfdbccc82ac2f0653de571f8371df04633727e714b71efd7e4a0140a
   
   file:  rust-std-x86_64-unknown-linux-gnu-1.77.2-h2c6d0dc_0.conda
   sha256: 73f7537db6bc0471135a85a261798abe77e7e83794f945a0355c4068973f31f6
   

   So, I tried cleaning out the cache, but that only caused the builds to break in a new way...

    Adding in variants from internal_defaults
   Adding in variants from config.variant
   Adding in variants from argument_variants
   Error: bad character '-' in package/version: publish-docs.1a240507

   Closing the PR for now.
5. Michael-J-Ward commented on May 8, 2024

   

   Michael-J-Ward

   on May 8, 2024

   Contributor

   More actions

   Ah, publish-docs is the tag @andygrove used to trigger the docs generation & publication.

   Apparently, conda doesn't like that for a package/version.
6. Michael-J-Ward

   added 2 commits that reference this issue on May 13, 2024

   ci: clean conda cache before building the packages

   ...

   aa49882

   ci: clean conda cache before building the packages

   ...

   a92d9ac
7. andygrove

   closed this as completedin #689on May 13, 2024