Skip to content

[Python][Packaging] Revisit Windows wheel Docker setup #45156

Closed

Description

@pitrou

Describe the enhancement requested

Our Windows wheel build setup currently uses Docker images based on https://hub.docker.com/r/abrarov/msvc-2019

This image is old and contains unwanted software.

We should instead try to build our own images from scratch.
It's possible to install the VS Build Tools in a container image:
https://learn.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2022
https://learn.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2022

Initial research suggests this command:

vs_buildtools.exe --quiet --wait --norestart \
    --add Microsoft.VisualStudio.Component.VC.CoreBuildTools \
    --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 \
    --installPath c:\tt

Component(s)

Continuous Integration, Packaging, Python

Activity

  1. pitrou commented on Jan 2, 2025

    @pitrou
    MemberAuthor
  2. amoeba commented on Feb 5, 2025

    @amoeba
    Member

    I'm looking at this today and have made some progress. My image so far is only 7.57GB whereas the existing wheel image is 26.8GB which is a nice savings. So far I have,

    # escape=`
    
    # build using
    # docker build -t buildtools -m 2GB --file buildtools.dockerfile .
    
    # Use the latest Windows Server Core 2022 image.
    FROM mcr.microsoft.com/windows/servercore:ltsc2022
    
    # Restore the default Windows shell for correct batch processing.
    SHELL ["cmd", "/S", "/C"]
    
    RUN `
      # Download the Build Tools bootstrapper.
      curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
      `
      # Install Build Tools with the Microsoft.VisualStudio.Workload.AzureBuildTools workload, excluding workloads and components with known issues.
      && (start /w vs_buildtools.exe --quiet --wait --norestart --nocache `
      --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" `
      --add Microsoft.VisualStudio.Component.VC.CoreBuildTools `
      --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
      || IF "%ERRORLEVEL%"=="3010" EXIT 0) `
      `
      # Cleanup
      && del /q vs_buildtools.exe
    
    SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
    
    # Install choco CLI
    # from https://chocolatey.org/install#completely-offline-install
    RUN `
      Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
    
    # Install CMake and Ninja
    # TODO: ARG cmake=3.31.2
    RUN choco install --no-progress -r -y cmake --version=3.31.2 --installargs 'ADD_CMAKE_TO_PATH=System'
    RUN choco install --no-progress -r -y gzip wget ninja
    
    # Define the entry point for the docker container.
    # This entry point starts the developer command prompt and launches the PowerShell shell.
    ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\Common7\\Tools\\VsDevCmd.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]
    

    This currently starts with a warning "The system cannot find the file specified." which I think means more components are needed so I'm looking at that now.

  3. pinned this issue on Feb 5, 2025
  4. amoeba commented on Feb 5, 2025

    @amoeba
    Member

    Installing a Win10/11 SDK fixes the warning at startup, --add Microsoft.VisualStudio.Component.Windows11SDK.

  5. pitrou commented on Feb 5, 2025

    @pitrou
    MemberAuthor

    Thanks a lot for working on this @amoeba ! Just a question: is there a reason to pin this issue?

  6. unpinned this issue on Feb 5, 2025
  7. amoeba commented on Feb 5, 2025

    @amoeba
    Member

    Hrm, no and I'm not sure how I did that. Maybe there's a shortcut? Either way, it's unpinned now.

  8. self-assigned this
    on Feb 6, 2025
  9. added a commit that references this issue on Feb 25, 2025
  10. added this to the 20.0.0 milestone on Feb 25, 2025
  11. amoeba commented on Feb 25, 2025

    @amoeba
    Member

    Issue resolved by pull request 45442
    #45442

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions