Repository navigation
Cannot create directory ‘/opt/hostedtoolcache’ after recent release #459
Description
Description:
The following error start happening after the recent action release:
2022-07-12T14:12:47.7158848Z ##[group]Run actions/setup-python@v4
2022-07-12T14:12:47.7159230Z with:
2022-07-12T14:12:47.7159527Z python-version: 3.x
2022-07-12T14:12:47.7160059Z token: ***
2022-07-12T14:12:47.7160394Z update-environment: true
2022-07-12T14:12:47.7162682Z ##[endgroup]
2022-07-12T14:12:48.0166854Z Version 3.x was not found in the local cache
2022-07-12T14:12:48.3072207Z Version 3.x is available for downloading
2022-07-12T14:12:48.3074130Z Download from "https://github.com/actions/python-versions/releases/download/3.10.5-2452607758/python-3.10.5-linux-20.04-x64.tar.gz"
2022-07-12T14:12:49.1279090Z Extract downloaded archive
2022-07-12T14:12:49.1380950Z [command]/usr/bin/tar xz --warning=no-unknown-keyword -C /actions-runner/_work/_temp/2d83bfeb-a252-4083-95bf-6f30cffc5507 -f /actions-runner/_work/_temp/c2619dbf-23ad-46f0-9b07-baf06258c9c8
2022-07-12T14:12:52.2679935Z Execute installation script
2022-07-12T14:12:52.2887302Z Check if Python hostedtoolcache folder exist...
2022-07-12T14:12:52.2888180Z Creating Python hostedtoolcache folder...
2022-07-12T14:12:52.2921018Z ##[error]mkdir:
2022-07-12T14:12:52.2937518Z ##[error]cannot create directory ‘/opt/hostedtoolcache’
2022-07-12T14:12:52.2939104Z ##[error]: Permission denied
2022-07-12T14:12:52.2941609Z ##[error]The process '/usr/bin/bash' failed with exit code 1
2022-07-12T14:12:52.3294942Z Post job cleanup.
Action version:
v4
Platform:
- Ubuntu
- macOS
- Windows
Runner type:
- Hosted
- Self-hosted
Tools version:
- 3.10.5
Repro steps:
setup-python:
runs-on: self-hosted
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.x
Expected behavior:
Won't error.
Actual behavior:
It errors.
Activity
IvanZosimov commentedon Jul 12, 2022
Hi, @kenorb 👋 Thank you for that issue, we will investigate it and get back to you with updates.
dmitry-shibanov commentedon Jul 12, 2022
Hello @kenorb. Could you please take a look at this pull request. I think it can be related to your issue.
techman83 commentedon Jul 13, 2022
@dmitry-shibanov yes, it would be directly related. RUNNER_TOOL_CACHE on GitHub hosted vs Self Hosted is different.
Self Hosted
Hosted
This variable is a default, I'd expect whilst we might want to make sure it exists, replacing it with a fixed path might cause consumers of the action to experience unexpected behaviour.
We're experience similar issues, as the runner is intentionally not running as root, so will not be able to write to that path.
dsame commentedon Jul 13, 2022
@techman83
The best solution is just to create
/opt/hostedtoolcachewith the write permissions on the self-hosted manually as it is said in README:Alternatively, you can choose the installation directory to any location by setting AGENT_TOOLSDIRECTORY env. variable, but it should not be used because of:
techman83 commentedon Jul 13, 2022
@dsame yes, that is a work around. Why would an action choose to override this variable as opposed to being a consumer of it? If for instance there is another workflow step that uses this path before, and after the python actions setup, the behaviour would be unexpected.
I'm not sure I follow what the benefit of overriding this default is, and what is gained by trying to assume what GitHub would have set it to.I see what you are saying here, that it relates to actions/python-versions. However reading through the source code, it uses RUNNER_TOOL_CACHE environment variable if AGENT_TOOLSDIRECTORY is not provided.
So, realistically there shouldn't be a requirement to override this variable. Rather when AGENT_TOOLSDIRECTORY is specified, use that path instead of RUNNER_TOOL_CACHE.
Along with that, actions/python-versions is likely to be run before actions/setup-python, so even allowing write to
/opt/hostedtoolcachewon't help, as the build output will actually be in/home/actions/actions-runner/_work/_tool.dsame commentedon Jul 13, 2022
@techman83
This is not workaround - this is the necessary host agent preparation step described in the README. The github hosted images already have the directory, but for self-images an user must create it manually.
The reason why the python must be installed there described in the README
Unfortunatelly
RUNNER_TOOL_CACHEon the self-hosted agent points to/home/actions/actions-runner/_work/_toolwhich leads to unpredictable issues and this is why we have to override it to point to the correct location/opt/hostedtoolcacheIn the previous versions of
setup-pythonit was necessary to assign AGENT_TOOLSDIRECTORY explicitly to/opt/hostedtoolcachebut it seemed to be error prone redundant requirement and now AGENT_TOOLSDIRECTORY has default value/opt/hostedtoolcacheand should never be changed.But in some environments an user may have not ability to create
/opt/hostedtoolcachedirectory and then, assuming he knows that he is doing, it is still possible to install python to other location.Thank you for your question, feel free to ask more if you still have some problems - it really helps to improve the code.
techman83 commentedon Jul 14, 2022
@dsame - Apologies if I came across as blunt yesterday, still getting over covid and had already shaved many yaks. Having read through
actions/python-versionsthis does not appear necessary. I have forked and done a quick hack to test my theory.Log
Files in the local docker container connected to my test org:
So to me it doesn't look necessary, and adds extra steps when configuring a self hosted runner, along with breaking existing workflows (though there is always that one guy). One of the benefits is normally that the runner is pretty self contained.
techman83 commentedon Jul 14, 2022
And for completeness to check hosted (I haven't checked self hosted Mac/Windows, I have not convenient access to either).
Linux Hosted:
Windows Hosted:
Mac Hosted:
dsame commentedon Jul 14, 2022
@techman83
As you already have noted, the python is installed in
_work/_tooldirectory ifAGENT_TOOLSDIRECTORYis not specified:ls -lah _work/_tool/Python/3.11.0-beta.4/This works in many cases, but the README warns:
compiled from source in /opt/hostedtoolcache/ with the [--enable-shared](https://github.com/actions/python-versions/blob/94f04ae6806c6633c82db94c6406a16e17decd5c/builders/ubuntu-python-builder.psm1#L35) flag, which makes them non-relocatableInstalling python in the directory other then
/opt/hostedtoolcache/works in many cases, but always there's a probability of unpredictable misbehaviour as soon as some shared libraries loaded. To avoid this the default installation forced to be/opt/hostedtoolcache/although with AGENT_TOOLSDIRECTORY it is still possible to choose another installation directory.Does it make sense now?
techman83 commentedon Jul 14, 2022
@dsame not really. I am aware of the warning, but this doesn't match the behaviour of the
actions/python-versionsthat this change has been introduced to address. The *nix helper in python-versions is:Internally they use a
TOOLCACHE_ROOTvariable, which they set toRUNNER_TOOL_CACHEwhenAGENT_TOOLSDIRECTORYis not provided. If the same is implemented here, actions-python will behave in the same manner, and respect the path that is configured as part of the self hosted runner options.Part of the use case (not one we utilise) is for runners without internet access. You can see references to the documentation for this path. Other use cases maybe mounting a shared volume for all your runners, so they can access the same tool cache.
Overriding this path within an action is unexpected and doesn't match GitHub's documented use of that variable.
To be explicit, that documentation specifically calls out
setup-pythonas a use case, which this is a breaking change for.fix: Self-Hosted Runner Tool Cache
23 remaining items
docs: Agent Tool Cache
fix: Self-Hosted Tool Cache
docs: Agent Tool Cache
fix: Self-Hosted Tool Cache
fix: Mac Toolpath
fix: Mac Toolpath
docs: Agent Tool Cache
fix: Self-Hosted Tool Cache
fix: Mac Toolpath
MarkArts commentedon Aug 2, 2022
We where having this issue while setting up our own runners with the ubuntu example in: https://github.com/philips-labs/terraform-aws-github-runner.
We could fix it by making these changes to the ubuntu example:
https://github.com/philips-labs/terraform-aws-github-runner/pull/2302/files
docs: Agent Tool Cache
fix: Self-Hosted Tool Cache
fix: Mac Toolpath