Repository navigation
Working via docker.sock does not necessarily mean containers are running on localhost #90
Description
In some build setups it may be the case that the build is triggered via Jenkins which itself runs in a docker container. With respect to testcontainers that means that a dockerized Jenkins runs unit tests which in turn start docker containers.
In order to be able to start a docker container from within a docker container, the currently proposed "solution" (it's merely a workaround) is to mount the file /var/run/docker.sock and the binary /usr/bin/docker into the Jenkins container.
This works fine with testcontainers. It detects a working docker.sock file and continues with starting containers. However if those containers do expose ports, testcontainers seems to wait for the port to become available. In the scenario described above it waits on localhost:port if docker communication via UNIX socket has been detected implying that if a socket is used, then the docker daemon must also run on localhost.
This implication is not true when doing docker in docker by the help of mounted socket files.
As a result, testcontainers times out waiting for the exposed port to become available.
Our current workaround for this is to not mount the socket and the binary but instead installing docker client into the container and passing the environment variables DOCKER_HOST, DOCKER_TLS_VERIFY and DOCKER_CERT_PATH to the Jenkins container.
Is it possible to implement some kind of switch that is able to influence the "wait for port to become available" behavior?
Activity
rnorth commentedon Mar 15, 2016
Hi Jan,
Thanks for this - that's a good point. I'm wondering, since the port availability wait can tend to be quite important, if there might be another option.. How about an environment variable that would be able to override the container address detection?
e.g. in
DockerClientFactory.instance().dockerHostIpAddress():if using /var/run/docker.sock, container address = 'localhost' unless
TESTCONTAINERS_CONTAINER_HOSTis set, in which case return the value of thatThat way the wait-for-start behaviour can carry on working (using
getContainerIpAddress()), plus tests can remain independent (callinggetContainerIpAddress()themselves).Would that help in your case?
Thanks
Richard
JanMosigItemis commentedon Mar 15, 2016
That sounds like it would solve the problem. However I tend to like properties more than environment variables. This way it's easier to integrate with Jenkins and Maven, e. g. I can run a build like
And it doesn't pollute the build server's environment.
If such a feature is going to be integrated it may be a good idea to mark it as experimental for some time since I am not exactly sure if it does really make sense that way. I mean: If it wasn't for the docker in docker setup, then socket->localhost, ip->someOtherHost holds true und is perfectly reasonable. The new feature may cause things to become more complicated (also more powerful) and it must be seen if similar situations do exist where it comes in handy. Maybe provisioning Jenkins Slaves with Docker or running a whole container as test would be another use case for this.
rnorth commentedon Mar 15, 2016
Yes, I think I'd probably want an equivalent system property for any environment variable setting anyway, so that would be OK.
I do think it is worth thinking more about where this would be useful, as you suggest. Another scenario I can imagine that might need something similar would be use of Docker Swarm. In that case the env var/property would be too simplistic to relate to where the container actually ends up. So maybe something programmatic might be a better idea - at least gives users the option of implementing more sophisticated behaviour when it's genuinely needed...
I'll have a bit more of a think about it and maybe propose some other approaches tomorrow.
Thanks for your input!
Richard
rnorth commentedon Mar 20, 2016
I'm thinking I'll keep this reasonably simple for now:
TESTCONTAINERS_CONTAINER_HOSTor sysproptestcontainers.container.hostis set, that value will always be returned fromgetContainerIpAddress()(no matter what)DOCKER_HOSTaddress unless using a unix socket, in which case it will return localhostI think this gives necessary flexibility without complicating things too much...
This doesn't cater for the swarm example, but in such a scenario
getContainerIpAddress()is unlikely to be useful anyway - there's only so much it can do. Instead some other project/environment specific method should be written separately and used in its place, outside of the testcontainers library.I'll set this up in a branch for review.
Refs #90: Proposed solution for environments where container host can…
rnorth commentedon Mar 26, 2016
Closing this in conjunction with PR #92 for the time being.