GitHub Actions Runner Controller vs your local network

[GitHub Actions workflow services troubleshooting]

If you use GHA workflows with services inside your own managed runners you might see some interesting behavior every time you run a job (which means it can often fail without any obvious reasons). GHA services run in a customer docker network within your Runner whereas the Docker daemon running as DinD will create a new network with the following command:

Create local container network
 /usr/local/bin/docker network create --label 60e226 github_network_3090e8095e8749b6946b408fe63e5944        

If not specified, docker daemon will pick from existing private network ranges. This can lead to conflicts in networking with unpredictable results (e.g. external networking not working) if it picks the same networks as your cluster is using. Easy fix which is buried in actions-runner-controller code is to use

DOCKER_DEFAULT_ADDRESS_POOL_BASE and DOCKER_DEFAULT_ADDRESS_POOL_SIZE environment variables [1] [2].

Thanks to Marian Soltys and Tomas Hulata for helping with troubleshooting this issue!

[1] https://github.com/actions/actions-runner-controller/pull/1971

[2] https://github.com/actions/actions-runner-controller/blob/3ede9b5a0159a5e0703ccae6eebfdc89defe2b8f/docs/using-entrypoint-features.md