Best practices for building docker images with GitLab CI
Using a generic .gitlab-ci.yml file that you can drop in
March 4, 2019
Updated in 2021
At CALLR, we have been using GitLab for quite a while. We are also using more and more Docker containers. In this post, I’ll show you how we build docker images with a simple .gitlab-ci.yml file.
Let’s not waste any time.
The GitLab CI yaml configuration file
Here is a .gitlab-ci.yml file that you can drop in directly without any modification in a project with a working Dockerfile.
It will:
- build a docker image for each git commit, tagging the docker image with the commit SHA
- tag the docker image “latest” for the “master” branch
- keep in sync git tags with docker tags
All docker images will be pushed to the GitLab Container Registry.
| # This is a GitLab CI configuration to build the project as a docker image | |
| # The file is generic enough to be dropped in a project containing a working Dockerfile | |
| # Author: Florent CHAUVEAU <florent.chauveau@gmail.com> | |
| # Mentioned here: https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/ | |
| # do not use "latest" here, if you want this to work in the future | |
| image: docker:20 | |
| stages: | |
| - build | |
| - push | |
| variables: | |
| # fill those if you have a proxy in your environment | |
| http_proxy: "" | |
| https_proxy: "" | |
| no_proxy: "" | |
| # Use this if your GitLab runner does not use socket binding | |
| # services: | |
| # - docker:dind | |
| before_script: | |
| # docker login asks for the password to be passed through stdin for security | |
| # we use $CI_REGISTRY_PASSWORD here which is a special variable provided by GitLab | |
| # https://docs.gitlab.com/ce/ci/variables/predefined_variables.html | |
| - echo -n $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY | |
| Build: | |
| stage: build | |
| script: | |
| # fetches the latest image (not failing if image is not found) | |
| - docker pull $CI_REGISTRY_IMAGE:latest || true | |
| # builds the project, passing proxy variables, using OCI labels | |
| # notice the cache-from, which is going to use the image we just pulled locally | |
| # the built image is tagged locally with the commit SHA, and then pushed to | |
| # the GitLab registry | |
| - > | |
| docker build | |
| --pull | |
| --build-arg http_proxy=$http_proxy | |
| --build-arg https_proxy=$https_proxy | |
| --build-arg no_proxy=$no_proxy | |
| --cache-from $CI_REGISTRY_IMAGE:latest | |
| --label "org.opencontainers.image.title=$CI_PROJECT_TITLE" | |
| --label "org.opencontainers.image.url=$CI_PROJECT_URL" | |
| --label "org.opencontainers.image.created=$CI_JOB_STARTED_AT" | |
| --label "org.opencontainers.image.revision=$CI_COMMIT_SHA" | |
| --label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME" | |
| --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | |
| . | |
| - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | |
| # Here, the goal is to tag the "master" branch as "latest" | |
| Push latest: | |
| variables: | |
| # We are just playing with Docker here. | |
| # We do not need GitLab to clone the source code. | |
| GIT_STRATEGY: none | |
| stage: push | |
| only: | |
| # Only "master" should be tagged "latest" | |
| - master | |
| script: | |
| # Because we have no guarantee that this job will be picked up by the same runner | |
| # that built the image in the previous step, we pull it again locally | |
| - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | |
| # Then we tag it "latest" | |
| - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest | |
| # Annnd we push it. | |
| - docker push $CI_REGISTRY_IMAGE:latest | |
| # Finally, the goal here is to Docker tag any Git tag | |
| # GitLab will start a new pipeline everytime a Git tag is created, which is pretty awesome | |
| Push tag: | |
| variables: | |
| # Again, we do not need the source code here. Just playing with Docker. | |
| GIT_STRATEGY: none | |
| stage: push | |
| only: | |
| # We want this job to be run on tags only. | |
| - tags | |
| script: | |
| - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA | |
| - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME | |
| - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME |
Best practices
-
Do not use “latest” nor “stable” images when using a CI. Why? Because you want reproducibility. You want your pipeline to work in 6 month. Or 6 years. Latest images will break things. Always target a version. Of course, the same applies to the base image in your Dockerfile. Hence
image: docker:20here. -
To speed-up your docker builds, pull the “latest” image (
$CI_REGISTRY_IMAGE:latest) before building , and then build with--cache-from $CI_REGISTRY_IMAGE:latest. This will make sure docker has the latest image and can leverage layer caching. Chances are you did not change all layers, so the build process will be very fast. -
When building, use
--pullto always attempt to pull a newer version of the image. Because you are targeting a specific version, this makes sure you have the latest (security) updates of that version. -
In the push jobs, tell GitLab not to clone the source code with
GIT_STRATEGY: none. Since we are just playing with docker pull/push, we do not need the source code. This will speed things up as well. -
Finally, keep your Git tags in sync with your Docker tags. If you have not automated this, you have probably found yourself in the situation of wondering “which git tag is this image again?”. No more. Use GitLab “tags” pipelines.
Further reading
- Building Docker images with GitLab CI/CD from GitLab