This folder contains various docker images used in taskcluster as well as other misc docker images which may be useful for hacking on gecko.
Each folder describes a single docker image.
These images depend on one another, as described in the FROM line at the top of the Dockerfile in each folder.
Images could either be an image intended for pushing to a docker registry, or one that is meant either for local testing or being built as an artifact when pushed to vcs.
There are two types of images that can be used for task. Those that are prebuilt and pushed to a docker registry, and images that are built on push and uploaded and indexed as a task artifact.
Images that are meant to be prebuilt and pushed to a docker registry should container a 'VERSION' and optionally 'REGISTRY' file. See the docker image section for more details.
Images that are intended to be saved as an artifact of a task do not need to specify the 'VERSION' or 'REGISTRY' files. In this environment no registry is provided, and VERSION is ignored to prevent poisoning other images that have been loaded.
These images can still be build locally by running the steps below, but will be tagged as 'latest'. This will overwrite any previous image named the same.
When using task images within in-tree task definitions, the format is:
image:
type: 'task-image'
path: 'public/image.tar'
taskId: '{{#task_id_for_image}}builder{{/task_id_for_image}}'When the decision task executes, it will calculate the sha256 hash of the contents of the image directory (Note: this is the contents of only the context directory, not the image contents) and then determine if an image has already been indexed for that branch and hash. If no image exists, a task will be included in the graph to build the image and have it indexed for other tasks. All tasks in the graph requiring the image will require the image task to complete if one has been created.
This context directory will be uploaded as an artifact of the decision task allowing dependent image building tasks to download the context as an artifact rather than needing to pull from vcs.
As an additional convenience, and a precaution of loading images per branch, if an image exists for mozilla-central that has been built with the same context hash, that image will be used for the dependent tasks. This is to ensure there are not multiple images built/used that were built from the same context.
Images that are built on push and uploaded as an artifact of a task will be indexed under the following namespaces.
- {index}.docker.images.v1.{project}.{image_name}.latest
- {index}.docker.images.v1.{project}.{image_name}.pushdate.{year}.{month}-{day}-{pushtime}
- {index}.docker.images.v1.{project}.{image_name}.hash.{context_hash}
Not only can images be browsed by the pushdate and context hash, but the 'latest' namespace is meant to view the latest built image. This functions similarly to the 'latest' tag for docker images that are pushed to a registry.
Deprecation Warning: Use of prebuilt images should only be used for base images (those that other images will inherit from), or private images that must be stored in a private docker registry account. Existing public images will be converted to images that are built on push and any newly added image should follow this pattern.
These are images that are intended to be pushed to a docker registry and used by specifying the folder name in task definitions. This information is automatically populated by using the 'docker_image' convenience method in task definitions.
Example: image: {#docker_image}builder{/docker_image}
Each image has a version, given by its VERSION file. This should be bumped when any changes are made that will be deployed into taskcluster.
Then, older tasks which were designed to run on an older version of the image can still be executed in taskcluster, while new tasks can use the new version.
Each image also has a REGISTRY, defaulting to the REGISTRY in this directory, and specifying the image registry to which the completed image should be uploaded.
Generally images can be pulled from the registry rather then build locally, but for developing new images its often helpful to hack on them locally.
To build an image, invoke build.sh with the name of the folder (without a trailing slash):
./build.sh baseThis is a tiny wrapper around building the docker images via docker build -t $REGISTRY/$FOLDER:$FOLDER_VERSION
Note: If no "VERSION" file present in the image directory, the tag 'latest' will be used and no registry user will be defined. The image is only meant to run locally and will overwrite any existing image with the same name and tag.
On completion, if the image has been tagged with a version and registry, build.sh gives a
command to upload the image to the registry, but this is not necessary until the image
is ready for production usage. Docker will successfully find the local, tagged image
while you continue to hack on the image definitions.
The docker image primitives are very basic building block for constructing an "image" but generally don't help much with tagging it for deployment so we have a wrapper (./build.sh) which adds some sugar to help with tagging/versioning... Each folder should look something like this:
- your_amazing_image/
- your_amazing_image/Dockerfile: Standard docker file syntax
- your_amazing_image/VERSION: The version of the docker file
(required* used during tagging)
- your_amazing_image/REGISTRY: Override default registry
(useful for secret registries)
In some image folders you will see .env files these can be used in
conjunction with the --env-file flag in docker to provide a
environment with the given environment variables. These are primarily
for convenience when manually hacking on the images.
You will also see a system-setup.sh script used to build the image.
Do not replicate this technique - prefer to include the commands and options directly in the Dockerfile.