Much of the time, the images returned from the ListImagesWithOptions
API are sorted according to timestamp. As a cheap and simple
optimisation for this case, when we're constructing a result, cache
both the slice of image.Info records and those records sorted by
This means, for any given request, images that occur in more than one
container will be sorted by timestamp only once.
Only calculate container fields that are requested
The ListImagesWithOptions API method lets you supply a list of the
fields you want for each container, so that you can cut down on the
size of the response when you don't care about some fields. But all
the fields are calculated, whatever you ask for, which involves a lot
of sorting and filtering.
This commit makes the procedure calculating the fields _only_
calculate the fields that were actually requested. Some of the
calculations depend on the same intermediate results; so, the approach
is to lazily calculate and cache intermediate results.
Before this change, we only accepted image repository metadata if a manifest was
present for all image tags (which is to be expected without inconsistencies).
However, enforcing fully consistent repository is a bit too strict, since:
1. Popular repositories may actually miss tag manifests (e.g. `bitnami/redis`),
which has been broken for months.
2. Users may not be able to enforce such consistency.
3. The offending image tags can be ignored if filtered out by container
image annotation patterns.
This change allows for the repository cache warmer to accept partial repository
information. Consistency is only enforced after tags are filtered by the
container image pattern annotations.
In particular, this change adds:
* A new storage format in the repository cache, splitting the information into
tags and image metadata.
* More tolerance towards tags missing metadata when listing images (e.g. `fluxctl list-images`).
* Tolerance towards tags missing metadata during auto-releases, provided that
those tags are not matched by container image pattern annotations.
It's worth noting that we can do better for `semver` patterns, since they don't
depend on image metadata for sorting (just on the tags themselves). `semver`
patterns could tolerate tags without metadata, even after filtering. This
however, is not provided by this change.
Currently, containers can be tagged in manifests to filter what image
tags are considered when doing automated releases. Filtering is done by
specifying a wildcard glob. An optional prefix `glob:` can be used.
This PR adds support for tag filters based on [semantic versioning]
by using the prefix `semver:` instead. Version constraints can be
specified that filter images. Since versions have an implicit ordering
this also changes the way images are sorted when trying to determine the
newest image. For glob filtering this falls back to image creation date.