We decided to clean-up runner and remove all the potential garbage
we get when we start runner - we also move docker to a separate
/mnt partitiion available on the runners - because for unit testing
especially we need as much space as possible. Also when we build
images, it is best if we use clean, pristine docker engine located
on the /mnt folder.
All of our other jobs are ok with small disk space and sometimes that cleanup
can take up to a minute (usually it is 10-20s)
This change (experimentally) removes all cleanup from all jobs
except the unit tests and image building to see if we can get stability
and speed at the same time.
You can specify commit hash to apply when you build documentation.
This allows to regenerate past version of the documentation by
checking out the exact version tag that was used back then and
applying the commit hash with fixes.
This might help in fixing issues like
https://github.com/apache/airflow/issues/53646
After https://github.com/eclipse-csi/octopin/issues/45 is fixed,
we can now switch off the exclusion for workflow files in yamllint
as the comments added by octopin now have 2 spaces as expected by
default by yamllint.
Apache Software Foundation security team is going to introduce the rule
that all actions should be pinned to hash commit (because security)
and recommends to use eclipse's octopin to automatically make sure
that your actions are pinned to commit rather than version (adding
version as a comment).
This PR integrates octopin and converts all our actions to use
hashes (even the standard ones that we considered as "safe" so far.
When we are starting the runner on CI we are cleaning it up and
preparing the /mnt directory to be used for all kind of disk
storage we need during CI build. This is happening in the step
that was named "cleanup docker" but it did more than that.
The script and all references to it are renamed, also more diagnostics
is printed just before we restore CI image from cache to see how
much space we have available, because for some reason we seem to
have problems with disk space there in some jobs
The release workflow now will run separately for each image - which
means that if both AMD / ARM images of the same python version have
finished, the merge step for that Python version will run immediately
rather than waiting for all Python versions to complete. This means
that some images might be available a bit faster and that even if
a single image releaase will fail for some reason, the other images
will appear before we re-run that failed image job.
It also adds the possibility of overriding the python version
images - we can now additionally filter which image versions should
be run and option to disable automated "latest tag" setting.
