Feature build performance · Issue #21 · devcontainers/spec (original) (raw)

This proposal is an enhancement idea to the current dev container features proposal.

Currently features are built in a single layer of the image, so when a feature is added, removed or its options are modified, all features need to be rebuilt.

Building each feature in a separate layer would partially improve the use of the layer cache, but when the user changes a feature, that would still trigger a rebuild of all the feature layers coming after that.

A more promising approach is to use a multi-stage build and build each feature in a separate stage. One additional advantage is that these stages can be built in parallel. The challenge with it is that a feature needs to build its 'layer' in a separate folder, so in the last build stage we can copy all these feature layers (folders) from the different feature stages. Not all features might be simple to implement in this way, but we can keep supporting the existing way of building features while we work on updating popular features to the new model.

A feature would come with a set of scripts that are run in the following order:

• validate-prereqs: Runs as root in a separate build stage using a "build image".
  • This can install build time dependencies.
• acquire: Runs as regular user after validate-prereqs in the build image.
  • This needs to copy the result to a 'layer' folder that is self-contained in that it can be copied to a different image and run there.
• configure: Runs as root in the final image build after the 'layer' from acquire is copied over. Based on a "run image".
  • This can run additional configuration or even installation steps. This will run on each rebuild, so doing less here is better.

A new-style feature could omit the configure and validate-prereqs scripts if it doesn't need them. Old-style features could come only with a configure script.

The build image and the run image must be binary compatible in that binaries built for the build image must be able to run in the run image. (E.g., Debian/Ubuntu vs. Alpine. x64 vs. ARM.)

On dependencies: Plain runtime dependencies between features don't affect this approach. When there are build-time dependencies between features, we can declare these on the feature (separate to runtime dependencies) and let the depending feature's stage run with the layer from its dependencies copied in. (This will reduce parallelization of building stages. Unclear if build-time dependencies are common.)