Substituting variable values (original) (raw)

Use substitutions in your build config file to substitute specific variables at build time.

Substitutions are helpful for variables whose value isn't known until build time, or to re-use an existing build request with different variable values.

Cloud Build provides built-in substitutions or you can define your own substitutions. Use substitutions in your build's steps and imagesto resolve their values at build time.

This page explains how to use default substitutions or define your ownsubstitutions.

Using default substitutions

Cloud Build provides the following default substitutions for all builds:

Cloud Build provides the following default substitutions for builds invoked by triggers:

Cloud Build provides the following GitHub-specific default substitutions available for pull request triggers:

If a default substitution is not available (such as with sourceless builds, or with builds that use storage source), then occurrences of the missing variable are replaced with an empty string.

When starting a build using gcloud builds submit, you can specify variables that would normally come from triggered builds with the--substitutions argument. Specifically, you can manually provide values for:

For example, the following command uses the TAG_NAME substitution:

gcloud builds submit --config=cloudbuild.yaml \
    --substitutions=TAG_NAME="test"

The following example uses the default substitutions $BUILD_ID, $PROJECT_ID, $PROJECT_NUMBER, and $REVISION_ID.

YAML

steps:
# Uses the ubuntu build step:
# to run a shell script; and
# set env variables for its execution
- name: 'ubuntu'
  args: ['bash', './myscript.sh']
  env:
  - 'BUILD=$BUILD_ID'
  - 'PROJECT_ID=$PROJECT_ID'
  - 'PROJECT_NUMBER=$PROJECT_NUMBER'
  - 'REV=$REVISION_ID'

# Uses the docker build step to build an image called my-image
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-image', '.']

# my-image is pushed to Container Registry
images:
- 'gcr.io/$PROJECT_ID/my-image'

JSON

{
  "steps": [{
      "name": "ubuntu",
      "args": [
        "bash",
        "./myscript.sh"
      ],
      "env": [
        "BUILD=$BUILD_ID",
        "PROJECT_ID=$PROJECT_ID",
        "PROJECT_NUMBER=$PROJECT_NUMBER",
        "REV=$REVISION_ID"
      ]
    }, {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/$PROJECT_ID/my-image", "."]
    }],
  "images": [
    "gcr.io/$PROJECT_ID/my-image"
  ]
}

The example below shows a build request using the docker build step to build an image, then pushes the image to Container Registry using the default$PROJECT_ID substitution:

In this example:

YAML

steps:
- name: gcr.io/cloud-builders/docker
  args: ["build", "-t", "gcr.io/$PROJECT_ID/cb-demo-img", "."]
images:
- gcr.io/$PROJECT_ID/cb-demo-img

JSON

{
  "steps": [{
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/$PROJECT_ID/cb-demo-img", "."]
    }],
  "images": [
    "gcr.io/$PROJECT_ID/cb-demo-img"
  ]
}

Using user-defined substitutions

You can also define your own substitutions. User-defined substitutions must conform to the following rules:

You can specify variables in one of two ways: $_FOO or ${_FOO}:

To use the substitutions, use the --substitutionsargument in the gcloud command or specify them in the config file.

The following example shows a build config with two user-defined substitutions called _NODE_VERSION_1 and _NODE_VERSION_2:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build',
         '--build-arg',
         'node_version=${_NODE_VERSION_1}',
         '-t',
         'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}',
         '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build',
         '--build-arg',
         'node_version=${_NODE_VERSION_2}',
         '-t',
         'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}',
         '.']
substitutions:
    _NODE_VERSION_1: v6.9.1 # default value
    _NODE_VERSION_2: v6.9.2 # default value
images: [
    'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}',
    'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}'
]

JSON

{
    "steps": [{
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "--build-arg",
            "node_version=${_NODE_VERSION_1}",
            "-t",
            "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}",
            "."
        ]
    }, {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "--build-arg",
            "node_version=${_NODE_VERSION_2}",
            "-t",
            "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}",
            "."
        ]
    }],
    "substitutions": {
        "_NODE_VERSION_1": "v6.9.1"
        "_NODE_VERSION_1": "v6.9.2"
    },
    "images": [
        "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}",
        "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}"
    ]
}

To override the substitution value you specified in the build config file, use the --substitutions flag in the gcloud builds submit command. Note that substitutions are a mapping of variables to values rather than arrays or sequences. You can override default substitution variable values except for $PROJECT_ID and $BUILD_ID. The following command overrides the default value for _NODE_VERSION_1 specified in the build config file above:

gcloud builds submit --config=cloudbuild.yaml \
  --substitutions=_NODE_VERSION_1="v6.9.4",_NODE_VERSION_2="v6.9.5" .

By default, the build returns an error if there's a missing substitution variable or a missing substitution. However, you can set the ALLOW_LOOSE option to skip this check.

The following snippet prints "hello world" and defines an unused substitution. Because the ALLOW_LOOSE substitution option is set, the build will be successful despite the missing substitution.

YAML

steps:
- name: 'ubuntu'
  args: ['echo', 'hello world']
substitutions:
    _SUB_VALUE: unused
options:
    substitutionOption: 'ALLOW_LOOSE'

JSON

{
    "steps": [
    {
        "name": "ubuntu",
        "args": [
            "echo",
            "hello world"
        ]
    }
    ],
    "substitutions": {
        "_SUB_VALUE": "unused"
},
    "options": {
        "substitution_option": "ALLOW_LOOSE"
    }
}

If your build is invoked by a trigger, the ALLOW_LOOSE option is set by default. In this case, your build will not return an error if there is a missing substitution variable or a missing substitution. You cannot override the ALLOW_LOOSE option for builds invoked by triggers.

If the ALLOW_LOOSE option is not specified, unmatched keys in your substitutions mapping or build request will result in error. For example, if your build request includes $_FOO and the substitutions mapping doesn't define _FOO, you will receive an error after running your build or invoking a trigger if your trigger includes substitution variables.

The following substitution variables always contain a default empty-string value even if you don't set the ALLOW_LOOSE option:

When defining a substitution variable, you aren't limited to static strings. You also have access to the event payload that invoked your trigger. These are available as payload bindings. You can also apply bash parameter expansions on substitution variables and store the resulting string as a new substitution variable. To learn more, seeUsing payload bindings and bash parameter expansions in substitutions.

Dynamic substitutions

You can reference the value of another variable within a user-defined substitution by setting the dynamicSubstitutions option to true in your build config file. If your build is invoked by a trigger, thedynamicSubstitutions field is always set to true and does not need to be specified in your build config file. If your build is invoked manually, you must set the dynamicSubstitutions field to true for bash parameter expansions to be interpreted when running your build.

The following build config file shows the substitution variable ${_IMAGE_NAME} referencing the variable, ${PROJECT_ID}. ThedynamicSubstitutions field is set to true so the reference is applied when invoking a build manually:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', '${_IMAGE_NAME}', '.']
substitutions:
    _IMAGE_NAME: 'gcr.io/${PROJECT_ID}/test-image'
options:
    dynamicSubstitutions: true

JSON

{
   "steps": [
      {
         "name": "gcr.io/cloud-builders/docker",
         "args": [
            "build",
            "-t",
            "${_IMAGE_NAME}",
            "."
         ]
      }
   ],
   "substitutions": {
      "_IMAGE_NAME": "gcr.io/${PROJECT_ID}/test-image"
   },
   "options": {
      "dynamic_substitutions": true
   }
}

For more information, see Applying bash parameter expansions.

Mapping substitutions to environment variables

Scripts don't directly support substitutions, but they support environment variables. You can map substitutions to environment variables, either automatically all at once, or manually by defining every environment variable yourself.

Map substitutions automatically

YAML

steps:  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Hello $_USER"  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Your project ID is $PROJECT_ID"  
options:  
  automapSubstitutions: true  
substitutions:  
  _USER: "Google Cloud"

JSON

{  
  "steps": [  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Hello $_USER'"  
    },  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'"  
    }  
  ],  
  "options": {  
    "automap_substitutions": true  
  },  
  "substitutions": {  
    "_USER": "Google Cloud"  
  }  
}

YAML

steps:  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Hello $_USER"  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Your project ID is $PROJECT_ID"  
  automapSubstitutions: true  
substitutions:  
  _USER: "Google Cloud"

JSON

{  
  "steps": [  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Hello $_USER'"  
    },  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'",  
      "automap_substitutions": true  
    }  
  ],  
  },  
  "substitutions": {  
    "_USER": "Google Cloud"  
  }

Additionally, you can make the substitutions available as environment variables in the entire build, then ignore them in one step. SetautomapSubstitutions to true at the build level, then set the same field to false in the step where you want to ignore the substitutions. In the following example, even though mapping substitutions is enabled at the build level, the project ID will not be printed in the second step, becauseautomapSubstitutions is set to false in that step:

YAML

steps:  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Hello $_USER"  
- name: 'ubuntu'  
  script: |  
    #!/usr/bin/env bash  
    echo "Your project ID is $PROJECT_ID"  
  automapSubstitutions: false  
options:  
  automapSubstitutions: true  
substitutions:  
  _USER: "Google Cloud"

JSON

{  
  "steps": [  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Hello $_USER'"  
    },  
    {  
      "name": "ubuntu",  
      "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'",  
      "automap_substitutions": false  
    }  
  ],  
  "options": {  
    "automap_substitutions": true  
  },  
  },  
  "substitutions": {  
    "_USER": "Google Cloud"  
  }

Map substitutions manually

You can manually map the substitutions to environment variables. Every environment variable is defined at the step level using the envfield, and the scope of the variables is restricted to the step where they are defined. This field takes a list of keys and values.

The following example shows how to map the substitution $PROJECT_ID to the environment variable BAR:

YAML

steps:
- name: 'ubuntu'
  env:
  - 'BAR=$PROJECT_ID'
  script: 'echo $BAR'

JSON

{
  "steps": [
    {
      "name": "ubuntu",
      "env": [
        "BAR=$PROJECT_ID"
      ],
      "script": "echo $BAR"
    }
  ]
}

What's next