How to use Secret Manager in Google Cloud Build (GCP)

What is Google Cloud Build?

Cloud Build is a service that executes your builds on Google Cloud.
Cloud Build can import source code from various repositories or cloud storage spaces, execute a build to your specifications, and produce artifacts such as Docker containers or Java archives.
Cloud build allows you to run your docker images on the cloud with minimum configuration.

Setting up the deployment pipeline using Google cloud triggers and Github

After setting up your google account, go to the google cloud console and search for “cloud build”. In the cloud build page, click on the triggers menu
you should see this page:

Click on the “Create trigger” button to enter the information of the trigger that you want to create.

In the event section, select “push to branch” to invoke the build trigger every time you push to the “master” branch. And in the source section, connect your GitHub account and assign the repository of your project to it.

In the configuration section, you specify the config file (yaml or json). the trigger uses this file to run the steps you need to build and deploy your project. (you can rename the config file if you want)

Hit the create button and you’re good to go.

Let’s create the project

In this article I use NestJS

npm i -g @nestjs/cli
nest new project-name

After creating the project we need to write a Dockerfile.

FROM node:16.14.0-alpine3.14

RUN apk update
RUN apk upgrade

ENV API_ROOT /kittyapp/api

WORKDIR ${API_ROOT}

COPY ./*.json ./

RUN npm install
RUN npm install pm2 -g

COPY ./src ./src

RUN npm run build

EXPOSE 8080

CMD ["pm2-runtime", "dist/main.js"]

Now we’re done with Dockerfile

let’s create cloudbuild.yaml file to run the pipeline every time we push to the master branch.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest', '-f', 'Dockerfile', '.' ]
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'push', 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest' ]
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: bash
  args: [
  '-c',
  'gcloud run deploy kittyapp-io-apis --image gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest 
  --region europe-west2
  --cpu 2 --memory 512Mi --port 8080 --allow-unauthenticated --platform managed
  '
  ]
images:
- 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest'
timeout: 3600s

cloudbuild.yaml explanation:
First, the Google build, builds an image from Dockerfile, Then push the image and runs a docker container using gcloud (cloud SDK).
when you push to the master branch the trigger will be invoked and your app will be ready to use.

What is Google Secret Manager?

Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud.
In most projects, environment variables such as API keys, Database passwords, etc are sensitive and should not be available to everyone who is working on the project. To achieve this goal we must use the Secret Manager.

In the Secret Manager menu, click on the “Create Secret” button and enter the name and value of the secret.

How to access Secrets in cloudbuild.yaml file?

Here is a document on How to use secrets in cloud builds
https://cloud.google.com/build/docs/securing-builds/use-secrets

Using Secrets in runtime

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest', '-f', 'Dockerfile', '.' ]
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'push', 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest' ]
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: bash
  args: [
  '-c',
  'gcloud run deploy kittyapp-io-apis --image gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest 
  --region europe-west2 
  --set-env-vars _PORT=$$SECRET_PORT 
  --set-env-vars DB_HOST=$$SECRET_DB_HOST 
  --set-env-vars DB_NAME=$$SECRET_DB_NAME
  --cpu 2 --memory 512Mi --port $$SECRET_PORT --allow-unauthenticated --platform managed
  '
  ]
  
  secretEnv: [
    'SECRET_PORT',
    'SECRET_DB_NAME', 
    'SECRET_DB_HOST'
  ]
images:
- 'gcr.io/YOUR_PROJECT_ID/kittyapp-io-apis:latest'
timeout: 3600s

availableSecrets:
  secretManager:
  - versionName: projects/YOUR_PROJECT_ID/secrets/PORT/versions/latest
    env: 'SECRET_PORT'
  - versionName: projects/YOUR_PROJECT_ID/secrets/DB_HOST/versions/latest
    env: 'SECRET_DB_HOST'
  - versionName: projects/YOUR_PROJECT_ID/secrets/DB_NAME/versions/latest
    env: 'SECRET_DB_NAME'

In order to define a secret:
After all the build steps, add an availableSecrets field to specify the secret version and environment variables to use for your secret. You can include substitution variables in the value of the secretVersion field. You can specify more than one secret in a build.

• In the build step where you want to specify the secret:
• Add an entrypoint field pointing to bash to use the bash tool in the build step. This is required to refer to the environment variable for the secret.
• Add an secretEnv field specifying the environment variable.
• In the args field, add a -c flag as the first argument. Any string you pass after -c is treated as a command. For more information on running bash commands with -c, see the bash documentation.
• When specifying the secret in the args field, specify it using the environment variable prefixed with $$.
• Use --set-env-varsflag to set environment variables in runtime.

These variables can be accessed using process.envin NestJs app.

Thanks for reading this article 💖. Liked the article? have some feedback or suggestions? leave a like and a comment. This will help me understand better and write more amazing articles for you 🙂.