Solvedcloud builders How do I run kubectl after image push?
✔️Accepted Answer
Hey!
Good news, the up-scoping feature is now live.
That means that you can add permission for Container Engine to your Cloudbuild service account, and then you can use kubectl directly in your build step:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ["build", "-t", "gcr.io/my-project/frontend", "."]
- name: 'gcr.io/cloud-builders/docker'
args: ["push", "gcr.io/my-project/frontend"]
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |
gcloud components install kubectl
gcloud container clusters get-credentials <my-cluster> --zone <my-zone>
kubectl set image deployment/frontend frontend=gcr.io/my-project/frontend
Enjoy!
Other Answers:
To expand on the above solution, I think these steps will work at this time.
In addition to the above link, this thread was useful Here. In particular a comment by jlowdermilk.
You begin by going to IAM > Service accounts and creating a creating a service account and downloading the json key file. ex. cloudbuild@my-project.iam.gserviceaccount.com
Then you add this service account to IAM and give it project editor access (the same as cloud builder, the difference is that later on this account will have the missing scopes).
- On a machine with gcloud:
export CLOUDSDK_CONTAINER_USE_APPLICATION_DEFAULT_CREDENTIALS=true
gcloud container clusters get-credentials mycluster
kubectl config view --minify --flatten > kubeconfig
-
Copy these this file and the json key file to Cloud Storage. I removed all privleges except 'Owner' and then granted the service account (cloudbuild@my-project.iam.gserviceaccount.com) read access. Not that this matters because this whole process is not very secure.
-
Then add a build step to cloudbuild.yaml
- name: 'gcr.io/cloud-builders/docker'
args: ["push", "gcr.io/$PROJECT_ID/imagename:$BRANCH_NAME-$REVISION_ID"]
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |
gcloud components install kubectl
gsutil cp gs://owner_secrets/cloud_build/$PROJECT_ID-cloudbuild.json .
gsutil cp gs://owner_secrets/cloud_build/$PROJECT_ID-kubeconfig .
export GOOGLE_APPLICATION_CREDENTIALS=$PROJECT_ID-cloudbuild.json
export KUBECONFIG=$PROJECT_ID-kubeconfig
kubectl set image deployment/deploy_name imagename=gcr.io/$PROJECT_ID/imagename:$BRANCH_NAME-$REVISION_ID --namespace=$BRANCH_NAME
That seemed to work for me at this time.
In case people are curious, here is my cloudbuild.yaml file that builds my image, pushes it to docker registry, then updates kubernetes:
My kubernetes file references my image like this:
- image: gcr.io/my-google-project/my-project:${TAG}
And is then set up with the following cloudbuild.yaml
steps:
- name: 'gcr.io/$PROJECT_ID/envsubst'
args: ['/workspace/kubernetes.yaml', '/workspace/kubernetes.yaml']
env:
- 'TAG=$TAG_NAME'
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '--build-arg', 'PROJECT_ID=$PROJECT_ID', '--cache-from', 'gcr.io/$PROJECT_ID/my-project:latest', '-t', 'gcr.io/$PROJECT_ID/my-project:$TAG_NAME', '-t', 'gcr.io/$PROJECT_ID/my-project:latest', '.']
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/my-project:$TAG_NAME']
- name: 'gcr.io/cloud-builders/kubectl'
env: ['CLOUDSDK_COMPUTE_ZONE=us-central1-a', 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster']
args: ['apply', '-f', '/workspace/kubernetes.yaml']
images: ['gcr.io/$PROJECT_ID/my-project:latest']
The envsubst Dockerfile:
FROM alpine:3.6
MAINTAINER Nick Richardson <nick.richardson@mediapixeldesign.com>
RUN apk --update add gettext-dev
ADD envsubst-file.sh /
RUN chmod +x /envsubst-file.sh
ENTRYPOINT ["/envsubst-file.sh"]
The envsubst-file.sh:
#!/bin/sh
FILENAME=$1
FILENAME_NEW=$2
if [ ! $FILENAME ]
then
echo 'No filename argument provided'
exit -1
fi
if [ ! $FILENAME2 ]
then
FILENAME_NEW=$FILENAME
fi
echo "Processing $FILENAME ..."
envsubst < $FILENAME > $FILENAME-bak
echo "Finished modifying file"
mv $FILENAME-bak $FILENAME_NEW
Yes, there are plans for this, landing soon...
At the moment, you will have trouble using kubectl
from within the container builder service, since the credentials used do not have the proper scopes. We will address this issue soon.
For pushing before running another step, you can always run docker push
as its own step (either instead of or in addition to listing the image in the images
field).
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/foo', '.']
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/foo']
- name: 'gcr.io/cloud-builders/kubectl'
args: ['run', 'foo-pod', '--image=gcr.io/$PROJECT_ID/foo']
images: ['gcr.io/$PROJECT_ID/foo']
(for an upcoming kubectl
step that is waiting on the scope issue)
It would seem that a CI/CD pipeline would require the ability to deploy an image into the gke cluster after it was built/tested/pushed.
Without this, it seems like Builder is incomplete.