Solvedcloud builders Mounting files in /workspace to docker run steps

This is somewhat related to #345.

Following the steps in this comment works totally fine for me, and I'm able to view the decrypted contents of that file. The trouble happens when I attempt to mount that file as a bind volume in later steps.

I might simply be missing what the absolute path of files in /workspace actually are.

Example cloudbuild.yaml (this results in cat: read error: Is a directory)

    - name: 'alpine'
      args: ['ls', '-la']
    - name: ''
      args: ['kms', 'decrypt', '--location=global', '--keyring=<ring>', '--key=<key>', '', '--plaintext-file=.hidden.txt']
    - name: 'alpine'
      args: ['ls', '-la']
    - name: 'alpine'
      args: ['cat', '.hidden.txt']
    - name: ''
      args: ['run', '-v', '/workspace/.hidden.txt:/data/.hidden.txt', 'alpine', 'cat', '/data/.hidden.txt']

Trying with relative ./.hidden.txt is also getting me nowhere.

The goal is to get to the point where I can compose up everything for integration testing as described here. The compose cloud builder behaves identically to the vanilla docker one in this scenario.


16 Answers

✔️Accepted Answer

Ah, ok; I think I've got it.

For folks coming here after-the-fact: you do still need to mount the volume normally (-v) in addition to using the volumes cloudbuild property. The trick is using the name rather than the path of that volume. You can still drop that volume wherever you want inside the container.

- name: 'ubuntu'
  - name: 'vol1'
    path: '/persistent_volume'
  entrypoint: 'bash'
  - '-c'
  - |
        echo "Hello, world!" > /persistent_volume/file
- name: ''
  - name: 'vol1'
    path: '/persistent_volume'
  args: ['run', '-v', 'vol1:/data', 'alpine', 'cat', 'data/file']

Running locally results in the below (command and full stdout included).

18:56 $ cloud-build-local --config=cloudbuild.yaml --dryrun=false .
2018/09/21 18:56:32 Warning: The server docker version installed (18.06.1-ce) is different from the one used in GCB (17.12.0-ce)
2018/09/21 18:56:32 Warning: The client docker version installed (18.06.1-ce) is different from the one used in GCB (17.12.0-ce)
Using default tag: latest
latest: Pulling from cloud-builders/metadata
Digest: sha256:7b1cc5e18ed10a2138a7c6c1db586fd1fe9ecadc24ff45fb731003bd4407afe6
Status: Image is up to date for
2018/09/21 18:56:39 Started spoofed metadata server
2018/09/21 18:56:39 Build id = localbuild_ac999140-ca4b-4978-8667-764b175a95ad
2018/09/21 18:56:39 status changed to "BUILD"
Starting Step #0
Step #0: Already have image (with digest): ubuntu
Finished Step #0
Starting Step #1
Step #1: Already have image (with digest):
Step #1: Hello, world!
Finished Step #1
2018/09/21 18:56:43 Failed to delete volume "vol1": exit status 1
2018/09/21 18:56:43 status changed to "DONE"

Related Issues:

cloud builders Mounting files in /workspace to docker run steps
Ah ok; I think I've got it This is somewhat related to #345 Following the steps in this comment work...
cloud builders How do I run kubectl after image push?
Hey! Good news the up-scoping feature is now live That means that you can add permission for Contain...
cloud builders Fetch repositories with submodules
You can use the source repositories of gcloud instead of github It is unclear how repositories with ...
cloud builders Document how to connect to a database
Cloud Build uses cloudbuild Docker network so we can minimize use of raw docker run without Docker C...
cloud builders Google Cloud Build for Github does not fetch submodules
Try this (I use BitBucket but change the url appropriately): It works for me Thanks ...
bazel Bazel 0.8.1: Build fails with "Xcode version must be specified to use an Apple CROSSTOOL" on OSX High Sierra
FYI: I encountered similar issue recently This is similar to #3063 but not quite the same so I'm add...
bazel Ubuntu 16.04 PPA key expired
I can confirm that reinstalling the key resolves the issue; sudo apt-get update runs successfully Th...
msbuild Cannot find reference assemblies for .NET 3.5 or lower using core msbuild
I got this working by adding the following to the .csproj file: Unfortunately dotnet/sdk supports ta...
msbuild Add support for non-string assembly attributes
Sorry for being late to the party on this I know it's closed but I think it's worth considering reop...
msbuild "MSB3823: Non-string resources require the property GenerateResourceUsePreserializedResources to be set to true" if TargetFramework=net48
I did not quite understand We have a few projects that muti target .net core and the old fashioned ....
bazel Error while building tensorflow 0.11.0 - cache (directory not empty)
I attempted both solutions suggest by @sfincke and @yselivonchyk but without luck Finally ...
bazel Bazel installer stopped working after d3f8efc, if project has .bazelversion file
We would like to kindly request this issue to be reopened as it does appears to have broken both Baz...
bazel a way at start a repl
@damienmg I think this is pretty standard for any build tool whose language has a repl: Clojure: lei...
tensorflow windows wheel failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
I haven't seen this error before Try set CUDA_VISIBLE_DEVICES=0 Hi ...
bazel Compatibility with Xcode and Command Line Developer Tools on macOS
I've just hit this on a fresh Bazel clone (bc5a9b1): Environment: macOS 10.14.6 Xcode 10.3 I can con...
msbuild Setting BaseIntermediateOutputPath correctly in a SDK-based project is hard
To summarize if I want to override my intermediate and output folders NuGet restore drops the projec...
msbuild MSBuild ToolsVersions missing from hklm\software\Microsoft\MSBuild\ToolsVersions registry key
This a critical issue All previous versions of MSBuild left a registry entry in hklm\software\Micros...
bazel bazel-built protoc segfaults on mac os monterey
I have had success working around this with the command line option --incompatible_linkopts_to_linkl...
msbuild Warning AL1073 when .resx files is compiled under x64
To be clear for those hitting this the workaround should be very similar to the workaround in the OP...
pants 2.3.0rc0 - ProcessExecutionFailure: Process 'Extracting plugin locations' failed with exit code 2
@jsirois Thanks! Can confirm this works for me now I should also note that in ...
assemble yfm in partials doesn't work
Since you're using the built-in Handlebars syntax for the partial: {{> foo }} assemble does not merg...
bazel Make maven_jar and friends smarter by re-using previously fetched artifacts across different projects
0590483 now lets you use --experimental_repository_cache=$HOME/some/path to cache downloaded artifac...
laradock Mysql. The server requested authentication method unknown to the client [caching_sha2_password]
alter user 'username'@'localhost' identified with mysql_native_password by 'password'; would fix it....
nvidia docker OpenCV Docker error "ImportError: cannot open shared object file: No such file or directory"
I fixed this problem on with (using solution above): ...
compose Docker-compose up failing because "port is already allocated"
I ran into the same issue today (with a postgres container) and despite having tried docker-compose ...
moby The name "/data-container-name" is already used by container <hash>. You have to remove (or rename) that container to be able to reuse that name.
I have a helper function to nuke everything so that our Continuous blah cycle can be tested erm.. co...
compose Compose error "HTTP request took too long to complete"
By simply restarting the docker service via sudo service docker restart I was able to get the aforem...
compose error on launching docker-compose by piping to sh ( echo 'docker-compose ... ' | sh )
I could get it to work by adding the -T parameter to not create a Pseudo-TTY docker-compose exec -T ...
compose docker-compose up fails if network attached to container is removed
Thanks for the report! I think there are several things to note here: First and foremost ...
nvidia docker docker: Error response from daemon: Unknown runtime specified nvidia.
I've also installed correctly but forgot to restart daemon in ubuntu it may resolve your error. ...
compose Error when trying to run docker-compose up. "oci runtime error: container_linux.go:247..."
you gotta make the an executable before building the image: otherwise it cant b...
laradock SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client
+1 I'm having the same problem here. Info: Docker version ($ docker --version): Docker version 17.12...
compose docker-compose up doesn't pull down latest image if the image exists locally
Imagine that git didn't have pull because git fetch && git merge origin/master is functionally ident...
nvidia docker could not select device driver "" with capabilities: [[gpu]].
Hello! If you didn't already make sure you've installed the nvidia-container-toolkit If this doesn't...
moby docker-engine 1.10.2-0~trusty can't install on clean Ubuntu 64-bit 14.04.3
I seem to have resolved this by putting deb trusty main in /etc/...
moby Docker service update --image "could not accessed on a registry to record its digest"
When updating services that need credentials to pull the image you need to pass --with-registry-auth...
laradock MySQL Container fails to start
I had the same issue last night I think it's the mysql version problem What I did was edited laradoc...
compose Docker Compose mounts named volumes as 'root' exclusively
Actually I come here with news it seems what I am trying to achieve is doable but I don't know if th...
compose INTERNAL ERROR: cannot create temporary directory!
Confirming this happened to me Today Was running low on space: After removing a container.. it works...
cookiecutter django No support for python3? I am getting: invalid syntax: raise ValueError, "No frame marked with %s." % fname
For me the issue was that I installed the environ package instead of the django-environ package. ...
compose docker-compose up -d doesn't expose ports when defined with build directive
oh you didn't specify but I'm assuming you're using run instead of up? If so you need --service-port...
compose How does compose chooses subnet for default network?
I'm also running into this issue Another way around this is to set the default-address-pools in your...
ddev In WSL2 ddev start fails at docker-credential-desktop.exe, "error listing credentials"
I had to set credsStore: in my ~/.docker/config.json .. it was previously set to credentials.exe ...
docker touch: cannot touch ‘/var/jenkins_home/copy_reference_file.log’: Permission denied
as mentioned there you need to figure out your volume mapping permissions ie. I have the same issue ...
kubernetes ingress 413 Request Entity Too Large
FYI the annotation has changed and is now: Also I had to restart the nginx pod for the effect to tak...
moby docker daemon unable to access registry - Client.Timeout exceeded while awaiting headers
I found out that the problem might be in /etc/resolv.conf I had: but moving the non-working (yet) 10...
cli How to skip one stage from multi-stage docker build
Docker 18.06 has been released Description We have Multistage docker build that creates rpm in each ...
compose ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint ltg_db_1
The following worked for me when i do the following : ± docker-compose up Starting ltg_db_1 ERROR: f...