I'm trying to give special permissions to a file located inside a container but I'm getting a "No such file or directory" error message.
The Dockerfile basically runs a R Script that generates an output.pptx file located inside an output folder created inside the container.
I want to send that output into a s3 bucket but for some reason it isn't finding the file inside the container.
# Make the output directory
RUN mkdir output
# Process main file
CMD ["RScript", "Script.R"]
# install AWS CLI
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
RUN unzip awscliv2.zip
RUN ./aws/install -i /usr/local/bin/aws -b /usr/local/bin
# run AWS CLI to push output file to s3 folder
RUN chmod 775 ./output/*.pptx
RUN aws s3 cp ./output/*.pptx s3://bucket
Could this be related to the path I'm using for the file?
(Edited to fix a word-swap brain-fart in the first version.)
I get the idea that there is a misunderstanding of how the image should be used. That is, a DOCKERFILE creates an image, and the CMD is not actually run when building the image.
Up front:
an image is really just a tarball with filesystems; multiple "layers" are there, to indicate the layers of the build process (which can be squashed); an image has no "running" component, no processes are active in an image; and
a container is an image that is in a running state. It might be the CMD you specify, or it might be something else (e.g., docker run -it --rm myimage /bin/bash to run a bash shell with the container as the filesystem/environment). When the running command finishes and exits, the container is stopped.
Typically, you create an image "once" (security updates and such notwithstanding), and then run it as needed (either manually or via some trigger, e.g., cron or CI triggers). That might look like
docker run --rm myimage # using the default `CMD`
docker run --rm myimage R Script.R # specifying the command manually
with a couple assumptions:
the image has R installed and within the PATH ... though you could specify the full /path/to/bin/R instead; and
the default working directory (dockerfile's WORKDIR directive) contains Script.R ... or you can specify the full internal path to Script.R
With this workflow, I suggest a few changes:
from the DOCKERFILE, remove the lines after # run AWS CLI;
add to Script.R steps to copy the file to your S3 instance, either using the awscli you installed in the image, or by using the aws.s3 R package (which might preclude the need to install the awscli);
I don't use AWS S3, but I suspect that you need credentials to be able to access the bucket; there are many ways for dealing with images and "secrets" like S3 credentials, the most naïve approaches involve hard-coding the credentials into the container, which is a security risk; others involve "docker secrets" or environment variables. For instance,
docker run -it --rm -e "S3TOKEN=asldkfjlaskdf"
though even that might be intercepted by neighboring users on the docker host.
Related
This is a question related to using docker to run scripts of RStudio. The problem I'm having is that the person evaluating the results I'm getting wants to have it so that they type in ./test.sh in the command window, that runs my R script, and a csv of my results prints out to the local directory.
Is it possible to get the rocker rstudio to appear in the command window as opposed to having to log into the browser to use rstudio? It seems all the resources online say something along the lines of put -p 8002:8787 and -d in the docker line of code, but this makes it so you have to go to a local browser to actually run your R script.
I've found this code snippet works, but is there an alternative to using \bin\bash at the end so that the rstudio commands can just stay in the window?
$ docker run -e PASSWORD=MYPASSWORD -v "$(pwd):/data:ro" -v "$(pwd):/workdir" -it thatguy /bin/bash
Or, better yet, is there a way to put this docker run command in my Dockerfile so that when I run my $ docker build this line just runs automatically?
Due to the need to direct shiny-server logs to stdout so that "docker logs" (and monitoring utilities relying on it) can see them i'm trying to do some kind of :
tail -f <logs_directory>/*
Which works as needed when no new files are added to the directory, the problem is shiny-server dynamically creates files in this directory which we need to automatically consider.
I found other users have solved this via the xtail package, the problem is i'm using Centos and xtail is not available for centos.
The question is , is there any "clean" way of doing this via standard tail command without needing xtail ? or maybe there exists an equivalent package to xtail for centos?
You will probably find it easier to use the docker run -v option to mount a host directory into the container and collect logs there. Then you can use any tool you want that collects log files out of a directory (logstash is popular but far from the only option) to collect those log files.
This also avoids the problem of having to both run your program and a log collector inside your container; you can just run the service as the main container process, and not have to do gymnastics with tail and supervisord and whatever else to try to keep everything running.
I would like to build a container which shows up in its startup log its build datetime. Is there a way to have that information injected from my build machine into the container ?
The output of each RUN step during a build is the changes to the filesystem. So you can output the date to a file in your image. And the logs from the container are just the stdout from commands you run. So you can cat out the date inside your entrypoint.
In code, you'd have at the end of your Dockerfile:
RUN date >/build-date.txt
And inside an entrypoint script:
#!/bin/sh
#.... Initialization steps
echo Image built: $(cat /build-date.txt)
#.... More initialization steps
# run the command
exec "$#"
You could use an ARG to pass in the current build timestamp at build time. You'd have to docker build --build-arg build-date=$(date) or something like that. Having done that, you can refer to the argument using something similar to shell variable syntax at build time.
This is probably more useful if you have a significant build step in your Dockerfile (you're compiling a Go application) or if the metadata you're trying to embed is harder to generate programmatically (the source control version stamp, the name of the person running the build command).
I am running Shiny-Server (to run web applications built in R) in a Docker container. I have an application where user can upload some files. It's working, but on server OS I needed to give write and read permissions to the user "shiny". The problem is that everytime I need to do something with the container (like restart, or simply stop and start) I lose the change made on folder's permissions, which come back to default.
I tried to use docker commit and docker run again on container, using the new image, but it did not work. So now I am searching if I can use docker run and docker exec togheter, doing something like this: docker run <docker commands to run shiny-server> exec -it bash <bash commands to change folder permissions>.
Is it possible? Does anyone has a good solution for this case?
Thanks.
I am using Docker to run an Rscript which saves the output to the containers /home folder.
Is there some way I can:
A) Access a shell inside that container as the script is running? Trying to attach from another terminal just brings me to the same output as the 'taken-over' original terminal. I would like to access the file-system while it runs. -SOLVED
B) Grab the output saved into /home on the container and put it on my local machine? Does the data 'dissapear' once the container dies, or is there some way to access it after the script finishes and the container closes?
Thanks!
a) you can use docker exec -it yourservername bash assuming you have bash on the image
b) you can set-up a volume which maps a directory on your machine to a directory in the container. Everything that gets written out or changed in the volume will be available locally. Check out the volume documentation
You can also restart the container and pull out the output if you haven't removed the container.