I'm trying to write a Makefile in such a way that all of the steps in my recipe can call scripts a list of directories. As an example, I have node commands that are in a node_modules/.bin directory, but there could just as well be Python commands from a Python virtual environment as well, etc.
I have node installed, as well as TypeScript installed via npm in my /node_modules. Given my directory /tmp/test123, I wrote the following Makefile at /tmp/test123/Makefile as an example:
PATH = /tmp/test123/node_modules/.bin:$(PATH)
.PHONY: compile
compile:
env | grep "PATH"
tsc --help
However, if I try to run it, it outputs:
$ make compile
env | grep "PATH"
PATH=/tmp/test123/node_modules/.bin:... # the rest of my PATH
tsc --help
make: tsc: No such file or directory
make: *** [compile] Error 1
But if I change the last line to cd . && tsc --help, then compile runs successfully.
Why is make unable to find the tsc command in the PATH? Why do I need to run cd . && in front of it? Is there any better workaround than writing cd . && in front of all of my recipe steps?
Related
How do I execute a command using RScript myfile.R so that it uses the renv environment of the project/directory it's in, NOT my default environment?
There are a couple ways:
Ensure your working directory is set to the root of your renv project, and that the renv project's auto-loader is active. (You can set up the auto-loader by calling renv::activate() from R in that project.)
In your script, explicitly call renv::load("/path/to/project") to load the requested project.
If neither of these methods suffice, please file an issue at https://github.com/rstudio/renv/issues.
I recently had a similar problem but the answer by #kevin-ushey was insufficient. Here's the background. I need to be able to run Rscript from any directory (Because I had several statistical models which were to be called from a Docker file, forcing a Docker file to have WORKDIR many times is just too cumbersome when you have long files with several Rscript calls. Moreover, some of these models are called several times in different bash files, making it cumbersome to cd to the directory before every Rscript call). We needed something akin to conda activate where any Rscript call would just use the activated 'renv environment' by default, regardless of what your working directory is. Here's a dummy example:
Install renv with install.packages('renv').
Create dummy folder with a dummy script containing the beepr library (just for the sake of the example) and initialize the renv environment:
mkdir ~/renv_test/
cd ~/renv_test/
echo "library(beepr); print('success')" >> test.R
Rscript -e "renv::init()"
Create a Docker image with the code below:
FROM rocker/r-base
ENV PROJ_ROOT='/usr/local/src/renv_test'
ENV RENV_DIR='/usr/local/.renv/'
COPY . $PROJ_ROOT
# Copy the projects renv infrastructure to RENV_DIR and remove all traces of renv from PROJ_ROOT
RUN mkdir -p $RENV_DIR/renv/ && \
cp $PROJ_ROOT/renv.lock $RENV_DIR && \
cp $PROJ_ROOT/renv/activate.R $RENV_DIR/renv/ && \
echo "source('renv/activate.R')" >> $RENV_DIR/.Rprofile && \
cd $RENV_DIR && \
Rscript -e "renv::restore()" && \
cd $PROJ_ROOT && Rscript -e "renv::deactivate()" && \
rm -rf renv/ renv.lock
# Set RENV_DIR's restore library as the default library
RUN echo $(cd $RENV_DIR && Rscript -e "cat(paste0('R_LIBS=', renv::paths\$library()), sep = '\n')") >> $HOME/.Renviron
# Run any script from any directory as if you had 'renv activated'
CMD Rscript $PROJ_ROOT/test.R
Here's a summary of the approach:
Copy the project to the docker image
Copy the renv infrastructure to a separate folder (here ~/.renv/) and restore the project there.
Eliminate all traces of renv from the project folder (this is so we don't mess up the path of the library if for some reason we execute a script from the root of this project).
Edit .Renviron so that it contains the restored library path in ~/.renv as the default library. This ensures that any new R session will use that library as the first option.
Execute any R scripts located in the project folder without having to cd or WORKDIR (docker) to the project folder.
If you build and run the previous Docker image, you should get a success statement even though we never cd to the project folder:
docker build -t renv_test .
docker run renv_test
[1] "success"
I believe a simpler way than the above answers:
Rscript -e 'renv::run("/path/to/myscript.R")'
It will pick up the renv environment from the base path. You can also specify the environment using the project parameter.
I am trying to compile R locally using a bash file, but it fails and shows the following error:
tar: command not found
cd: R-3.2.5: No such file or directory
./configure: No such file or directory
make: *** No targets specified and no makefile found. Stop.
make: *** No rule to make target `install'. Stop.
R-3.2.5/lib64/R/bin/R: No such file or directory
sed: command not found
mv: command not found
tar: command not found
Bellow is the bash file I am submitting:
#!/bin/bash
tar -xzvf R-3.2.5.tar.gz
cd R-3.2.5
./configure --prefix=$(pwd)
make
make install
cd ..
R-3.2.5/lib64/R/bin/R -e 'install.packages(c("BGLR"),
repos="http://cran.cnr.berkeley.edu/", dependencies = TRUE)'
sed -i '/^R_HOME_DIR=/c R_HOME_DIR=$(pwd)/R' R-3.2.5/lib64/R/bin/R
mv R-3.2.5/lib64/R ./
tar -czvf R.tar.gz R/
When I run the same command lines directly on terminal it works fine, but when I try to run them using a bash file it fails.
Does anyone have an idea how to make it work?
The bash instance used to run the script doesn't seem to have the $PATH variable correctly set, so it can't find tar and the other commands.
Try replacing the 1st line with #!/bin/bash -l. Add echo Path: $PATH as the 2nd line and see if one of the directory listed actually contains tar. You should get something like /bin:/sbin:/usr/bin/.
For building my target I have a list of prerequisites contained in a file list.txt and a script for generating this file generate-list.sh.
I need the script to be executed as first thing every time I invoke the make in order to have the list.txt updated and to give ti make the right list of prerequisites.
prebuild:
touch list.txt
.SECONDEXPANSION:
exe: prebuild $$(shell cat list.txt)
touch exe
<files in list.txt>:
<rules for generating these files>
In this way when I run make I first get an error from cat saying that list.txt does not exist, then list.txt is generated but since the cat failed the prerequisites contained in list.txt are not generated.
One method you could use, given that generate_list.sh must be executed at the very start every time, would be to explicitly execute it using the shell function. This would mean altering your makefile to something like
$(shell ./generate_list.sh > /dev/null)
.SECONDEXPANSION:
exe: $(shell cat list.txt)
touch exe
#echo $?
<files in list.txt>:
<rules for generating these files>
Executing this makefile produces
$ make
touch exe
deps.c test.c
where my generate_list.sh file contains
#!/bin/bash
touch test.c deps.c
echo deps.c test.c > list.txt
echo 'Created prerequisites list.'
Notes
/dev/null is included in $(shell ./generate_list.sh > /dev/null) incase your generate_list.sh produces an output as this would cause an error in make of
$ make
GNUmakefile:1: *** missing separator. Stop.
otherwise.
#echo $? shows that all of the prerequisites in list.txt are now included as prerequisites of exe.
Alternate Method Based on Auto Dependency Generation
What you are attempting to do is very similar to automatic dependency generation which can be accomplished using the -include directive in make. For future usage you may want to consider going down this route and altering your generate_list.sh script to create a makefile that can be included in your main makefile.
I am trying to build a software following some instructions on line.
cd app
mkdir -p build/Release
cd build/Release
cmake ../..
make installpyplusplus && cmake .
make .
My questions:
What does "../.." after cmake do or mean?
What is the significance of the dot after make?
what will 'make installpyplusplus && cmake .' ?
The cmake application looks for one file in special, called CMakeLists.txt.
So by running:
cmake ../..
It's like saying to cmake that the CMakeLists.txt is two directories below, like described by MERose.
The cmake command creates many files at your current working directory (CWD, the directory you ran the command from), and among them is a file called Makefile, which has rules about which files to compile/build/copy/write/whatever and how to do it.
So when you run:
make .
You are telling the make application that the Makefile file is at your CWD. It's the same as running:
make
That looks for the Makefile file at your CWD.
Concluding, . is the CWD, and .. is one level below.
EX: If your CWD is /Users/yourname/
. represents /Users/yourname/
.. represents /Users/
../. represents /Users/
../.. represents /
And so on...
what will 'make installpyplusplus && cmake .' ?
When you use && the commands will be executed sequentially if the first command returns true (exit status zero). So, in the case you said, make installpyplusplus will be run, and after it's done (it can create a CMakeLists.txt, I don't know what you are running), if it returns true, the command cmake . will be run, and if the CMakeLists.txt is there, it will run properly.
BONUS:
If you run:
make -j4
You will separate the build process in 4 instances (you can change 4 by anything you want)! Multi-threading magic will make it build faster if you have more than one processor core available :)
I'm building a shell script that does a gradle build and then run my grunt build after it is done and I'm storing this shell script in a subdirectory but the the graddle wrapper is in the root directory of the file. My shell script looks like this
sh .././gradle ../build.gradle clean build
grunt build
So when I call the script from the root directory it looks like this
$ sh gradle fullbuild
It errors out saying that
Execution failed for task ':fullbuild'. > Process 'command 'sh'' finished with non-zero exit value 127
When I do the first command from the script in the sub directory it says
FAILURE: Could not determine which tasks to execute. * What went wrong: Task '../build.gradle' not found in root project 'tools'.
Can I call gradle from a subdirectory and use that directories build.gradle file?
With the -b flag you can indicate which build file to use. Try
sh ../gradle -b ../build.gradle clean build
You could also specify the -p flag instead which indicates which directory is the project dir.