Suppose I have a project for which I have developed an R package. The hierarchy might look something like this.
/project
---Makefile
---workflow.R
---test.R
---/mypackage
---DESCRIPTION
---NAMESPACE
---/R
---func1.R
---func2.R
workflow.R depends on the latest version of mypackage being installed. However, I only want to re-build the package if any file inside of it has been modified.
Currently, in my Makefile, I have:
PACKAGE=$(wildcard mypackage/**/*)
all: install test workflow
install: $(PACKAGE)
R CMD INSTALL mypackage
workflow: install
Rscript workflow.R
test: install
Rscript test.R
However, this will re-install the package every time I run make test, even if nothing inside the package has changed. Is there a clean way to avoid this?
The install rule does not create a file named install in the current directory, so make tries to remake it each time. This looks like it should be a .PHONY target, but that itself won't fix the issue as it will still execute the recipes.
One solution is to have another rule that creates a stub file:
.PHONY: all install test workflow
all: install test workflow
install: install.done
install.done: $(PACKAGE)
R CMD INSTALL mypackage
touch $#
Or you could just make install the stub file itself and make it a non-.PHONY rule.
It sounds like you want to treat the installation as an intermediate step. You can do this by adding
.INTERMEDIATE: install
to your makefile.
The make manual explains (link):
If an ordinary file b does not exist, and make considers a target that depends on b, it invariably creates b and then updates the target from b. But if b is an intermediate file, then make can leave well enough alone. It won’t bother updating b, or the ultimate target, unless some prerequisite of b is newer than that target or there is some other reason to update that target.
Related
Meson/Ninja provide an easy method to run a script at install time.
For example, this line will tell Meson to run the glib-compile-schemas command to compile the GSettings on Linux (system configuration options).
meson.add_install_script('glib-compile-schemas', schemas_dir)
(this command will be automatically run when the user executes ninja install)
How can I tell Meson to run a custom command at uninstall?
In this specific case I would like to delete (or at least reset to default) the key-value pairs in GSettings. To reset them, I have found that the command is gsettings reset-recursively <path> (successfully tested in terminal).
Adding custom uninstall script is still being discussed, it's proposed quite some time ago but not yet implemented. It looks this task is typically left for package manager (and therefore to corresponding packaged scripts).
But I agree, there is some illogical asymmetry in case of meson install command. As a workaround, you can create your own target:
run_target('my-uninstall', command : ['scripts/uninstall.sh'])
The drawbacks, of course, are that it should be invoked explicitly, cannot override, append or rename internal uninstall target and script should have executable permissions.
The internal, reserved uninstall target, however, does revert all explicit install operations:
Meson allows you to uninstall an install step by invoking the
uninstall target. This will remove all files installed as part of
install. Note that this does not restore the original files. This also
does not undo changes done by custom install scripts (because they can
do arbitrary install operations).
I want to be able to import or using a package that I'm writing in a directory ~/projects/ExamplePkg from my main Julia REPL / from another project or environment.
By ]foo I mean "use the foo command at the Julia Pkg REPL". Type ] at the Julia REPL to enter the Pkg REPL. Use ]help <command name> for more info or check the link below.
Ensure that your package has a Project.toml that gives it a UUID and names it (generate one with ]generate from the Julia REPL or with the PkgTemplates package) and that it is in a git repo with at least one commit including all the relevant files.
Then choose how you would like to use the package.
You probably want to run ]dev ~/projects/ExamplePkg:
If dev is used on a local path, that path to that package is recorded and used when loading that package. The path will be recorded relative to the project file, unless it is given as an absolute path.
If you use dev and you change the dependencies in the dev'd package, then you should probably run ]resolve in all environments that depend on the package.
Or you can run ]add ~/projects/ExamplePkg:
Instead of giving a URL of a git repo to add we could instead have given a local path to a git repo. This works similarly to adding a URL. The local repository will be tracked (at some branch) and updates from that local repo are pulled when packages are updated. Note that changes to files in the local package repository will not immediately be reflected when loading that package. The changes would have to be committed and the packages updated in order to pull in the changes.
In Julia versions <1.4: If you accidentally ]add a package before the git repo is set up correctly then you might get ERROR: GitError(Code:EUNBORNBRANCH, Class:Reference, reference 'refs/heads/master' not found). Unfortunately, Julia will probably have cached the bad repo, and you will need to remove that from ~/.julia/clones/<gibberish>/. You can find the dir to remove with grep: $ grep ExamplePkg ~/.julia/clones/*/config.
Documentation: https://julialang.github.io/Pkg.jl/v1/managing-packages/
you can try
path_to_package = "~/projects/ExamplePkg"
push!(LOAD_PATH,path_to_package)
# then use it, ExamplePkg is the package's name
using ExamplePkg
But you have to run codes above whenever you restart Julia.
reference is Workflow tips-Julia Documentation
Problem Definition
I'm attempting to adapt these rosjava installation instructions so that I can include rosjava on a target image built by the BitBake build system. I'm using the jethro branch of Poky.
Implementation Attempt: Build From .deb with package_deb.bbclass
According to the installation instructions, all that really needs to be done to install rosjava is the following:
sudo apt-get install ros-indigo-rosjava
Which works perfectly fine on my build machine. I figured that if I can just point to a .deb and use the Poky metadata class package_deb, it would do all the heavy lifting for me, so I produced the following simple recipe adapted on this posting on the Yocto Project mailing list:
inherit package_deb
SRC_URI = "http://packages.ros.org/ros/ubuntu/pool/main/r/ros-indigo-rosjava/ros-indigo-rosjava_0.2.1-0trusty-20160207-031808-0800_amd64.deb"
SRC_URI[md5sum] = "2020ccc8b4a67dd918a9a2c426eece0b"
SRC_URI[sha256sum] = "ab9493fabe1285b0d21aab031348d0d733d116b0b2470bae90025709b303b649"
The relevant part of the errors I get during the above recipe's do_unpack are:
| no entry data.tar.gz in archive
|
| gzip: stdin: unexpected end of file
| tar: This does not look like a tar archive
| tar: Exiting with failure status due to previous errors
| DEBUG: Python function base_do_unpack finished
| DEBUG: Python function do_unpack finished
The following command produces the output below:
$ ar t python-rosdistro_0.4.5-1_all.deb
debian-binary
control.tar.gz
data.tar.xz
You can see here that there's a data.tar.xz, not data.tar.gz. What can I do to remedy this error and install from this particular .deb?
I've included package_deb in my PACKAGE_CLASSES variable and package-management in my IMAGE_FEATURES. I've tried other methods of installation which have all failed; I thought this method in particular would be very useful to know how to implement.
Update - 3/22
I'm attempting to circumvent the problems with the method above by doing my installation through a ROOTFS_POSTPROCESS_COMMAND which I've adapted from forum posts like this
install_rosjava() {
${STAGING_BINDIR_NATIVE}/dpkg \
--root=${IMAGE_ROOTFS}/ \
--admindir=${IMAGE_ROOTFS}/var/lib/dpkg/ \
-L /var/cache/apt/archives/ros-indigo-rosjava_0.2.1-0trusty-20160207-031808-0800_amd64.deb
}
ROOTFS_POSTPROCESS_COMMAND += " install_rosjava() ; "
However, this fails due to dpkg not being a command found within the ${STAGING_BINDIR_NATIVE} path. The Yocto Project Reference Manual states that:
STAGING_BINDIR_NATIVE Specifies the path to the /usr/bin subdirectory of the sysroot directory for the build host.
Taking a look inside this directory yields a lot of commands but not dpkg (The recipe depends on the dpkg package, and this command can be found in my target rootfs after the build is finished; I've also tried pointing to ${IMAGE_ROOTFS}/usr/bin/dpkg which yields the same results). From what I understand of the BitBake process, this command may be in another sysroot, but I must admit that this is where my understanding breaks down.
Can I adjust this method so that it works, or will I need to start from scratch on an installation from source?
Perhaps there's a different method entirely which I could consider?
If you really want to install their deb directly then your rootfs postprocess is one solution. It doesn't work because depending on dpkg will build you a dpkg for the target but you want a dpkg that will run on the host. Add a dependency on dpkg-native to your image.
Though personally I'd either inherit bin_package and extract the deb they provide then re-package it as a standard package in OE, or ideally write a proper recipe to build rosjava and submit it to meta-ros (https://github.com/bmwcarit/meta-ros).
package_deb is where the packaging machinery for deb packages is stored, it's not something you'd inherit in a recipe but should be listed in PACKAGE_CLASSES.
When you put a .deb in a SRC_URI the fetcher will try to unpack it so you can access the contents: the assumption is that you're going to repack the contents as a native Yocto recipe.
If that's what you want to do then first you'll need to fix the unpack logic (in bitbake/lib/bb/fetch2/__init__.py) to handle .debs with xz-compressed data. This is a bug in bitbake and a bug report and/or patch would be appreciated.
The alternative would be to use their deb directly but I don't recommend that as it's likely the dependencies don't match. The best long-term solution would be to build it from source directly instead of attempting to use a package for another distro.
I am one of the author of the R package xgboost. When I tried to pack it with R CMD build xgboost, there are 5 new compiled .o files inside src/.
I double checked that these files are not in the directory before. And strangely, this problem disappears when I run R CMD build --no-build-vignettes xgboost. I suspect there are something related to the process of compiling the vignettes. However we don't want to sacrifice the vignettes for it.
How can this be fixed? Thanks!
NOTE: To reproduce the problem, please run the following command in your console:
git clone https://github.com/tqchen/xgboost && cd xgboost && make Rpack
where make Rpack is a combination of file operations that move files to the correct places and R CMD operations. Edit the Rpack part in Makefile to control this flow.
OK after some attempts the following solution works for me:
Create a file named .Rbuildignore
Fill it with the file names (patterns) that you don't need.
An example is this file: https://github.com/dmlc/xgboost/blob/master/R-package/.Rbuildignore
I'm building an R package and need to build a jni library for OSX (called myPackage.jnilib) as part of my build process and then have R's automatic installation mechanisms put it inside the libs directory of my package.
The problem is that R's default is to try and build an object called myPackage.so. I'd like to be able to customize this but can't see how.
I can get part of the way by subverting R's mechanisms using a phony "all" target in Makevars (described here) and then copying the file to the inst directory of my package. This is OK for my own local uses but generates headaches when trying to build universal binaries and isn't very portable. I'm currently preparing the package for CRAN so this method isn't likely to work.
I can see two potential solutions but haven't got either to work yet
Copy my library manually to the libs directory of my package during installation. Since this directory is created on the fly, how would I find out what it is from within Makevars or a configure script
The best solution: Tell R CMD SHLIB the name of my output file so I can use R's normal package mechanisms and let it copy the file to the right directory.
In case anyone else encounters this problem I'm posting my own workaround here.
I define targets in my Makevars and copy the libraries directly (ie answer 1). The variable R_LIBRARY_DIR provides the temporary location where the package is being built.
My Makevars now looks something like this
OBJECTS =
LIBSINSTDIR=$(R_LIBRARY_DIR)/myPackage/libs/
#ARCHFLAG is set in the configure script to i386 or ppc as appropriate
JNIINSTDIR=$(LIBSINSTDIR)/#ARCHFLAG#/
.PHONY: all
all: $(SHLIB) jnilib
jnilib: object1.o object2.o
$(CXX) -bundle $(JAVA_LIBS) $(JAVA_CPPFLAGS) -o libmyPackage.jnilib object1.o object2.o
mkdir -p $(JNIINSTDIR)
cp libmyPackage.jnilib $(JNIINSTDIR)