I am working on a remote server using RStudio. This server has no access to the Internet. I would like to install the package "stringi." I have looked at this stackoverflow article, but whenever I use the command
install.packages("stringi_0.5-5.tar.gz",
configure.vars="ICUDT_DIR=/my/directory/for/icudt.zip")
It simply tries to access the Internet, which it cannot do. Up until now I have been using Tools -> Install Packages -> Install from Packaged Archive File. However, due to this error, I can no longer use this method.
How can I install this package?
If you have no internet access on local machines, you can build a distributable source package that includes all the required
ICU data files (for off-line use) by omitting some relevant lines in
the .Rbuildignore file. The following command sequence should do the trick:
wget https://github.com/gagolews/stringi/archive/master.zip -O stringi.zip
unzip stringi.zip
sed -i '/\/icu..\/data/d' stringi-master/.Rbuildignore
R CMD build stringi-master
Assuming the most recent development version is 1.3.1,
a file named stringi_1.3.1.tar.gz is created in the current working directory.
The package can now be installed (the source bundle may be propagated via
scp etc.) by executing:
R CMD INSTALL stringi_1.3.1.tar.gz
or by calling install.packages("stringi_1.3.1.tar.gz", repos=NULL),
from within an R session.
For a Linux machine the easiest way is from my point of view:
Download the release you need from Rexamine in tar.gz format to your local pc. In opposition to the version on CRAN it already contains the icu55\data\ folder.
Move the archive to your target linux machine without internet access
run R CMD INSTALL stringi-1.0-1.tar.gz (in case of release 1.0-1)
You provided the wrong value of configure.vars.
It indicates that you have to give the directory's name, not a final file name.
Correct your code to the following:
install.packages("stringi_0.5-5.tar.gz",
configure.vars="ICUDT_DIR=/my/directory/for/")
Regards,
Sean
Follow the steps below
Download icudt55l.zip seperately from server where you have internet access with
wget http://www.mini.pw.edu.pl/~gagolews/stringi/icudt55l.zip
Copy the downloaded packages to the server where you want to install stringi
Execute the following command
R CMD INSTALL --configure-vars='ICUDT_DIR=/tmp/ALL' stringi_1.1.6.tar.gz
icudt55l.zip is copied to /tmp/ALL
The suggestion from #gagolews almost worked for me. Here's what actually did the trick with RStudio.
Download the master.zip file that will save as stringi-master.zip.
Unzip the file onto your desktop. The unzipped folder should be stringi-master.
Edit the .Rbuildignore file by removing ^src/icu55/data and ^src/icu61/data or similar lines.
Move the folder from your desktop to the home directory of your server.
Create a New Project in RStudio with ~/stringi-master as the Existing Directory
From RStudio's menu, select Build and Build Source Package. (You may need to first select Configure Build Tools. For Project build tools choose Package then select OK.)
It should create a tar.gz file, in the following format: stringi_x.x.(x+1).tar.gz. For example, if the current version of stringi is 1.5.3, it will create version 1.5.4. (I received a few warnings that didn't seem to affect the outcome.)
Move the newly created package to your local repository. Update the repository index. And install the package.
Related
I am building a windows docker image and use a package manager called choco for automating package installation. I would like to avoid downloading packages from the internet every time I modify and rebuild the dockerfile. Locally we run an Artifactory server that purpose But the package I am installing has hardcoded url value which is pointing to microsoft.com
In the package installation script here https://community.chocolatey.org/packages/visualcpp-build-tools#files
$packageName = 'visualcpp-build-tools'
$installerType = 'EXE'
$url = 'https://download.microsoft.com/download/5/A/8/5A8B8314-CA70-4225-9AF0-9E957C9771F7/vs_BuildTools.exe'
$checksum = 'E77D433C44F3D0CBF7A3EFA497101DE93918C492C2EBCAEC79A1FAF593D419BC'
How can I overcome this hardcoded Url value and force the install script to download the package from my local artifactory server?
Before discovering the hardcoded Url value inside the package installation script, I tried using the --source option with choco install as described in the guide here https://jfrog.com/blog/artifactory-as-a-caching-mechanism-for-package-managers/ but I noticed that the package is still downloaded from the internet.
Packages on the Chocolatey Community Repository often can't include the original binaries due to licenses and distribution rights - leading to each user downloading the file from the source URL.
However, you can manually re-create the package with the downloaded file inside - to do so, you should:
Extract the relevant content of the nupkg file (e.g. the tools directory and nuspec file)
Download the file specified in URL (which is the file being downloaded repeatedly during the package installation) to the tools directory (or somewhere else within the package)
Replace the URL in URL with the relative path to the downloaded file
Run choco pack $PathToNuspecFile (in the root of the extracted files)
Upload the resultant nupkg file to your Artifactory instance
Alternatively, you could host the binary file on your Artifactory server and update URL to point to that. This would have the potential benefit of not downloading the full size if you had logic in the package that would short-circuit the installation before the actual install step, e.g. if you had gigantic MSU file you only wanted to download if you needed to apply it.
Finally, one of the parts of a licensed copy of Chocolatey for Business is the Package Internalizer.
This automates the process described above, allowing you to simply call choco download visualcpp-build-tools --internalize.
I am working on Azure databricks and it's compute server is Ubuntu 18.04. I want to install arrow R package but without internet access because of security reasons. I downloaded arrow tar file on my MacBook that has internet access and made it available in ubuntu for manual installation. I performed following steps:
Re-installed build-essential by downloading it from this link and uploaded to ubuntu and executed following bash command to make it available: sudo dpkg -i /dbfs/FileStore/tables/build_essential_12_4ubuntu1_amd64.deb
Installed cpp11 as it is dependency mentioned on cran: R CMD INSTALL /dbfs/FileStore/tables/arrow_dir/cpp11_0_3_1.tar.gz
Downloaded arrow_4.0.1.tar.gz from here and made it available on ubuntu.
Here I see required C++ dependencies to be available on ubuntu before installing the arrow package. How can I install these dependencies without access to internet?
Thanks for reading my question.
Note: A solution is suggested below and after execution of ./thirdparty/download_dependencies.sh $HOME/arrow-thirdparty I get:
# Environment variables for offline Arrow build
export ARROW_ABSL_URL=/root/arrow-thirdparty/absl-0f3bb466b868b523cf1dc9b2aaaed65c77b28862.tar.gz
export ARROW_AWSSDK_URL=/root/arrow-thirdparty/aws-sdk-cpp-1.8.133.tar.gz
export ARROW_AWS_CHECKSUMS_URL=/root/arrow-thirdparty/aws-checksums-v0.1.10
export ARROW_AWS_C_COMMON_URL=/root/arrow-thirdparty/aws-c-common-v0.5.10.tar.gz
export ARROW_AWS_C_EVENT_STREAM_URL=/root/arrow-thirdparty/aws-c-event-stream-v0.1.5
export ARROW_BOOST_URL=/root/arrow-thirdparty/boost-1.75.0.tar.gz
export ARROW_BROTLI_URL=/root/arrow-thirdparty/brotli-v1.0.9.tar.gz
export ARROW_BZIP2_URL=/root/arrow-thirdparty/bzip2-1.0.8.tar.gz
export ARROW_CARES_URL=/root/arrow-thirdparty/cares-1.17.1.tar.gz
export ARROW_GBENCHMARK_URL=/root/arrow-thirdparty/gbenchmark-v1.5.2.tar.gz
export ARROW_GFLAGS_URL=/root/arrow-thirdparty/gflags-v2.2.2.tar.gz
export ARROW_GLOG_URL=/root/arrow-thirdparty/glog-v0.4.0.tar.gz
export ARROW_GRPC_URL=/root/arrow-thirdparty/grpc-v1.35.0.tar.gz
export ARROW_GTEST_URL=/root/arrow-thirdparty/gtest-1.10.0.tar.gz
export ARROW_JEMALLOC_URL=/root/arrow-thirdparty/jemalloc-5.2.1.tar.bz2
export ARROW_LZ4_URL=/root/arrow-thirdparty/lz4-v1.9.3.tar.gz
export ARROW_MIMALLOC_URL=/root/arrow-thirdparty/mimalloc-v1.7.2.tar.gz
export ARROW_ORC_URL=/root/arrow-thirdparty/orc-1.6.6.tar.gz
Failed downloading https://github.com/google/protobuf/releases/download/v3.14.0/protobuf-all-3.14.0.tar.gz
Would it help to use the script mentioned in the link below to download the dependencies and put them somewhere you can then install them from?
There's some instructions here: https://arrow.apache.org/docs/developers/cpp/building.html#offline-builds
I've pasted them below in case the link expires, but you may want to check it for the most up to date version of these instructions.
To enable offline builds, you can download the source artifacts yourself and use environment variables of the form ARROW_$LIBRARY_URL to direct the build system to read from a local file rather than accessing the internet.
To make this easier for you, we have prepared a script thirdparty/download_dependencies.sh which will download the correct version of each dependency to a directory of your choosing. It will print a list of bash-style environment variable statements at the end to use for your build script.
# Download tarballs into $HOME/arrow-thirdparty
$ ./thirdparty/download_dependencies.sh $HOME/arrow-thirdparty
You can then invoke CMake to create the build directory and it will use the declared environment variable pointing to downloaded archives instead of downloading them (one for each build dir!).
Starting in arrow 6.0.0, the package should successfully install from source when offline. It will have only basic features: you'll be able to work with Arrow data and feather files, but features like Parquet reading, S3, and compression libraries won't be available. There is also a new utility function, create_package_with_all_dependencies(), that you can run on a machine connected to the internet in order to produce a "fat" source package containing all third-party C++ dependencies. You can then copy this to your airgapped server. See https://arrow.apache.org/docs/r/reference/create_package_with_all_dependencies.html for details.
I have developed a solution using R and want to transfer it to the production server (CentOS 7) which has no Internet connection to install packages. To facilitate installation of packages, I used packrat to bundle the packages I used in my R script to the project.
Using packrat::bundle(), I have created a tar file of the project and moved the file to the server and untar the zip file.
According to a post in Blogger, once I open the project, When R is started from that directory, Packrat will do its magic and make sure the software environment is the same as on the source machine.
However, when I open the project in Server (using R-Studio Server 0.99), nothing happens and it throws error of unknown packages.
When manually execute the "packarat/init.R" file below error is thrown
Error in ensurePackageSymlink(source, target) :
Target '/home/R_Projects/prjName/packrat/lib-R/base' already exists and is not a symlink
Well, I found the problem and solve it. The symlink error is related to centOS (it is not related to R). I just simply removed all the folders inside the
/home/R_Projects/prjName/packrat/lib-R
Because these folder exist, the packrat is unable to create symlink with the same name inside the lib-R folder. If I remove them, it will create a link (shortcut) to the actual folder where the r package is located.
Hope it helps future readers.
I have a computer behind a very restrictive proxy server it only allows me to surf the web and download programs it does not allow programs like the Atom text editor to download it's packages.
My question is how do I install them using only browser based downloads?
It is certainly possible:
Find the package you want to install, for example the activate-power-mode package.
Click on the Repo button to go to the GitHub repository.
Click Releases towards the top of the UI, then click on the most recent release, 0.4.1 in this case.
Download the source code release in either Zip or GZip depending on your platform.
Extract the content of the archive to a known permanent location, I have chosen:
C:\Source\Atom
Run the following command from your terminal / command prompt (make sure to include quotes around the path):
apm link "C:\Source\Atom\activate-power-mode-0.4.1"
Restart or Reload Ctrl-Alt-R Atom and the package will now be installed.
You can alternatively extract the package directly to your ~/.atom/packages folder however you will have to rename the folder to exactly match the name of the package, additionally uninstalling the package from Atom will delete the files which could be annoying if it is an accidental deletion.
Because of package dependencies a safest bet is this:
Install package normally on connected computer
Copy contents from your ~/.atom/packages
Paste contents to ~/.atom/packages on offline computer
Restart Atom
At least this worked for me like a charm.
The answer of Richard Slater is informative and the answer of Andriy Buday could look less professional. But, in my case, the answer of Andriy Buday was also very important.
I tried to install two packages atom-beautify and prettier-atom by following the answer of Richard Slater and had some problems of not being able to find some modules. It was not only me who had these problems. Consider checking the following links.
The issue of "cannot find module event-kit"
https://github.com/Glavin001/atom-beautify/issues/1734
https://github.com/Glavin001/atom-beautify/issues/1366#issuecomment-269716306
When I decompressed a file (atom-beautify-0.30.3.tar.gz) I received from GitHub respository, I could find out directories like appveyor, docs, and examples. But I could not find out a directory named node_modules which was present when I installed this package atom-beautify using Atom Editor online.
To check if the absence of directory node_modules is the only problem, I went through the following steps.
Start Atom Editor.
Install atom-beautify using Atom Editor online like the answer of Andriy Buday suggests.
Close Atom Editor.
Move atom-beautify directory from ~/.atom/packages (that was %HOMEDIRECTORY%%HOMEPATH%.atom\packages in my case because I used cmd on Windows 10) to somewhere else.
Decompress atom-beautify-0.30.3.tar.gz and move or copy atom-beautify-0.30.3 directory from this decompressed result into %HOMEDIRECTORY%%HOMEPATH%.atom\packages as the answer of Andriy Buday suggests.
Rename directory %HOMEDIRECTORY%%HOMEPATH%.atom\packages\atom-beautify-0.30.3 to %HOMEDIRECTORY%%HOMEPATH%.atom\packages\atom-beautify as the answer of Richard Slater suggests.
Move or copy node_modules directory from the directory moved at step 4 into %HOMEDIRECTORY%%HOMEPATH%.atom\packages\atom-beautify.
Start Atom Editor.
I found that no error message appeared and that package atom-beautify worked properly, thus I am thinking that absence of node_modules directory was the only problem of the file atom-beautify-0.30.3.tar.gz I received from GitHub repository.
I am afraid if it is normal that directory node_modules is not contained in the file atom-beautify-0.30.3.tar.gz downloaded from GitHub repository because of any rules I do not know yet, like placing directories like node_modules somewhere else. If there really are such rules and somebody tells me about such rules by adding an answer or a comment here, I will appreciate it a lot.
I am not sure if it is same with all other packages, but I found that it was same at least with package prettier-atom.
I wish it helps somebody.
+++++++++++++++++++++++++++
I found why the directory node_modules was not contained in atom-beautify-0.30.3.tar.gz.
I checked answers of the following link.
How can I manually download packages for atom editor and install them (manually)?
Answer by D3181 included a link to a page of http://discuss.atom.io/ (I could get a helpful answer by Alchiadus from the link) and suggested running apm install in the package's directory. If it is necessary to use a file downloaded from GitHub repository like atom-beautify-0.30.3.tar.gz, it is necessary to run apm install in the package's directory before copying or moving into %HOMEDIRECTORY%%HOMEPATH%\.atom\package (~/.atom/package in case of *nix) of the offline computer.
Decompress the file downloaded from GitHub repository like atom-beautify-0.30.3.tar.gz.
Go into the directory like atom-beautify-0.30.3 of the decompressed result.
Run apm install on an online computer. (If the directory of apm.cmd is not in PATH, run {directory of apm.cmd}\apm.cmd install.)
Rename directory like atom-beautify-0.30.3 to the correct name of the package like atom-beautify.
Move directory with the correct name of the package like atom-beautify into %HOMEDIRECTORY%%HOMEPATH%\.atom\packages of the offline computer.
Run Atom Editor on the offline computer and check if the package works properly.
It seems normal that the directory node_modules is not included the the compressed file downloaded from GitHub repository.
Normally I install packages using:
install.packages("foo")
and a Repo over the internet. But I have a new machine now where I want to replicate the packages from my existing installation without having to pull everything off the internet all over again. (I've a ton of packages and slow internet access)
Both machines are Windows and run the same R version. (2.13.1)
Is there a way to do this? Closest I can get is I know I can install from local zip files using:
install.packages("pathtozip", repos = NULL)
But does R store all Zips somewhere? I found a few in locations like:
C:\Documents and Settings\foouser\Local Settings\Temp\RtmpjNKkyp\downloaded_packages
But not all.
Any tips?
The function .libPaths will give you a vector of all the libraries on your machine. Run this on your old machine to find all of them. You can simply copy all these files into the libraries on your new machine (run .libPaths on it too to find out where).
Alternatively, if you want to set up a real repository (i.e. basically a CRAN mirror) on your computer or on a network drive you can update, you can put binary or source packages into a folder and run tools::write_PACKAGES on that folder. You can them run install.packages using the contriburl argument and point it to your repository folder.
All packages that you have installed are stored in a folder called win-library\r-version, for example,
C:\Users\Ehsan\Documents\R\win-library\2.15 so, it is enough to copy all the folders inside 2.15 to the same folder in your new machine. because you have the same version of R you do not need to update them by update.packages().
On your original computer, run
write.csv(unique(data.frame(installed.packages())[,1]),"packages.csv",row.names=F)
Save this .csv into the working directory of your new computer, then run
install.packages(as.character(read.csv("packages.csv")[,1]))
You can check what your working directory is using getwd().