I'm barely new to CI/CD piplines with GIT and hope that my questions is related to that functionality. I want to achieve the following:
Whenever a new commit (or tag) of a specific package in our companies local git server is created, I want to execute a shell script on a remote server. This shell script should download this new version of the package and install it into the global R-Installation, making it available for different processes that use this R environment. So far this is more like a manual step but I really would like to automate the process. Is this possible?
Thanks!
Related
I am new to AEM, and would like to know how to take author instance into local instance. I can take the pushed changes made in project by using git command, then I can build the local by using mvn command. However I can't see the changes that I made in aem prod on local aem.
Can you help me regarding this. Thank you in advance!
If you are talking how you can have the changes made in pages in any of the environment to your local instance, then just create a content package in crxde in the environment from where you want the changes. After creating the package, download it and install in your local instance crxde.
The more direct approach is via the Package Manager, you create a package, enter the desired filters, then build and download. There are automated ways, by using the package manager with curl commands, Grabbit, etc. For local development probably you do not need to bring all the content, especially if the DAM is too big.
I have built a Dashboard that has multiple calculations using R Scripts and now I want to publish this to our internal Server. I get the following error message:
"This worksheet contains R scripts, which cannot be viewed on the target platform until the administrator configures an Rserve connection."
From what I understand, the administrator has to configure the Rserve, but what about the rest of the installed packages I have in use? Should the administrator install those too and every time I use a new package, I should inform him to install that particular package?
You need to install the packages on the server that your script will use. Then make sure to start Rserve there and connect your Tableau workbook to the server where Rserve is used (using Rserve connection in Tableau).
Tableau described the process pretty well:
http://kb.tableau.com/articles/knowledgebase/r-implementation-notes
I have a very beginner question. I've just been reading through some of the documentation regarding Amazon's EMR. Before I sign up etc. I just wanted to ask about using R in it.
I have one R module that calls several other modules, and then, just before it finishes running, saves several variables as .txt files.
My rather basic question is, can I do this in Amazon's EMR? And will I be able to access the .txt output files? Finally, my R script reads in some data from Excel spreadsheets. Will it still be able to do this from the EMR if I upload the Excel files into the system?
Thanks
Mike
#Mike, Answers to your 3 questions below
Running R on EMR: Yes you can.
You can run R programs on EMR once you have installed R on the EMR instance. I assume that you would write MapReduce moules if you plan to use multi-instance cluster. If you program is just about a "plain" R program then you may have to just use one sizable instance. I would rather use an EC2 instance with R AMI (look for Louis Aslett).
Moving output files:
Yes you can. It is possible to transfer your program output from EMR to S3 storage bucket of your choice. You will have to add a step calling S3DistCp command to move the files. An example from my project -
--jar /home/hadoop/lib/emr-s3distcp-1.0.jar --args '--src,hdfs:///contents,--dest,s3://<bucket-name>/'
Reading spreadsheets: AFAIK, If you are able to do this on local installation of R, then you should also be able to do it on EMR. You have to ensure that the necessary packages/libraries are installed during the bootstrap process.
I am able to install squeezy-cran and rmr2 on an EMR instance with all their dependencies (RCpp, reshap2, digest, RJSONIO, functional etc.). I am still unable to call the R program as a step. I am having to use SSH session and run R CMD commands on the shell prompt. Being on Windows, putty.exe works for me.
I love github and RStudio for workflow. Recently, I've created a project template that makes directories and scripts etc. and would like to create locally and push to github.
In the past I created a repo for a project via https://github.com/ used version control in RStudio to create the local repo and then dump all files I already had there.
This seems wasteful of time. How can one to take the directory/repo that's already in RStudio with a .Rproj file and upload to github with out first creating the shell repo at https://github.com/?
I think this could save time in the workflow.
I thought I could just follow the directions -here- (under Adding version control to a project) to add version control but this doesn't allow me to push to github (nor should it because how does RStudio know which git site you want to push to).
The only way you could create a repository on github directly from your computer, without having to create it with their website first, would be to create a remote branch directly from git on your system. This is possible on some git installation, but not on Github.
However, Github provides an API that allows to create the repository from the command line, via a call to curl for example. You will find information on how to do it in this answer (not tested) :
curl -u 'USER:PASS' https://api.github.com/user/repos -d '{"name":"REPO"}'
git remote add origin git#github.com:USER/REPO.git
git push origin master
But I don't think you will be able to do it directly from RStudio : you will need to put your project under version control, and then to execute the three commands provided in the answer in a shell.
Have you seen hub?
hub create
git push -u origin master
will do the job for you, once hub is configured to access your GitHub account. If you want the project to be called different from the name of the parent directory, use
hub create projectname
The general usage is
hub create [NAME] [-p] [-d DESCRIPTION] [-h HOMEPAGE]
(-p -- private repository), and you can access many more GitHub features with this tool.
I am trying to create a setup procedure which installs my entire web application. I am using Visual Studio's Setup and Deployment Project. So far I've gotten it to deploy my website to the Inetpub folder, and I've also added some custom actions which allow it to run some SQL and setup my database.
The last thing I have to integrate into the setup process is my two SSIS packages. Not only do these need to be installed, they also need to be scheduled to run nightly.
The packages are simple, and don't reference anything unusual. They are just 2 .dtsx files.
So far, I've seen that I can use something called dtutil in order to create a dtsinstall.exe file which can be run to install the SSIS packages to either the file system or the database.
First of all, is this the easiest way to do it? And secondly, how would I go about scheduling the packages to be run nightly?
One caveat is that I need this to install silently, without prompting the user for any input.
First, create a new job in SQL Agent. In the steps panel, click New, give it a name, and for the job type, select SQL Server Integration Services Package. For the Package source, select File system, and point it to where you want the file to live.
Then you can select Schedules from the left panel, and configure how often you want it to run.
After that, you should be able to handle deployment of the SSIS package by copying the .dtsx file to the location you specified when you created the job.