Using Maven for R projects - r

I am beginning work on a project that makes heavy use of R. I've used R in the past, but only in a casual mode, whereas I'm now interested in following a more rigid practice of test/source control/continuous integration. I'm hoping to use Maven with this project if possible (having been pleased with how this manages packages with Java), but I can't find any evidence that it is possible to use Maven with R. Is it possible to create an R project with Maven, and if so, where can I find steps to help me get started?
I've found this question and this question, but they don't mention R.

Well you could make use of the Maven Exec Plugin with the resource, filter and jar plugins to fake an R project type.
You can do something like this to set up multiple R goals with the exec plugin so that compile, filter, test, et al have some kind of R call associated.
http://maven.apache.org/plugins/maven-resources-plugin/

Related

How auto package installation of Grunt, Karma, package.json in one process

This is more of a question without code so, please forgive me.
Is there a way programatically create a Package.json, karma.conf.js and gruntfile.js without building them individually.
I want to automate the process without having to build each one individually.
Also, the same for creating a Spec.js file for describe testing
You could look into Yeoman generators to see if any of the pre-existing ones are in line with what you are aiming to create. It's a fairly straight forward node tool that scaffolds out an app skeleton for you.
http://yeoman.io/generators/
If not, there's a fair bit of information available on how to write your own generator here:
http://yeoman.io/authoring/

How to write the scripts for Qt Installer

I'm trying to write my first Qt installer and having trouble. In my installer, in one of the packages, I need to run an .exe file at the end of the installation and add an environment variable as well.
I think the way to do it is with the script option in the package.xml file but I don't know how to write that script, I cannot find a step by step explanation of how to do it anywhere.
Can someone help?
I have been recently suffering the same fate as yourself but have managed, for the most part, to achieve what you are trying to do from the Qt installer framework examples and documentation found here http://qt-project.org/doc/qtinstallerframework-1.5/index.html and http://doc.qt.io/qtinstallerframework/qtifwexamples.html.
The information provided on these pages is a little ambiguous but with a little experimentation you should be able to create the install script that you require.
The package.xml file you would like to implement the install script requires that you add a
<Script>installscript.qs</Script>
tag and that the installscript.qs its self should be located in the meta folder of that package. The installscript.qs will then contain, among a few other things, a
component.addOperation("Execute".....)
command which will execute your required .exe. The same process for the environment variable could be implemented using the execute operation to run a batch file which creates the variable on the users system.
I have been trying over the last few weeks to implement the custom operations detailed in the qt installer framework documentation but have so far been unsuccessful so if you make any progress in this area feel free to pass the information on ;)

designing large projects in OCaml [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What is a best practices to write large software projects in OCaml?
How do you structure your projects?
What features of OCaml should and should not be used to simplify code management? Exceptions? First-class modules? GADTs? Object types?
Build system? Testing framework? Library stack?
I found great recommendations for haskell and I think it would be good to have something similar for OCaml.
I am going to answer for a medium-sized project in the conditions that I am familiar with, that is between 100K and 1M lines of source code and up to 10 developers. This is what we are using now, for a project started two months ago in August 2013.
Build system and code organization:
one source-able shell script defines PATH and other variables for our project
one .ocamlinit file at the root of our project loads a bunch of libraries when starting a toplevel session
omake, which is fast (with -j option for parallel builds); but we avoid making crazy custom omake plugins
one root Makefile contains all the essential targets (setup, build, test, clean, and more)
one level of subdirectories, not two
most subdirectories build into an OCaml library
some subdirectories contain other things (setup, scripts, etc.)
OCAMLPATH contains the root of the project; each library subdirectory produces a META file, making all OCaml parts of the projects accessible from the toplevel using #require.
only one OCaml executable is built for the whole project (saves a lot of linking time; still not sure why)
libraries are installed via a setup script using opam
local opam packages are made for software that it not in the official opam repository
we use an opam switch which is an alias named after our project, avoiding conflicts with other projects on the same machine
Source-code editing:
emacs with opam packages ocp-indent and ocp-index
Source control and management:
we use git and github
all new code is peer-reviewed via github pull requests
tarballs for non-opam non-github libraries are stored in a separate git repository (that can be blown away if history gets too big)
bleeding-edge libraries existing on github are forked into our github account and installed via our own local opam package
Use of OCaml:
OCaml will not compensate for bad programming practices; teaching good taste is beyond the scope of this answer. http://ocaml.org/learn/tutorials/guidelines.html is a good starting point.
OCaml 4.01.0 makes it much easier than before to reuse record field labels and variant constructors (i.e. type t1 = {x:int} type t2 = {x:int;y:int} let t1_of_t2 ({x}:t2) : t1 = {x} now works)
we try to not use camlp4 syntax extensions in our own code
we do not use classes and objects unless mandated by some external library
in theory since OCaml 4.01.0 we should prefer classic variants over polymorphic variants
we use exceptions to indicate errors and let them go through happily until our main server loop catches them and interprets them as "internal error" (default), "bad request", or something else
exceptions such as Exit or Not_found can be used locally when it makes sense, but in module interfaces we prefer to use options.
Libraries, protocols, frameworks:
we use Batteries for all commodity functions that are missing from OCaml's standard library; for the rest we have a "util" library
we use Lwt for asynchronous programming, without the syntax extensions, and the bind operator (>>=) is the only operator that we use (if you have to know, we do reluctantly use camlp4 preprocessing for better exception tracking on bind points).
we use HTTP and JSON to communicate with 3rd-party software and we expect every modern service to provide such APIs
for serving HTTP, we run our own SCGI server (ocaml-scgi) behind nginx
as an HTTP client we use Cohttp
for JSON serialization we use atdgen
"Cloud" services:
we use quite a lot of them as they are usually cheap, easy to interact with, and solve scalability and maintenance problems for us.
Testing:
we have one make/omake target for fast tests and one for slow tests
fast tests are unit tests; each module may provide a "test" function; a test.ml file runs the list of tests
slow tests are those that involve running multiple services; these are crafted specifically for our project, but they cover as much as possible as a production service. Everything runs locally either on Linux or MacOS, except for cloud services for which we find ways to not interfere with production.
Setting this all up is quite a bit of work, especially for someone not familiar with OCaml. There is no framework taking care of all that yet, but at least you get the choice of the tools.
OASIS
To add to Pavel answer:
Disclaimer: I am the author of OASIS.
OASIS also has oasis2opam that can help to create OPAM package quickly and oasis2debian to create Debian packages. This is extremly useful if you want to create a 'release' target that automate most of the tasks to upload a package.
OASIS is also shipped with a script called oasis-dist.ml that creates automatically tarball for upload.
Look all this in https://github.com/ocaml.org.
Testing
I use OUnit to do all my tests. This is simple and pretty efficient if you are used to xUnit testing.
Source control/management
Disclaimer: I am the owner/maintainer of forge.ocamlcore.org (aka forge.o.o)
If you want to use git, I recommend to use github. This is really efficient for review.
If you use darcs or subversion, you can create an account on forge.o.o.
In both case having a public mailing list where you send all commit notification is a must have, so that everyone can see them and review them. You can use either Google groups or a mailing list on forge.o.o.
I recommend to have a nice web (github or forge.o.o) page with OCamldoc documentation build everytime you commit. If you have a huge code base this will help you to use the OCamldoc generated documentation right from the beginning (and fix it quickly).
I recommend to create tarballs when you reach a stable stage. Don't just rely on checking out the latest git/svn version. This tip has saved me hours of work in the past. As said by Martin, store all your tarballs in a central place (a git repository is a good idea for that).
This one probably doesn't answer your question completely, but here is my experience regarding build environment:
I really appreciate OASIS. It has a nice set of features, helping not only to build the project, but also to write documentation and support test environment.
Build system
OASIS generates setup.ml file from the specification (_oasis file), which works basically as a building script. It accepts -configure, -build, -test, -distclean flags. I quite used to them while working with different GNU and other projects that usually use Makefiles and I find it convenient that it is possible to use all of them automatically here.
Makefiles. Instead of generating setup.ml, it is also possible to generate Makefile with all options described above available.
Structure
Usually my project that is built by OASIS has at least three directories: src, _build, scripts and tests.
In the former directory all source files are stored in one directory: source (.ml) and interface (.mli) files are stored together. May be if the project is too large, it is worth introducing more subdirectories.
The _build directory is under the influence of OASIS build system. It stores both source and object files there and I like that build files are not interfered with source files, so I can easily delete it in case something goes wrong.
I store multiple shell scripts in the scripts directory. Some of them are for test execution and interface file generation.
All input and output files for tests I store in a separate directory.
Interfaces/Documentation
The use of interface files (.mli) has both advantages and drawbacks for me. It really helps to find type errors, but if you have them, you have to edit them as well when making changes or improvements in your code. Sometimes forgetting this causes nasty errors.
But the main reason why I like interface files is documentation. I use ocamldoc to generate (OASIS supports this feature with -doc flag) html pages with documentation automatically. In my opinion it is enough to write comments describing each function in the interface and not to insert comments in the middle of code. In OCaml functions are usually short and concise and if there is a necessity to insert extra comments there, may be it is better to split the function.
Also be aware of -i flag for ocamlc. The compiler can automatically generate interface file for a module.
Tests
I didn't find a reasonable solution for supporting tests (I would like to have some ocamltest application), that's why I am using my own scripts for executing and verifying use cases. Fortunately, OASIS supports executing custom commands when setup.ml is run with -test flag.
I don't use OASIS for a long time and if anyone knows any other cool features, I would like also to know about them.
Also, it you are not aware of OPAM, it is definitely worth looking at. Without it installing and managing new packages is a nightmare.

Lemur Toolkit tutorial, installation and vs indri

I am new to IR and I was wondering whether it's better to use lemur or indri. lemur seems to have more features to me although indri seems more popular somehow..
and how to start, i cant seem to find any good basic tutorials... i need to know how to install and start a simple project
Lemur is a set of tools to experiment with search. Indri is a search index that comes with Lemur. So use Indri.
There is good introduction here.
I never found a good tutorial on how to install it but, it wasn't that hard.
Download and install the source
Create an Index
Query the Index
After you install the source it will install an executable called: buildindex. To use build index you will need to create an XML Parameter file. The documentation for these files is given here. You can use the same Parameter file for both index building and searching. This page lists the parts you need for index building. The tutorial listed at the top of this answer has example parameter files on slide 16 and 18.
If you want to use it for indexing and querying, you can use indri which is part of Lemur project. The indri installation folder (after you download and install the correct version from here has a lot of source codes itself.
For example you can find all the source codes of java in :
installationfolder\Indri\Indri 5.6\src\swig\src\java\lemurproject\indri
There is a sample application for index building in java:
installationfolder\Indri\Indri 5.6\src\swig\src\java\IndexUI.jar,
and one for retrieval process:
installationfolder\Indri\Indri 5.6\src\swig\src\java\RetUI.jar

Using Gradle with R project

I am beginning work on a project that makes heavy use of R. I've used R in the past, but only in a casual mode, whereas I'm now interested in following a more rigid practice of test/source control/continuous integration. I'm hoping to use Gradle with this project if possible, but I can't find any evidence that it is possible to use Gradle with R. Is it possible to create an R project with Gradle, and if so, where can I find steps to help me get started?
There appears to be a third-party plugin (https://github.com/jamiefolson/gradle-plugin-r). Alternatively, if R provides some command-line tools or Ant tasks, you can call into them from Gradle.
We have been working on gradle-R-plugin. (link to source is here https://github.com/arekbee/gradle-R-plugin). We have used it for CI/CD with TeamCity. gradle-R-plugin is base on devtools R package. I am happy to help you with this plugin.

Resources