I want to use testthat for integration tests in an R package being installed into a Jupyter notebook environment. Practically, that means I don't want the tests run when the package is installed, but rather manually later, when the system is running.
I think that means I shouldn't put tests into a tests/testthat directory, but rather the R directory.
So, I still want all the expect_ functions, but how do I run a test? I'm not sure if I can specify a file or directory, because it will be after the package is installed, and I don't know what the current working directory or install directory will be.
From the testthat package . . .
If you're using testthat in a package, you should put your tests in
tests/testthat. Each test file should start with test and end in .R or
.r. To ensure R CMD check runs your tests, place the following code in
tests/testthat.R:
library(testthat) ;
library(yourpackage)
test_check("yourpackage")
It sounds like you should follow all the setup but leave the testthat.R file out of the directory. You can run the test_check("yourpackage") from the console or a script.
Related
devtools::check() is a wonderful tool, but it can take a lot of time on big packages.
If I have an error in one step, I often would like to check this step only, without having to compute all the previous ones.
For instance, I have this annoying no visible binding for global variable 'xxx' note when running the checking R code for possible problems step. I'm not sure which call of xxx is causing the note, so I would love to run only this step.
On this particular matter, I tried codetools::checkUsagePackage("mypackage") but there are far too many false positives so that is not CRAN-like.
Is there any way to run a single step of devtools::check() at a time?
Only a partial answer. You can set some of the arguments of devtools::check to FALSE to skip some specific checks/parts and you can also pass arguments to the underlying R CMD check (via args) to skip even more.
Have a look at the R CMD check utility help for the parameters you can pass to it:
$ R CMD check --help
Usage: R CMD check [options] pkgs
Check R packages from package sources, which can be directories or
package 'tar' archives with extension '.tar.gz', '.tar.bz2',
'.tar.xz' or '.tgz'.
A variety of diagnostic checks on directory structure, index and
control files are performed. The package is installed into the log
directory and production of the package PDF manual is tested.
All examples and tests provided by the package are tested to see if
they run successfully. By default code in the vignettes is tested,
as is re-building the vignette PDFs.
Options:
-h, --help print short help message and exit
-v, --version print version info and exit
-l, --library=LIB library directory used for test installation
of packages (default is outdir)
-o, --output=DIR directory for output, default is current directory.
Logfiles, R output, etc. will be placed in 'pkg.Rcheck'
in this directory, where 'pkg' is the name of the
checked package
--no-clean do not clean 'outdir' before using it
--no-codoc do not check for code/documentation mismatches
--no-examples do not run the examples in the Rd files
--no-install skip installation and associated tests
--no-tests do not run code in 'tests' subdirectory
--no-manual do not produce the PDF manual
--no-vignettes do not run R code in vignettes nor build outputs
--no-build-vignettes do not build vignette outputs
--ignore-vignettes skip all tests on vignettes
--run-dontrun do run \dontrun sections in the Rd files
--run-donttest do run \donttest sections in the Rd files
--use-gct use 'gctorture(TRUE)' when running examples/tests
--use-valgrind use 'valgrind' when running examples/tests/vignettes
--timings record timings for examples
--install-args= command-line args to be passed to INSTALL
--test-dir= look in this subdirectory for test scripts (default tests)
--no-stop-on-test-error do not stop running tests after first error
--check-subdirs=default|yes|no
run checks on the package subdirectories
(default is yes for a tarball, no otherwise)
--as-cran select customizations similar to those used
for CRAN incoming checking
The following options apply where sub-architectures are in use:
--extra-arch do only runtime tests needed for an additional
sub-architecture.
--multiarch do runtime tests on all installed sub-archs
--no-multiarch do runtime tests only on the main sub-architecture
--force-multiarch run tests on all sub-archs even for packages
with no compiled code
By default, all test sections are turned on.
Report bugs at <https://bugs.R-project.org>.
R CMD check automatically runs tests located in tests/ directory. However running the tests this way requires building the package first. After that R CMD check goes through various different sanity checks before finally reaching the tests at the end.
Question: Is there a way to run those tests without having to build or install the package first?
NOTE: without using testthat or other non-standard packages.
To summarise our discussion.
To my knowledge there is no standard alternative to R CMD check for unit testing provided by base R
Typically for unit testing, I source everything under R/ (and dyn.load everything under source/) and then source everything under tests/ (actually, I also use the Example sections of the help pages in the man/ directory as test cases and compare their outcome to those from previous package versions)
I assume that these are the basic testing functionalities provided by devtools and testthat. If you expect to develop multiple packages and want to stay independent from non-base-R, I'd recommed to automate the above processes with custom scripts/packages.
I'd recomment looking into http://r-pkgs.had.co.nz/tests.html.
When building R packages, we use testthat to write tests. We have 2 files: a test file for the specific package (specific.R), and one that we use to make sure all packages continue to work together and the overall result is fine (overall.R). Both tests are currently run when we push to github or create a PR through Travis, which implicitly runs this line of code(R CMD check *tar.gz). check runs all the tests in the test folder, and thus both files are run.
Now, I'm a bit new to testing... but it seems that we have more or less recreated the difference b/w a unit test and an integration test.
Considering that the tests for overall.R do take a lot longer to run, we would like to restrict it so that they only run when we do a pull-request to the package (when we have introduced new functionality on a different dev branch) while the package-specific tests keep running every time we commit/push to the repo.
Is this possible in github or Travis?
You could put your overall.R script into a different directory and then specify this folder as the new tests directory for pull-request hooks, but this will then only run your integration tests and not the unit tests. See R CMD check --help
R CMD check --test-dir integration_tests package.tar.gz
When we use testthat in R, the package looks in tests/testthat directory and execute all the files that starts with 'test'... but if i have this type of organization:
tests
--/testthat
----/case1
------/Test1.R
------/Test2.R
----/case2
------/Test3.R
----/..
How can I execute all the tests in a unique command or file?
I try to change the path of testthat tests to 'tests/testthat/**/*.R' but testthat not iterate over directories.
Anybody knows a solution?
I'm new to R and I'm having some trouble to get the testthat unit test package work via devtools::test().
I've setup a package and created a test case under the .\tests\testthat folder. My R source code files are located at .\R.
When I run:
testthat::test_dir("./tests/testthat/")
The test ran successfully.
However when I tried to run the test via
devtools::test()
Instead of running the test cases, it tried to run my source code files located under .\R.
How can I get devtools::test() to just run my test cases?
Thank you for your help.
BTW, there is little documentation about how to setup and use testthat which is very frustrating as a new R user.
test() (re)loads your package before running your tests. That's why you see your package source code being executed.