R's testthat package has a number of functions for running tests: https://testthat.r-lib.org/reference/index.html#run-tests. However, the most coarse level you can filter tests seems to be at a file level, since there is a test_file() function that doesn't have any filtering arguments, and test_dir() has a filter argument but it is only used to filter by filename.
However I frequently want to only run a single test, because it's new, or because I know it's relevant to a change I just made.
Is there a way, in the R console or in RStudio to run a single testthat test? If not, is there some other recommended solution to this problem such as putting each test in it's own file (this seems pretty painful though)?
Related
tl;dr I want to run devtools::test() on a package and have it skip tests etc. as though it were running on CRAN, but I can't figure out how.
As I understand it, testthat::skip_on_cran() checks for the environment variable NOT_CRAN, which should be set to a value of "true" if the tests are not being run on CRAN (to back this up, the underlying test function testthat:::on_cran() is equal to
!identical(Sys.getenv("NOT_CRAN"), "true")
I am trying to use skip_on_cran() to skip some tests. I want to confirm that these tests will actually be skipped on CRAN. I have a line
cat("ON CRAN:", testthat:::on_cran(), "\n")
in my test file so that I can see what R/testthat thinks is going on.
The environment variables get set the way I want (i.e., the output includes ON CRAN: FALSE)/the tests get skipped properly) if I use
source([testfile], echo = TRUE)
(i.e., without doing anything special to set or unset the NOT_CRAN environment variable beforehand) or
withr::with_envvar(c(NOT_CRAN = "false"),
devtools::test_active_file("tests/testthat/test-bootMer.R"))
(if I run test_active_file() without wrapping it, I get ON CRAN: FALSE).
However, I don't see a way to run all the tests (via devtools::test()) in a similar way. In other words, I can't figure out how to run devtools::test() in "ON CRAN" mode. test() doesn't have an explicit argument for this (it has ... which is "additional arguments passed to wrapped functions", but I can't see anything relevant digging downward), and using withr::with_envvar() doesn't seem to help. devtools::check() does have an explicit env_vars argument, but I would like to be able to run the tests without going through the whole package check procedure ...
I'm sorry this isn't fully reproducible; if requested I can try to build a minimal package that will display the behaviour ...
Not sure if you still have this question but I put this line at the top of my first test file and it works in R 4.2.1.
Sys.setenv(NOT_CRAN='skip')
I am using the tinytest package for unit testing and wanted to modify some of the tests which did not work out. To boil down the problem I give the following example. I define a new function
expect_equal_new <- function(a,b){expect_equal(a,b)}
If I put expect_equal_new and expect_equal in a test file and run test_all() from the tinytest package, only the result of expect_equal appears in the summary.
Why is that the case? Is there a way that test_all() also finds expect_equal_new?
How can I run testthat in 'auto' mode such that when I'm editing files in my R folder, only specific tests are re-run?
I have a lot of tests and some are slower than others. I need to be able to run specific tests or else I'll be waiting for up to 15 minutes for my test suite to complete. (I'd like to make the test suite faster, but that's not a realistic option right now.)
Ideally, I'd like to specify a grep expression to select the tests I want. In the JavaScript world, MochaJs and Jest both support grepping to select tests by name or by file.
Alternatively, I'd be OK with being able to specify a file directly - as long as I can do it with "auto test" support.
Here's what I've found so far with testthat:
testthat::auto_test_package runs everything at first, but only re-runs a specific test file if you edit that test file. However, if you edit any code in the R folder, it re-runs all tests.
testthat::auto_test accepts a path to a directory of test-files to test. However, testthat doesn't seem to support putting tests into different subdirectories if you want to use devtools::test or testthat::auto_test_package. Am I missing something?
testthat::test_file can run the tests from one file, but it doesn't support "auto" re-running the tests with changes.
testthat::test_dir has a filter argument, but it only filters files, not tests; it also doesn't support "auto" re-running tests
Versions:
R: 3.6.2 (2019-12-12)
testthat: 2.3.1
Addendum
I created a simple repo to demo the problem: https://github.com/generalui/select_testthat_tests
If you open that, run:
renv::restore()
testthat::auto_test_package()
It takes forever because one of the tests is slow. If I'm working on other tests, I want to skip the slow tests and only run the tests I've selected. Grepping for tests is a standard feature of test tools, so I'm sure R must have a way. testthat::test_dir has a filter option to filter files, but how do you filter on test-names, and how do you filter with auto_test_package? I just can't find it.
How do you do something like this in R:
testthat::auto_test_package(filter = 'double_it')
And have it run:
"double_it(2) == 4"
"double_it(3) == 6"
BUT NOT
"work_hard returns 'wow'"
Thanks!
I have a suite of unit tests in R spread out across numerous files. These tests all run just fine via the 'testthat' package.
However, I cannot seem to find a way to filter tests run by the package's 'context' name. For example, if I have tests spread across different files that all fall under a context with the name "Integration," how can I get R to only run those?
I am aware of the 'filter' argument in the test() function that allows you to filter by test file name, but I am looking to filter by context in this case. I was not able to find anything related to such functionality in the documentation, so any help would be appreciated!
Link to package -> https://cran.r-project.org/web/packages/testthat/testthat.pdf
I am creating an R package, and found it useful to break parts of the logic in one file into internal helper functions, which I define in the same file. I have sort of a special case where my function decides which helper function to use via match.fun(). Since they won't be useful to other functions or people, I don't want to put these in separate files, and I don't want to export them.
All my testthat cases pass using test_dir(). When I don't export these functions, my testthat cases fail during R CMD check.
"object 'helperfunction1' of mode 'function' was not found", quote(get(as.character(FUN),
mode = "function", envir = envir)))
After looking at this post, I am able to get things to work if I explicitly export or add export entries to NAMESPACE, but again I don't want to export these.
Is there a better way to do this and doesn't require me to export? (I'll admit that the source of the issue may be match.fun() and am open to other ways of calling functions at runtime.)
From memory it wasn't clear in the documentation last time I read it (it may have changed), but it will work correctly (without having to export) so long as everything is in the right directories:
You should have a file:
tests/run-all.R
That looks like:
library(testthat)
library(myPackage)
test_package("myPackage")
Then your individual test files should be in the directory inst/tests
These will be run when you do R CMD check, otherwise you can call test_package("myPackage") in R manually.