I am writing an R package with the following lint test:
context("Require no code style errors")
library(lintr)
test_that("Package has no lintr errors", {
lintr::expect_lint_free()
})
The tests pass with `devtools::check():
$ Rscript -e "devtools::check()"
...
─ checking tests ...
✔ Running ‘testthat.R’ [61s/65s]
...
0 errors ✔ | 0 warnings ✔ | 0 notes ✔
and the lint-free test fails with devtools::test():
$ Rscript -e "devtools::test()"
...
Testing PosteriorBootstrap
...
✖ | 0 1 | Require no code style errors [6.2 s]
────────────────────────────────────────────────────────────────────────────────
test_lintr.R:5: failure: Package has no lintr errors
Not lint free
tests/testthat/test_anpl.R:112:1: style: Trailing whitespace is superfluous.
^~
...
OK: 20
Failed: 1
Warnings: 0
Skipped: 0
The problem is that Github and Travis are set to reject pull requests that fail the tests, and that if I run devtools::test() after devtools::check(), all the other tests run twice.
How can I get devtools::check() to run the lintr test?
This problem is a known issue:
This is not a bug in devtools (but maybe in lintr). devtools::check() runs the check in a temporary directory, but lint_package() assumes it is being run in a package directory, therefore there are no source files to be linted. ... you can confirm this with devtools::check(check_dir = "."), which should produce linting failures if devtools::test() does.
The solution proposed, written in May 2015, no longer works. The issue is now locked and closed, so it's unlikely to be addressed.
I suggest running the check and narrowing the test only to lintr:
Rscript -e "devtools::check();devtools::test_file(file = 'testthat/test_lintr.R')"
Related
I'm running my package through R CMD check and the only (remaining) warning is the following:
W checking for unstated dependencies in 'tests' (4.4s)
'::' or ':::' import not declared from: 'devtools'
After getting confused for a long time about this seemingly nonsensical warning, I realized it's coming from my "test manager" script (see reason for its need below). This is file pkg/tests/testthat.R, while the tests themselves are in pkg/tests/testthat/.
# testthat.R
sink(stderr(), type = "output")
x <- tryCatch(
{
x <- data.frame(devtools::test()) # here's the problem!
as.numeric(sum(x$failed) > 0)
},
error = function(e) {
1
}
)
sink(NULL, type = "output")
cat(1)
If I comment out this entire file, the R CMD check warning vanishes.
And then the weird part: if I replace devtools::test() with just test(), the R CMD check warning vanishes.
However, the purpose of this "manager" script is to be to be called (via Rscript) by a git pre-commit hook. This way, I can run all my tests to ensure the commit is stable. Due to this, I can't use test(), since devtools isn't loaded when the script is run via Rscript.
I tried a few things to satisfy both R CMD check and being called by Rscript:
Using library(devtools) doesn't work (throws a package not found error);
Moving testthat.R out of the /tests/ folder and into the top-level. This kills the R CMD check warning, but it now instead throws a note: Non-standard file/directory found at top level: 'testthat.R', so not exactly satisfactory (especially since keeping it in the /tests/ directory seems more logically consistent);
Testing for a function which has been apparently loaded by R CMD check to determine behavior. Since using a naked test() works, I assumed devtools was loaded, so prepended the following to the file (and used runTests on the problematic line). The logic being, if we can find test(), use it. If we can't, then this probably isn't R CMD check, so we can use the full name.
if (length(find("test")) == 0) {
runTests <- devtools::test()
} else {
runTests <- test()
}
Unfortunately, this just made things worse: the warning remains and we also get an error on the if-else block:
> if (length(find("test")) == 0) {
+ runTests <- devtools::test()
+ } else {
+ runTests <- test()
+ }
Error in loadNamespace(name) : there is no package called 'devtools'
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Why devtools::test() throws an error here and just a warning on the problematic line is beyond me.
Similarly, using testthat::skip(). Also doesn't work.
So, what can I do to satisfy both R CMD check and being called by Rscript? Is there a way to tell R CMD check to ignore this file?
For the record, this is my git pre-commit hook, in case it can be reformulated to solve this problem some other way
#!/bin/sh
R_USER="D:/Users/wasabi/Documents"
export R_USER
# check that Rscript is accessible via PATH; fail otherwise
command -v Rscript >/dev/null || {
echo "Rscript must be accessible via PATH. Commit aborted.";
exit 1;
};
# check whether there are unstaged changes. If so, stash them.
# This allows the tests to run only on previously committed or
# indexed (added on this commit) changes.
hasChanges=$(git diff)
if [ -n "$hasChanges" ]; then
git stash push --keep-index
fi
exitCode=$(Rscript tests/testthat.R)
# remember to unstash any unstaged changes
if [ -n "$hasChanges" ]; then
git stash pop
fi
exit $exitCode
The solution is to simply add tests/testthat.R to .Rbuildignore (either by hand in the form of a regular expression or using usethis::use_build_ignore("tests/testthat.R")).
If you actually run R CMD check, the warning will still appear (since it runs on the source files, and therefore ignores .Rbuildignore, unless you run it on the binary itself).
But the "Check Package" command in RStudio relies on devtools::check(), which builds the package first and then checks the binary, therefore not getting the error. And since that's how my team and I will actually be running the checks, it's sufficient.
Solution inspired by this question.
I am trying to add Codecov support via library(covr) to my personal R package sesh.
When I check locally the coverage tests run and report without incident:
covr::package_coverage()
sesh Coverage: 68.75%
R/executeDevtoolDocument.R: 0.00%
R/sesh.R: 69.23%
But when it runs on Travis it encounters an error for missing token:
$ Rscript -e 'covr::codecov()'
Error in if (nzchar(token)) { : argument is of length zero
Calls: <Anonymous>
Execution halted
The R CMD check runs successfully on Travis.
The contents of my .travis.yml:
language: R
matrix:
include:
- r: release
after_success: Rscript -e 'covr::codecov()'
r_github_packages:
- r-lib/covr
And a link to the most recent Travis report.
I have tried to faithfully follow the covr README for getting set up. And the README says Travis is supported without needing CODECOV_TOKEN, so I have not tried to pass one yet.
What am I missing here?
Following is my .travis.yml
language: r
cache: packages
script:
- R CMD build .
- R CMD check *tar.gz
r_github_packages:
- r-lib/covr
after_success:
- Rscript -e 'covr::codecov()'
Adding the repository upload token to codecov.yml avoids the error and successfully runs the coverage report.
codecov:
token: a1c53d1f-266f-47bc-bb23-3b3d67c57b2d
The token is found in the 'Settings(tab) >>> General(sidebar)' menu on the Codecov page for the repo (which is only visible once you are logged in).
I am running two pipeline stages on my GitLab cl (homemade Docker contatiner with R-base on Ubuntu:16.04). The only track of the error is in the codecov step (while R check is successful). This is the error message and command (on GitLab):
$ Rscript -e 'covr::package_coverage(type="tests", quiet = FALSE)'
(...)
* DONE (mypkg)
Running specific tests for package ‘mypkg’
Running ‘testthat.R’
Error: Failure in `/tmp/RtmpGgElCC/R_LIBS94b18abb4/mypkg/mypkg-tests/testthat.Rout.fail`
As usual, I can not replicate this error locally. No other message related to the error is shown. Moreover, I can not find a way to retrieve that log file. Is it possible?
Use a Codecov token:
# your .gitlab-ci.yml, ending with:
- apt-get install —yes git
- R -e 'covr::codecov(token = "yourtoken")'
Get your token from https://codecov.io/your_name/your_project/settings.
See my own implementation at https://gitlab.com/msberends/AMR :)
I have an R package that was passing all Travis CI builds successfully and randomly started to fail. I've pondered where the problem is at but I can't seem to fix it.
Here is the travi.yaml from the package
Here is the failing log of Travis-CI
Here is the last log the passed (about 2-3 logs behind the last fail)
I think the fail might be related to line 1495 from the last log that passed:
The command "for name in $(find "${RCHECK_DIR}" -type f -name "*out");do echo ">>> Filename: ${name} <<<";cat ${name};done" exited with 0.
which does not appear in the failed log from above. After thoroughly checking my tests with devtools:
R CMD check results
0 errors | 0 warnings | 0 notes
I don't know what it might be. I thought it was related to some examples taking too much time (some examples pass the 5 second thresholds, so I comment them out but the failing persists. I tried posting this in the Rstudio community but no luck so far.
Any ideas?
I am testing an R package called eutradeflows on travis. The package contains test programmed with testthat and I would like to see the output of devtools::test() in travis.
There was a line in the main travis log saying :
Status: 4 NOTEs
See ‘/home/travis/build/stix-global/eutradeflows/eutradeflows.Rcheck/00check.log’
for details
From this answer, I learned that its possible to display a file in the travis log. In .travis.yml I have asked travis to print that file after the test:
- cat /home/travis/build/stix-global/eutradeflows/eutradeflows.Rcheck/00check.log
But it doesn't contain the result of testthat tests.
How can I display the output of testthat tests in travis?
This is particularly important since I have skip instructions in the tests and I would like to know which tests have been skipped.
To display the result of testthat tests,
add this to .travis.yml:
r_binary_packages:
- devtools
- roxygen2
after_success:
- Rscript -e 'devtools::install();devtools::test()'
I used a slightly heavier weight fix to ensure that build and test only happened once. In the build matrix I put the following in for the script
script: |
R CMD build .
R CMD INSTALL RMINC*.tar.gz
R CMD check --as-cran --no-install RMINC*.tar.gz
cat RMINC.Rcheck/00check.log
Rscript -e "\
testr <- devtools::test(); \
testr <- as.data.frame(testr); \
if(any(testr\$error) || any(testr\$warning > 0)) \
stop('Found failing tests') \
"
pass=$?
if [[ $pass -ne 0 || $(grep -i "WARNING\|ERROR" RMINC.Rcheck/00check.log) != "" ]]; then
(exit 1)
else
(exit 0)
fi
This runs R CMD check and properly displays build output, failing on warnings or errors in either check or test phases.