I have a very simple knitr Rnw script. When I run it in RStudio, the Environment Pane shows that the global environment is empty although the script compiles into pdf correctly, and the variable is evaluated correctly.
\documentclass{article}
\begin{document}
<<settings, echo=FALSE>>=
library(knitr)
a<-1+10
#
The outcome is equal to \Sexpr{a}.
\end{document}
This worked fine always until recently. I wonder whether this has to do with some RStudio settings or knitr options. Variables in a regular R script show up fine in the environment pane. For more complex knitr projects, being able to look at the variables can make work much much easier.
Normally when you knit a document by clicking knit in RStudio, it is run in a separate R process, and the variables are deleted when done. Your code chunks won't be able to see variables in your environment, and those variables won't get modified.
There are ways to run in your current process: run each chunk as code, or run rmarkdown::render("somefilename.Rmd") in your main process. Then your document can see your current workspace and make modifications there.
For debugging, the 2nd approach is convenient, but for reproducibility of the final result, you should run the separate R process.
In the end, this is what worked for me. Instead of clicking on "Compile pdf" button in RStudio, I ran the following in the console:
knitr::knit2pdf("file.Rnw", envir = globalenv())
This had been recommended here:
knitr: why does nothing appear in the "environment" panel when I compile my .Rnw file
Related
TL;DR
What are the (possibly unwanted) side-effects of using knit()/knit2pdf() instead of the "Compile PDF"1 button in RStudio?
Motivation
Most users of knitr seem to write their documents in RStudio and compile the documents using the "Compile PDF" / "Knit HTML" button. This works smoothly most of the time, but every once a while there are special requirements that cannot be achieved using the compile button. In these cases, the solution is usually to call knit()/knit2pdf()/rmarkdown::render() (or similar functions) directly.
Some examples:
How to knit/Sweave to a different file name?
Is there a way to knitr markdown straight out of your workspace using RStudio?
Insert date in filename while knitting document using RStudio Knit button
Using knit2pdf() instead of the "Compile PDF" button usually offers a simple solution to such questions. However, this comes at a price: There is the fundamental difference that "Compile PDF" processes the document in a separate process and environment whereas knit2pdf() and friends don't.
This has implications and the problem is that not all of these implications are obvious. Take the fact that knit() uses objects from the global environment (whereas "Compile PDF" does not) as an example. This might be obvious and the desired behavior in cases like the second example above, but it is an unexpected consequence when knit() is used to overcome problems like in example 1 and 3.
Moreover, there are more subtle differences:
The working directory might not be set as expected.
Packages need to be loaded.
Some options that are usually set by RStudio may have unexpected values.
The Question and it's goal
Whenever I read/write the advice to use knit2pdf() instead of "Compile PDF", I think "correct, but the user should understand the consequences …".
Therefore, the question here is:
What are the (possibly unwanted) side-effects of using knit()/knit2pdf() instead of the "Compile PDF" button in RStudio?
If there was a comprehensive (community wiki?) answer to this question, it could be linked in future answers that suggest using knit2pdf().
Related Questions
There are dozens of related questions to this one. However, they either propose only code to (more or less) reproduce the behavior of the RStudio button or they explain what "basically" happens without mentioning the possible pitfalls. Others look like being very similar questions but turn out to be a (very) special case of it. Some examples:
Knit2html not replicating functionality of Knit HTML button in R Studio: Caching issue.
HTML outputs are different between using knitr in Rstudio & knit2html in command line: Markdown versions.
How to convert R Markdown to HTML? I.e., What does “Knit HTML” do in Rstudio 0.96?: Rather superficial answer by Yihui (explains what "basically" happens) and some options how to reproduce the behavior of the RStudio button. Neither the suggested Sys.sleep(30) nor the "Compile PDF" log are insightful (both hints point to the same thing).
What does “Knit HTML” do in Rstudio 0.98?: Reproduce behavior of button.
About the answer
I think this question raised many of the issues that should be part of an answer. However, there might be many more aspects I don't know about which is the reason why I am reluctant to self-answer this question (though I might try if nobody answers).
Probably, an answer should cover three main points:
The new session vs. current session issue (global options, working directory, loaded packages, …).
A consequence of the first point: The fact that knit() uses objects from the calling environment (default: envir = parent.frame()) and implications for reproducibility. I tried to tackle the issue of preventing knit() from using objects from outside the document in this answer (second bullet point).
Things RStudio secretly does …
… when starting an interactive session (example) --> Not available when hitting "Compile PDF"
… when hitting "Compile PDF" (anything special besides the new session with the working directory set to the file processed?)
I am not sure about the right perspective on the issue. I think both, "What happens when I hit 'Compile PDF' + implications" as well as "What happens when I use knit() + implications" is a good approach to tackle the question.
1 The same applies to the "Knit HTML" button when writing RMD documents.
First of all, I think this question is easier to answer if you limit the scope to the "Compile PDF" button, because the "Knit HTML" button is a different story. "Compile PDF" is only for Rnw documents (R + LaTeX, or think Sweave).
I'll answer your question following the three points you suggested:
Currently RStudio always launch a new R session to compile Rnw documents, and first changes the working directory to the directory of the Rnw file. You can imagine the process as a shell script like this:
cd path/to/your-Rnw-directory
Rscript -e "library(knitr); knit('your.Rnw')"
pdflatex your.tex
Note that the knitr package is always attached, and pdflatex might be other LaTeX engines (depending on your RStudio configurations for Sweave documents, e.g., xelatex). If you want to replicate it in your current R session, you may rewrite the script in R:
owd = setwd("path/to/your-Rnw-directory")
system2("Rscript", c("-e", shQuote("library(knitr); knit('your.Rnw')"))
system2("pdflatex", "your.tex")
setwd(owd)
which is not as simple as knitr::knit('path/to/your.Rnw'), in which case the working directory is not automatically changed, and everything is executed in the current R session (in the globalenv() by default).
Because the Rnw document is always compiled in a new R session, it won't use any objects in your current R session. This is hard to replicate only through the envir argument of knitr::knit() in the current R session. In particular, you cannot use knitr::knit(envir = new.env()) because although new.env() is a new environment, it has a default parent environment parent.frame(), which is typically the globalenv(); you cannot use knitr::knit(envir = emptyenv()), either, because it is "too clean", and you will have trouble with objects even in the R base package. The only reliable way to replicate what the "Compile PDF" button does is what I said in 1: system2("Rscript", c("-e", shQuote("library(knitr); knit('your.Rnw')")), in which case knit() uses the globalenv() of a new R session.
I'm not entirely sure about what RStudio does for the repos option. It probably automatically sets this option behind the scenes if it is not set. I think this is a relatively minor issue. You can set it in your .Rprofile, and I think RStudio should respect your CRAN mirror setting.
Users have always been asking why the Rnw document (or R Markdown documents) are not compiled in the current R session. To us, it basically boils down to which of the following consequences is more surprising or undesired:
If we knit a document in the current R session, there is no guarantee that your results can be reproduced in another R session (e.g., the next time you open RStudio, or your collaborators open RStudio on their computers).
If we knit a document in a new R session, users can be surprised that objects are not found (and when they type the object names in the R console, they can see them). This can be surprising, but it is also a good and early reminder that your document probably won't work the next time.
To sum it up, I think:
Knitting in a new R session is better for reproducibilty;
Knitting in the current R session is sometimes more convenient (e.g., you try to knit with different temporary R objects in the current session). Sometimes you also have to knit in the current R session, especially when you are generating PDF reports programmatically, e.g., you use a (for) loop to generate a series of reports. There is no way that you can achieve this only through the "Compile PDF" button (the button is mostly only for a single Rnw document).
BTW, I think what I said above can also apply to the Knit or Knit HTML buttons, but the underlying function is rmarkdown::render() instead of knitr::knit().
I'm using sweave package to make a report based on my R code. However, since some code chunks take too much time to process, I'm planning to use cacheSweave package to avoid this issue.
In cacheSweave's vignette, it says I need to specify a driver
Sweave("foo.Rnw", driver = cacheSweaveDriver)
However, I would like to keep using the "Compile PDF" button inside RStudio, so that it automatically runs Sweave command and pdflatex as well.
How do I tell RStudio to use that specific driver when calling Sweave function?
The expected result is that when I process the following ".Rnw" code twice (example based on code taken from cacheSweave's vignette), the second time will be much faster since data is cached.
\documentclass{article}
\begin{document}
\SweaveOpts{concordance=TRUE}
<cache=TRUE>>=
set.seed(1)
x <- local({
Sys.sleep(10)
rnorm(100)
})
results <- mean(x)
#
\end{document}
Sweave function help says *Environment variable SWEAVE_OPTIONS can be used to override the initial options set by the driver*. So I tried the following command in RStudio console,
Sys.setenv(SWEAVE_OPTIONS="driver=cacheSweaveDriver")
then "Compile PDF" twice again, but no success.
Solution:
As posted in this "ghost" blog, I created a file named .Rprofile in my working directory with the following content:
library(utils)
library(cacheSweave)
assignInNamespace("RweaveLatex", cacheSweave::cacheSweaveDriver, "utils")
This is a terribly simple request, and I can't believe I haven't found the solution to this yet, but I've been searching for it far and wide without luck.
I have an .Rnw file loaded up in Emacs, I use M-n s to compile it.
Everything works well, and it even opens an R buffer. Great. But that buffer
is entirely useless: it doesn't contain the objects that I just sourced!
Example minimal .Rnw file:
\documentclass{article}
\begin{document}
<<>>=
foo <- "bar"
#
\end{document}
Using M-n s, I now have a new R-buffer with a session loaded up, but:
> foo
Error: object 'foo' not found
That is disappointing. I would like to play around with the data interactively.
How do I achieve that? I don't want to be sourcing the file line-by-line, or
region-by-region with C-c C-c or something similar every time I change my code.
Ideally, it should be just like RStudio's source function, that leaves me with
a fully prepared R session.
I haven't tried this with sweave yet, only with knitr.
EDIT: the eval=TRUE chunk option does not seem to result in the correct behaviour.
This behaviour was recently changed in ESS. Now sweave and knitr are executed directly in the global environment, as if when you write it yourself at command line. So wait for a couple of more weeks till ESSv13.09 is out or use the development version.
Alternatively, you can also set ess-swv-processing-command to "%s(%s)" and you will get the same result, except automatic library loading.
For the record, knitr (in contrast to sweave) evaluates everything in it's own environment unless you instruct it otherwise.
[edit: Something went wrong. I don't see the correct .ess_weave any more. Probably some git commit messup again. So it is not fixed in 13.09. Fixing it now. Sorry.]
Open an interactive R session, and then call Sweave directly, I believe like this (untested though). knitr works in the same way, though you need to load the knitr library first.
> Sweave("yourfile.Rnw")
There is some potential for peril here, though. If you call Sweave in a session after doing other things, your code can use things previously in the workspace, thus making your results unreproducible.
Trying to write my second Latex file
\documentclass{article}
\title{simple Sweave script}
\usepackage{/Users/Brenden/Documents/R/R-2.15.1/share/texmf/tex/latex/Sweave}
\begin{document}
\maketitle
This is a simple script to demonstrate Sweave. Let's
begin by generating some results
<<>>=
x=rnorm(30)
y=rnorm(30)
mean(x)
cor(x,y)
#
and follow that up with some random text
\end{document}
File saved with .Rnw as extension. I could then use Sweave under R to convert the file to tex. I then run pdflatex under cmd to get my .pdf file. R is stored under /Users/Brenden/Documents. MiKTex is stored under /Users/Brenden/Desktop.
It is said that usually the line of "usepackage" is not needed, as when I run Sweave under R, a line of "usepackage{Sweave}" will be added to the tex file which is stored under /Users/Brenden/Documents. However, if I don't put in userpackage line, when I run pdflatex under cmd (either under /Documents or /Desktop), I got a messsage that says "Sweave.sty not found". So I have been through this trouble by always adding a line of usepackage with a detailed path to help circumvent the problem. Although the "Sweave.sty not found" problem is circumvented, I do notice that when I run pdflatex under cmd, I got the response
LaTeX Warning: You have requested package '/Users/Brenden/Documents/R/R-
2.15.1/share/texmf/tex/latex/Sweave', but the package provides "Sweave'.
Then it runs through several files with .sty under MiKTex
(C:\Users\Brenden\Desktop\MiKTex\tex\latex\base\ifthen.sty)
(C:\Users\Brenden\Desktop\MiKTex\tex\latex\graphics\graphicx.sty)
(C:\Users\Brenden\Desktop\MiKTex\tex\latex\graphics\keyval.sty)
(C:\Users\Brenden\Desktop\MiKTex\tex\latex\graphics\graphics.sty)
(C:\Users\Brenden\Desktop\MiKTex\tex\latex\graphics\trig.sty)
...
to eventually create a .pdf
From another post on Stackoverflow, it is said, "That path is set automatically when you install LaTeX; it's your personal texmf tree and you can put any style or class files you want LaTeX to find there. The directory names don't matter, it searches recursively in them for Sweave.sty". To me, however, clearly my MiKTex could not find the Sweave.sty unless I specify the path. And even with the path specification, LaTex still gives me a warning. Could someone explain to me where I screwed up (during installing MiKTex, perhaps? ) so that I could help MiKtex find its way to Sweave without my specifying the path?
Thank you.
In the MiKTeX Settings program, there is a tab called "Roots". It is there that you can specify where additional .sty files can be found. In your case, I believe that you will want to add C:\Users\Brenden\Documents\R\R-2.15.1\share\texmf as an additional root. Doing this, any of the TeX programs will be able to find Sweave.sty whether run from inside R or from the command line.
It is annoying to be always distracted by little devils like this (new concept texmf tree, endless environment variables TEXINPUTS, SWEAVE_STYLEPATH_DEFAULT, ...). You will never worry about LaTeX packages if you use knitr, and if you like the Sweave style, you can simply put render_sweave() in the document like:
\documentclass{article}
\title{simple Sweave script}
<<setup, include=FALSE>>=
render_sweave()
#
\begin{document}
\maketitle
This is a simple script to demonstrate Sweave. Let's
begin by generating some results
<<>>=
x=rnorm(30)
y=rnorm(30)
mean(x)
cor(x,y)
#
and follow that up with some random text
\end{document}
Save it as, say, test.Rnw and compile it with
library(knitr); knit('test.Rnw') # or knit2pdf('test.Rnw')
And I guess you are probably better off without render_sweave() (i.e. use the default LaTeX style in knitr).
I suspect the path referred to in the other post is the path to your personal texmf tree, which is not where Sweave puts it. One option is to move it to your personal texmf tree, but the usual method nowadays, I believe, is to run pdflatex from within R, like this:
texi2dvi("foo.tex", pdf = TRUE)
I think you can also have Sweave add the full path automatically by adding
export SWEAVE_STYLEPATH_DEFAULT="TRUE"
to your .bashrc file, or whatever the equivalent would be on your system.
At the command line, R CMD sets up the environment with variables appropriate for running R, for instance on Windows
C:\Users\mtmorgan> "c:\Program Files\R\R-2.15.1\bin\R" CMD set
includes (this is not found if one types set at the command line)
TEXINPUTS=.;c:/PROGRA~1/R/R-215~1/.1/share/texmf/tex/latex;
and so
C:\Users\mtmorgan> "c:\Program Files\R\R-2.15.1\bin\R" CMD pdflatex myfile.tex
will find Sweave.sty. Also, tools::texi2dvi can be invoked from within R.
I eventually end up with the following solution:
install.packages("tinytex")
require("tinytex")}
install_tinytex(force = TRUE)
Sweave( "Report.Rnw" , encoding = "utf8" )
xelatex('Report.tex')
This code install TinyTex, then you can compile with pdflatex(), xelatex() or lualatex().
When compiling with sweave/pgfsweave, every time a figure is created in R it is shown in a graphics windows (during the sweave compilation process). This is helpful in many cases as I can see what the figures look like as the document is being compiled.
But when I compile through ssh a large document this can be very slow. Is there a way to tell sweave/pgfsweave to avoid displaying the figure during the compilation (I still want the figure in the final pdf document though).
For interactive sessions, the figs.only Sweave option controls this behavior. To plot figures only to the target graphics files (and not to a console graphical window) set figs.only=TRUE.
As explained in the RweaveLatex help file:
figs.only: logical (‘FALSE’). By default each figure chunk is run
once, then re-run for each selected type of graphics. That
will open a default graphics device for the first figure
chunk and use that device for the first evaluation of all
subsequent chunks. If this option is true, the figure chunk
is run only for each selected type of graphics, for which a
new graphics device is opened and then closed.
As with other Sweave options, you can set this option: (1) for the current compilation (e.g. Sweave("example.Rnw", figs.only=TRUE); (2) within the .Rnw file, using \SweaveOpts{figs.only=TRUE}; or (3) as a global default, by putting SWEAVE_OPTIONS="figs.only=TRUE" in, e.g., $R_HOME/etc/Renviron.site
figs.only is the correct way to go, and I also want to mention the default graphical device in R here:
For now you may look at this: http://yihui.name/en/2010/12/a-special-graphics-device-in-r-the-null-device/
After R 2.14.1 (not released yet) you will be able to set the default device to a null PDF device, which is both safe and fast: https://github.com/yihui/knitr/issues/9
If you sweave from the command line instead of in an interactive session, graphics aren't produced in an interactive graphic window.
You can run R from the command line by just typing R CMD Sweave mydoc.Rnw or via a batch file, or a makefile for larger projects. I've started to use makefiles for many of my sweave documents as it handles dependencies, can clear up after itself and much more.
One option could be
<<label=myplotlabel, fig=TRUE, include=FALSE>>=
graph code
#
then
\begin{figure}[h]
\includegraphics[width=6cm, height=6cm]{myplotlabel}
\caption{My Plot}
\label{fig:label}
\end{figure}