I've seen similar questions on Stack Overflow but virtually no conclusive answers, and certainly no answer that worked for me.
What is the easiest way to access and use objects (regression fits, data frames, other objects) that are located in the global R environment in the Markdown (Rstudio) script.
I find it surprising that there is no easy solution to this, given the tendency of the RStudio team to make things comfortable and effective.
Thanks in advance.
For better or worse, this omission is intentional. Relying on objects created outside the document makes your document less reproducible--that is, if your document needs data in the global environment, you can't just give someone (or yourself in two years) the document and data files and let them recreate it themselves.
For this reason, and in order to perform the render in the background, RStudio actually creates a separate R session to render the document. That background R session cannot see any of the environments in the interactive R session you see in RStudio.
The best way around this problem is to take the code you used to create the contents of your global environment and move it inside your document (you can use echo = FALSE if you don't want it to show up in the document). This makes your document self-contained and reproducible.
If you can't do that, there are a few approaches you can take to use the data in the global environment directly:
Instead of using the Knit HTML button, type rmarkdown::render("your_doc.Rmd") at the R console. This will knit in the current session instead of a background session. Alternatively:
Save your global environment to an .Rdata file prior to rendering (use R's save function), and load it in your document.
Well, in my case i found the following solution:
(1) Save your Global Environmental in a .Rdata file inside the same folder where you have your .Rmd file. (You just need click at disquet picture that is on "Global Environmental" panel)
(2) Write the following code in your script of Rmarkdown:
load(file = "filename.RData") # it load the file that you saved before
and stop suffering.
Going to RStudio´s 'Tools' and 'Global options' and visiting the 'R Markdown' tab, you can make a selection in 'Evaluate chunks in directory', there select the option 'Documents' and the R Markdown knitting engine will be accessing the global environment as plain R code does. Hope this helps those who search this info!
The thread is old but in case anyone's still looking for a solution (as I was):
You can pass an envir parameter to the render() (or knit() function) so that it can access objects from the environment it was called from.
rmarkdown::render(
input = input_rmd,
output_file = output_file,
envir = parent.frame()
)
I have the same problem myself. Some stuff is pretty time consuming to reproduce every time.
I think there could be another answer. What if you save your environment with the save.image() function to a different file than the standard .Rdata one. Then, bring it back with load().
To be sure you are using the same data, use the md5sum() from tools.
Cheers, Cord
I think I solved this problem by referring to the package explicitly in the code that is being knitted. Using the yarrr package, for example, I loaded the dataframe "pirates" using data(pirates). This worked fine at the console and within an Rstudio code chunk, but with knitr it failed following the pattern in the question above. If, however, I loaded the data into memory by creating an object using pirates <- yarrr::pirates, the document then knitted cleanly to HTML.
You can load the script in the desired environment as follows:
```{r, include=FALSE}
source("your-script.R", local = knitr::knit_global())
# or sys.source("your-script.R", envir = knitr::knit_global())
```
Next in the R Markdown document, you can use objects created in these scripts (e.g., data objects or functions).
https://bookdown.org/yihui/rmarkdown-cookbook/source-script.html
One option that I have not yet seen is the use of parameters.
This chapter goes through a simple example of how to do this.
Is there any way to compile knitr subfiles separately? What I have in mind is something like the package subfiles for latex just in combination with R/knitr/Sweave?
This would be great in case one has two exercises a first exercise with heavy computations and
don't want to compile the entire exercise always while working and testing the second one.
The patchDVI package does this for Sweave. I imagine it would be possible (maybe even easy) to modify it to do the same for knitr.
For example, in Sweave, you define variables in a chunk like so:
<<>>=
.TexRoot <- "main.tex"
.SweaveFiles <- c("subfile1.Rnw", "subfile2.Rnw")
#
and after Sweave is finished running that file, patchDVI will check whether the files subfile1.Rnw and subfile2.Rnw also need to be run, then will run LaTeX on the main.tex file once everything is up to date.
You don't need to do anything difficult, just use the cache options. Lots of details here, but it's probably as simple as specifying cache = T in the chunk options of your first exercise.
This paper inspired me to check out Emac's org-mode a bit and currently I try to assess what's more suitable for writing my documents: knitr/Sweave (I'm mainly using R to do my programming) or org-mode.
What I really like about knitr is the option to externalize the actual source (watch out: the declaration of labels/names in the R script seems to have changed from ## ---- label ------- to ## #knitr label; see ?read_chunk) and "link" them to the actual literate programming/reproducible research document (as opposed to actually writing the code in that very document):
"Import" with
<<import-external, cache=FALSE>>=
read_chunk('foo-bar.R') # has label/name 'foo-bar'
#
and "re-use" by referencing the respective labels with
<<foo-bar>>=
#
Question
Is this also possible in org-mode or am I bound to putting the actual code into the .org document?
I found this, but I did not find any specific notion of linking/importing external source code files and be able to execute them by having the linked code inside
#+BEGIN_SRC R
<linked code>
#+END_SRC
Background
I do see that this approach might contrast the general paradigm of literate programing to some extend. But I like to work in a somewhat "atomic" style and thus it feels more natural to me to keep the files separated at first and then mash everything together dynamically.
Would named code blocks help?
#+NAME: foo-bar
#+BEGIN_SRC R
source(foo-bar.R)
#+END_SRC
And then evaluate (i.e. load) the code when you actually need it:
#+CALL: foo-bar()
See manual for more details.
I keep R and Rnw files separate, then load the R data/plots with load("file.R") in the first Sweave chunk. Is there a way that I can print the sourced R file to an appendix without executing all of the code? (i.e., the code is slow enough that I don't want to source() it in an echo=TRUE chunk).
Thanks!
Update -- actually, I don't think my source() idea works.
How about using a Latex package?
Add into your header
\usepackage{fancyvrb}
Then
\VerbatimInput{yourRfile.R}
You can use highlight package to output nicely formatted, colorful code:
highlight("myRfile.R", renderer = renderer_latex(document = F))
But don't forget to put in your latex doc the lengthy preamble which you get with document=T.
You can experiment with code directly:
highlight(output="test.tex",
parser.output = parser(text = deparse(lm)),
renderer = renderer_latex(document = T))
And get
I usually solve this by:
\begin{appendix}
\section{Appendix A}
\subsection{R session information}
<<SessionInforamtaion,echo=F,eval=T,results=tex>>=
toLatex(sessionInfo())
#
\subsection{The simulation's source code}
<<SourceCode,echo=F,eval=T>>=
Stangle(file.path("Projectpath","RnwFile.Rnw"))
SourceCode <- readLines(file.path("Projectpath","Codefile.R"))
writeLines(SourceCode)
#
\end{appendix}
Using this you have to think of a maximum numbers of characters per line.
Separating R and Rnw files sort of defeats the purpose of literate programming. My own approach is to include the code chunks at the appropriate place in the text. If my audience isn't interested in the code, then I might mark it as
<<foo, echo=FALSE>>=
x <- 1:10
#
I might assemble the code in an appendix as
<<appendix-foo, eval=FALSE>>=
<<foo>>
#
which I admit is a bit of a kludge and error prone (forgotten chunks). One quickly wants to bundle the document with supporting material (data sets, useful helper functions, non-R scripts) into an R package, and these are not difficult to create. Building the package automatically creates the pdf and Stangle'd R file, which is exactly what you want. Package building can be a slow process, but installing the package does not require that the vignettes be rebuilt and so is fast and convenient for whomever you're giving the package to.
For twiddling with formatting / text, I use a global option \SweaveOpts{eval=FALSE}.
The task is to create a file (word, rtf, pdf, html, or whatever) that will capture the output of R (e.g: not the code that created the output), into that format (including text and images).
The way of doing this should involve as little change to the original R script as possible.
If I had cared only for the text or images, then I would use ?sink, or ?pdf. But I don't know how to combine the two into one output in an easy way.
I know there is a way to export R output using r2wd, but it involves too much medaling in the original code for my taste (I imagine the same is true for the sweave solution, although I don't have experience with it to tell)
Here is a sample code for future examples:
START.text.and.image.recording("output.file") # this is the function I am looking for
x <- rnorm(100)
y <- jitter(x)
print(summary(x))
print(head(data.frame(x,y)))
cor(x,y)
plot(x,y)
print(summary(lm(y~x)))
STOP.text.and.image.recording("output.file") # this is the function I am looking for
Update: I was asked way not Sweave, or other options from ReproducibleResearch task view.
The reasons are:
I don't (yet) know LaTeX
Even knowing LaTeX, I want something with simple defaults to simply dump all the outputs together, and in order. "simply" means - as little extra code/file management overhead as possible.
I understand that something like sweave or brew are more scalable, but I am looking to see if there is a more "simple" solution for smaller projects/scripts.
As of 2012 knitr provides a perfect solution to this problem.
For example, create a file with an rmd extension. Wrap your code in a couple of commands as follows:
```{r}
x <- rnorm(100)
y <- jitter(x)
print(summary(x))
print(head(data.frame(x,y)))
cor(x,y)
plot(x,y)
print(summary(lm(y~x)))
```
You can convert it into a self-contained HTML file in several ways. In RStudio you just press a single button Knit HTML.
This is the HTML file produced; to actually view how the HTML displays in a browser, save the file and open it.
Images code, and output are interweaved as you might expect.
Of course, you can and typically would divide up your file into multiple R code chunks. But the point is, you don't have to.
Here are another couple of examples I've created:
Getting started with R Markdown
Case study in using R Markdown
If you know LaTeX, sweave will likely be your best bet. odfWeave is a similar mechanism but for embedding the code in an OpenOffice.org file. For HTML there is the R2html package. But all will likely require you to break the code up a little bit to get the best out of the systems. Alternatively, your sweave/odfweave/html template could source the data generation aspects of the script in a single code chunk, with the output display (print() statements) placed where required. Your graphics could also be called within the script to produce the figures to embed in the document as separate files, which you then include by hand in the template.
For example (and this isn't a full .Rnw file for running through sweave) in a sweave file you'd put something like this high up in the template which sources the main part of the R script that will do the analysis and generate the R objects:
<<run_script, eval=TRUE, echo=FALSE, results=hide>>=
source("my_script.R")
#
Then you will need to insert code chunks where you want printed output:
<<disp_output, eval=TRUE, echo=FALSE, results=verbatim>>=
## The results=verbatim is redundant as it is the default, as is eval=TRUE
print(summary(x)) ## etc
#
Then you will need chunks to insert the figures.
Separating your analysis code from the output (printed and/or figures) is probably good practice as well, especially if the analysis code is expensive in compute terms. You can run it once - or even cache it - whilst updating the output/display code as you need to.
Example Sweave File
Using csgillespie's example sweave file I would set things up like this. First the my_script.R file containing the core analysis code:
x <- rnorm(100)
y <- jitter(x)
corXY <- cor(x,y)
mod.lm <- lm(y~x)
Then the Sweave file
\documentclass[12pt]{article}
\usepackage{Sweave}
\begin{document}
An introduction
<<run_analysis, eval=TRUE,echo=FALSE, results=hide>>=
source("my_script.R")
#
% Later
Here are the results of the analysis
<<show_printed_output, echo=FALSE>>=
summary(x))
head(data.frame(x,y))
#
The correlation between \texttt{x} and \texttt{y} is:
<<print_cor, echo=FALSE>>=
corXY
#
Now a plot
\begin{figure}[h]
\centering
<<echo=FALSE, eval=TRUE, fig=TRUE, width=6, height=4>>=
plot(x,y)
#
\caption{\textit{A nice plot.}}
\end{figure}
\end{document}
What you seem to be wanting doesn't exist; a simple way of combining R code and output into a document file. That is if you don't consider sweave and its ilk simple. You might need to rethink what you want to do or how you arrange your analysis and graphics and output code, but you are likely best served looking at one of the suggested options (sweave, odfweave, brew, R2html).
HTH
I would encourage you to use Sweave, but a rudimentary functionality that is not pretty can be achieved with sink().
A regular txt file:
sink(file = "test.txt", type = "output")
summary(cars)
sink()
or add some HTML tags:
sink(file = "tal_test.html", type = "output")
cat("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"", "\n")
cat("\"http://www.w3.org/TR/html4/strict.dtd\">", "\n")
cat("<HTML>", "\n")
cat("<HEAD>", "\n")
cat("<TITLE>My first HTML document</TITLE>", "\n")
cat("</HEAD>", "\n")
cat("<BODY>", "\n")
summary(cars)
cat("</BODY>", "\n")
cat("</HTML>", "\n")
sink()
I wrote a script called Roux about a year ago which does this. I wanted to be able to create HTML transcripts from running an R script, including any images, without having to change the script.
You call Roux from the command line, like this:
roux example.R
and roux will:
run the script in R (requiring the Roux package first automatically)
syntax highlight the .Rout output using Pygments
insert images in the correct location
the Roux R package is a very small R package which modifies plot() and some other functions to automatically write to a random filename rather than the default interactive graphics device.
I have used this a lot, and it works really well for me, although I'm sure if more people use it with new packages then minor issues will arise, most likely that you'll have a different function which generates a graph and Roux won't know that it should open a PNG device for you.
Since speaking with Tal about this I have updated and improved the code, and it's now up here:
http://bitbucket.org/ananelson/roux/src
so if you run into any issues, please report them to the issue tracker there on Bitbucket.
I have added support for LaTeX transcripts so you can easily create PDFs which have the transcript of your R script including images. (You can see an example if you look in the example-output directory, find the "raw" link to download it.)
You do need to have Python and the Pygments python library intalled. If you have an older version of Python and run into any issues, please let me know.
I wrote about Roux on my blog but didn't publicize it that much because my efforts have been focused on a new project called Dexy which is intended as a replacement for Sweave. If you want more flexibility and control or are interested in literate documentation then you might want to check out Dexy too.
You mentioned sweave in your question but not really why it isn't suitable. Your question seems perfect for Sweave. In fact, your example code could have came from the second Sweave example.
Example Sweave file
If you know Latex then Sweave isn't that difficult. Here's your example file as a Sweave file:
\documentclass[12pt,BCOR3mm,DIV16]{scrreprt}
\usepackage{Sweave}
\begin{document}
An introduction
<<eval=TRUE,echo=TRUE>>=
x <- rnorm(100)
y <- jitter(x)
print(summary(x))
print(head(data.frame(x,y)))
cor(x,y)
#
Now a plot
\setkeys{Gin}{width=0.5\textwidth}
\begin{figure}[h]
\centering
<<echo=FALSE, eval=TRUE, fig=TRUE, width=6, height=4>>=
plot(x,y)
#
\caption{\textit{A nice plot.}}
\end{figure}
\end{document}
Under linux, just save the file as tmp.Rnw. Then
R CMD Sweave tmp.Rnw
pdflatex tmp.tex
There is also LyX, which has an Sweave interface. The R / LyX / Sweave interface code is on CRAN at http://cran.fhcrc.org/contrib/extra/lyx/. LyX itself is in most of the Linux distros. All of this magic can be made to work on Windows, but it's definitely non-trivial. On Windows, I'd recommend Inference for R from Blue Reference for literate R progamming.
Well, I just remind that I was using Asciidoc for short reporting or editing webpage. Now there's an R plugin (ascii on CRAN), which allows to embed R code into an asciidoc document. The syntax is quite similar to Markdown or Textile, so you'll learn it very fast.
Output are (X)HTML, Docbook, LaTeX, and of course PDF through one of the last two backends.
Unfortunately, I don't think you can wrap all your code into a single statement. However, it supports a large number of R objects, see below.
> methods(ascii)
[1] ascii.anova* ascii.aov* ascii.aovlist* ascii.cast_df*
[5] ascii.character* ascii.coxph* ascii.CrossTable* ascii.data.frame*
[9] ascii.default* ascii.density* ascii.describe* ascii.describe.single*
[13] ascii.factor* ascii.freqtable* ascii.ftable* ascii.glm*
[17] ascii.htest* ascii.integer* ascii.list* ascii.lm*
[21] ascii.matrix* ascii.meanscomp* ascii.numeric* ascii.packageDescription*
[25] ascii.prcomp* ascii.sessionInfo* ascii.simple.list* ascii.smooth.spline*
[29] ascii.summary.aov* ascii.summary.aovlist* ascii.summary.glm* ascii.summary.lm*
[33] ascii.summary.prcomp* ascii.summary.survfit* ascii.summary.table* ascii.survdiff*
[37] ascii.survfit* ascii.table* ascii.ts* ascii.zoo*
Non-visible functions are asterisked
This is in light of romunov's answer, but still. You can just write your own print that wraps the output in some HTML formatting and embeds the output to a HTML file. The same can be done with pictures with Data URI scheme, for instance by using img function from base64 R package.
You can use the R2HTML package to output a session to html and there are some similar functions in the TeachingDemos package (see txtStart) for output to enhanced text and word (via R2wd). Non-graphics commands will be included in the file automatically and the current plot can be inserted by a single command.
Through the wonders of twitter, someone reached out and sent me a link to this page, regarding a package called "roux". It was created a year ago, and I have never heard about it (apparently neither have most of you).
This package seems to do exactly what I was looking for in my question, although the installation seems non trivial.
I hope to play with this solution and also to see if other R members might go into this project to better enhance R.
good suggestion by #znmeb to try Lyx - a more word-like front end for LaTeX, and as the documentation points out, there is a good article of its use with Sweave on page 2 of this edition of R news
This is how I did it in Ubuntu 10.04 follwoing the guidelines in the lyx sweave repository:
sudo apt-get install lyx
cd ~./lyx
wget http://cran.fhcrc.org/contrib/extra/lyx/preferences
cd layouts
wget http://cran.fhcrc.org/contrib/extra/lyx/literate*
wget http://cran.fhcrc.org/contrib/extra/lyx/literate-article.layout
wget http://cran.fhcrc.org/contrib/extra/lyx/literate-book.layout
wget http://cran.fhcrc.org/contrib/extra/lyx/literate-report.layout
wget http://cran.fhcrc.org/contrib/extra/lyx/literate-scrap.inc
cd ~/texmf/tex
wget http://www.biostat.jhsph.edu/~rpeng/ENAR2009/Sweave.sty
start Lyx
Preferences -> Reconfigure
restart Lyx
File -> new
Document -> Settings -> Document Class -> article (Sweave noweb)
useful links:
lyx sweave repository
R news article about Lyx and Sweave