I'm working in RMarkdown, trying to render a document that has some UTF-8 characters in it. When I push the "Knit" button in RStudio, everything works great. But when I use the render() function, the UTF-8 gets lost. Here's a short snippet of reproducible code:
---
output: html_document
---
Total nitrogen (µg/L)
Water temperature (°C)
Pushing the Knit button gives me the correct output, whether I view it in RStudio or in Chrome. But if I render the file with render(), I get:
Total nitrogen (µg/L)
Water temperature (°C)
I'm working in Windows, which may be the source of much of the problem. Here's my locale info.
Sys.getlocale("LC_ALL")
[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
I've tried adding a code chunk with "options(encoding = 'UTF-8')" but it doesn't help. I'm using pwalk() to generate 36 reports automatically with different parameters, so I need to get this working with render().
You can force encoding:
render("test.html",encoding="UTF-8")
You can also set the encoding using you R terminal:
options(encoding = 'UTF-8')
render("test.html")
I was considering this as a comment because it does not necessarily answer your question, however, it is too long to get lost there.
Firstly, using the knit button in RStudio does call render, so all things being equal, both running from the console and via the GUI will produce the same output.
your environment
An important note from jjallaire in an old closed issue on Github:
when RStudio calls render it is in a fresh R process rather than in the global environment of the current session (which is what you would git when calling render at the console)
A good question that provides context is here.
initial conclusion
If the document renders correctly using the GUI button and not from the console, then there is something in your environment causing the encoding to be read incorrectly.
Try from a clean session, if it still produces the same output then that would suggest an issue in the the environment at startup. Check the encoding...
getOption("encoding")
# [1] "native.enc"
Instead of placing options(encoding = "UTF-8") in a code chunk, execute it before your render statement. You can check that it has changed by running the getOption as above again and confirm it now returns # [1] "UTF-8"
Related
In R Markdown if I have a chunk coded with the normal r chunk wrappers,
```{r}
[R code]
```
A play button will appear. However, when I run a chunk wrapped like so for other language engines:
```{python}
[python code]
```
or replacing python replaced with bash/sh, I don't get a play button on the right.
Is there a package I need to install? I restarted Rstudio, and even downloaded the Lesson 5: https://rmarkdown.rstudio.com/lesson-5.html , which even shows those play buttons, and I still don't get it. I am on a mac, and saw that someone on a Windows computer was able to see the play buttons.
I think you need to install the package "knitr". Try to check that first.
Second, you might need to state explitly where your interpreter for Python is, for instance. Use
{python, engine.path = '/usr/bin/python3'}
YOUR CODE
to see if it works.
How can I force RStudio (v1.1.383) to evaluate R chunks always in console (instead of inline) when working with Rmarkdown documents, using a script?
I know I can set up Output chunks in Console by clicking on it:
According to this RStudio support post I could also un-check 'Show output inline for all R Markdown document' under 'Tools -> Global Options...':
But, is there a way to do it from a command line?
The reason I ask is that, I often work on my university machines and they all restore to defaults after each reset. Each time when in class, we have to manually go thru menus.
Knowing how to do it via a console command would as useful as starting each of my classes with
rm(list=ls())
There's not currently an elegant way to do this. This preference is stored inside an internal RStudio state file, in %localappdata%\RStudio-Desktop\monitored\user-settings. If you're sufficiently motivated you can write a script which sets the rmd_chunk_output_inline preference, but it's going to be unpleasant.
One thing you can do is set the chunk output type in the YAML header, like this:
---
editor_options:
chunk_output_type: console
---
You could also use an R Markdown document template with this set up for you (maybe your script could write this out).
Finally, there's an open issue for this on RStudio's github page which you might comment on and/or vote for:
https://github.com/rstudio/rstudio/issues/1607
I have an RMarkdown document that includes 'special characters', such as ë. If I render the document using RStudio Server's "knit document" button, it renders fine. When I render it by using the RStudio Server button to source another R script that calls RMarkdown's render function, it also renders fine.
However, from some reason that's beyond me (but hopefully not for long), I get different results when that same R script is called by index.php using:
$results = shell_exec("R --file='/home/username/public_html/some/subdirectories/process.R' --no-save 2>&1");
When I do this, in the resulting .html file, the special symbols (I guess the unicode symbols) are replaced by <U+00EB>. I've tried to look up whether this is some kind of variation of HTML elements that I didn't know about yet, but I have been unable to find anything about this.
(note: any link to a place where I can learn more about this (and, while we're at it, why my browser doesn't show it as, for example, the ë it represents, is also greatly appreciated!)
Reproducable example
Contents of example.php:
<?php
shell_exec("R --file='/home/username/public_html/subdirectory/example.R' --no-save 2>&1");
?>
Contents of example.R (this is what I needed on my server):
workingPath <- "/home/username/public_html/subdirectory";
### Set path to RStudio's pandoc version
Sys.setenv(PATH=paste(Sys.getenv("PATH"),
"/usr/lib/rstudio-server/bin/pandoc",
sep=":"));
### Set HOME and LANG
Sys.setenv(HOME = '/home/username');
Sys.setenv(LANG = 'en_US.UTF-8');
require(rmarkdown);
renderResults <-
render(file.path(workingPath, 'example.Rmd'),
output_file = file.path(workingPath, 'example.html'),
intermediates_dir = file.path(workingPath, 'tmp'),
encoding="UTF-8");
Contents of example.Rmd:
---
title: 'Reproducable example'
output: html_document
---
```{r}
cat("This is an ë symbol.");
```
Results of this example:
When I run this from R Studio, I get:
cat("This is an ë symbol.");
## This is an ë symbol.
When I run this from PHP, I get:
cat("This is an ë symbol.");
## This is an <U+00EB> symbol.
(note how, interestingly, the echo'ed ë does show up normally...)
I now resorted to doing a str_replace in the index.php file, but that's not ideal.
I've checked the render manual, but I can't find anything about this behavior.
I've also looked at specifying options for pandoc in the YAML header of the .Rmd file, but the only thing that seems to come close is the --ascii option, and that doesn't do anything. The R Studio RMarkdown page doesn't provide any hints, either.
Could it perhaps have to do with environment variables that are set in RStudio? I already had to set:
Sys.setenv(HOME = '/home/oupsyusr');
Sys.setenv(LANG = 'en_US.UTF-8');
in the R script to get Pandoc going in the first place when called in the R script called from the PHP shell; but if this is the problem, how can I figure out which settings RStudio sets to which values, or more accurately, which of those are important? I ran:
Sys.getenv()
From within R Studio, and that shows quite a list. I recognize none of the entries as having to do with encoding or so.
Or, does knitr cause this? When I store and inspect the .md file, the Unicode element things already show up. However, the knitr help page with chunk options doesn't say anything about unicode or encoding in general.
Does anybody know where this is documented, or does anybody happen to have encountered this situation before?
I'm running RStudio 0.99.903 and R 3.3.1 on CentOS 6.8.
Usually, issues of this form (where unicode characters are converted to a unicode code point representation, e.g. <U+00EB> in this case) are caused by an attempt to run R within a non-UTF-8 locale.
Typically, this can be verified by checking the output of Sys.getlocale("LC_ALL"). If you see a C locale reported, then you likely need to enforce a UTF-8 locale with something like:
Sys.setlocale("LC_ALL", "en_US.UTF-8")
substituting the particular UTF-8 locale flavor based on your desired language. (For reference, the set of available locales can usually be queried from a terminal with something like locale -a).
I'm utilizing a Macbook pro running "El Capitan" and RStudio is version 0.99.902. I'm writing a Rmd document. I want to utilize the snippets that RStudio has built in and create my own also. By clicking Preference => Code; I can see that "Enable code snippets" is checked. However, while trying to utilize any snippet the completion is not performed. If I typed just r I should get this block of code, but nothing hapen
snippet r
```{r ${1:label}, ${2:options}}
${0}
```
I also create a simple snippet:
snippet dthen
%>%
None of the markdown snippet seems to work. Do I'm doing something wrong or any setting has to be done?
You can invoke code snippets in Markdown in RStudio using Shift+Tab after typing the snippet, but it will not currently prompt you to expand the snippet, either with Tab or by just waiting. I thought this was a bug, but the discussion in this GitHub issue says it's intended, just still-to-be-documented.
In Tools > Global Options > Code > Editing > Edit Snippets, go to the section "Markdown" and paste your snippet there. Save, OK, apply. Type r in your Rmd document (not within in a R code chunk) and click Shift+Tab. If it still does not work, try to create a new "R markdown" file (File > New file > R markdown) and check if it works there.
TL;DR
What are the (possibly unwanted) side-effects of using knit()/knit2pdf() instead of the "Compile PDF"1 button in RStudio?
Motivation
Most users of knitr seem to write their documents in RStudio and compile the documents using the "Compile PDF" / "Knit HTML" button. This works smoothly most of the time, but every once a while there are special requirements that cannot be achieved using the compile button. In these cases, the solution is usually to call knit()/knit2pdf()/rmarkdown::render() (or similar functions) directly.
Some examples:
How to knit/Sweave to a different file name?
Is there a way to knitr markdown straight out of your workspace using RStudio?
Insert date in filename while knitting document using RStudio Knit button
Using knit2pdf() instead of the "Compile PDF" button usually offers a simple solution to such questions. However, this comes at a price: There is the fundamental difference that "Compile PDF" processes the document in a separate process and environment whereas knit2pdf() and friends don't.
This has implications and the problem is that not all of these implications are obvious. Take the fact that knit() uses objects from the global environment (whereas "Compile PDF" does not) as an example. This might be obvious and the desired behavior in cases like the second example above, but it is an unexpected consequence when knit() is used to overcome problems like in example 1 and 3.
Moreover, there are more subtle differences:
The working directory might not be set as expected.
Packages need to be loaded.
Some options that are usually set by RStudio may have unexpected values.
The Question and it's goal
Whenever I read/write the advice to use knit2pdf() instead of "Compile PDF", I think "correct, but the user should understand the consequences …".
Therefore, the question here is:
What are the (possibly unwanted) side-effects of using knit()/knit2pdf() instead of the "Compile PDF" button in RStudio?
If there was a comprehensive (community wiki?) answer to this question, it could be linked in future answers that suggest using knit2pdf().
Related Questions
There are dozens of related questions to this one. However, they either propose only code to (more or less) reproduce the behavior of the RStudio button or they explain what "basically" happens without mentioning the possible pitfalls. Others look like being very similar questions but turn out to be a (very) special case of it. Some examples:
Knit2html not replicating functionality of Knit HTML button in R Studio: Caching issue.
HTML outputs are different between using knitr in Rstudio & knit2html in command line: Markdown versions.
How to convert R Markdown to HTML? I.e., What does “Knit HTML” do in Rstudio 0.96?: Rather superficial answer by Yihui (explains what "basically" happens) and some options how to reproduce the behavior of the RStudio button. Neither the suggested Sys.sleep(30) nor the "Compile PDF" log are insightful (both hints point to the same thing).
What does “Knit HTML” do in Rstudio 0.98?: Reproduce behavior of button.
About the answer
I think this question raised many of the issues that should be part of an answer. However, there might be many more aspects I don't know about which is the reason why I am reluctant to self-answer this question (though I might try if nobody answers).
Probably, an answer should cover three main points:
The new session vs. current session issue (global options, working directory, loaded packages, …).
A consequence of the first point: The fact that knit() uses objects from the calling environment (default: envir = parent.frame()) and implications for reproducibility. I tried to tackle the issue of preventing knit() from using objects from outside the document in this answer (second bullet point).
Things RStudio secretly does …
… when starting an interactive session (example) --> Not available when hitting "Compile PDF"
… when hitting "Compile PDF" (anything special besides the new session with the working directory set to the file processed?)
I am not sure about the right perspective on the issue. I think both, "What happens when I hit 'Compile PDF' + implications" as well as "What happens when I use knit() + implications" is a good approach to tackle the question.
1 The same applies to the "Knit HTML" button when writing RMD documents.
First of all, I think this question is easier to answer if you limit the scope to the "Compile PDF" button, because the "Knit HTML" button is a different story. "Compile PDF" is only for Rnw documents (R + LaTeX, or think Sweave).
I'll answer your question following the three points you suggested:
Currently RStudio always launch a new R session to compile Rnw documents, and first changes the working directory to the directory of the Rnw file. You can imagine the process as a shell script like this:
cd path/to/your-Rnw-directory
Rscript -e "library(knitr); knit('your.Rnw')"
pdflatex your.tex
Note that the knitr package is always attached, and pdflatex might be other LaTeX engines (depending on your RStudio configurations for Sweave documents, e.g., xelatex). If you want to replicate it in your current R session, you may rewrite the script in R:
owd = setwd("path/to/your-Rnw-directory")
system2("Rscript", c("-e", shQuote("library(knitr); knit('your.Rnw')"))
system2("pdflatex", "your.tex")
setwd(owd)
which is not as simple as knitr::knit('path/to/your.Rnw'), in which case the working directory is not automatically changed, and everything is executed in the current R session (in the globalenv() by default).
Because the Rnw document is always compiled in a new R session, it won't use any objects in your current R session. This is hard to replicate only through the envir argument of knitr::knit() in the current R session. In particular, you cannot use knitr::knit(envir = new.env()) because although new.env() is a new environment, it has a default parent environment parent.frame(), which is typically the globalenv(); you cannot use knitr::knit(envir = emptyenv()), either, because it is "too clean", and you will have trouble with objects even in the R base package. The only reliable way to replicate what the "Compile PDF" button does is what I said in 1: system2("Rscript", c("-e", shQuote("library(knitr); knit('your.Rnw')")), in which case knit() uses the globalenv() of a new R session.
I'm not entirely sure about what RStudio does for the repos option. It probably automatically sets this option behind the scenes if it is not set. I think this is a relatively minor issue. You can set it in your .Rprofile, and I think RStudio should respect your CRAN mirror setting.
Users have always been asking why the Rnw document (or R Markdown documents) are not compiled in the current R session. To us, it basically boils down to which of the following consequences is more surprising or undesired:
If we knit a document in the current R session, there is no guarantee that your results can be reproduced in another R session (e.g., the next time you open RStudio, or your collaborators open RStudio on their computers).
If we knit a document in a new R session, users can be surprised that objects are not found (and when they type the object names in the R console, they can see them). This can be surprising, but it is also a good and early reminder that your document probably won't work the next time.
To sum it up, I think:
Knitting in a new R session is better for reproducibilty;
Knitting in the current R session is sometimes more convenient (e.g., you try to knit with different temporary R objects in the current session). Sometimes you also have to knit in the current R session, especially when you are generating PDF reports programmatically, e.g., you use a (for) loop to generate a series of reports. There is no way that you can achieve this only through the "Compile PDF" button (the button is mostly only for a single Rnw document).
BTW, I think what I said above can also apply to the Knit or Knit HTML buttons, but the underlying function is rmarkdown::render() instead of knitr::knit().