Retrieve pictures from API with R - r

I have never worked with APIs so this is my first try, I don't even know if what I'm trying to do is possible.
I'm trying to obtain pictures of cells from the SwissBioPics API (https://www.npmjs.com/package/%40swissprot/swissbiopics%2Dvisualizer) and have them in my R session.
res <- httr::GET('https://www.swissbiopics.org/static/swissbiopics.js',
query = list(taxisid = '9606', sls= 'SL0073',gos = '0005641'))
result <- httr::content(res$content)
but I'm getting this error:
Error in httr::content(res$content) : is.response(x) is not TRUE
Any clues?

After many misadventures, I have your answer as promised!
Since it involves interactive imagery, courtesy of JavaScript and HTML, this solution must be run as a .Rmd file within RStudio. The interactive imagery can also be accessed in the eponymous .html file, output by knitr when you click the Knit button in RStudio.
Step 1: Project Setup
Create a new R project my_pics in RStudio, under a new directory. From within this project, create a new R Notebook (here my_book.Rmd), which should end up right next to my_pics.Rproj under the aforementioned directory.
Step 2: Supporting Files
Under that same directory, create a ./snippets subdirectory. The latter should contain the following two .txt files, copied (and corrected) from the swissbiopics-visualizer documentation:
templates.txt: the first code block given in the documentation. Reproduced here with necessary EOF and syntactically corrected comments:
<template id="sibSwissBioPicsStyle">
<style>
ul > li > a {
font-style:oblique;
}
ul.notpresent li > .subcell_description {
display:none;
}
</style>
</template>
<template id="sibSwissBioPicsSlLiItem">
<li class="subcellular_location">
<a class="subcell_name"></a> <!-- the class name is required and textContent will be set on it -->
<span class="subcell_description"></span> <!-- the class name is required and textContent will be set on it -->
</li>
</template>
custom_element.txt: the third code block given in the documentation. Reproduced here with necessary EOF:
<script type="module" src="https://www.swissbiopics.org/static/swissbiopics.js"></script>
<script defer>
if (! window.customElements.get("sib-swissbiopics-sl"))
window.customElements.define("sib-swissbiopics-sl", SwissBioPicsSL);
</script>
Mind you, these .txt files can just as easily be saved as .html files. Only the file extensions would need to be refactored, in the default values for the templates and custom_element parameters, for the make_html() function in the my_book.Rmd code below.
Step 3: Interact in RStudio
Now we are ready! In my_book.Rmd, write the following:
---
title: "R Notebook"
output: html_document
---
```{r}
library(htmltools)
library(readr)
library(rlang)
```
# Functions #
Here are the functions that do the trick. The snippets used by `make_html()` are copied from the [documentation](https://www.npmjs.com/package/%40swissprot/swissbiopics-visualizer#usage) for `swissbiopics-visualizer`, and (after fixing the HTML comments) pasted into `.txt` files (`templates.txt` and `custom_element.txt`) under the `./snippets` subdirectory, which lies within the directory containing this `.Rproj`.
```{r}
# Create comma-separated list from vectorized (or listed) items, safely escaped.
csl <- function(items) {
return(paste("\"", paste(htmltools::htmlEscape(unlist(items)), collapse = ",", sep = ""), "\"", sep = ""))
}
# Create the HTML for the interactive imagery given by the parameters. Assembly
# process is as described the documentation for 'swissbiopics-visualizer':
# https://www.npmjs.com/package/%40swissprot/swissbiopics-visualizer#usage
make_html <- function(# The NCBI taxonomy ID.
tax_id,
# The IDs of the cellular elements to highlight.
sl_ids,
# The filepath to (or raw HTML text of) the templates
# snippet.
templates = "./snippets/templates.txt",
# The filepath to (or raw HTML text of) the custom element
# snippet.
custom_element = "./snippets/custom_element.txt",
# Further arguments to 'readr::read_file()', which might
# be useful to process snippet encodings across platforms.
...) {
# Escape any strings supplied.
tax_id <- csl(tax_id[1])
sl_ids <- csl(sl_ids)
# Compile all the HTML snippets into a list:
elements <- list()
# Include the templates (as read)...
elements$templates <- readr::read_file(file = templates, ...)
# ...then include the line (created here) to target the right picture...
elements$identifier <- "<sib-swissbiopics-sl taxid=%s sls=%s></sib-swissbiopics-sl>"
elements$identifier <- sprintf(fmt = elements$identifier, tax_id, sl_ids)
# ...and finally include the definition (as read) for the custom element.
elements$custom_element <- readr::read_file(file = custom_element, ...)
# Append these snippets together, into the full HTML code.
return(paste(unlist(elements), collapse = "\n"))
}
# Display the interactive imagery given by the parameters, visible in both
# RStudio (crowded) and the R Markdown file (well laid out).
visualize <- function(# The NCBI taxonomy ID.
taxid = "9606",
# A list (or vector) of the UniProtKB subcellular location
# (SL) IDs for the cellular elements to highlight.
sls = list("SL0073"),
# Further arguments to 'make_html()'.
...
) {
# Embed the HTML text where this function is called.
return(htmltools::HTML(make_html(tax_id = taxid, sl_ids = sls, ...)))
}
```
# Results #
Here we `visualize()` the **interactive** image, also accessible on [SwissBioPics](https://www.swissbiopics.org):
```{r}
visualize(sls = list("SL0073", "SL0138"))
```
Note
Observe how (in this case) we "lazily" use the default value ("9606") for taxid, without having to specify it. Observe also how we can simultaneously highlight not one but multiple separate components, namely the Contractile vacuole ("SL0073") and the Cell cortex ("SL0138").
Now below that last chunk where visualize() is called
```{r}
visualize(sls = list("SL0073", "SL0138"))
```
you will see interactive output that looks like this:
Sadly, it appears extremely crowded in RStudio, and an HTML wizard might be needed to alter the supporting .txt (or .html) files, to achieve properly formatted HTML within this IDE.
Step 4: Embed in Reports
As with any .Rmd file, RStudio gives you the option to Knit the Markdown results into a .html file, which can be easily accessed and beautifully formatted as a report!
With my_book.Rmd open in RStudio, click the Knit button, and my_book.html should appear within that same directory. You can open this .html file in a web browser (I used Chrome) to see it in all its glory!
In Conclusion
With either of these two interactive images, you can hover to highlight the various components and layers of the diagram. Furthermore, clicking on any definition will take you by hyperlink to its profile on UnitProt.
Many of the remaining limitations are due to the swissbiopics-visualizer API itself. For example, there seems to be a malfunction in its mapping from GO IDs to SL IDs, via this dataset. As such, you should provide only SL codes to visualize().
That said, if you can wrangle that HTML and bend its layout to your will, the sky is the limit!
Enjoy!
Bonus
Here's a demo of the same interactive output, embedded right here in Stack Overflow! Unfortunately, it's unstable and horribly unwieldy in this environment, so I've left it as a mere "footnote":
<template id="sibSwissBioPicsStyle">
<style>
ul > li > a {
font-style:oblique;
}
ul.notpresent li > .subcell_description {
display:none;
}
</style>
</template>
<template id="sibSwissBioPicsSlLiItem">
<li class="subcellular_location">
<a class="subcell_name"></a> <!-- the class name is required and textContent will be set on it -->
<span class="subcell_description"></span> <!-- the class name is required and textContent will be set on it -->
</li>
</template>
<sib-swissbiopics-sl taxid="9606" sls="SL0073,SL0138" ></sib-swissbiopics-sl>
<script type="module" src="https://www.swissbiopics.org/static/swissbiopics.js"></script>
<script defer>
if (! window.customElements.get("sib-swissbiopics-sl"))
window.customElements.define("sib-swissbiopics-sl", SwissBioPicsSL);
</script>

You have to call the content-function on res, not res$content. Then you get raw content which needs to be converted e.g. via
base::rawToChar(content(res))
which results in a string containing some JS-code
base::rawToChar(content(res))
[1] "var SwissBioPics;SwissBioPics=(()=>....

I have only quickly looked at the website, but what about just downloading the files? It also goes through the API.
qurl = "https://www.swissbiopics.org/api/image/Chlamydomona_cells.svg"
fl = file.path(tempdir(), basename(qurl))
download.file(qurl, fl)
Once on disk, you can load the image in R, e.g. through the magick-package:
require(magick)
img = image_read_svg(fl)
print(img)

Related

Print HTML formatted text in Jupyter with R from the list of citations of loaded packages

I have a R variable containing some html content inside, for exemple :
myvar = "<h3>Section Title</h3>"
I would like to print it not as <h3>Section Title</h3> but as a formated h3 title in html or markdown that jupyter notebook understands.
I have been looking a bit everywhere and found htmlwidget or knitr but I feel like it needs a HTML file. What I would like to do is just displaying a variable.
I have tried also with htmltools package and the HTML() function, but no success...
myvar = "<h3>Section Title</h3>"
# I would expect that I can do something like:
print(HTML(myvar), format='html')
# But it doesn't work, I just get:
# => <h3>Section Title</h3>
And to be very clear, the goal is also that when I save the notebook as HTML or as PDF, the html is displayed formatted (and not as raw text)
In Fine the goal is to display the citations for loaded packages, in a aesthetic way. I know that I can use print(citation("packagename"), style='html') to output it as HTML but I can't find how to format the HTML properly. That's the reason my idea is to capture this output in a variable and output it as formatted HTML.
So I found the simplest option (I think) which doesn't require any other packages than htmltools which I think is by default installed.
How to display and format HTML in R
The key is the function htmlTemplate(text_ = var) :
myvar ="<h3>Test Title</h3>"
htmlTemplate(text_ = var)
This will be displayed as raw HTML in the console, but on a Jupyter Notebook it will be displayed formatted. I didn't Try it on RStudio but I would expect it to be displayed correctly as well (anyone to confirm ?)
Another solution to display the html content (as mentionned by #krassowski) Nevertheless this works only in Jupyter Notebook.
# We still need to call `HTML()` function inside
IRdisplay::display_html( HTML(myvar) )
How to pretty print citations for loaded packages in R
My final goal was to display the citations for loaded R packages in HTML
Using the .packages() I could retrieve the list of attached packages.
Then I loop on the list and for each package name (str) I can call citation(package) on the top. To get it as html I only found the print(citation(package), style='html').
Then we need to capture the output of this into a variable. The function capture.output(func) do the trick. See below:
library('htmltools')
# Create a function to output the html of all packages
list_pkgs_html = function(){
# Remove warning for the time of the function
w = getOption("warn")
options(warn = -1)
# For each package loaded
for (p in .packages()){
# Catch error in case there is no citation for one of them.
tryCatch({
# Display the name of the package as a title h3
cat("<h3>", p, '</h3>\n')
# Print the citation in html style
print(citation(p), style = "html")
})
}
# Restore the warning as before
options(warn = w)
}
# Call the function and capture the output to a variable.
capture.output(list_pkgs_html()) -> myvar
# Display the html result.
htmlTemplate(text_ = myvar)
Output:

RMarkdown Inline Code Format

I am reading ISL at the moment which is related to machine learning in R
I really like how the book is laid out specifically where the authors reference code inline or libraries for example library(MASS).
Does anyone know if the same effect can be achieved using R Markdown i.e. making the MASS keyword above brown when i reference it in a paper? I want to color code columns in data frames when i talk about them in the R Markdown document. When you knit it as a HTML document it provides pretty good formatting but when i Knit it to MS Word it seems to just change the font type
Thanks
I've come up with a solution that I think might address your issue. Essentially, because inline source code gets the same style label as code chunks, any change you make to SourceCode will be applied to both chunks, which I don't think is what you want. Instead, there needs to be a way to target just the inline code, which doesn't seem to be possible from within rmarkdown. Instead, what I've opted to do is take the .docx file that is produced, convert it to a .zip file, and then modify the .xml file inside that has all the data. It applies a new style to the inline source code text, which can then be modified in your MS Word template. Here is the code:
format_inline_code = function(fpath) {
if (!tools::file_ext(fpath) == "docx") stop("File must be a .docx file...")
cur_dir = getwd()
.dir = dirname(fpath)
setwd(.dir)
out = gsub("docx$", "zip", fpath)
# Convert to zip file
file.rename(fpath, out)
# Extract files
unzip(out, exdir=".")
# Read in document.xml
xml = readr::read_lines("word/document.xml")
# Replace styling
# VerbatimChar didn't appear to the style that was applied in Word, nor was
# it present to be styled. VerbatimStringTok was though.
xml = sapply(xml, function(line) gsub("VerbatimChar", "VerbatimStringTok", line))
# Save document.xml
readr::write_lines(xml, "word/document.xml")
# Zip files
.files = c("_rels", "docProps", "word", "[Content_Types].xml")
zip(zipfile=out, files=.files)
# Convert to docx
file.rename(out, fpath)
# Remove the folders extracted from zip
sapply(.files, unlink, recursive=TRUE)
setwd(cur_dir)
}
The style that you'll want to modify in you MS Word template is VerbatimStringTok. Hope that helps!

knitr/Rmd: Adding title page and text when converting to MS Word

I'm trying to write a report using the rmarkdown package and as it is, unfortunately, customary in my field reports are often submitted as MS Word documents. So I can't always rely on the power of LaTeX and have to be able convert my .Rmd to MS Word. Now, because I want to be able to create PDF and MS Word files from the same source file, I'm trying to find a general way to do this. I've got PDF working using the apa6 LaTeX-document class. The .Rmd will look something like this when creating a Word file:
---
title: My title
abstract: This is the abstract.
author: John Doe
affiliation: Unknown
note: Nothing to say.
output:
word_document:
reference_docx: myreference.docx
---
Lorem ipsum.
I can create a Word document from this but for obvious reasons my custom yaml-variables (e.g. abstract) will not be rendered in the document.
So basically, my problem is the following:
When creating a word document, how can I add a title page (including author names, affiliations, author notes, etc.) and another page with just the abstract before the document body ("Lorem ipsum")? The focus here is not to create pagebreaks (there are other open questions on this), but rather **is there a way to make pandoc use the custom yaml variables place them at the beginning of the document and assign styles to them?
The rmarkdown package provides an include() function but it only works with HTML and PDF documents.
I have found that it's possible to customize the content of the Markdown file (e.g. to add and modify a title page) generated by rmarkdown before submitting it to pandoc for the conversion to DOCX by using a preprocessor. Let's assume we are trying to add some information specified in a YAML parameter note just before the abstract (support for abstracts has in the meantime been added to pandoc).
To do so, we first need a preprocessor function that reads the input file and parses the YAML front matter, and customizes the input file:
my_pre_processor <- function(metadata, input_file, runtime, knit_meta, files_dir, output_dir, from) {
# Identify YAML front matter delimiters
input_text <- readLines(input_file, encoding = "UTF-8")
yaml_delimiters <- grep("^(---|\\.\\.\\.)\\s*$", input_text)
if(length(yaml_delimiters) >= 2 &&
(yaml_delimiters[2] - yaml_delimiters[1] > 1) &&
grepl("^---\\s*$", input_text[yaml_delimiters[1]])) {
yaml_params <- yaml::yaml.load(paste(input_text[(yaml_delimiters[1] + 1):(yaml_delimiters[2] - 1)], collapse = "\n"))
} else yaml_params <- NULL
# Modify title page
custom_lines <- c(
"NOTE:"
, metadata$note
, "\n\n"
, "# Abstract"
, "\n"
, metadata$abstract
, "\n"
)
## Add modified title page components after YAML front matter
augmented_input_text <- c(custom_lines, input_text[(yaml_delimiters[2] + 1):length(input_text)])
# Remove redundant default abstract
yaml_params$abstract <- NULL
# Add modifications to input file
augmented_input_text <- c("---", yaml::as.yaml(yaml_params), "---", augmented_input_text)
input_file_connection <- file(input_file, encoding = "UTF-8")
writeLines(augmented_input_text, input_file_connection)
close(input_file_connection)
NULL
}
Now we need to define a custom format that utilizes our preprocessor:
my_word_document <- function(...) {
config <- rmarkdown::word_document(...)
# Preprocessor functions are adaptations from the RMarkdown package
# (https://github.com/rstudio/rmarkdown/blob/master/R/pdf_document.R)
pre_processor <- function(metadata, input_file, runtime, knit_meta, files_dir, output_dir, from = .from) {
# save files dir (for generating intermediates)
saved_files_dir <<- files_dir
args <- my_pre_processor(metadata, input_file, runtime, knit_meta, files_dir, output_dir, from)
args
}
config$pre_processor <- pre_processor
config
}
Now, you can use the custom format when rendering R Markdown documents as follows:
rmarkdown::render("./foo/bar.Rmd", output_format = my_word_document())

Send/share sensitive R slidify presentation via email or other secure methods

I would like to send to someone else a presentation that I have created using R and slidfy, but it contains sensitive information, so putting it up onto github pages using the gh-pages branch and then sending the URL isn't really an option, since all github pages are public, as suggested here.
Pushing it up to the glimmer shiny server as well seems a bit unsecure too...(I would ideally like to do this for free so setting up a server to host the one presentation seems a bit cumbersome and overkill for my purposes)
I don't think dropbox would work either since any URL link that is produced if someone else types it into an address bar, would probably be able to download it and view the sensitive information...
Is there a way of sending the presentation (by email or other methods) that contains all the necessary files for it to work, so that a person who doesn't use R can open it and view it easily. (i.e. without having to send them a zip file of all the files (i.e. the assets and libraries and figure folders etc), asking them to unzip it and then opening the index.html file)?
Edit
I forgot to mention that the presentations include nvd3 and morrisjs graphs too, making it difficult to bring over all the files in one go...
Edit2
Given all the libraries used are public libraries, is there a way for it to reference a URL instead of a local drive?
Here is how you would do it with Slidify. There are two tricks to use.
Specify mode: standalone in your YAML front matter. This makes sure that all slide related JS and CSS assets are served from an online CDN, and also that all static images are converted into data URLs.
Use n1$print('mychart', include_assets = TRUE, cdn = TRUE) when you print the chart in your knitr code chunk. This makes sure that all chart relate assets are included and served from an online CDN. Note that for each library, you should use include_assets only once, so that you don't duplicate.
This approach is not very robust since you are linking to multiple JS libraries in a single file, and as a result there could be conflicts. Case in point, MorrisJS does not play well with Google IO2012, since Google IO2012 uses requireJS and for some reason raises conflicts.
You can also use the same code chunks in RStudio Presentations and save them as standalone HTML. Here is the same presentation in RPres format.
---
title : Standalone Presentation with Slidify
author : Ramnath Vaidyanathan
mode : standalone
---
## Plain Text
This is a slide with plain text
> 1. Point 1
> 2. Point 2
> 3. Point 3
---
## R Plot
```{r message = F}
require(ggplot2)
qplot(wt, mpg, data = mtcars)
```
---
## NVD3 Plot
```{r results = 'asis', comment = NA, message = F, echo = F}
require(rCharts)
n1 <- nPlot(mpg ~ wt, data = mtcars, type = 'scatterChart', group = 'gear')
n1$print('chart2', include_assets = TRUE, cdn = TRUE)
```
<style>
.rChart {
height: 500px;
}
</style>
---
## Another NVD3 Plot
```{r results = 'asis', comment = NA, message = F, echo = F}
require(rCharts)
n2 <- nPlot(mpg ~ cyl, data = mtcars, type = 'scatterChart')
n2$print('chart3')
```
Another option is to
1) Open the HTML file in Chrome,
2) Choose the option to print
3) Save it as .pdf.
It's not perfect for all cases but definately a decent option to consider.
I think your last method is the safest.
Do you really want to use slidify? With the latest (preview) version of Rstudio you can create HTML5 presentations on the fly. And it is much easier than slidify. To distribute the slideshow you just have to mail the standalone output html file (if you aren't using fancy things like nvd3 or latex).
Here is some more information.
link1
link2
link3

Custom CSS with knitr and markdown in R

I found this great tutorial on how to modify the css formatting of a HTML report created with markdown and knitr in Rstudio. The post can be found here.
I was hoping to build on this concept and mimic the layout of the page here by using the same css. I tried to simply copy/paste/combine the two css files I found when I viewed the page's source.
Any help you can lend would be greatly appreciated! This is my first attempt and doing anything CSS.
This is the method provided by RStudio: http://www.rstudio.com/ide/docs/authoring/markdown_custom_rendering
options(rstudio.markdownToHTML =
function(inputFile, outputFile) {
require(markdown)
markdownToHTML(inputFile, outputFile, stylesheet='custom.css')
}
)
I've never been able to get that working properly so I do it a little differently:
I do this by creating the standard output file, then dropping the header and css code at the top in R:
tmp <- readLines("your.html")
tmp <- tmp[-c(1:50)] # or however many lines it is before the css ends
write(tmp,"your.html")
Then I use pandoc to add my own css in a standalone file
system("pandoc -s -S your.html -c your.css -o output.html")
Outside of RStudio (may work in it too - I'm not sure as I don't use it much), you can use option 'markdown.HTML.stylesheet' to set a custom style sheet. It will then import everything from your .css file into the newly created html file.
Here is an example:
## Set file names
htmlName <- "test.html"
rmdName <- gsub("html","Rmd", htmlName)
stylesheetName <- 'style.css'
## Generate rmd file from R
sink(file = rmdName, type='output')
cat('\n<textarea maxlength="3000" cols="70">')
cat("Hello World!")
cat('</textarea>\n')
sink()
## Generate style sheet from R
sink(file = stylesheetName, type='output')
cat("textarea {color: #a10000; }\n")
sink()
## Set knitr options and knit html
require(knitr)
options(markdown.HTML.stylesheet = stylesheetName)
knit2html(rmdName, output = htmlName)

Resources