Use multiple command chains with piping - r

EDIT: I reworked the question to make it clearer and integrate what I found by myself
Pipes are a great way to make the code more readable when using a single command chain
In some cases however, I feel one is forced to be inconsistent to its philosophy, either by creating unnecessary temp variables, mixing piping and embedded parenthesis, or defining custom functions.
See this SO question for example, where OP wants to know how to convert colnames to lower case with pipes: Dplyr or Magrittr - tolower?
I'll forget about the existence of names<- to make my point
There's basically 3 ways to do it:
Use a temp variable
temp <- df %>% names %>% tolower
df %>% setNames(temp)
Use embedded parenthesis
df %>% setNames(tolower(names(.)))
Define custom function
upcase <- function(df) {names(df) <- tolower(names(df)); df}
df %>% upcase
I think it would be more consistent to be able to do something like this:
df %T>% # create new branch with %T%>%
{names(.) %>% tolower %as% n} %>% # parallel branch assigned to alias n, then going back to main branch with %>%
setNames(n) # combine branches
For more complex cases, it is in my opinion more readable than the 3 examples above and I'm not polluting my workspace.
So far I've been able to come quite close, I can type:
df %T>%
{names(.) %>% tolower %as% n} %>%
setNames(A(n));fp()
OR (a little tribute to old school calculators)
df %1% # puts lhs in first memory slot (notice "%1%", I define these up to "%9%")
names %>%
tolower %>%
setNames(M(1),.);fp() # call the first stored value
(see code at bottom)
My issues are the following:
I create a new environment in my global environment, and I have to flush it manually with fp(), it's quite ugly
I'd like to get rid of this A function, but I don't understand well enough the environment structure of pipe chains to do so
Here's my code :
It creates an environment named PipeAliasEnv for aliases
%as% creates an alias in an isolated environment
%to% creates a variable in the calling environment
A calls an alias
fp removes all objects from PipeAliasEnv
This is the code that I used and a reproducible example solved in 4 different ways:
library(magrittr)
alias_init <- function(){
assign("PipeAliasEnv",new.env(),envir=.GlobalEnv)
assign("%as%" ,function(value,variable) {assign(as.character(substitute(variable)),value,envir=PipeAliasEnv)},envir=.GlobalEnv)
assign("%to%" ,function(value,variable) {assign(as.character(substitute(variable)),value,envir=parent.frame())},envir=.GlobalEnv)
assign("A" ,function(variable) { get(as.character(substitute(variable)), envir=PipeAliasEnv)},envir=.GlobalEnv)
assign("fp" ,function(remove_envir=FALSE){if(remove_envir) rm(PipeAliasEnv,envir=.GlobalEnv) else rm(list=ls(envir=PipeAliasEnv),envir=PipeAliasEnv)},envir=.GlobalEnv) # flush environment
# to handle `%i%` and M(i) notation, 9 should be enough :
sapply(1:9,function(i){assign(paste0("%",i,"%"),eval(parse(text=paste0('function(lhs,rhs){lhs <- eval(lhs)
rhs <- as.character(substitute(rhs))
str <- paste("lhs %>%",rhs[1],"(",paste(rhs[-1],collapse=","),")")
assign("x',i,'",lhs,envir=PipeAliasEnv)
eval(parse(text= str))}'))),envir=.GlobalEnv)})
assign("M" ,function(i) { get(paste0("x",as.character(substitute(i))), envir=PipeAliasEnv)},envir=.GlobalEnv)
}
alias_init()
# using %as%
df <- iris %T>%
{names(.) %>% toupper %as% n} %>%
setNames(A(n)) %T>%
{. %>% head %>% print}(.) ;fp()
# still using %as%, choosing another main chain
df <- iris %as% dataset %>%
names %>%
toupper %>%
setNames(A(dataset),.) %T>%
{. %>% head %>% print}(.);fp()
# using %to% (notice no assignment on 1st line)
iris %T>%
{names(.) %>% toupper %as% n} %>%
{setNames(.,A(n))} %to% df %>% # no need for '%T>%' and '{}' here
head %>% print;fp()
# or using the old school calculator fashion (probably the clearest for this precise task)
df <- iris %1%
names %>%
toupper %>%
setNames(M(1),.) %T>%
{. %>% head %>% print}(.);fp()
My question in short:
How do I get rid of A and fp ?
Bonus: %to% doesn't work when inside {}, how can I solve this ?

Related

How to properly parse (?) mdsets in expss within a loop?

I'm new to R and I don't know all basic concepts yet. The task is to produce a one merged table with multiple response sets. I am trying to do this using expss library and a loop.
This is the code in R without a loop (works fine):
#libraries
#blah, blah...
#path
df.path = "C:/dataset.sav"
#dataset load
df = read_sav(df.path)
#table
table_undropped1 = df %>%
tab_cells(mdset(q20s1i1 %to% q20s1i8)) %>%
tab_total_row_position("none") %>%
tab_stat_cpct() %>%
tab_pivot()
There are 10 multiple response sets therefore I need to create 10 tables in a manner shown above. Then I transpose those tables and merge. To simplify the code (and learn something new) I decided to produce tables using a loop. However nothing works. I'd looked for a solution and I think the most close to correct one is:
#this generates a message: '1' not found
for(i in 1:10) {
assign(paste0("table_undropped",i),1) = df %>%
tab_cells(mdset(assign(paste0("q20s",i,"i1"),1) %to% assign(paste0("q20s",i,"i8"),1)))
tab_total_row_position("none") %>%
tab_stat_cpct() %>%
tab_pivot()
}
Still it causes an error described above the code.
Alternatively, an SPSS macro for that would be (published only to better express the problem because I have to avoid SPSS):
define macro1 (x = !tokens (1)
/y = !tokens (1))
!do !i = !x !to !y.
mrsets
/mdgroup name = !concat($SET_,!i)
variables = !concat("q20s",!i,"i1") to !concat("q20s",!i,"i8")
value = 1.
ctables
/table !concat($SET_,!i) [colpct.responses.count pct40.0].
!doend
!enddefine.
*** MACRO CALL.
macro1 x = 1 y = 10.
In other words I am looking for a working substitute of !concat() in R.
%to% is not suited for parametric variable selection. There is a set of special functions for parametric variable selection and assignment. One of them is mdset_t:
for(i in 1:10) {
table_name = paste0("table_undropped",i)
..$table_name = df %>%
tab_cells(mdset_t("q20s{i}i{1:8}")) %>% # expressions in the curly brackets will be evaluated and substituted
tab_total_row_position("none") %>%
tab_stat_cpct() %>%
tab_pivot()
}
However, it is not good practice to store all tables as separate variables in the global environment. Better approach is to save all tables in the list:
all_tables = lapply(1:10, function(i)
df %>%
tab_cells(mdset_t("q20s{i}i{1:8}")) %>%
tab_total_row_position("none") %>%
tab_stat_cpct() %>%
tab_pivot()
)
UPDATE.
Generally speaking, there is no need to merge. You can do all your work with tab_*:
my_big_table = df %>%
tab_total_row_position("none")
for(i in 1:10) {
my_big_table = my_big_table %>%
tab_cells(mdset_t("q20s{i}i{1:8}")) %>% # expressions in the curly brackets will be evaluated and substituted
tab_stat_cpct()
}
my_big_table = my_big_table %>%
tab_pivot(stat_position = "inside_columns") # here we say that we need combine subtables horizontally

how to unquote (!!) inside `map` inside `mutate`

I'm modifying nested data frames inside of foo with map2 and mutate, and I'd like to name a variable in each nested data frame according to foo$name. I'm not sure what the proper syntax for nse/tidyeval unquotation would be here.
My attempt:
library(tidyverse)
foo <- mtcars %>%
group_by(gear) %>%
nest %>%
mutate(name = c("one", "two", "three")) %>%
mutate(data = map2(data, name, ~
mutate(.x, !!(.y) := "anything")))
#> Error in quos(...): object '.y' not found
I want the name of the newly created variable inside the nested data frames to be "one", "two", and "three", respectively.
I'm basing my attempt off the normal syntax I'd use if I was doing a normal mutate on a normal df, and where name is a string:
name <- "test"
mtcars %>% mutate(!!name := "anything") # works fine
If successful, the following line should return TRUE:
foo[1,2] %>% unnest %>% names %>% .[11] == "one"
This seems to be a feature/bug (not sure, see linked GitHub issue below) of how !! works within mutate and map. The solution is to define a custom function, in which case the unquoting works as expected.
library(tidyverse)
custom_mutate <- function(df, name, string = "anything")
mutate(df, !!name := string)
foo <- mtcars %>%
group_by(gear) %>%
nest %>%
mutate(name = c("one", "two", "three")) %>%
mutate(data = map2(data, name, ~
custom_mutate(.x, .y)))
foo[1,2] %>% unnest %>% names %>% .[11] == "one"
#[1] TRUE
You find more details on GitHub under issue #541: map2() call in dplyr::mutate() error while standalone map2() call works; note that the issue has been closed in September 2018, so I am assuming this is intended behaviour.
An alternative might be to use group_split instead of nest, in which case we
avoid the unquoting issue
nms <- c("one", "two", "three")
mtcars %>%
group_split(gear) %>%
map2(nms, ~.x %>% mutate(!!.y := "anything"))
This is because of the timing of unquoting. Nesting tidy eval functions can be a bit tricky because it is the very first tidy eval function that processes the unquoting operators.
Let's rewrite this:
mutate(data = map2(data, name, ~ mutate(.x, !!.y := "anything")))
to
mutate(data = map2(data, name, function(x, y) mutate(x, !!y := "anything")))
The x and y bindings are only created when the function is called by map2(). So when the first mutate() runs, these bindings don't exist yet and you get an object not found error. With the formula it's a bit harder to see but the formula expands to a function taking .x and .y arguments so we have the same problem.
In general, it's better to avoid complex nested logic in your code because it makes it harder to read. With tidy eval that's even more complexity, so best do things in steps. As an added bonus, doing things in steps requires creating intermediate variables which, if well named, help understand what the function is doing.

How to resolve 'Don't know how to pluck from a closure' error in R

The code below works if I remove the Sys.sleep() from within the map() function. I tried to research the error ('Don't know how to pluck from a closure') but i haven't found much on that topic.
Does anyone know where I can find documentation on this error, and any help on why it is happening and how to prevent it?
library(rvest)
library(tidyverse)
library(stringr)
# lets assume 3 pages only to do it quickly
page <- (0:18)
# no need to create a list. Just a vector
urls = paste0("https://www.mlssoccer.com/players?page=", page)
# define this function that collects the player's name from a url
get_the_names = function( url){
url %>%
read_html() %>%
html_nodes("a.name_link") %>%
html_text()
}
# map the urls to the function that gets the names
players = map(urls, get_the_names) %>%
# turn into a single character vector
unlist() %>%
# make lower case
tolower() %>%
# replace the `space` to underscore
str_replace_all(" ", "-")
# Now create a vector of player urls
player_urls = paste0("https://www.mlssoccer.com/players/", players )
# define a function that reads the 3rd table of the url
get_the_summary_stats <- function(url){
url %>%
read_html() %>%
html_nodes("table") %>%
html_table() %>% .[[3]]
}
# lets read 3 players only to speed things up [otherwise it takes a significant amount of time to run...]
a_few_players <- player_urls[1:5]
# get the stats
tables = a_few_players %>%
# important step so I can name the rows I get in the table
set_names() %>%
#map the player urls to the function that reads the 3rd table
# note the `safely` wrap around the get_the_summary_stats' function
# since there are players with no stats and causes an error (eg.brenden-aaronson )
# the output will be a list of lists [result and error]
map(., ~{ Sys.sleep(5)
safely(get_the_summary_stats) }) %>%
# collect only the `result` output (the table) INTO A DATA FRAME
# There is also an `error` output
# also, name each row with the players name
map_df("result", .id = "player") %>%
#keep only the player name (remove the www.mls.... part)
mutate(player = str_replace(player, "https://www.mlssoccer.com/players/", "")) %>%
as_tibble()
tables <- tables %>% separate(Match,c("awayTeam","homeTeam"), extra= "drop", fill = "right")
purrr::safely(...) returns a function, so your map(., { Sys.sleep(5); safely(get_the_summary_stats) }) is returning functions, not any data. In R, a "closure" is a function and its enclosing environment.
Tilde notation is a tidyverse-specific method of more-terse anonymous functions. Typically (e.g., with lapply) one would use lapply(mydata, function(x) get_the_summary_stats(x)). In tilde notation, the same thing is written as map(mydata, ~ get_the_summary_stats(.))
So, re-write to:
... %>% map(~ { Sys.sleep(5); safely(get_the_summary_stats)(.); })
From comments by #r2evans

Using lapply instead of repeating code

I would like to know how to use lapply and/or for loops to have more concise code.
This is what I currently have and it works.
MLFreq <- MLlyrics %>%
unnest_tokens(word, line) %>%
anti_join(stop_words) %>%
ungroup() %>%
count(word)
MLpct <- sum(albumList2$MLlyrics$n) / sum(MLFreq$n)
ViewFreq <- ViewLyrics %>%
unnest_tokens(word, line) %>%
anti_join(stop_words) %>%
ungroup() %>%
count(word)
Viewpct <- sum(albumList2$ViewLyrics$n) / sum(ViewFreq$n)
#... repeating 6 times with different data frames
I've been trying
Freq <- lapply(albumList2, function(df){
df %>% unnest_tokens(word, line) %>%
anti_join(stop_words) %>%
ungroup()%>%
count(word) %>%
sum(albumList2$df$n) / sum(df$n)
})
and
for (i in 1:length(albumList2)) {
unnest_tokens(word, line) %>%
anti_join(stop_words) %>%
ungroup()%>%
count(word) %>%
print(sum(albumList2$i$n) / sum(i$n))
}
but the lapply brings
Error in check_input(x) : Input must be a character vector of any length or
a list of character vectors, each of which has a length of 1.
and the for loop brings
no applicable method for 'unnest_tokens_' applied to an object of class
"function"
For reference albumList2 contains a list of data frames (MLlyrics, ViewLyrics, etc...)
I was originally going to leave it as is but just read something along the lines of "If you use the same code 3 times, loop over it"
The problem with the lapply example is that the list that you are looping over is a nested list instead of single list.
Also, references of the kind: sum(albumList2$df$n) / sum(df$n) & print(sum(albumList2$i$n) / sum(i$n)) would not work.
i is just a number ranging from 1 to length(albumList2). Saying you want 1$n or albumList2$1$n does not make sense.
You should read about indexing in lists and nested lists here and here. Please add some dummy data that everyone can test and help you better.

Combining pipes and the magrittr dot (.) placeholder

I am fairly new to R and I am trying to understand the %>% operator and the usage of the " ." (dot) placeholder. As a simple example the following code works
library(magrittr)
library(ensurer)
ensure_data.frame <- ensures_that(is.data.frame(.))
data.frame(x = 5) %>% ensure_data.frame
However the following code fails
ensure_data.frame <- ensures_that(. %>% is.data.frame)
data.frame(x = 5) %>% ensure_data.frame
where I am now piping the placeholder into the is.data.frame method.
I am guessing that it is my understanding of the limitations/interpretation of the dot placeholder that is lagging, but can anyone clarify this?
The "problem" is that magrittr has a short-hand notation for anonymous functions:
. %>% is.data.frame
is roughly the same as
function(.) is.data.frame(.)
In other words, when the dot is the (left-most) left-hand side, the pipe has special behaviour.
You can escape the behaviour in a few ways, e.g.
(.) %>% is.data.frame
or any other way where the LHS is not identical to .
In this particular example, this may seem as undesirable behaviuour, but commonly in examples like this there's really no need to pipe the first expression, so is.data.frame(.) is as expressive as . %>% is.data.frame, and
examples like
data %>%
some_action %>%
lapply(. %>% some_other_action %>% final_action)
can be argued to be clearner than
data %>%
some_action %>%
lapply(function(.) final_action(some_other_action(.)))
This is the problem:
. = data.frame(x = 5)
a = data.frame(x = 5)
a %>% is.data.frame
#[1] TRUE
. %>% is.data.frame
#Functional sequence with the following components:
#
# 1. is.data.frame(.)
#
#Use 'functions' to extract the individual functions.
Looks like a bug to me, but dplyr experts can chime in.
A simple workaround in your expression is to do .[] %>% is.data.frame.

Resources