Automatically generate new variable names using dplyr mutate - r

I would like to create variable names dynamically while using dplyr; although, I’d be fine with a non-dplyr solution as well.
For Example:
data(iris)
library(dplyr)
iris <- iris %>%
group_by(Species) %>%
mutate(
lag_Sepal.Length = lag(Sepal.Length),
lag_Sepal.Width = lag(Sepal.Width),
lag_Petal.Length = lag(Petal.Length)
) %>%
ungroup
head(iris)
Sepal.Length Sepal.Width Petal.Length Petal.Width Species lag_Sepal.Length lag_Sepal.Width
(dbl) (dbl) (dbl) (dbl) (fctr) (dbl) (dbl)
1 5.1 3.5 1.4 0.2 setosa NA NA
2 4.9 3.0 1.4 0.2 setosa 5.1 3.5
3 4.7 3.2 1.3 0.2 setosa 4.9 3.0
4 4.6 3.1 1.5 0.2 setosa 4.7 3.2
5 5.0 3.6 1.4 0.2 setosa 4.6 3.1
6 5.4 3.9 1.7 0.4 setosa 5.0 3.6
Variables not shown: lag_Petal.Length (dbl)
But, instead of doing this three times, I want to create 100 of these “lag” variables that take in the name: lag_original variable name. I’m trying to figure out how to do this without typing the new variable name 100 times, but I’m coming up short.
I’ve looked into this example and this example elsewhere on SO. They are similar, but I’m not quite able to piece together the specific solution I need. Any help is appreciated!
Edit
Thanks to #BenFasoli for the inspiration. I took his answer and tweaked it just a bit to get the solution I needed.
I also used This RStudio Blog and This SO post. The "lag" in the variable name is trailing instead of leading, but I can live with that.
My final code is posted here in case it’s helpful to anyone else:
lagged <- iris %>%
group_by(Species) %>%
mutate_at(
vars(Sepal.Length:Petal.Length),
funs("lag" = lag)) %>%
ungroup
# A tibble: 6 x 8
Sepal.Length Sepal.Width Petal.Length Petal.Width Species Sepal.Length_lag Sepal.Width_lag
<dbl> <dbl> <dbl> <dbl> <fctr> <dbl> <dbl>
1 5.1 3.5 1.4 0.2 setosa NA NA
2 4.9 3.0 1.4 0.2 setosa 5.1 3.5
3 4.7 3.2 1.3 0.2 setosa 4.9 3.0
4 4.6 3.1 1.5 0.2 setosa 4.7 3.2
5 5.0 3.6 1.4 0.2 setosa 4.6 3.1
6 5.4 3.9 1.7 0.4 setosa 5.0 3.6
# ... with 1 more variables: Petal.Length_lag <dbl>

You can use mutate_all (or mutate_at for specific columns) then prepend lag_ to the column names.
data(iris)
library(dplyr)
lag_iris <- iris %>%
group_by(Species) %>%
mutate_all(funs(lag(.))) %>%
ungroup
colnames(lag_iris) <- paste0('lag_', colnames(lag_iris))
head(lag_iris)
lag_Sepal.Length lag_Sepal.Width lag_Petal.Length lag_Petal.Width lag_Species
<dbl> <dbl> <dbl> <dbl> <fctr>
1 NA NA NA NA setosa
2 5.1 3.5 1.4 0.2 setosa
3 4.9 3.0 1.4 0.2 setosa
4 4.7 3.2 1.3 0.2 setosa
5 4.6 3.1 1.5 0.2 setosa
6 5.0 3.6 1.4 0.2 setosa

Here is a data.table approach. I chose columns with numbers in this case. What you want to do is to choose column names and create new column names in advance. Then, you apply shift(), which works like lag() and lead() in the dplyr package, to each of the columns you chose.
library(data.table)
# Crate a df for this demo.
mydf <- iris
# Choose columns that you want to apply lag() and create new colnames.
cols = names(iris)[sapply(iris, is.numeric)]
anscols = paste("lag_", cols, sep = "")
# Apply shift() to each of the chosen columns.
setDT(mydf)[, (anscols) := shift(.SD, 1, type = "lag"),
.SDcols = cols]
Sepal.Length Sepal.Width Petal.Length Petal.Width Species lag_Sepal.Length lag_Sepal.Width
1: 5.1 3.5 1.4 0.2 setosa NA NA
2: 4.9 3.0 1.4 0.2 setosa 5.1 3.5
3: 4.7 3.2 1.3 0.2 setosa 4.9 3.0
4: 4.6 3.1 1.5 0.2 setosa 4.7 3.2
5: 5.0 3.6 1.4 0.2 setosa 4.6 3.1
---
146: 6.7 3.0 5.2 2.3 virginica 6.7 3.3
147: 6.3 2.5 5.0 1.9 virginica 6.7 3.0
148: 6.5 3.0 5.2 2.0 virginica 6.3 2.5
149: 6.2 3.4 5.4 2.3 virginica 6.5 3.0
150: 5.9 3.0 5.1 1.8 virginica 6.2 3.4
lag_Petal.Length lag_Petal.Width
1: NA NA
2: 1.4 0.2
3: 1.4 0.2
4: 1.3 0.2
5: 1.5 0.2
---
146: 5.7 2.5
147: 5.2 2.3
148: 5.0 1.9
149: 5.2 2.0
150: 5.4 2.3

Since you're also happy with a non-dplyr, try this:
lagger <- function(x, n) c(rep(NA,n), head(x,-n) )
iris[paste0("lag_", names(iris) )] <- lapply(iris, lagger, n=1)
head(iris,2)[-(1:5)]
# lag_Sepal.Length lag_Sepal.Width lag_Petal.Length lag_Petal.Width lag_Species
#1 NA NA NA NA NA
#2 5.1 3.5 1.4 0.2 1

Related

How to slice a dataset into multiple dataset in R

For this example, I'm going to use iris dataset built-in in R.
How can I avoid the copy and pasting of the syntax below to have the same output?
package
library(dplyr)
Input
head(iris)
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#1 5.1 3.5 1.4 0.2 setosa
#2 4.9 3.0 1.4 0.2 setosa
#3 4.7 3.2 1.3 0.2 setosa
#4 4.6 3.1 1.5 0.2 setosa
#5 5.0 3.6 1.4 0.2 setosa
#6 5.4 3.9 1.7 0.4 setosa
Manual Solution
I have to subset my dataset based on the name of the column names.
I know how to do this "manually" but it would require a lot of copying and pasting on my current dataset.
Sepal <- iris %>% select(contains("Sepal"))
Petal <- iris %>% select(contains("Petal"))
Output
head(Sepal)
# Sepal.Length Sepal.Width
# 1 5.1 3.5
# 2 4.9 3.0
# 3 4.7 3.2
# 4 4.6 3.1
# 5 5.0 3.6
# 6 5.4 3.9
head(Petal)
# Petal.Length Petal.Width
# 1 1.4 0.2
# 2 1.4 0.2
# 3 1.3 0.2
# 4 1.5 0.2
# 5 1.4 0.2
# 6 1.7 0.4
How can I automatize this process? I think I can use the purrr package here. But I couldn't find a way to do it.
You can use
library(tidyverse)
map(set_names(c("Sepal", "Petal")), ~ select(iris, starts_with(.x)))
output (head)
$Sepal
Sepal.Length Sepal.Width
1 5.1 3.5
2 4.9 3.0
3 4.7 3.2
4 4.6 3.1
5 5.0 3.6
6 5.4 3.9
$Petal
Petal.Length Petal.Width
1 1.4 0.2
2 1.4 0.2
3 1.3 0.2
4 1.5 0.2
5 1.4 0.2
6 1.7 0.4
An option is also to use split.default on the substring of column names to return a named list of data.frames
library(dplyr)
library(stringr)
head(iris) %>%
select(-Species) %>%
split.default(str_remove(names(.), "\\..*"))
$Petal
Petal.Length Petal.Width
1 1.4 0.2
2 1.4 0.2
3 1.3 0.2
4 1.5 0.2
5 1.4 0.2
6 1.7 0.4
$Sepal
Sepal.Length Sepal.Width
1 5.1 3.5
2 4.9 3.0
3 4.7 3.2
4 4.6 3.1
5 5.0 3.6
6 5.4 3.9

Re-order rows of a R dataframe based on a column/ label

My current dataframe in R has the following dimensions
nrows=605
ncol: 1514
The first column indicates the class/ label and my dataset has only two classes namely: setosa and iris.
test[1:5,]
class id1 id2...
1: setosa 2 4.....
2: setosa 2 5 .....
3: setosa 5 4 .....
4: iris 5 9......
5: iris 7 9 ....
However the dataframe is ordered as of now : ie. Rows 2- row 233 of my dataframe correspond to class setosa and class iris is from 234 until end. I want the dataset to be rearranged so that the samples are mixed up.
The expected output should be in following form:
If I do df[1:10,] ie. 10 lines of dataframe ,I should be able to see samples of both iris and setosa. Any ideas or suggestion on how to do this?
library( tidyverse )
iris[1:10,]
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# 1 5.1 3.5 1.4 0.2 setosa
# 2 4.9 3.0 1.4 0.2 setosa
# 3 4.7 3.2 1.3 0.2 setosa
# 4 4.6 3.1 1.5 0.2 setosa
# 5 5.0 3.6 1.4 0.2 setosa
# 6 5.4 3.9 1.7 0.4 setosa
# 7 4.6 3.4 1.4 0.3 setosa
# 8 5.0 3.4 1.5 0.2 setosa
# 9 4.4 2.9 1.4 0.2 setosa
# 10 4.9 3.1 1.5 0.1 setosa
df <- iris %>%
group_by( Species ) %>%
mutate( id = row_number() ) %>%
arrange( id ) %>%
select ( -id )
df[1:10,]
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fct>
# 1 5.1 3.5 1.4 0.2 setosa
# 2 7 3.2 4.7 1.4 versicolor
# 3 6.3 3.3 6 2.5 virginica
# 4 4.9 3 1.4 0.2 setosa
# 5 6.4 3.2 4.5 1.5 versicolor
# 6 5.8 2.7 5.1 1.9 virginica
# 7 4.7 3.2 1.3 0.2 setosa
# 8 6.9 3.1 4.9 1.5 versicolor
# 9 7.1 3 5.9 2.1 virginica
# 10 4.6 3.1 1.5 0.2 setosa

select the first and last row within in a data frame?

Is there a function in BASE R that could show the first and last rows within in a data frame? I know the functions like ropls::strF and print an object in data.table could do this. It is not like this topic Select first and last row from grouped data
ropls::strF(iris)
#Sepal.Length Sepal.Width ... Petal.Width Species
#numeric numeric ... numeric factor
#nRow nCol size NAs
#150 5 0 Mb 0
#Sepal.Length Sepal.Width ... Petal.Width Species
#1 5.1 3.5 ... 0.2 setosa
#2 4.9 3 ... 0.2 setosa
#... ... ... ... ... ...
#149 6.2 3.4 ... 2.3 virginica
#150 5.9 3 ... 1.8 virginica
library(data.table)
a <- as.data.table(iris)
a
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#1: 5.1 3.5 1.4 0.2 setosa
#2: 4.9 3.0 1.4 0.2 setosa
#3: 4.7 3.2 1.3 0.2 setosa
#4: 4.6 3.1 1.5 0.2 setosa
#5: 5.0 3.6 1.4 0.2 setosa
#---
#146: 6.7 3.0 5.2 2.3 virginica
#147: 6.3 2.5 5.0 1.9 virginica
#148: 6.5 3.0 5.2 2.0 virginica
#149: 6.2 3.4 5.4 2.3 virginica
#150: 5.9 3.0 5.1 1.8 virginica
As others said in the comments, there isn't a function in base R to do this, but it's straightforward enough to write a function that binds together the first N rows and last N rows.
head_and_tail <- function(x, n = 1) {
rbind(
head(x, n),
tail(x, n)
)
}
head_and_tail(iris, n = 3)
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 5.1 3.5 1.4 0.2 setosa
#> 2 4.9 3.0 1.4 0.2 setosa
#> 3 4.7 3.2 1.3 0.2 setosa
#> 148 6.5 3.0 5.2 2.0 virginica
#> 149 6.2 3.4 5.4 2.3 virginica
#> 150 5.9 3.0 5.1 1.8 virginica
Created on 2018-12-22 by the reprex package (v0.2.1)

dplyr: how to reference columns by column index rather than column name using mutate?

Using dplyr, you can do something like this:
iris %>% head %>% mutate(sum=Sepal.Length + Sepal.Width)
Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum
1 5.1 3.5 1.4 0.2 setosa 8.6
2 4.9 3.0 1.4 0.2 setosa 7.9
3 4.7 3.2 1.3 0.2 setosa 7.9
4 4.6 3.1 1.5 0.2 setosa 7.7
5 5.0 3.6 1.4 0.2 setosa 8.6
6 5.4 3.9 1.7 0.4 setosa 9.3
But above, I referenced the columns by their column names. How can I use 1 and 2 , which are the column indices to achieve the same result?
Here I have the following, but I feel it's not as elegant.
iris %>% head %>% mutate(sum=apply(select(.,1,2),1,sum))
Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum
1 5.1 3.5 1.4 0.2 setosa 8.6
2 4.9 3.0 1.4 0.2 setosa 7.9
3 4.7 3.2 1.3 0.2 setosa 7.9
4 4.6 3.1 1.5 0.2 setosa 7.7
5 5.0 3.6 1.4 0.2 setosa 8.6
6 5.4 3.9 1.7 0.4 setosa 9.3
You can try:
iris %>% head %>% mutate(sum = .[[1]] + .[[2]])
Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum
1 5.1 3.5 1.4 0.2 setosa 8.6
2 4.9 3.0 1.4 0.2 setosa 7.9
3 4.7 3.2 1.3 0.2 setosa 7.9
4 4.6 3.1 1.5 0.2 setosa 7.7
5 5.0 3.6 1.4 0.2 setosa 8.6
6 5.4 3.9 1.7 0.4 setosa 9.3
I'm a bit late to the game, but my personal strategy in cases like this is to write my own tidyverse-compliant function that will do exactly what I want. By tidyverse-compliant, I mean that the first argument of the function is a data frame and that the output is a vector that can be added to the data frame.
sum_cols <- function(x, col1, col2){
x[[col1]] + x[[col2]]
}
iris %>%
head %>%
mutate(sum = sum_cols(x = ., col1 = 1, col2 = 2))
An alternative to reusing . in mutate that will respect grouping is to use dplyr::cur_data_all(). From help(cur_data_all)
cur_data_all() gives the current data for the current group (including grouping variables)
Consider the following:
iris %>% group_by(Species) %>% mutate(sum = .[[1]] + .[[2]]) %>% head
#Error: Problem with `mutate()` column `sum`.
#ℹ `sum = .[[1]] + .[[2]]`.
#ℹ `sum` must be size 50 or 1, not 150.
#ℹ The error occurred in group 1: Species = setosa.
If instead you use cur_data_all(), it works without issue:
iris %>% mutate(sum = select(cur_data_all(),1) + select(cur_data_all(),2)) %>% head()
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species Sepal.Length
#1 5.1 3.5 1.4 0.2 setosa 8.6
#2 4.9 3.0 1.4 0.2 setosa 7.9
#3 4.7 3.2 1.3 0.2 setosa 7.9
#4 4.6 3.1 1.5 0.2 setosa 7.7
#5 5.0 3.6 1.4 0.2 setosa 8.6
#6 5.4 3.9 1.7 0.4 setosa 9.3
The same approach works with the extract operator ([[).
iris %>% mutate(sum = cur_data()[[1]] + cur_data()[[2]]) %>% head()
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum
#1 5.1 3.5 1.4 0.2 setosa 8.6
#2 4.9 3.0 1.4 0.2 setosa 7.9
#3 4.7 3.2 1.3 0.2 setosa 7.9
#4 4.6 3.1 1.5 0.2 setosa 7.7
#5 5.0 3.6 1.4 0.2 setosa 8.6
#6 5.4 3.9 1.7 0.4 setosa 9.3
What do you think about this version?
Inspired by #SavedByJesus's answer.
applySum <- function(df, ...) {
assertthat::assert_that(...length() > 0, msg = "one or more column indexes are required")
mutate(df, Sum = apply(as.data.frame(df[, c(...)]), 1, sum))
}
iris %>%
head(2) %>%
applySum(1, 2)
#
### output
#
Sepal.Length Sepal.Width Petal.Length Petal.Width Species Sum
1 5.1 3.5 1.4 0.2 setosa 8.6
2 4.9 3.0 1.4 0.2 setosa 7.9
#
### you can select and sum more then two columns by the same function
#
iris %>%
head(2) %>%
applySum(1, 2, 3, 4)
#
### output
#
Sepal.Length Sepal.Width Petal.Length Petal.Width Species Sum
1 5.1 3.5 1.4 0.2 setosa 10.2
2 4.9 3.0 1.4 0.2 setosa 9.5
To address the issue that #pluke is asking about in the comments, dplyr doesn't really support column index.
Not a perfect solution, but you can use base R to get around this
iris[1] <- iris[1] + iris[2]
This can now (packageVersion("dplyr") >= 1.0.0) be done very nicely with the combination of dplyr::rowwise() and dplyr::c_across().
library(dplyr)
packageVersion("dplyr")
#> [1] '1.0.10'
iris %>%
head %>%
rowwise() %>%
mutate(sum = sum(c_across(c(1, 2))))
#> # A tibble: 6 × 6
#> # Rowwise:
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species sum
#> <dbl> <dbl> <dbl> <dbl> <fct> <dbl>
#> 1 5.1 3.5 1.4 0.2 setosa 8.6
#> 2 4.9 3 1.4 0.2 setosa 7.9
#> 3 4.7 3.2 1.3 0.2 setosa 7.9
#> 4 4.6 3.1 1.5 0.2 setosa 7.7
#> 5 5 3.6 1.4 0.2 setosa 8.6
#> 6 5.4 3.9 1.7 0.4 setosa 9.3
Created on 2022-11-01 with reprex v2.0.2

Creating a random sample from a dataframe with a nested structure

This question builds from the SO post found here
I am trying to extract a random sample of rows in a data frame using a nesting condition.
Using the following dummy dataset (modified from iris):
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 5.3 2.9 1.5 0.2 setosa
5 5.2 3.7 1.3 0.2 virginica
6 4.7 3.2 1.5 0.2 virginica
7 3.9 3.1 1.4 0.2 virginica
8 4.7 3.2 1.3 0.2 virginica
9 4.0 3.1 1.5 0.2 versicolor
10 5.0 3.6 1.4 0.2 versicolor
11 4.6 3.1 1.5 0.2 versicolor
12 5.0 3.6 1.5 0.2 versicolor
The code below works fine to take a simple sample of 2 rows:
iris[sample(nrow(iris), 2), ]
However, what I would like to do is to take a sample of 2 rows for each level of a specific variable. For example create a random sample of 2 rows for each level of the variable 'Species', like that:
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
4 5.3 2.9 1.5 0.2 setosa
6 4.7 3.2 1.5 0.2 virginica
7 3.9 3.1 1.4 0.2 virginica
11 4.6 3.1 1.5 0.2 versicolor
12 5.0 3.6 1.5 0.2 versicolor
Thanks for your help!
Very easy with dplyr:
library(dplyr)
iris %>%
group_by(Species) %>%
sample_n(size = 2)
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# 1 4.6 3.4 1.4 0.3 setosa
# 2 5.2 3.5 1.5 0.2 setosa
# 3 6.5 2.8 4.6 1.5 versicolor
# 4 5.7 2.8 4.5 1.3 versicolor
# 5 5.8 2.8 5.1 2.4 virginica
# 6 7.7 2.6 6.9 2.3 virginica
You can group by as many columns as you'd like
CO2 %>% group_by(Type, Treatment) %>% sample_n(size = 2)

Resources