I have this tibble
# Data
set.seed(1)
x <- tibble(values = round(rnorm(20, 10, 10), 0),
index = c(0,0,1,1,1,0,1,0,1,1,1,1,1,1,0,
1,1,0,0,0))
x
#> # A tibble: 20 x 2
#> values index
#> <dbl> <dbl>
#> 1 4 0
#> 2 12 0
#> 3 2 1
#> 4 26 1
#> 5 13 1
#> 6 2 0
#> 7 15 1
#> 8 17 0
#> 9 16 1
#> 10 7 1
#> 11 25 1
#> 12 14 1
#> 13 4 1
#> 14 -12 1
#> 15 21 0
#> 16 10 1
#> 17 10 1
#> 18 19 0
#> 19 18 0
#> 20 16 0
I'd like to create groups where the value in the index column are consecutive ones. The final aim is to compute the sum per each group.
This is the expected tibble is someting like:
# A tibble: 20 x 3
values index group
<dbl> <dbl> <chr>
1 4 0 NA
2 12 0 NA
3 2 1 A
4 26 1 A
5 13 1 A
6 2 0 NA
7 15 1 B
8 17 0 NA
9 16 1 C
10 7 1 C
11 25 1 C
12 14 1 C
13 4 1 C
14 -12 1 C
15 21 0 NA
16 10 1 D
17 10 1 D
18 19 0 NA
19 18 0 NA
20 16 0 NA
Thank you in advance for your advice.
You could use cumsum() on runs identified by rle(), replacing the values where index is zero with NA. If there are more than 26 IDs it will need a minor modification.
library(dplyr)
x2 <- x %>%
mutate(id = LETTERS[replace(with(rle(index),
rep(cumsum(values), lengths)), index == 0, NA)])
Giving:
# A tibble: 20 x 3
values index id
<dbl> <dbl> <chr>
1 4 0 NA
2 12 0 NA
3 2 1 A
4 26 1 A
5 13 1 A
6 2 0 NA
7 15 1 B
8 17 0 NA
9 16 1 C
10 7 1 C
11 25 1 C
12 14 1 C
13 4 1 C
14 -12 1 C
15 21 0 NA
16 10 1 D
17 10 1 D
18 19 0 NA
19 18 0 NA
20 16 0 NA
To sum the values:
x2 %>%
group_by(id) %>%
summarise(sv = sum(values))
# A tibble: 5 x 2
id sv
* <chr> <dbl>
1 A 41
2 B 15
3 C 54
4 D 20
5 NA 109
An option with data.table
library(data.table)
setDT(x)[, group := LETTERS[as.integer(factor((NA^!index) *rleid(index)))]]
x
# values index group
# 1: 4 0 <NA>
# 2: 12 0 <NA>
# 3: 2 1 A
# 4: 26 1 A
# 5: 13 1 A
# 6: 2 0 <NA>
# 7: 15 1 B
# 8: 17 0 <NA>
# 9: 16 1 C
#10: 7 1 C
#11: 25 1 C
#12: 14 1 C
#13: 4 1 C
#14: -12 1 C
#15: 21 0 <NA>
#16: 10 1 D
#17: 10 1 D
#18: 19 0 <NA>
#19: 18 0 <NA>
#20: 16 0 <NA>
Or similar logic in dplyr
library(dplyr)
x %>%
mutate(group = LETTERS[as.integer(factor((NA^!index) *rleid(index)))])
# A tibble: 20 x 3
# values index group
# <dbl> <dbl> <chr>
# 1 4 0 <NA>
# 2 12 0 <NA>
# 3 2 1 A
# 4 26 1 A
# 5 13 1 A
# 6 2 0 <NA>
# 7 15 1 B
# 8 17 0 <NA>
# 9 16 1 C
#10 7 1 C
#11 25 1 C
#12 14 1 C
#13 4 1 C
#14 -12 1 C
#15 21 0 <NA>
#16 10 1 D
#17 10 1 D
#18 19 0 <NA>
#19 18 0 <NA>
#20 16 0 <NA>
Related
Here is a part of the sample data :
dat<-read.table (text=" ID Time B1 T1 Q1 W1 M1
1 12 12 0 12 11 9
1 13 0 1 NA NA NA
2 10 12 0 6 7 8
2 14 0 1 NA NA NA
1 16 16A 0 1 2 4
1 14 0 1 NA NA NA
2 14 16A 0 5 6 7
2 7 0 1 NA NA NA
1 7 20 0 5 8 0
1 7 0 1 NA NA NA
2 9 20 0 7 8 1
2 9 0 1 NA NA NA
", header=TRUE)
I want to update value 1 In column T1 for repeated IDs. For the first repeated IDs, should be a value of 1, and for the second repeated IDs, a value of 2; and for the third repeated IDs, a value of 3 and so on. I also want to replace NA with blank cells. here is the expected outcome:
ID Time B1 T1 Q1 W1 M1
1 12 12 0 12 11 9
1 13 0 1
2 10 12 0 6 7 8
2 14 0 1
1 16 16A 0 1 2 4
1 14 0 2
2 14 16A 0 5 6 7
2 7 0 2
1 7 20 0 5 8 0
1 7 0 3
2 9 20 0 7 8 1
2 9 0 3
You could use an ifelse across with cumsum per group like this:
library(dplyr)
dat %>%
group_by(ID, B1) %>%
mutate(across(T1, ~ ifelse(.x == 1, cumsum(.x), T1)))
#> # A tibble: 12 × 7
#> # Groups: ID, B1 [8]
#> ID Time B1 T1 Q1 W1 M1
#> <int> <int> <chr> <int> <int> <int> <int>
#> 1 1 12 12 0 12 11 9
#> 2 1 13 0 1 NA NA NA
#> 3 2 10 12 0 6 7 8
#> 4 2 14 0 1 NA NA NA
#> 5 1 16 16A 0 1 2 4
#> 6 1 14 0 2 NA NA NA
#> 7 2 14 16A 0 5 6 7
#> 8 2 7 0 2 NA NA NA
#> 9 1 7 20 0 5 8 0
#> 10 1 7 0 3 NA NA NA
#> 11 2 9 20 0 7 8 1
#> 12 2 9 0 3 NA NA NA
Created on 2023-01-14 with reprex v2.0.2
With data.table
library(data.table)
setDT(dat)[T1 ==1, T1 := cumsum(T1), .(ID, B1)]
-output
> dat
ID Time B1 T1 Q1 W1 M1
1: 1 12 12 0 12 11 9
2: 1 13 0 1 NA NA NA
3: 2 10 12 0 6 7 8
4: 2 14 0 1 NA NA NA
5: 1 16 16A 0 1 2 4
6: 1 14 0 2 NA NA NA
7: 2 14 16A 0 5 6 7
8: 2 7 0 2 NA NA NA
9: 1 7 20 0 5 8 0
10: 1 7 0 3 NA NA NA
11: 2 9 20 0 7 8 1
12: 2 9 0 3 NA NA NA
A similar to my data is:
dat1<-read.table (text=" ID Rat Garden Class Time1 Time2 Time3
1 12 12 0 15 16 20
1 13 0 1 NA NA NA
2 13 11 0 18 12 16
2 9 0 1 NA NA NA
1 6 13 0 17 14 14
1 7 0 2 NA NA NA
2 4 14 0 17 16 12
2 3 0 2 NA NA NA
", header=TRUE)
dat2<-read.table (text=" ID Value1 Value2
1 6 7
2 5 4
", header=TRUE)
I want to insert the values of dat2 to dat1 in the Time1 column. In front of numbers 1 and 2 in the class column.
I get the following outcome.
ID Rat Garden Class Time1 Time2 Time3
1 12 12 0 15 16 20
1 13 0 1 6
2 13 11 0 18 12 16
2 9 0 1 5
1 6 13 0 17 14 14
1 7 0 2 7
2 4 14 0 17 16 12
2 3 0 2 4
We may group by 'ID', and replace the 'Time1' where the NA values occur with the unlisted 'dat2' 'Value' columns where the ID matches
library(dplyr)
dat1 %>%
group_by(ID) %>%
mutate(Time1 = replace(Time1, is.na(Time1),
unlist(dat2[-1][dat2$ID == cur_group()$ID,]))) %>%
ungroup
-output
# A tibble: 8 × 7
ID Rat Garden Class Time1 Time2 Time3
<int> <int> <int> <int> <int> <int> <int>
1 1 12 12 0 15 16 20
2 1 13 0 1 6 NA NA
3 2 13 11 0 18 12 16
4 2 9 0 1 5 NA NA
5 1 6 13 0 17 14 14
6 1 7 0 2 7 NA NA
7 2 4 14 0 17 16 12
8 2 3 0 2 4 NA NA
Here is a wild ride:
First we pull the values as a vector from dat2.
Then we put alternating an NA into the vector until it gets column length of dat1 and
finally we use coalesce after cbind:
library(dplyr)
library(tidyr)
vector <- dat2 %>%
pivot_longer(-ID) %>%
arrange(name) %>%
pull(value)
col_x <- c(sapply(vector, c, rep(NA, 1)))
cbind(dat1, col_x) %>%
mutate(col_x = lag(col_x)) %>%
mutate(Time1= coalesce(Time1, col_x), .keep="unused")
ID Rat Garden Class Time1 Time2 Time3
1 1 12 12 0 15 16 20
2 1 13 0 1 6 NA NA
3 2 13 11 0 18 12 16
4 2 9 0 1 5 NA NA
5 1 6 13 0 17 14 14
6 1 7 0 2 7 NA NA
7 2 4 14 0 17 16 12
8 2 3 0 2 4 NA NA
I have a huge dataset and wanted to create a binary dummy variable indicating whether a value is observed before. Here is the sample data set.
data.frame(
id = c(rep("A",3),rep("B",3),rep("C",3)),
time = rep(seq(1:3),3),
item = c(11,12,13,11,11,13,22,11,22))
From the dataset, here is the desired column,
observed_b4 = c(NA,0,0,NA,1,0,NA,0,1)
For each group, I want to have information about whether item is observed before or not. I can do it with for-loop but the data size is too big to do.
Using duplicated:
base:
cbind(x, flag = as.integer(duplicated(paste(x$id, x$item))))
# id time item flag
# 1 A 1 11 0
# 2 A 2 12 0
# 3 A 3 13 0
# 4 B 1 11 0
# 5 B 2 11 1
# 6 B 3 13 0
# 7 C 1 22 0
# 8 C 2 11 0
# 9 C 3 22 1
or dplyr:
library(dplyr)
x %>%
group_by(id) %>%
mutate(flag = as.integer(duplicated(item)))
## A tibble: 9 x 4
## Groups: id [3]
# id time item flag
# <chr> <int> <dbl> <int>
#1 A 1 11 0
#2 A 2 12 0
#3 A 3 13 0
#4 B 1 11 0
#5 B 2 11 1
#6 B 3 13 0
#7 C 1 22 0
#8 C 2 11 0
#9 C 3 22 1
A solution with base R that uses: ave and duplicated.
ave allows you to apply a function over df$item for each group made by df$id. duplicated checks whether an item was already shown. ave returns automatically a numeric vector (the name class of the input vector).
df$observed_b4 <- ave(df$item, df$id, FUN = duplicated)
df
#> id time item observed_b4
#> 1 A 1 11 0
#> 2 A 2 12 0
#> 3 A 3 13 0
#> 4 B 1 11 0
#> 5 B 2 11 1
#> 6 B 3 13 0
#> 7 C 1 22 0
#> 8 C 2 11 0
#> 9 C 3 22 1
However, to get exactly what you're looking for, you can use this:
df$observed_b4 <- ave(df$item, df$id, FUN = function(x) replace(duplicated(x),1,NA))
df
#> id time item observed_b4
#> 1 A 1 11 NA
#> 2 A 2 12 0
#> 3 A 3 13 0
#> 4 B 1 11 NA
#> 5 B 2 11 1
#> 6 B 3 13 0
#> 7 C 1 22 NA
#> 8 C 2 11 0
#> 9 C 3 22 1
We could group by 'id', 'item', create a logical vector with row_number() and coerce it to binary (+)
library(dplyr)
df1 %>%
group_by(id, item) %>%
mutate(flag = +(row_number() != 1))
-output
# A tibble: 9 x 4
# Groups: id, item [7]
# id time item flag
# <chr> <int> <dbl> <int>
#1 A 1 11 0
#2 A 2 12 0
#3 A 3 13 0
#4 B 1 11 0
#5 B 2 11 1
#6 B 3 13 0
#7 C 1 22 0
#8 C 2 11 0
#9 C 3 22 1
I have a data frame and wish to sort specific columns alphabetically in dplyr. I know I can use the code below to sort all columns, but I would only like to sort columns C, B and A alphabetically. I tried using the across function as I would effectively like to select columns C:A, but this did not work.
df <- data.frame(1:16)
df$Testinfo1 <- 1
df$Band <- 1
df$Alpha <- 1
df$C <- c(10,12,14,16,10,12,14,16,10,12,14,16,10,12,14,16)
df$B <- c(10,0,0,0,12,12,12,12,0,14,NA_real_,14,16,16,16,16)
df$A <- c(1,1,1,1,1,1,1,1,1,1,1,14,NA_real_,NA_real_,NA_real_,16)
df
df %>%
select(sort(names(.)))
A Alpha B Band C Testinfo1 X1.16
1: 1 1 10 1 10 1 1
2: 1 1 0 1 12 1 2
3: 1 1 0 1 14 1 3
4: 1 1 0 1 16 1 4
5: 1 1 12 1 10 1 5
6: 1 1 12 1 12 1 6
7: 1 1 12 1 14 1 7
8: 1 1 12 1 16 1 8
9: 1 1 0 1 10 1 9
10: 1 1 14 1 12 1 10
11: 1 1 NA 1 14 1 11
12: 14 1 14 1 16 1 12
13: NA 1 16 1 10 1 13
14: NA 1 16 1 12 1 14
15: NA 1 16 1 14 1 15
16: 16 1 16 1 16 1 16
My desired output is below:
X1.16 Testinfo1 Band Alpha A B C
1: 1 1 1 1 1 10 10
2: 2 1 1 1 1 0 12
3: 3 1 1 1 1 0 14
4: 4 1 1 1 1 0 16
5: 5 1 1 1 1 12 10
6: 6 1 1 1 1 12 12
7: 7 1 1 1 1 12 14
8: 8 1 1 1 1 12 16
9: 9 1 1 1 1 0 10
10: 10 1 1 1 1 14 12
11: 11 1 1 1 1 NA 14
12: 12 1 1 1 14 14 16
13: 13 1 1 1 NA 16 10
14: 14 1 1 1 NA 16 12
15: 15 1 1 1 NA 16 14
16: 16 1 1 1 16 16 16
You can use relocate() (from dplyr 1.0.0 onwards):
library(dplyr)
vars <- c("C", "B", "A")
df %>%
relocate(all_of(sort(vars)), .after = last_col())
If you are passing a character vector of names you should wrap it in all_of() (which will error if any variables are missing) or any_of() which won't.
You can do
sortcols <- c("A","B","C")
library(dplyr)
df %>%
select(-sortcols, sort(sortcols))
The -sortcols part selects everything but the columns you want to sort and then you put the columns you want after those.
A base R option for a case which may or may not exist. If the columns that you want to sort are not at the end of the dataframe.
We add a new column D which you don't want to change the position of.
df$D <- 1:16
cols_to_sort <- c('A', 'B', 'C')
inds <- match(cols_to_sort, names(df))
cols <- seq_along(df)
cols[cols %in% inds] <- inds
df[cols]
# X1.16 Testinfo1 Band Alpha A B C D
#1 1 1 1 1 1 10 10 1
#2 2 1 1 1 1 0 12 2
#3 3 1 1 1 1 0 14 3
#4 4 1 1 1 1 0 16 4
#5 5 1 1 1 1 12 10 5
#6 6 1 1 1 1 12 12 6
#7 7 1 1 1 1 12 14 7
#8 8 1 1 1 1 12 16 8
#9 9 1 1 1 1 0 10 9
#10 10 1 1 1 1 14 12 10
#11 11 1 1 1 1 NA 14 11
#12 12 1 1 1 14 14 16 12
#13 13 1 1 1 NA 16 10 13
#14 14 1 1 1 NA 16 12 14
#15 15 1 1 1 NA 16 14 15
#16 16 1 1 1 16 16 16 16
Here is the code for my example dataset.
df = data.frame("group" =c(rep(1,5),rep(1,6),rep(2,4),rep(2,3)), "time" = c(rep(NA,5),seq(1,6),rep(NA,4),seq(1,3)), "p" = seq(1,18) )
group time p
1 1 NA 1
2 1 NA 2
3 1 NA 3
4 1 NA 4
5 1 NA 5
6 1 1 6
7 1 2 7
8 1 3 8
9 1 4 9
10 1 5 10
11 1 6 11
12 2 NA 12
13 2 NA 13
14 2 NA 14
15 2 NA 15
16 2 1 16
17 2 2 17
18 2 3 18
I would like to figure out how to apply a function by group to only the values that have time then append the result as a new column in the data frame. Here is my example function I would like to apply.
pfunc <- function(p){
p+5
}
The output I am hoping to obtain would look as follows.
group time p new_p
1 1 NA 1 NA
2 1 NA 2 NA
3 1 NA 3 NA
4 1 NA 4 NA
5 1 NA 5 NA
6 1 1 6 11
7 1 2 7 12
8 1 3 8 13
9 1 4 9 14
10 1 5 10 15
11 1 6 11 16
12 2 NA 12 NA
13 2 NA 13 NA
14 2 NA 14 NA
15 2 NA 15 NA
16 2 1 16 21
17 2 2 17 22
18 2 3 18 23
You can try this:
library(dplyr)
df %>% group_by(group) %>%
mutate(pnew=ifelse(is.na(time),time,time+5))
# A tibble: 18 x 4
# Groups: group [2]
group time p pnew
<dbl> <int> <int> <dbl>
1 1 NA 1 NA
2 1 NA 2 NA
3 1 NA 3 NA
4 1 NA 4 NA
5 1 NA 5 NA
6 1 1 6 6
7 1 2 7 7
8 1 3 8 8
9 1 4 9 9
10 1 5 10 10
11 1 6 11 11
12 2 NA 12 NA
13 2 NA 13 NA
14 2 NA 14 NA
15 2 NA 15 NA
16 2 1 16 6
17 2 2 17 7
18 2 3 18 8
Update
You can use this function:
increase <- function(data,n)
{
data %>% group_by(group) %>%
mutate(pnew=ifelse(is.na(time),time,time+n)) -> result
return(result)
}
increase(df,n = 10)
# A tibble: 18 x 4
# Groups: group [2]
group time p pnew
<dbl> <int> <int> <dbl>
1 1 NA 1 NA
2 1 NA 2 NA
3 1 NA 3 NA
4 1 NA 4 NA
5 1 NA 5 NA
6 1 1 6 11
7 1 2 7 12
8 1 3 8 13
9 1 4 9 14
10 1 5 10 15
11 1 6 11 16
12 2 NA 12 NA
13 2 NA 13 NA
14 2 NA 14 NA
15 2 NA 15 NA
16 2 1 16 11
17 2 2 17 12
18 2 3 18 13
Update 2
I hope this helps:
df %>% group_by(group) %>% rowwise() %>% mutate(pnew=ifelse(is.na(time),NA,pfunc(time)))
# A tibble: 18 x 4
# Rowwise: group
group time p pnew
<dbl> <int> <int> <dbl>
1 1 NA 1 NA
2 1 NA 2 NA
3 1 NA 3 NA
4 1 NA 4 NA
5 1 NA 5 NA
6 1 1 6 6
7 1 2 7 7
8 1 3 8 8
9 1 4 9 9
10 1 5 10 10
11 1 6 11 11
12 2 NA 12 NA
13 2 NA 13 NA
14 2 NA 14 NA
15 2 NA 15 NA
16 2 1 16 6
17 2 2 17 7
18 2 3 18 8