Related
This question already has answers here:
Numbering rows within groups in a data frame
(10 answers)
Closed 1 year ago.
I am trying to create an additional variable (new variable-> flag) that will number the repetition of observation in my variable starting from 0.
dataset <- data.frame(id = c(1,1,1,2,2,4,6,6,6,7,7,7,7,8))
intended results will look like:
id flag
1 0
1 1
1 2
2 0
2 1
4 0
6 0
6 1
6 2
7 0
7 1
7 2
7 3
8 0
Thank You!
You may try
dataset$flag <- unlist(sapply(rle(dataset$id)$length, function(x) seq(1,x)-1))
id flag
1 1 0
2 1 1
3 1 2
4 2 0
5 2 1
6 4 0
7 6 0
8 6 1
9 6 2
10 7 0
11 7 1
12 7 2
13 7 3
14 8 0
data.table:
library(data.table)
setDT(dataset)[, flag := rowid(id) - 1]
dataset
id flag
1: 1 0
2: 1 1
3: 1 2
4: 2 0
5: 2 1
6: 4 0
7: 6 0
8: 6 1
9: 6 2
10: 7 0
11: 7 1
12: 7 2
13: 7 3
14: 8 0
Base R:
dataset$flag = sequence(rle(dataset$id)$lengths) - 1
dataset
id flag
1 1 0
2 1 1
3 1 2
4 2 0
5 2 1
6 4 0
7 6 0
8 6 1
9 6 2
10 7 0
11 7 1
12 7 2
13 7 3
14 8 0
Another base option:
transform(dataset,
flag = Reduce(function(x, y) y * x + y, duplicated(id), accumulate = TRUE))
id flag
1 1 0
2 1 1
3 1 2
4 2 0
5 2 1
6 4 0
7 6 0
8 6 1
9 6 2
10 7 0
11 7 1
12 7 2
13 7 3
14 8 0
dplyr -
library(dplyr)
dataset %>% group_by(id) %>% mutate(flag = row_number() - 1)
# id flag
# <dbl> <dbl>
# 1 1 0
# 2 1 1
# 3 1 2
# 4 2 0
# 5 2 1
# 6 4 0
# 7 6 0
# 8 6 1
# 9 6 2
#10 7 0
#11 7 1
#12 7 2
#13 7 3
#14 8 0
Base R with similar logic
transform(dataset, flag = ave(id, id, FUN = seq_along) - 1)
another way to reach what you expect but writing a little more
x <- dataset %>%
group_by(id) %>%
summarise(nreg=n())
df <- data.frame()
for(i in 1:nrow(x)){
flag <- data.frame(id = rep( x$id[i], x$nreg[i] ),
flag = seq(0, x$nreg [i] -1 )
)
df <- rbind(df, flag)
}
Let's say I have the following table:
> df <- data.frame("1"=c(9,10,11,10,11,9,10,10,9,11), "2"=c(1,1,2,2,1,2,1,2,2,1), "3"=c(3,1,0,0,3,3,3,3,1,0))
> df
X1 X2 X3
1 9 1 3
2 10 1 1
3 11 2 0
4 10 2 0
5 11 1 3
6 9 2 3
7 10 1 3
8 10 2 3
9 9 2 1
10 11 1 0
How do I find the sum of all the 3's in the column X3, given the criteria that the value in column X1 must be 9, and the value in column X2 is 1?
We can use == with & to create a logical vector, get the sum and multiply by 3
with(df, 3 * sum(X3 == 3 & X1 == 9 & X2 == 1))
#[1] 3
Or another option is
3 * sum(do.call(paste0, df) == '913')
I have the following dataframe (df):
A B T Required col (window = 3)
1 1 0 1
2 3 0 3
3 4 0 4
4 2 1 1 4
5 6 0 0 2
6 4 1 1 0
7 7 1 1 1
8 8 1 1 1
9 1 0 0 1
I would like to add the required column, as followed:
Insert in the current row the previous row value of A or B.
If in the last 3 (window) rows most of time the content of A column is equal to T column - choose A, otherwise - B. (There can be more columns - so the content of the column with the most times equal to T will be chosen).
What is the most efficient way to do it for big data table.
I changed the column named T to be named TC to avoid confusion with T as an abbreviation for TRUE
library(tidyverse)
library(data.table)
df[, newcol := {
equal <- A == TC
map(1:.N, ~ if(.x <= 3) NA
else if(sum(equal[.x - 1:3]) > 3/2) A[.x - 1]
else B[.x - 1])
}]
df
# N A B TC newcol
# 1: 1 1 0 1 NA
# 2: 2 3 0 3 NA
# 3: 3 4 0 4 NA
# 4: 4 2 1 1 4
# 5: 5 6 0 0 2
# 6: 6 4 1 1 0
# 7: 7 7 1 1 1
# 8: 8 8 1 1 1
# 9: 9 1 0 0 1
This works too, but it's less clear, and likely less efficient
df[, newcol := shift(A == TC, 1:3) %>%
pmap_lgl(~sum(...) > 3/2) %>%
ifelse(shift(A), shift(B))]
data:
df <- fread("
N A B TC
1 1 0 1
2 3 0 3
3 4 0 4
4 2 1 1
5 6 0 0
6 4 1 1
7 7 1 1
8 8 1 1
9 1 0 0
")
Probably much less efficient than the answer by Ryan, but without additional packages.
A<-c(1,3,4,2,6,4,7,8,1)
B<-c(0,0,0,1,0,1,1,1,0)
TC<-c(1,3,4,1,0,1,1,1,0)
req<-rep(NA,9)
df<-data.frame(A,B,TC,req)
window<-3
for(i in window:(length(req)-1)){
equal <- sum(df$A[(i-window+1):i]==df$TC[(i-window+1):i])
if(equal > window/2){
df$req[i+1]<-df$A[i]
}else{
df$req[i+1]<-df$B[i]
}
}
Suppose I have a df with 3 columns, group1, group2 & variable
set.seed(1)
group1 = c(rep(1,5),rep(2,5),rep(3,5),rep(4,5))
group2 = c("A","B","C","D","B","C","C","B","C","A","B","D")
variable = c(as.integer(rnorm(20,2)**3))
df=data.frame(group1, group2, variable)
I added the column 'min1' which states if the value of b within 'group1' is also present in group1(x-1). Vice Versa with plus1. Below the total data frame:
group1 group2 variable min1 plus1
1 1 A 3 0 0
2 1 B 11 0 1
3 1 C 2 0 1
4 2 D 47 0 1
5 2 B 13 1 1
6 2 C 2 1 1
7 3 C 16 1 0
8 3 B 21 1 1
9 3 C 18 1 0
10 4 A 5 0 0
11 4 B 44 1 0
12 4 D 14 0 0
Now I want to do calculations such as max() and sum() (but also some more exotic ones) on the variables but not just on all values within their own group1 & group2 combination, but including the values of the group before (or after it). The min1 example is shown below.
group1_min1 group2_min1 sum_min1 max_min1
1 2 B 24 13
2 2 C 4 2
3 3 C 36 18
4 3 B 34 21
5 4 B 65 44
Note that for group1_min1(3),group2_min1(C) three values are used: rows 6,7&9 (2,16&18).
I tried using group_by and summarize within dplyr, something like:
group_by(group1, group2) %>%
summarize_each(funs(sum, max))
EDIT:
I found a solution to add the sum to the original df:
sum_min1 = c()
j=0
for (j in 1:(length(df$group1))){
if (df[j,"min1"] == 0){sum_min1 = c(sum_min1,0)} else {
sum_min1 = c(sum_min1,(sum(df[which((df[,"group1"] == df[j,"group1"] | df[,"group1"] == (df[j,"group1"]-1)) & df[,"group2"]==(df[j,"group2"])),"variable"])))
}
}
df = cbind(df,sum_min1)
This delivers the output:
group1 group2 variable min1 plus1 sum_min1
1 1 A 3 0 0 0
2 1 B 11 0 1 0
3 1 C 2 0 1 0
4 2 D 47 0 0 0
5 2 B 13 1 1 24
6 2 C 2 1 1 4
7 3 C 16 1 0 36
8 3 B 21 1 1 34
9 3 C 18 1 0 36
10 4 A 5 0 0 0
11 4 B 44 1 0 65
12 4 D 14 0 0 0
However this seems to be a very crude way and may take long on big data sets, also in reality there are multiple variables and multiple functions. Also it might be a problem because I want to do some user-defined functions which include a for loop for all the values.
Is there a more elegant way to do this?
Sorry for anything I do wrong, I am new to R and StackOverflow and not a native speaker.
# Data
set.seed(1)
group1 = c(rep(1,3),rep(2,3),rep(3,3),rep(4,3))
group2 = c("A","B","C","D","B","C","C","B","C","A","B","D")
variable = c(as.integer(rnorm(12,2)**3))
df=data.frame(group1, group2, variable)
For the first part-
df$min1 <- sapply(seq(nrow(df)), function(x)
{
if(df[x, "group1"] == 1){0} else {
max(df[x, "group2"] %in% df[df$group1 == df[x,"group1"] - 1,"group2"])}
})
df$plus1 <- sapply(seq(nrow(df)), function(x)
{
if(df[x, "group1"] == max(df$group1){0} else {
max(df[x, "group2"] %in% df[df$group1 == df[x,"group1"] + 1,"group2"])}
})
Second part
df$sum_min1 <- sapply(seq(nrow(df)), function(x)
{
if(df[x, "group1"] == 1){0}else{
sum(df[df$group1 == df[x,"group1"] &
df$group2 == df[x,"group2"],"variable"],
df[df$group1 == df[x,"group1"] - 1 &
df$group2 == df[x,"group2"],"variable"])}
})
Suppose I have an R dataframe that looks like this, where end.group signifies the end of a unique group of observations:
x <- data.frame(end.group=c(0,0,1,0,0,1,1,0,0,0,1,1,1,0,1))
I want to return the following, where group.count is a running count of the number of observations in a group, and group is a unique identifier for each group, in number order. Can anyone help me with a piece of R code to do this?
end.group group.count group
0 1 1
0 2 1
1 3 1
0 1 2
0 2 2
1 3 2
1 1 3
0 1 4
0 2 4
0 3 4
1 4 4
1 1 5
1 1 6
0 1 7
1 2 7
You can create group by using cumsum and rev. You need rev because you have the end points of the groups.
x <- data.frame(end.group=c(0,0,1,0,0,1,1,0,0,0,1,1,1,0,1))
# create groups
x$group <- rev(cumsum(rev(x$end.group)))
# re-number groups from smallest to largest
x$group <- abs(x$group-max(x$group)-1)
Now you can use ave to create group.count.
x$group.count <- ave(x$end.group, x$group, FUN=seq_along)
x <- data.frame(end.group=c(0,0,1,0,0,1,1,0,0,0,1,1,1,0,1))
ends <- which(as.logical(x$end.group))
ends2 <- c(ends[1],diff(ends))
transform(x, group.count=unlist(sapply(ends2,seq)), group=rep(seq(length(ends)),times=ends2))
end.group group.count group
1 0 1 1
2 0 2 1
3 1 3 1
4 0 1 2
5 0 2 2
6 1 3 2
7 1 1 3
8 0 1 4
9 0 2 4
10 0 3 4
11 1 4 4
12 1 1 5
13 1 1 6
14 0 1 7
15 1 2 7