I have the following individual data and I want to make an unique household identifier. Every individual already has its rank in household so basically rank 1 marks the start of the new household.
e.g.
rank name
1 John
2 Lisa
3 Stu
1 Phil
1 Mike
1 Florence
2 George
3 David
4 Diana
1 Eleanor
The result I am looking for is this:
rank name id
1 John 1
2 Lisa 1
3 Stu 1
1 Phil 2
1 Mike 3
1 Florence 4
2 George 4
3 David 4
4 Diana 4
1 Eleanor 5
There are around 320 000 individuals, so group id should go from 1 to sum(df$rank[rank = 1]) or something similar. Any other type of unique ID also works, it doesn't have to be seq(1,n,1).
df$id <- cumsum(df$rank == 1)
# rank name id
# 1 1 John 1
# 2 2 Lisa 1
# 3 3 Stu 1
# 4 1 Phil 2
# 5 1 Mike 3
# 6 1 Florence 4
# 7 2 George 4
# 8 3 David 4
# 9 4 Diana 4
# 10 1 Eleanor 5
As #Andre Elrico noted, if rank is NA for any rows, the method above would give you NA for id in all subsequent rows, so you could use the option below instead if you know rank may be NA (but not when it should be 1).
df$id <- cumsum(df$rank %in% 1)
Data used:
df <- read.table(text = '
rank name
1 John
2 Lisa
3 Stu
1 Phil
1 Mike
1 Florence
2 George
3 David
4 Diana
1 Eleanor
', header = T)
Related
I have a data.frame with some individual, like this:
Animal Score Weight
John 5 4
John 5 3
John 5 3
Peter 3 2
Peter 3 2
Louis 4 2
Louis 4 4
Louis 4 1
Sammy 3 2
Sammy 3 2
Sammy 3 2
John 1 5
John 1 5
John 1 5
And I would like to select the TOP 40% of the best animals, based on the score, and maintaining all measures of the TOP 40%, like this:
Animal Score Weight
John 5 4
John 5 3
John 5 3
Louis 4 2
Louis 4 4
Louis 4 1
I tried this code:
top40=subset(df, Score > quantile(Score, prob = 1 - 40/100))
but didn't work, I selected based only on the Score value, like this:
Animal Score Weight
John 5 4
John 5 3
John 5 3
Louis 4 2
Assuming each animal has a single score which is repeated, and you want to select animals whose score is in the top 40%, one option is
subset(df, Score > quantile(unique(Score), 1 - 40/100))
# Animal Score Weight
# 1: John 5 4
# 2: John 5 3
# 3: John 5 3
# 4: Louis 4 2
# 5: Louis 4 4
# 6: Louis 4 1
If you don't use unique (as in all the other answers currently), you get what I would think is an unexpected result. Without unique, the animals in the "top 40% by Score" change even when the scores are the same, just because some animals have more rows.
df[c(rep(1, 10), 2:nrow(df)),] %>%
filter(
Score > Score %>% quantile(0.4)
)
# Animal Score Weight
# 1 John 5 4
# 2 John 5 4
# 3 John 5 4
# 4 John 5 4
# 5 John 5 4
# 6 John 5 4
# 7 John 5 4
# 8 John 5 4
# 9 John 5 4
# 10 John 5 4
# 11 John 5 3
# 12 John 5 3
Use this:
animals %>%
filter(
Score > Score %>% quantile(0.4)
)
BTW, animals is your dataframe.
You can use :
n <- nrow(data)
head(data[order(data$Score, decreasing = TRUE) , ] , round(n*0.4) )
Suppose I have two data frames df1 and df2 as follows
Df1
Id Price Profit Month
10 5 2 1
10 5 3 2
10 5 2 3
11 7 3 1
11 7 1 2
12 0 0 1
12 5 1 2
Df2
Id Name
9 Kane
10 Jack
10 Jack
11 Will
12 Matt
13 Lee
14 Han
Now I want to insert a new column in Df1 named Name and get its value from Df2 based on matching Id
So modified Df1 will be
Id Price Profit Month Name
10 5 2 1 Jack
10 5 3 2 Jack
10 5 2 3 Jack
11 7 3 1 Will
11 7 1 2 Will
12 0 0 1 Matt
12 5 1 2 Matt
df1 <- data.frame(Id=c(10L,10L,10L,11L,11L,12L,12L),Price=c(5L,5L,5L,7L,7L,0L,5L),Profit=c(2L,3L,2L,3L,1L,0L,1L),Month=c(1L,2L,3L,1L,2L,1L,2L),stringsAsFactors=F);
df2 <- data.frame(Id=c(9L,10L,10L,11L,12L,13L,14L),Name=c('Kane','Jack','Jack','Will','Matt','Lee','Han'),stringsAsFactors=F);
df1$Name <- df2$Name[match(df1$Id,df2$Id)];
df1;
## Id Price Profit Month Name
## 1 10 5 2 1 Jack
## 2 10 5 3 2 Jack
## 3 10 5 2 3 Jack
## 4 11 7 3 1 Will
## 5 11 7 1 2 Will
## 6 12 0 0 1 Matt
## 7 12 5 1 2 Matt
use left_join in dplyr
library(dplyr)
left_join(df1, df2, "Id")
eg:
> left_join(df1, df2)
Joining by: "Id"
Id Price Profit Month Name
1 10 5 2 1 Jack
2 10 5 3 2 Jack
3 10 5 2 3 Jack
4 11 7 3 1 Will
5 11 7 1 2 Will
6 12 0 0 1 Matt
7 12 5 1 2 Matt
Data wrangling cheatsheet by RStudio is a very helpful resource.
Here is an option using data.table
library(data.table)
setDT(Df1)[unique(Df2), on = "Id", nomatch=0]
# Id Price Profit Month Name
#1: 10 5 2 1 Jack
#2: 10 5 3 2 Jack
#3: 10 5 2 3 Jack
#4: 11 7 3 1 Will
#5: 11 7 1 2 Will
#6: 12 0 0 1 Matt
#7: 12 5 1 2 Matt
Or as #Arun mentioned in the comments, we can assign (:=) the "Name" column after joining on "Id" to reflect the changes in the original dataset "Df1".
setDT(Df1)[Df2, Name:= Name, on = "Id"]
Df1
A simple base R option could be merge()
merge(Df1,unique(Df2), by="Id")
# Id Price Profit Month Name
#1 10 5 2 1 Jack
#2 10 5 3 2 Jack
#3 10 5 2 3 Jack
#4 11 7 3 1 Will
#5 11 7 1 2 Will
#6 12 0 0 1 Matt
#7 12 5 1 2 Matt
The function unique() is used here because of the duplicate entry in Df2 concerning "Jack". For the example data described in the OP the option by="Id" can be omitted, but it might be necessary in a more general case.
Here is a reproducible example of my data. For the following data frame:
df <- data.frame(Subject = c('John', 'John', 'John', 'John','Mary', 'Mary', 'Mary', 'Mary'),
SNR = c(-4,-4,0,4,0,4,4,8))
I would like to add a column 'rank' that provides a ranking for SNR by Subject, so that it would look like this:
Subject SNR Rank
John -4 1
John -4 1
John 0 2
John 4 3
Mary 0 1
Mary 4 2
Mary 4 2
Mary 8 3
I have tried using:
dfNew <- transform(df, Rank = ave(SNR, Subject, FUN = function(x) rank(x, ties.method = "first")))
But I get the following:
Subject SNR Rank
John -4 1
John -4 2
John 0 3
John 4 4
Mary 0 1
Mary 4 2
Mary 4 3
Mary 8 4
I have also tried using the different ties.method options, but none give me what I am looking for (i.e., ranking only from 1-3).
Any help would be much appreciated!
Using aggregate and factor in base R:
ag <- aggregate(SNR~Subject, df, function(x) as.numeric(factor(x)))
df$rank <- c(t(ag[,-1]))
Subject SNR rank
1 John -4 1
2 John -4 1
3 John 0 2
4 John 4 3
5 Mary 0 1
6 Mary 4 2
7 Mary 4 2
8 Mary 8 3
Another base R method:
transform(df1, Rank = ave(SNR, Subject, FUN = function(x) cumsum(c(TRUE, head(x, -1) != tail(x, -1)))))
gives:
Subject SNR Rank
1 John -4 1
2 John -4 1
3 John 0 2
4 John 4 3
5 Mary 0 1
6 Mary 4 2
7 Mary 4 2
8 Mary 8 3
If your dataframe is not ordered yet, you should order it first with df1 <- df1[order(df1$SNR),] for this method to give the correct result.
library(dplyr)
df %>%
arrange(Subject, SNR) %>%
group_by(Subject) %>%
mutate(rank=dense_rank(SNR))
of course credit to #rich-scriven for mentioning dense_rank()
A bit dirty but it seems to work:
library(dplyr)
df %>% group_by(Subject) %>% mutate(Rank = as.numeric(as.factor(SNR)))
Subject SNR Rank
<fctr> <dbl> <dbl>
1 John -4 1
2 John -4 1
3 John 0 2
4 John 4 3
5 Mary 0 1
6 Mary 4 2
7 Mary 4 2
8 Mary 8 3
OK, check out this data frame...
customer_name order_dates order_values
1 John 2010-11-01 15
2 Bob 2008-03-25 12
3 Alex 2009-11-15 5
4 John 2012-08-06 15
5 John 2015-05-07 20
Lets say I want to add an order variable that Ranks the highest order value, by name, by max order date, using the last order date at the tie breaker. So, ultimately the data should look like this:
customer_name order_dates order_values ranked_order_values_by_max_value_date
1 John 2010-11-01 15 3
2 Bob 2008-03-25 12 1
3 Alex 2009-11-15 5 1
4 John 2012-08-06 15 2
5 John 2015-05-07 20 1
Where everyone's single order gets 1, and all subsequent orders are ranked based on the value, and the tie breaker is the last order date getting priority.
In this example, John's 8/6/2012 order gets the #2 rank because it was placed after 11/1/2010. The 5/7/2015 order is 1 because it was the biggest. So, even if that order was placed 20 years ago, it should be the #1 Rank because it was John's highest order value.
Does anyone know how I can do this in R? Where I can Rank within a group of specified variables in a data frame?
Thanks for your help!
The top rated answer (by cdeterman) is actually incorrect. The order function provides the location of the 1st, 2nd, 3rd, etc ranked values not the ranks of the values in their current order.
Let’s take a simple example where we want to rank, starting with the largest, grouping by customer name. I have included a manual ranking so we can check the values
> df
customer_name order_values manual_rank
1 John 2 5
2 John 5 2
3 John 9 1
4 John 1 6
5 John 4 3
6 John 3 4
7 Lucy 4 4
8 Lucy 9 1
9 Lucy 6 3
10 Lucy 2 6
11 Lucy 8 2
12 Lucy 3 5
If I run the code suggested by cdeterman I get the following incorrect ranks:
> df %>%
+ group_by(customer_name) %>%
+ mutate(my_ranks = order(order_values, decreasing=TRUE))
Source: local data frame [12 x 4]
Groups: customer_name [2]
customer_name order_values manual_rank my_ranks
<fctr> <dbl> <dbl> <int>
1 John 2 5 3
2 John 5 2 2
3 John 9 1 5
4 John 1 6 6
5 John 4 3 1
6 John 3 4 4
7 Lucy 4 4 2
8 Lucy 9 1 5
9 Lucy 6 3 3
10 Lucy 2 6 1
11 Lucy 8 2 6
12 Lucy 3 5 4
Order is used to re-order dataframes into decreasing or increasing order. What we actually want is to run the order function twice, with the second order function giving us the actual ranks we want.
> df %>%
+ group_by(customer_name) %>%
+ mutate(good_ranks = order(order(order_values, decreasing=TRUE)))
Source: local data frame [12 x 4]
Groups: customer_name [2]
customer_name order_values manual_rank good_ranks
<fctr> <dbl> <dbl> <int>
1 John 2 5 5
2 John 5 2 2
3 John 9 1 1
4 John 1 6 6
5 John 4 3 3
6 John 3 4 4
7 Lucy 4 4 4
8 Lucy 9 1 1
9 Lucy 6 3 3
10 Lucy 2 6 6
11 Lucy 8 2 2
12 Lucy 3 5 5
You can do this pretty cleanly with dplyr
library(dplyr)
df %>%
group_by(customer_name) %>%
mutate(my_ranks = order(order(order_values, order_dates, decreasing=TRUE)))
Source: local data frame [5 x 4]
Groups: customer_name
customer_name order_dates order_values my_ranks
1 John 2010-11-01 15 3
2 Bob 2008-03-25 12 1
3 Alex 2009-11-15 5 1
4 John 2012-08-06 15 2
5 John 2015-05-07 20 1
This can be achieved with ave and rank. ave passes the proper groups to rank. The result from rank is reversed due to the requested order:
with(x, ave(as.numeric(order_dates), customer_name, FUN=function(x) rev(rank(x))))
## [1] 3 1 1 2 1
In base R you can do this with the slightly unwieldy
transform(df,rank=ave(1:nrow(df),customer_name,
FUN=function(x) order(order_values[x],order_dates[x],decreasing=TRUE)))
customer_name order_dates order_values rank
1 John 2010-11-01 15 3
2 Bob 2008-03-25 12 1
3 Alex 2009-11-15 5 1
4 John 2012-08-06 15 2
5 John 2015-05-07 20 1
where order is provided both the primary and tie-breaker values for each group.
df %>%
group_by(customer_name) %>%
arrange(customer_name,desc(order_values)) %>%
mutate(rank2=rank(order_values))
Similar to #t-himmel's answer, you can get the ranks with data.table.
dt[ , rnk := order(order(order_values, decreasing = TRUE)), customer_name ]
I have a df as follows
names numbers
1 john -3
2 john -2
3 john -1
4 john 1
5 john 2
6 mary -2
7 mary -1
8 mary 1
9 mary 2
10 mary 3
11 tom -1
12 tom 1
13 tom 2
14 tom 3
I want to limit the df to people who have a value that begins with -3. Then I want to do the same for -2 and then the same again for people who start off with a value of -1. My end result would be three dfs, one each for john, mary and tom given that they all have different starting values (-3,-2 and -1).
e.g., for mary
names numbers
6 mary -2
7 mary -1
8 mary 1
9 mary 2
10 mary 3
my real dataframe has about 10,000 people in it so I can't just filter out by name as i'm doing here. I'd like a way of doing it by number, something like
df1<-df[df$number>=-3,] ##too simplistic
but this pulls in all the rows for everyone in the dataframe (logical considering they all have values > -3). I want the code to limit the resultant df to just the person who had a starting value of -3 and then all their values underneath that as shown for mary above.
Thank you in advance!
I would use ave to calculate the first number for each group, then split on it.
df$first <- ave(df$numbers, df$names, FUN=function(x) x[1])
split(df, f = df$first)
yields:
$`-3`
i names numbers first
1 1 john -3 -3
2 2 john -2 -3
3 3 john -1 -3
4 4 john 1 -3
5 5 john 2 -3
$`-2`
i names numbers first
6 6 mary -2 -2
7 7 mary -1 -2
8 8 mary 1 -2
9 9 mary 2 -2
10 10 mary 3 -2
$`-1`
i names numbers first
11 11 tom -1 -1
12 12 tom 1 -1
13 13 tom 2 -1
14 14 tom 3 -1