This question already has answers here:
Aggregate / summarize multiple variables per group (e.g. sum, mean)
(10 answers)
Closed 4 years ago.
I would wish to find the average per season for each year. Each year is observed 4 times. The seasons are two but are repeated twice as shown below
year=rep(c(1990:1992),each=4)
season=c("W","D","W","D","W","W","D","D","D","W","W","D")
temp=c(28,25,26,21,28,25,20,20,20,35,28,21)
df=data.frame(year,season,temp)
which gives
year season temp
1 1990 W 28
2 1990 D 25
3 1990 W 26
4 1990 D 21
5 1991 W 28
6 1991 W 25
7 1991 D 20
8 1991 D 20
9 1992 D 20
10 1992 W 35
11 1992 W 28
12 1992 D 21
i want to collapse this data to have the average of the two seasons for each year as below
year season avgtemp
1 1990 D 23.0
2 1990 W 27.0
3 1991 D 20.0
4 1991 W 25.1
5 1992 D 20.5
6 1992 W 31.5
How can i obtain this?
Try below:
aggregate(df[, 3], df[, 1:2], mean)
library(tidyvere)
df %>%
group_by(year,season) %>%
summarise(avgtemp=mean(temp))
# A tibble: 6 x 3
# Groups: year [?]
year season avgtemp
<int> <fct> <dbl>
1 1990 D 23
2 1990 W 27
3 1991 D 20
4 1991 W 26.5
5 1992 D 20.5
6 1992 W 31.5
Related
I have a dataframe like
year lwg_pred bon_pred lwg_prey bon_prey
1990 10 5 20 30
1991 1 2 3 4
1992 5 6 7 8
1993 9 10 11 12
1994 13 14 15 16
This is quite difficult to barplot as is, so I was trying to reorganize it so it would be like
year location type number
1990 lwg pred 10
1990 bon pred 5
1990 lwg prey 20
1990 bon prey 30
The end goal is to have a barplot that has x-axis as Location, y-axis with the number, fill with the type and faceted by the year.
I took a look at a previous post,
barplot_df3 <- barplot_df3 |>
pivot_longer(everything(),
names_sep = "_",
names_to = c("location", ".value"))
but I couldn't figure out how to keep year nor made it 'categorize' pred/prey
Got this:
location pred prey
lwg 10 20
...
I also took a look at transpose, but I don't think it will be of much help.
I would like some suggestions on either how to reorganize the data so it can be graphed or perhaps there was a way to graph it originally, and I was unaware. Thank you for your help in advance!
You don't need the magic ".value", you can just use the new column names you want.
library(tidyr)
barplot_df3 %>%
pivot_longer(-year, names_to = c("location", "type"), names_sep = "_")
will give
year location type value
<int> <chr> <chr> <int>
1 1990 lwg pred 10
2 1990 bon pred 5
3 1990 lwg prey 20
4 1990 bon prey 30
5 1991 lwg pred 1
6 1991 bon pred 2
7 1991 lwg prey 3
8 1991 bon prey 4
9 1992 lwg pred 5
10 1992 bon pred 6
11 1992 lwg prey 7
12 1992 bon prey 8
13 1993 lwg pred 9
14 1993 bon pred 10
15 1993 lwg prey 11
16 1993 bon prey 12
17 1994 lwg pred 13
18 1994 bon pred 14
19 1994 lwg prey 15
20 1994 bon prey 16
My dataset is:
CLASS YEAR VALUE
A 1990 4
A 1991 3
A 1992 7
B 1989 5
B 1990 23
B 1991 3
C 1990 7
C 1991 4
C 1992 6
I want to apply the CAGR formula for each class, I was trying with this code:
df <- df %>%
arrange(CLASS, YEAR) %>%
group_by(CLASS) %>%
mutate(cagr = ((VALUE / lag(VALUE, n)) ^ (1 / n)) - 1)
The dataset that I am using is quite huge, the issue is that I get the first n values of the first class as N/A but it does not happen for the other classes. Therefore I think that in this way the formula takes into account the values of the class above for the first n cases which is wrong.
See if this works for your CAGR:
library(dplyr)
library(xts)
df %>% group_by(CLASS) %>% mutate(cagr = (last(VALUE)/first(VALUE))^(1/(n()-1)) - 1)
# A tibble: 9 x 4
# Groups: CLASS [3]
CLASS YEAR VALUE cagr
<chr> <dbl> <dbl> <dbl>
1 A 1990 4 0.323
2 A 1991 3 0.323
3 A 1992 7 0.323
4 B 1989 5 -0.225
5 B 1990 23 -0.225
6 B 1991 3 -0.225
7 C 1990 7 -0.0742
8 C 1991 4 -0.0742
9 C 1992 6 -0.0742
I'm trying to get group "weighted" mean with multiple grouping variables and excluding own group value. This is related to my earlier post Get group mean with multiple grouping variables and excluding own group value, but when I applied it to my actual question (which is getting the weighted mean) I found out that it's much more complicated than getting the simple mean. Here's what I mean by that.
df <- data_frame(
state = rep(c("AL", "CA"), each = 6),
county = rep(letters[1:6], each = 2),
year = rep(c(2011:2012), 6),
value = c(91,46,37,80,33,97,4,19,85,90,56,94),
wt = c(1,4,3,5,1,4,5,1,5,5,4,1)
) %>% arrange(state, year)
For unweighted mean case, the following code (from the accepted answer of my earlier post) should work.
df %>%
group_by(state, year) %>%
mutate(q = (sum(value) - value) / (n()-1))
The desired variable new_val, which is the weighted mean, would be the following. For instance, the first two rows of new_val column are calculated as 37*3/4 + 33*1/4 = 36, 91*1/2 + 33*1/2 = 62.
# A tibble: 12 x 6
state county year value wt new_val
<chr> <chr> <int> <dbl> <dbl> <dbl>
1 AL a 2011 91 1 36
2 AL b 2011 37 3 62
3 AL c 2011 33 1 50.5
4 AL a 2012 46 4 87.6
5 AL b 2012 80 5 71.5
6 AL c 2012 97 4 64.9
7 CA d 2011 4 5 72.1
8 CA e 2011 85 5 27.1
9 CA f 2011 56 4 44.5
10 CA d 2012 19 1 90.7
11 CA e 2012 90 5 56.5
12 CA f 2012 94 1 78.2
I searched for similar posts with weighted mean in mind, but all the available ones were for the simple mean cases. Any comments would be greatly appreciated. Thank you!
We can use map_dbl to exclude current row in the calculation of weighted.mean
library(dplyr)
df %>%
group_by(state, year) %>%
mutate(new_val = purrr::map_dbl(row_number(),
~weighted.mean(value[-.x], wt[-.x])))
# state county year value wt new_val
# <chr> <chr> <int> <dbl> <dbl> <dbl>
# 1 AL a 2011 91 1 36
# 2 AL b 2011 37 3 62
# 3 AL c 2011 33 1 50.5
# 4 AL a 2012 46 4 87.6
# 5 AL b 2012 80 5 71.5
# 6 AL c 2012 97 4 64.9
# 7 CA d 2011 4 5 72.1
# 8 CA e 2011 85 5 27.1
# 9 CA f 2011 56 4 44.5
#10 CA d 2012 19 1 90.7
#11 CA e 2012 90 5 56.5
#12 CA f 2012 94 1 78.2
I have a list of dataframes that I need to be combined into a single one.
year<-1990:2000
v1<-1:11
v2<-20:30
df1<-data.frame(year,v1)
df2<-data.frame(year,v2)
ldf<-list(df1,df2)
I now want to unlist this dataframe and get
> head(df)
year v1 v2
1 1990 1 20
2 1991 2 21
3 1992 3 22
4 1993 4 23
Note that my question is different from the solution provided in a similar question, where the solution to that question was: `df <- ldply(ldf, data.frame)
Because what I am essentially looking for, is a more automatic way of doing this: df<-merge(df1,df2, by="year")
With more number of list elements, a convenient option is reduce with one of the join functions
library(tidyverse)
ldf %>%
reduce(inner_join, by = "year")
# year v1 v2
#1 1990 1 20
#2 1991 2 21
#3 1992 3 22
#4 1993 4 23
#5 1994 5 24
#6 1995 6 25
#7 1996 7 26
#8 1997 8 27
#9 1998 9 28
#10 1999 10 29
#11 2000 11 30
Is there anything wrong with:
df <- merge(ldf[[1]], ldf[[2]], by="year")
Or for a long list:
df1 <- ldf[[1]]
for (x in 2:length(ldf)) {
df1 <- merge(df1, ldf[[x]])
}
# year v1 v2
# 1 1990 1 20
# 2 1991 2 21
# 3 1992 3 22
# 4 1993 4 23
# 5 1994 5 24
# 6 1995 6 25
# 7 1996 7 26
# 8 1997 8 27
# 9 1998 9 28
# 10 1999 10 29
# 11 2000 11 30
I am completely new with R, and I tried googling a representative solution for my problem for some time, but haven't found an adequate answer so far, so I hope that asking for help might solve this one here.
I should merge two different size data sets (other includes annual data: df_f, and other monthly data: df_m). I should merge the smaller df_f to the larger df_m in a way that rows of df_f are merged conditionally with df_m.
Here is a descriptive example of my problem (with some very basic reproducible numbers):
first dataset
a <- c(1990)
b <- c(1980:1981)
c <- c(1994:1995)
aa <- rep("A", 1)
bb <- rep("B", 2)
cc <- rep("C", 2)
df1 <- data.frame(comp=factor(c(aa, bb, cc)))
df2 <- data.frame(year=factor(c(a, b, c)))
other.columns <- rep("other_columns", length(df1))
df_f <- cbind(df1, df2, other.columns ) # first dataset
second dataset
z <- c(10:12)
x <- c(7:12)
xx <- c(1:9)
v <- c(2:9)
w <- rep(1990, length(z))
e <- rep(1980, length(x))
ee <- rep (1981, length(xx))
r <- rep(1995, length(v))
t <- rep("A", length(z))
y <- rep("B", length(x) + length(xx))
u <- rep("C", length(v))
df3 <- data.frame(month=factor(c(z, x, xx, v)))
df4 <- data.frame(year=factor(c(w, e, ee, r)))
df5 <- data.frame(comp=factor(c(t, y, u)))
df_m <- cbind(df5, df4, df3) # second dataset
Output:
> df_m
comp year month
1 A 1990 10
2 A 1990 11
3 A 1990 12
4 B 1980 7
5 B 1980 8
6 B 1980 9
7 B 1980 10
8 B 1980 11
9 B 1980 12
10 B 1981 1
11 B 1981 2
12 B 1981 3
13 B 1981 4
14 B 1981 5
15 B 1981 6
16 B 1981 7
17 B 1981 8
18 B 1981 9
19 C 1995 2
20 C 1995 3
21 C 1995 4
22 C 1995 5
23 C 1995 6
24 C 1995 7
25 C 1995 8
26 C 1995 9
> df_f
comp year other.columns
1 A 1990 other_columns
2 B 1980 other_columns
3 B 1981 other_columns
4 C 1994 other_columns
5 C 1995 other_columns
I want to have the rows from df_f placed to df_m (store the data from df_f to new columns in df_m) according to the conditions comp, year, and month. Comp (company) needs to match always, but matching the year is conditional to month: if month is >6 then year is matched between datasets, if month is <7 then year + 1 (in df_m) is matched with year (in df_f). Note that a certain row in df_f should be placed into several rows in df_m according to the conditions.
The wanted output clarifies the problem and the goal:
Wanted output:
comp year month comp year other.columns
1 A 1990 10 A 1990 other_columns
2 A 1990 11 A 1990 other_columns
3 A 1990 12 A 1990 other_columns
4 B 1980 7 B 1980 other_columns
5 B 1980 8 B 1980 other_columns
6 B 1980 9 B 1980 other_columns
7 B 1980 10 B 1980 other_columns
8 B 1980 11 B 1980 other_columns
9 B 1980 12 B 1980 other_columns
10 B 1981 1 B 1980 other_columns
11 B 1981 2 B 1980 other_columns
12 B 1981 3 B 1980 other_columns
13 B 1981 4 B 1980 other_columns
14 B 1981 5 B 1980 other_columns
15 B 1981 6 B 1980 other_columns
16 B 1981 7 B 1981 other_columns
17 B 1981 8 B 1981 other_columns
18 B 1981 9 B 1981 other_columns
19 C 1995 2 C 1994 other_columns
20 C 1995 3 C 1994 other_columns
21 C 1995 4 C 1994 other_columns
22 C 1995 5 C 1994 other_columns
23 C 1995 6 C 1994 other_columns
24 C 1995 7 C 1995 other_columns
25 C 1995 8 C 1995 other_columns
26 C 1995 9 C 1995 other_columns
Thank you very much in advance! I hope the question is clear enough, it was somewhat difficult to explain it at least.
The basic idea to solve your problem is to add an extra column with the year that should be used for matching. I will use the package dpylr for this and other manipulation steps.
Before the tables can be combined, the numeric columns must be converted to be numeric:
library(dplyr)
df_m <- mutate(df_m, year = as.numeric(as.character(year)),
month = as.numeric(as.character(month)))
df_f <- mutate(df_f, year = as.numeric(as.character(year)))
The reason is that you want to be able to do numerical comparison with the month (month > 6) and subtract one from the year. You cannot do this with a factor.
Then I add the column to be used for matching:
df_m <- mutate(df_m, match_year = ifelse(month >= 7, year, year - 1))
And in the last step, I join the two tables:
df_new <- left_join(df_m, df_f, by = c("comp", "match_year" = "year"))
The argument by determines which columns of the two data frames should be matched. The output agrees with your result:
## comp year month match_year other.columns
## 1 A 1990 10 1990 other_columns
## 2 A 1990 11 1990 other_columns
## 3 A 1990 12 1990 other_columns
## 4 B 1980 7 1980 other_columns
## 5 B 1980 8 1980 other_columns
## 6 B 1980 9 1980 other_columns
## 7 B 1980 10 1980 other_columns
## 8 B 1980 11 1980 other_columns
## 9 B 1980 12 1980 other_columns
## 10 B 1981 1 1980 other_columns
## 11 B 1981 2 1980 other_columns
## 12 B 1981 3 1980 other_columns
## 13 B 1981 4 1980 other_columns
## 14 B 1981 5 1980 other_columns
## 15 B 1981 6 1980 other_columns
## 16 B 1981 7 1981 other_columns
## 17 B 1981 8 1981 other_columns
## 18 B 1981 9 1981 other_columns
## 19 C 1995 2 1994 other_columns
## 20 C 1995 3 1994 other_columns
## 21 C 1995 4 1994 other_columns
## 22 C 1995 5 1994 other_columns
## 23 C 1995 6 1994 other_columns
## 24 C 1995 7 1995 other_columns
## 25 C 1995 8 1995 other_columns
## 26 C 1995 9 1995 other_columns