I have the following data table
PIECE SAMPLE QC_CODE
1 1 1
2 1 NA
3 2 2
4 2 4
5 2 NA
6 3 6
7 3 3
8 3 NA
9 4 6
10 4 NA
and I would like to count the number of qc_code in each sample and return an output like this
SAMPLE SAMPLE_SIZE QC_CODE_COUNT
1 2 1
2 3 2
3 3 2
4 2 1
Where sample size is the count of pieces in each sample, and qc_code_count is the count of al qc_code that are no NA.
How would I go about this in R
You can try
library(dplyr)
df1 %>%
group_by(SAMPLE) %>%
summarise(SAMPLE_SIZE=n(), QC_CODE_UNIT= sum(!is.na(QC_CODE)))
# SAMPLE SAMPLE_SIZE QC_CODE_UNIT
#1 1 2 1
#2 2 3 2
#3 3 3 2
#4 4 2 1
Or
library(data.table)
setDT(df1)[,list(SAMPLE_SIZE=.N, QC_CODE_UNIT=sum(!is.na(QC_CODE))), by=SAMPLE]
Or using aggregate from base R
do.call(data.frame,aggregate(QC_CODE~SAMPLE, df1, na.action=NULL,
FUN=function(x) c(SAMPLE_SIZE=length(x), QC_CODE_UNIT= sum(!is.na(x)))))
data
df1 <- structure(list(PIECE = 1:10, SAMPLE = c(1L, 1L, 2L, 2L, 2L, 3L,
3L, 3L, 4L, 4L), QC_CODE = c(1L, NA, 2L, 4L, NA, 6L, 3L, NA,
6L, NA)), .Names = c("PIECE", "SAMPLE", "QC_CODE"), class = "data.frame",
row.names = c(NA, -10L))
Related
I´m having a data.frame of the following form:
ID Var1
1 1
1 1
1 3
1 4
1 1
1 0
2 2
2 2
2 6
2 7
2 8
2 0
3 0
3 2
3 1
3 3
3 2
3 4
and I would like to get there:
ID Var1 X
1 1 0
1 1 0
1 3 0
1 4 5
1 1 5
1 0 5
2 2 0
2 2 0
2 6 0
2 7 10
2 8 10
2 0 10
3 0 0
3 2 0
3 1 0
3 3 3
3 2 3
3 4 3
so in words: I´d like to calculate the sum of the variable in a window = 3, and then report the results obtained in the previous window. This should happen with respect to the IDs and thus the first three observations on every ID should be returned with 0, as there is no previous time period that could be reported.
For understanding: In the actual dataset each row corresponds to one week and the window = 7. So X is supposed to give information on the sum of Var1 in the previous week.
I have tried using some rollapply stuff, but always ended in an error and also the window would be a rolling window if I got that right, which is specifically not what I need.
Thanks for your answers!
In rollapply, the width argument can be a list which provides the offsets to use. In this case we want to use the points 3, 2 and 1 back for the first point, 4, 3 and 2 back for the second, 5, 4 and 3 back for the third and then recycle. That is, for a window width of k = 3 we would want the following list of offset vectors:
w <- list(-(3:1), -(4:2), -(5:3))
In general we can write w below in terms of the window width k. ave then invokes rollapply with that width list for each ID.
library(zoo)
k <- 3
w <- lapply(1:k, function(x) seq(to = -x, length = k))
transform(DF, X = ave(Var1, ID, FUN = function(x) rollapply(x, w, sum, fill = 0)))
giving:
ID Var1 X
1 1 1 0
2 1 1 0
3 1 3 0
4 1 4 5
5 1 1 5
6 1 0 5
7 2 2 0
8 2 2 0
9 2 6 0
10 2 7 10
11 2 8 10
12 2 0 10
13 3 0 0
14 3 2 0
15 3 1 0
16 3 3 3
17 3 2 3
18 3 4 3
Note
The input DF in reproducible form is:
DF <- structure(list(ID = c(1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L), Var1 = c(1L, 1L, 3L, 4L, 1L,
0L, 2L, 2L, 6L, 7L, 8L, 0L, 0L, 2L, 1L, 3L, 2L, 4L)),
class = "data.frame", row.names = c(NA, -18L))
We could group by 'ID', create a new grouping column with window size of 3 using gl, then get the summarized output by taking the sum of 'Var1' and placing the 'Var1' in a list, get the lag of 'X' and unnest
library(dplyr) #1.0.0
library(tidyr)
df1 %>%
# // grouping by ID
group_by(ID) %>%
# // create another group added with gl
group_by(grp = as.integer(gl(n(), 3, n())), .add = TRUE) %>%
# // get the sum of Var1, while changing the Var1 in a list
summarise(X = sum(Var1), Var1 = list(Var1)) %>%
# // get the lag of X
mutate(X = lag(X, default = 0)) %>%
# // unnest the list column
unnest(c(Var1)) %>%
select(names(df1), X)
# A tibble: 18 x 3
# Groups: ID [3]
# ID Var1 X
# <int> <int> <dbl>
# 1 1 1 0
# 2 1 1 0
# 3 1 3 0
# 4 1 4 5
# 5 1 1 5
# 6 1 0 5
# 7 2 2 0
# 8 2 2 0
# 9 2 6 0
#10 2 7 10
#11 2 8 10
#12 2 0 10
#13 3 0 0
#14 3 2 0
#15 3 1 0
#16 3 3 3
#17 3 2 3
#18 3 4 3
data
df1 <- structure(list(ID = c(1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L), Var1 = c(1L, 1L, 3L, 4L, 1L,
0L, 2L, 2L, 6L, 7L, 8L, 0L, 0L, 2L, 1L, 3L, 2L, 4L)), class = "data.frame",
row.names = c(NA,
-18L))
I am relatively new to R but slowly finding my way. I encountered a problem, however, and hope someone can help me.
Let's say I two dataframes (lets call them A and B), both containing survey responses. A contains all responses from the first set of people. B contains the responses of the second set of people, plus the people of the first set but with their responses set to NA. An example:
Dataframe A:
Household Individual Answer_A Answer_b
1 2 5 6
1 3 6 6
2 1 2 3
Dataframe B:
Household Individual Answer_A Answer_b
1 1 3 6
1 2 NA NA
1 3 NA NA
2 1 NA NA
2 2 4 7
I want to get one dataframe with all individuals and their responses:
Dataframe C:
Household Individual Answer_A Answer_b
1 1 3 6
1 2 5 6
1 3 6 6
2 1 2 3
2 2 4 7
If I only have two datasets I can use rbind.fill, with rbind.fill(B, A) to get dataframe C, as then the NAs in B are overwritten with answers in A.
But... if I would have to add a third dataset, D, that would consist of NAs for people in A and B, I would not be able to use this solution. What would I be able to do then? I've looked at intersect, outersect, different forms of join, but can't seem to think of a good solution.
Any thoughts?
Maybe you can left_join and then use coalesce
library(dplyr)
left_join(B, A, by = c("Household", "Individual")) %>%
mutate(Answer_A = coalesce(Answer_A.x, Answer_A.y),
Answer_B = coalesce(Answer_b.x, Answer_b.y)) %>%
select(-matches("\\.x|\\.y"))
# Household Individual Answer_A Answer_B
#1 1 1 3 6
#2 1 2 5 6
#3 1 3 6 6
#4 2 1 2 3
#5 2 2 4 7
data
A <- structure(list(Household = c(1L, 1L, 2L), Individual = c(2L,
3L, 1L), Answer_A = c(5L, 6L, 2L), Answer_b = c(6L, 6L, 3L)), class = "data.frame",
row.names = c(NA, -3L))
B <- structure(list(Household = c(1L, 1L, 1L, 2L, 2L), Individual = c(1L,
2L, 3L, 1L, 2L), Answer_A = c(3L, NA, NA, NA, 4L), Answer_b = c(6L,
NA, NA, NA, 7L)), class = "data.frame", row.names = c(NA, -5L))
Within a group, I want to find the difference between that row and the first time that user appeared in the data. For example, I need to create the diff variable below. Users have different number of rows each as in the following data:
df <- structure(list(ID = c(1L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 4L, 4L),
money = c(9L, 12L, 13L, 15L, 5L, 7L, 8L, 5L, 2L, 10L), occurence = c(1L,
2L, 3L, 4L, 1L, 2L, 3L, 1L, 1L, 2L), diff = c(NA, 3L, 4L,
6L, NA, 2L, 3L, NA, NA, 8L)), .Names = c("ID", "money", "occurence",
"diff"), class = "data.frame", row.names = c(NA, -10L))
ID money occurence diff
1 1 9 1 NA
2 1 12 2 3
3 1 13 3 4
4 1 15 4 6
5 2 5 1 NA
6 2 7 2 2
7 2 8 3 3
8 3 5 1 NA
9 4 2 1 NA
10 4 10 2 8
You can use ave(). We just remove the first value per group and replace it with NA, and subtract the first value from the rest of the values.
with(df, ave(money, ID, FUN = function(x) c(NA, x[-1] - x[1])))
# [1] NA 3 4 6 NA 2 3 NA NA 8
A dplyr solution, which uses the first function to get the first value and calculate the difference.
library(dplyr)
df2 <- df %>%
group_by(ID) %>%
mutate(diff = money - first(money)) %>%
mutate(diff = replace(diff, diff == 0, NA)) %>%
ungroup()
df2
# # A tibble: 10 x 4
# ID money occurence diff
# <int> <int> <int> <int>
# 1 1 9 1 NA
# 2 1 12 2 3
# 3 1 13 3 4
# 4 1 15 4 6
# 5 2 5 1 NA
# 6 2 7 2 2
# 7 2 8 3 3
# 8 3 5 1 NA
# 9 4 2 1 NA
# 10 4 10 2 8
Update
Here is a data.table solution provided by Sotos. Notice that no need to replace 0 with NA.
library(data.table)
setDT(df)[, money := money - first(money), by = ID][]
# ID money occurence diff
# 1: 1 0 1 NA
# 2: 1 3 2 3
# 3: 1 4 3 4
# 4: 1 6 4 6
# 5: 2 0 1 NA
# 6: 2 2 2 2
# 7: 2 3 3 3
# 8: 3 0 1 NA
# 9: 4 0 1 NA
# 10: 4 8 2 8
DATA
dput(df)
structure(list(ID = c(1L, 1L, 1L, 1L, 2L, 2L, 2L, 3L, 4L, 4L),
money = c(9L, 12L, 13L, 15L, 5L, 7L, 8L, 5L, 2L, 10L), occurence = c(1L,
2L, 3L, 4L, 1L, 2L, 3L, 1L, 1L, 2L)), .Names = c("ID", "money",
"occurence"), row.names = c(NA, -10L), class = "data.frame")
What is a good way to get the independent frequency counts of multiple columns using dplyr? I want to go from a table of values:
# A tibble: 7 x 4
a b c d
<int> <int> <int> <int>
1 1 2 1 3
2 1 2 1 3
3 2 2 5 3
4 3 2 4 3
5 3 3 2 3
6 5 3 4 3
7 5 4 2 1
to a frequency table like so:
# A tibble: 5 x 5
x a_n b_n c_n d_n
<int> <int> <int> <int> <int>
1 1 2 0 2 1
2 2 1 4 2 0
3 3 2 2 0 6
4 4 0 1 2 0
5 5 2 0 1 0
I'm still trying to get my head around dplyr, but it seems like this is something it could do. If it is easier to do with an add-on library, that is fine too.
For the same data set that you provided in the question this would be another solution (base-R):
myfreq <- sapply(df, function(x) table(factor(x, levels=unique(unlist(df)), ordered=TRUE)))
Output would be:
> myfreq
# a b c d
# 1 2 0 2 1
# 2 1 4 2 0
# 3 2 2 0 6
# 5 2 0 1 0
# 4 0 1 2 0
Using tabulate in base R:
apply(df,2,function(x) tabulate(x)[min(df):max(df)])
# a b c d
#[1,] 2 0 2 1
#[2,] 1 4 2 0
#[3,] 2 2 0 6
#[4,] 0 1 2 NA
#[5,] 2 NA 1 NA
library(dplyr)
library(reshape2)
df %>%
melt() %>%
dcast(value ~ variable, fun.aggregate=length)
# value a b c d
# 1 1 2 0 2 1
# 2 2 1 4 2 0
# 3 3 2 2 0 6
# 4 4 0 1 2 0
# 5 5 2 0 1 0
Data
df <- structure(list(a = c(1L, 1L, 2L, 3L, 3L, 5L, 5L), b = c(2L, 2L,
2L, 2L, 3L, 3L, 4L), c = c(1L, 1L, 5L, 4L, 2L, 4L, 2L), d = c(3L,
3L, 3L, 3L, 3L, 3L, 1L)), .Names = c("a", "b", "c", "d"), class = "data.frame", row.names = c("1",
"2", "3", "4", "5", "6", "7"))
library(tidyverse)
dt <- data.frame(a = c(1L, 1L, 2L, 3L, 3L, 5L, 5L), b = c(2L, 2L, 2L, 2L, 3L, 3L, 4L),
c = c(1L, 1L, 5L, 4L, 2L, 4L, 2L), d = c(3L, 3L, 3L, 3L, 3L, 3L, 1L))
dt2 <- dt %>%
mutate(ID = 1:n()) %>%
gather(Group, x, -ID) %>%
select(-ID) %>%
mutate(Group = paste(Group, "n", sep = "_")) %>%
count(Group, x) %>%
spread(Group, n, fill = 0L)
I am working with time series data and want to calculate the difference between the first and final measurement times, and put these numbers into a new and simpler dataframe. For example, for this dataframe
structure(list(time = c(1L, 2L, 3L, 4L, 1L, 2L, 3L, 4L), indv = c(1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L), value = c(1L, 3L, 5L, 8L, 3L, 4L,
7L, 8L)), .Names = c("time", "indv", "value"), class = "data.frame", row.names = c(NA,
-8L))
or
time indv value
1 1 1
2 1 3
3 1 5
4 1 8
1 2 3
2 2 4
3 2 7
4 2 8
I can use this code
ddply(test, .(indv), transform, value_change = (value[length(value)] - value[1]), time_change = (time[length(time)] - time[1]))
to give
time indv value value_change time_change
1 1 1 7 3
2 1 3 7 3
3 1 5 7 3
4 1 8 7 3
1 2 3 5 3
2 2 4 5 3
3 2 7 5 3
4 2 8 5 3
However, I would like to eliminate the redundant rows and make a new and simpler dataframe like this
indv time_change value_change
1 3 7
2 3 5
Does anyone have any clever way to do this?
Thanks!
Just replace transform with summarize. You can also make your code a little prettier by using head and tail:
ddply(test, .(indv), summarize,
value_change = tail(value, 1) - head(value, 1),
time_change = tail(time, 1) - head(time, 1))
For maximum readability, write a function:
change <- function(x) tail(x, 1) - head(x, 1)
ddply(test, .(indv), summarize, value_change = change(value),
time_change = change(time))