I'm trying to create SMA formula for multiple items in the same column. Here's an example of the data I'm working with.
Person Time Value
<chr> <dbl> <dbl>
1 A 1 14
2 A 2 13
3 A 3 17
4 A 4 9
5 A 5 20
6 A 6 5
7 B 1 17
8 B 2 11
9 B 3 18
10 B 4 10
11 B 5 10
12 B 6 20
13 C 1 5
14 C 2 5
15 C 3 11
16 C 4 12
17 C 5 12
18 C 6 9
What I'd like to be able to do is to create another column with the SMA formula for each person (A,B,C, etc.). In this case let's say SMA2. While it works for Person A, I can't get the formula to restart at Person B. Rather Person B's first SMA2 value has Person A's values with it.
Right now I've used this which does give me the SMA I want, just not restarted at each new person:
DataSet$SMA2<-SMA(DataSet$Value, 2)
Any help would be appreciated.
DataSet <- DataSet %>%
group_by(Person) %>%
mutate(sma2 = TTR::SMA(Value,2))
Still came up with this:
# A tibble: 18 x 4
# Groups: Person [3]
Person Time Value sma2
<chr> <dbl> <dbl> <dbl>
1 A 1 14 NA
2 A 2 13 13.5
3 A 3 17 15
4 A 4 9 13
5 A 5 20 14.5
6 A 6 5 12.5
7 B 1 17 11
8 B 2 11 14
9 B 3 18 14.5
10 B 4 10 14
11 B 5 10 10
12 B 6 20 15
13 C 1 5 12.5
14 C 2 5 5
15 C 3 11 8
16 C 4 12 11.5
17 C 5 12 12
18 C 6 9 10.5
Using dplyr, group_by person then use mutate. This will reset the calculations per person.
DataSet <- DataSet %>%
group_by(Person) %>%
mutate(sma2 = TTR::SMA(Value, 2))
# A tibble: 18 x 4
# Groups: Person [3]
Person Time Value sma2
<chr> <int> <int> <dbl>
1 A 1 14 NA
2 A 2 13 13.5
3 A 3 17 15
4 A 4 9 13
5 A 5 20 14.5
6 A 6 5 12.5
7 B 1 17 NA
8 B 2 11 14
9 B 3 18 14.5
10 B 4 10 14
11 B 5 10 10
12 B 6 20 15
13 C 1 5 NA
14 C 2 5 5
15 C 3 11 8
16 C 4 12 11.5
17 C 5 12 12
18 C 6 9 10.5
Related
I have a data frame that looks like this
> df
name time count
1 A 10 9
2 A 12 17
3 A 24 19
4 A 3 15
5 A 29 11
6 B 31 14
7 B 7 7
8 B 30 18
9 C 29 13
10 C 12 12
11 C 3 16
12 C 4 6
and for each name group (A, B, C) I would need to assign a category following the rules below:
if time<= 10 then category = 1
if 10 <time<= 20 then category = 2
if 20 <time<= 30 then category = 3
if time> 30 then category = 4
to have a data frame that looks like this:
> df_final
name time count category
1 A 10 9 1
2 A 12 17 2
3 A 24 19 3
4 A 3 15 1
5 A 29 11 3
6 B 31 14 4
7 B 7 7 1
8 B 30 18 3
9 C 29 13 3
10 C 12 12 2
11 C 3 16 1
12 C 4 6 1
after that I would need to sum the value in count based on their category. The ultimate data frame should loo like this:
> df_ultimate
name count category
1 A 24 1
2 A 17 2
3 A 30 3
4 A NA 4
5 B 7 1
6 B NA 2
7 B 18 3
8 B 14 4
9 C 22 1
10 C 12 2
11 C 13 3
12 C NA 4
I have tried to play around with summarise and group_by but without much success.
Thanks for your help
With cut + complete:
library(dplyr)
library(tidyr)
dat %>%
group_by(name, category = cut(time, breaks = c(-Inf, 10, 20, 30, Inf), labels = 1:4)) %>%
summarise(count = sum(count)) %>%
complete(category)
# # Groups: name [3]
# name category count
# 1 A 1 24
# 2 A 2 17
# 3 A 3 30
# 4 A 4 NA
# 5 B 1 7
# 6 B 2 NA
# 7 B 3 18
# 8 B 4 14
# 9 C 1 22
# 10 C 2 12
# 11 C 3 13
# 12 C 4 NA
I used this code to generate these random numbers (corresponds to an edge list for a graph) such that (Generating Random Graphs According to Some Conditions):
The first and last "nodes" are the same (e.g. starts at "1" and ends at "1")
Each node is visited exactly once
See below:
d = 15
relations = data.frame(tibble(
from = sample(data$d),
to = lead(from, default=from[1]),
))
> relations
from to
1 1 11
2 11 7
3 7 5
4 5 10
5 10 13
6 13 9
7 9 15
8 15 2
9 2 3
10 3 4
11 4 8
12 8 6
13 6 12
14 12 14
15 14 1
If I re-run this above code, it will (naturally) produce a different list:
relations
from to
1 6 9
2 9 2
3 2 5
4 5 8
5 8 13
6 13 1
7 1 14
8 14 3
9 3 11
10 11 12
11 12 7
12 7 15
13 15 4
14 4 10
15 10 6
Can I do something so that each time I generate a new random set of numbers, I can fix the first and last number to a specific number?
For instance, could I make it so that the first number and the last number are always "7"?
#example 1
from to
1 7 11
2 11 1
3 1 5
4 5 10
5 10 13
6 13 9
7 9 15
8 15 2
9 2 3
10 3 4
11 4 8
12 8 6
13 6 12
14 12 14
15 14 7
#example 2
from to
1 7 9
2 9 2
3 2 5
4 5 8
5 8 13
6 13 1
7 1 14
8 14 3
9 3 11
10 11 12
11 12 6
12 6 15
13 15 4
14 4 10
15 10 7
In the above examples (example 1, example 2), I took the first two random lists I made and manually replaced the first number and last number with 7 - and then replaced the replacement numbers as well.
But is there a way to "automatically" do this instead of making a manual correction?
For example, I think I figured out how to do this:
#run twice to make sure the output is correct
relations = data.frame(tibble(
from = sample(data$d),
to = lead(from, default=from[1]),
))
orig_first = relations[1,1]
relations[1,1] = 7
relations[15,2] = 7
relation = relations[-c(1,15),]
r1 = relations[1,]
r2 = relations[15,]
final_relation = rbind(r1, relation, r2)
#output 1 : seems correct (starts with 7, ends with 7, all nodes visited exactly once)
from to
1 7 8
2 8 4
3 4 7
4 7 13
5 13 1
6 1 14
7 14 6
8 6 9
9 9 11
10 11 10
11 10 12
12 12 2
13 2 5
14 5 15
15 15 7
#output 2: looks correct
from to
1 7 9
2 9 2
3 2 1
4 1 6
5 6 3
6 3 10
7 10 11
8 11 14
9 14 12
10 12 7
11 7 13
12 13 4
13 4 15
14 15 8
15 8 7
Am I doing this correctly? Is there an easier way to do this?
Thank you!
Here is a way to do this -
library(dplyr)
set.seed(2021)
d = 15
fix_num <- 7
relations = tibble(
from = c(fix_num, sample(setdiff(1:d, fix_num))),
to = lead(from, default=from[1]),
)
relations
# A tibble: 15 x 2
# from to
# <dbl> <dbl>
# 1 7 8
# 2 8 6
# 3 6 11
# 4 11 15
# 5 15 4
# 6 4 14
# 7 14 9
# 8 9 10
# 9 10 3
#10 3 5
#11 5 12
#12 12 13
#13 13 1
#14 1 2
#15 2 7
A dataframe DD has some missing rows. Based on the values in 'ID_raw' column I have duplicated the rows in order to replace the missing rows. Now I have to number the rows in such way that the first value in each group (column 'File') equals the value in the same row in the column 'ID_raw'. This will be a key in joining the dataframe with another one. Below a dummy example of the DD dataframe:
DD<-data.frame(ID_raw=c(1,5,7,8,5,7,9,13,3,6),Val=c(1,2,8,15,54,23,88,77,32,2),File=c("A","A","A","A","B","B","B","B","C","C"))
ID_raw Val File
1 1 1 A
2 5 2 A
3 7 8 A
4 8 15 A
5 5 54 B
6 7 23 B
7 9 88 B
8 13 77 B
9 3 32 C
10 6 2 C
So far I've successfully duplicated the rows, however, I have a problem in numbering the rows in such way, that they start from the same value as the value in ID_raw column for each group ('File').
DD$ID_diff <- 0
DD$ID_diff[1:nrow(DD)-1] <- as.integer(diff(DD$ID_raw, 1)) #values which tell how many times a row has to be duplicated
DD$ID_diff <- sapply(DD$ID_diff, function(x) ifelse(x<0, 0, x)) #replacement the values <0 (for the first rows in each 'File' group)
DD <- DD[rep(seq(nrow(DD)), DD$ID_diff), 1:ncol(DD)] #rows duplication
Based on the code above I receive this output:
ID_raw Val File ID_diff
1 1 1 A 4
1.1 1 1 A 4
1.2 1 1 A 4
1.3 1 1 A 4
2 5 2 A 2
2.1 5 2 A 2
3 7 8 A 1
5 5 54 B 2
5.1 5 54 B 2
6 7 23 B 2
6.1 7 23 B 2
7 9 88 B 4
7.1 9 88 B 4
7.2 9 88 B 4
7.3 9 88 B 4
9 3 32 C 3
9.1 3 32 C 3
9.2 3 32 C 3
I would like to receive this:
ID_raw Val File ID_diff ID_new
1 1 1 A 4 1
1.1 1 1 A 4 2
1.2 1 1 A 4 3
1.3 1 1 A 4 4
2 5 2 A 2 5
2.1 5 2 A 2 6
3 7 8 A 1 7
5 5 54 B 2 5
5.1 5 54 B 2 6
6 7 23 B 2 7
6.1 7 23 B 2 8
7 9 88 B 4 9
7.1 9 88 B 4 10
7.2 9 88 B 4 11
7.3 9 88 B 4 12
9 3 32 C 3 3
9.1 3 32 C 3 4
9.2 3 32 C 3 5
This is one option using dplyr based on the output of your code:
df %>%
group_by(File) %>%
mutate(ID_new = seq(1, n()) + first(ID_raw) - 1)
# A tibble: 18 x 5
# Groups: File [3]
ID_raw Val File ID_diff ID_new
<int> <int> <fct> <int> <dbl>
1 1 1 A 4 1
2 1 1 A 4 2
3 1 1 A 4 3
4 1 1 A 4 4
5 5 2 A 2 5
6 5 2 A 2 6
7 7 8 A 1 7
8 5 54 B 2 5
9 5 54 B 2 6
10 7 23 B 2 7
11 7 23 B 2 8
12 9 88 B 4 9
13 9 88 B 4 10
14 9 88 B 4 11
15 9 88 B 4 12
16 3 32 C 3 3
17 3 32 C 3 4
18 3 32 C 3 5
We can do this in the chain from the beginning itself i.e. instead of creating the 'ID_diff' and using sapply, directly use diff on the 'ID_raw', then uncount, grouped by 'File', create the sequence column
library(tidyverse)
DD %>%
mutate(ID_diff = pmax(c(diff(ID_raw), 0), 0)) %>%
uncount(ID_diff, .remove = FALSE) %>%
group_by(File) %>%
mutate(ID_new = seq(first(ID_raw), length.out = n(), by = 1))
# A tibble: 18 x 5
# Groups: File [3]
# ID_raw Val File ID_diff ID_new
# <dbl> <dbl> <fct> <dbl> <dbl>
# 1 1 1 A 4 1
# 2 1 1 A 4 2
# 3 1 1 A 4 3
# 4 1 1 A 4 4
# 5 5 2 A 2 5
# 6 5 2 A 2 6
# 7 7 8 A 1 7
# 8 5 54 B 2 5
# 9 5 54 B 2 6
#10 7 23 B 2 7
#11 7 23 B 2 8
#12 9 88 B 4 9
#13 9 88 B 4 10
#14 9 88 B 4 11
#15 9 88 B 4 12
#16 3 32 C 3 3
#17 3 32 C 3 4
#18 3 32 C 3 5
I have some data where each id is measured by different types which can be have different values type_val. The measured value is val. A small dummy data is like this:
df <- data.frame(id=rep(letters[1:2],6),
type=c(rep('t1',6), rep('t2',6)),
type_val=rep(c(1,1,2,2,3,3),2),
val=1:12)
Then df is:
id type type_val val
1 a t1 1 1
2 b t1 1 2
3 a t1 2 3
4 b t1 2 4
5 a t1 3 5
6 b t1 3 6
7 a t2 1 7
8 b t2 1 8
9 a t2 2 9
10 b t2 2 10
11 a t2 3 11
12 b t2 3 12
I need to spread/cast data so that all combinations of type and type_val for each id are row-wise. I think this must be a job for pkgs reshape2 or tidyr but I have completely failed to generate anything other than errors.
The outcome data structure - somewhat redundant - would be something like this (hope I got it right!) where pairs of type (as given by combinations of the type_val) are columns type_t1 and type_t2 , and their associated values (val in df) are val_t1 and val_t2 - columns names are of cause arbitrary :
id type_t1 type_t2 val_t1 val_t2
1 a 1 1 1 7
2 a 1 2 1 9
3 a 1 3 1 11
4 a 2 1 3 7
5 a 2 2 3 9
6 a 2 3 3 11
7 a 3 1 5 7
8 a 3 2 5 9
9 a 3 3 5 11
10 b 1 1 2 8
11 b 1 2 2 10
12 b 1 3 2 12
13 b 2 1 4 8
14 b 2 2 4 10
15 b 2 3 4 12
16 b 3 1 6 8
17 b 3 2 6 10
18 b 3 3 6 12
UPDATE
Note that (#Sotos)
> spread(df, type, val)
id type_val t1 t2
1 a 1 1 7
2 a 2 3 9
3 a 3 5 11
4 b 1 2 8
5 b 2 4 10
6 b 3 6 12
is not the desired output - it fails to deliver the wide format defined by combinations of type and type_val in df.
how about this:
df1=df[df$type=="t1",]
df2=df[df$type=="t2",]
DF=merge(df1,df2,by="id")
DF=DF[,-c(2,5)]
colnames(DF)<-c("id", "type_t1", "val_t1","type_t2", "val_t2")
Here is something more generic that will work with an arbitrary number of unique type:
library(dplyr)
# This function takes a list of dataframes (.data) and merges them by ID
reduce_merge <- function(.data, ID) {
return(Reduce(function(x, y) merge(x, y, by = ID), .data))
}
# This function renames the cols columns in .data by appending _identifier
batch_rename <- function(.data, cols, identifier, sep = '_') {
return(plyr::rename(.data, sapply(cols, function(x){
x = paste(x, .data[1, identifier], sep = sep)
})))
}
# This function creates a list of subsetted dataframes
# (subsetted by values of key),
# uses batch_rename() to give each dataframe more informative column names,
# merges them together, and returns the columns you'd like in a sensible order
multi_spread <- function(.data, grp, key, vals) {
.data %>%
plyr::dlply(key, subset) %>%
lapply(batch_rename, vals, key) %>%
reduce_merge(grp) %>%
select(-starts_with(paste0(key, '.'))) %>%
select(id, sort(setdiff(colnames(.), c(grp, key, vals))))
}
# Your example
df <- data.frame(id=rep(letters[1:2],6),
type=c(rep('t1',6), rep('t2',6)),
type_val=rep(c(1,1,2,2,3,3),2),
val=1:12)
df %>% multi_spread('id', 'type', c('type_val', 'val'))
id type_val_t1 type_val_t2 val_t1 val_t2
1 a 1 1 1 7
2 a 1 2 1 9
3 a 1 3 1 11
4 a 2 1 3 7
5 a 2 2 3 9
6 a 2 3 3 11
7 a 3 1 5 7
8 a 3 2 5 9
9 a 3 3 5 11
10 b 1 1 2 8
11 b 1 2 2 10
12 b 1 3 2 12
13 b 2 1 4 8
14 b 2 2 4 10
15 b 2 3 4 12
16 b 3 1 6 8
17 b 3 2 6 10
18 b 3 3 6 12
# An example with three unique values of 'type'
df <- data.frame(id = rep(letters[1:2], 9),
type = c(rep('t1', 6), rep('t2', 6), rep('t3', 6)),
type_val = rep(c(1, 1, 2, 2, 3, 3), 3),
val = 1:18)
df %>% multi_spread('id', 'type', c('type_val', 'val'))
id type_val_t1 type_val_t2 type_val_t3 val_t1 val_t2 val_t3
1 a 1 1 1 1 7 13
2 a 1 1 2 1 7 15
3 a 1 1 3 1 7 17
4 a 1 2 1 1 9 13
5 a 1 2 2 1 9 15
6 a 1 2 3 1 9 17
7 a 1 3 1 1 11 13
8 a 1 3 2 1 11 15
9 a 1 3 3 1 11 17
10 a 2 1 1 3 7 13
11 a 2 1 2 3 7 15
12 a 2 1 3 3 7 17
13 a 2 2 1 3 9 13
14 a 2 2 2 3 9 15
15 a 2 2 3 3 9 17
16 a 2 3 1 3 11 13
17 a 2 3 2 3 11 15
18 a 2 3 3 3 11 17
19 a 3 1 1 5 7 13
20 a 3 1 2 5 7 15
21 a 3 1 3 5 7 17
22 a 3 2 1 5 9 13
23 a 3 2 2 5 9 15
24 a 3 2 3 5 9 17
25 a 3 3 1 5 11 13
26 a 3 3 2 5 11 15
27 a 3 3 3 5 11 17
28 b 1 1 1 2 8 14
29 b 1 1 2 2 8 16
30 b 1 1 3 2 8 18
31 b 1 2 1 2 10 14
32 b 1 2 2 2 10 16
33 b 1 2 3 2 10 18
34 b 1 3 1 2 12 14
35 b 1 3 2 2 12 16
36 b 1 3 3 2 12 18
37 b 2 1 1 4 8 14
38 b 2 1 2 4 8 16
39 b 2 1 3 4 8 18
40 b 2 2 1 4 10 14
41 b 2 2 2 4 10 16
42 b 2 2 3 4 10 18
43 b 2 3 1 4 12 14
44 b 2 3 2 4 12 16
45 b 2 3 3 4 12 18
46 b 3 1 1 6 8 14
47 b 3 1 2 6 8 16
48 b 3 1 3 6 8 18
49 b 3 2 1 6 10 14
50 b 3 2 2 6 10 16
51 b 3 2 3 6 10 18
52 b 3 3 1 6 12 14
53 b 3 3 2 6 12 16
54 b 3 3 3 6 12 18
I have to sort a datapool with following structure into subgroups based on the value of 3 columns in R, but I cannot figure it out.
What I want to do is:
First, sort the datapool based on the column V1, the datapool should be divided into three subgroups according to the value of V1 (the value of V1 should be sorted by descending at first).
Sort each of the 3 subgroups into another 3 subgroups according to the value of V2, now we should have 9 subgroups.
Similarly, subdivide each of the 9 groups into 3 groups again,and resulting in 27 subgroups all together.
the following data is only a simple example, the data have 1545 firms.
Firm value V1 V2 V3
1 7 7 11 8
2 9 9 11 7
3 8 14 8 10
4 9 9 7 14
5 8 11 15 14
6 9 10 9 7
7 8 8 6 14
8 4 8 11 14
9 8 10 13 10
10 2 11 6 13
11 3 5 12 14
12 5 12 15 12
13 1 9 13 7
14 4 5 14 7
15 5 10 5 9
16 5 8 13 14
17 2 10 10 7
18 5 12 12 9
19 7 6 11 7
20 6 9 14 14
21 6 14 9 14
22 8 6 6 7
23 9 11 9 5
24 7 7 6 9
25 10 5 15 11
26 4 6 10 9
27 4 13 14 8
And the result should be:
Firm value V1 V2 V3
5 8 11 15 14
12 5 12 15 12
27 4 13 14 8
21 6 14 9 14
18 5 12 12 9
23 9 11 9 5
10 2 11 6 13
3 8 14 8 10
6 9 10 9 7
20 6 9 14 14
9 8 10 13 10
13 1 9 13 7
8 4 8 11 14
2 9 9 11 7
17 2 10 10 7
4 9 9 7 14
7 8 8 6 14
15 5 10 5 9
16 5 8 13 14
25 10 5 15 11
14 4 5 14 7
11 3 5 12 14
1 7 7 11 8
19 7 6 11 7
26 4 6 10 9
24 7 7 6 9
22 8 6 6 7
I have tried for a long time, also searched Google without success. :(
As #Codoremifa said, data.table can be used here:
require(data.table)
DT <- data.table(dat)
DT[order(V1),G1:=rep(1:3,each=9)]
DT[order(V2),G2:=rep(1:3,each=3),by=G1]
DT[order(V3),G3:=1:3,by='G1,G2']
Now your groups are labeled using the additional columns G1 and G2. To sort, so that it's easier to see the groups, use
setkey(DT,G1,G2,G3)
A couple of the OP's columns are just noise unrelated to the question; to verify that this works by eye, try DT[,list(V1,V2,V3,G1,G2,G3)]
EDIT: The OP did not specify a means of dealing with ties. I guess it makes sense to use the value in the later columns to break ties, so...
DT <- data.table(dat)
DT[order(rank(V1)+rank(V2)/100+rank(V3)/100^2),
G1:=rep(1:3,each=9)]
DT[order(rank(V2)+rank(V3)/100),
G2:=rep(1:3,each=3),by=G1]
DT[order(V3),
G3:=1:3,by='G1,G2']
setkey(DT,G1,G2,G3)
DT[27:1] (the result backwards) is
Firm value V1 V2 V3 G1 G2 G3
1: 5 8 11 15 14 3 3 3
2: 12 5 12 15 12 3 3 2
3: 27 4 13 14 8 3 3 1
4: 21 6 14 9 14 3 2 3
5: 9 8 10 13 10 3 2 2
6: 18 5 12 12 9 3 2 1
7: 10 2 11 6 13 3 1 3
8: 3 8 14 8 10 3 1 2
9: 23 9 11 9 5 3 1 1
10: 20 6 9 14 14 2 3 3
11: 16 5 8 13 14 2 3 2
12: 13 1 9 13 7 2 3 1
13: 8 4 8 11 14 2 2 3
14: 17 2 10 10 7 2 2 2
15: 2 9 9 11 7 2 2 1
16: 4 9 9 7 14 2 1 3
17: 15 5 10 5 9 2 1 2
18: 6 9 10 9 7 2 1 1
19: 11 3 5 12 14 1 3 3
20: 25 10 5 15 11 1 3 2
21: 14 4 5 14 7 1 3 1
22: 26 4 6 10 9 1 2 3
23: 1 7 7 11 8 1 2 2
24: 19 7 6 11 7 1 2 1
25: 7 8 8 6 14 1 1 3
26: 24 7 7 6 9 1 1 2
27: 22 8 6 6 7 1 1 1
Firm value V1 V2 V3 G1 G2 G3
Here is an answer using transform and then ddply from plyr. I don't address the ties, which really means that in case of a tie the value from the lowest row number is used first. This is what the OP shows in the example output.
First, order the dataset in descending order of V1 and create three groups of 9 by creating a new variable, fv1.
dat1 = transform(dat1[order(-dat1$V1),], fv1 = factor(rep(1:3, each = 9)))
Then order the dataset in descending order of V2 and create three groups of 3 within each level of fv1.
require(plyr)
dat1 = ddply(dat1[order(-dat1$V2),], .(fv1), transform, fv2 = factor(rep(1:3, each = 3)))
Finally order the dataset by the two factors and V3. I use arrange from plyr for typing efficiency compared to order
(finaldat = arrange(dat1, fv1, fv2, -V3) )
This isn't a particularly generalizable answer, as the group sizes are known in advance for the factors. If the V3 group size was larger than one, a similar process as for V2 would be needed.