Pairwise t.test multiple combinations in a Data Table - r

I have a Data Table with three columns: seller, product and price.
Example data:
seller product price
1: A banana 56
2: A lemon 94
3: A orange 84
4: A banana 11
5: A lemon 86
---
166: C orange 162
167: C banana 109
168: C orange 61
169: C banana 141
170: C orange 22
Code for the data
require (data.table)
DT <- data.table(seller = c(rep(c("A"),60),rep(c("B"),62),rep(c("C"),48)), product = c(rep(c("banana", "lemon", "orange"), 20), rep(c("banana", "lemon"), 31), rep(c("banana", "orange"), 24)),
price = c(56, 94, 84, 11, 86, 103, 151, 51, 117, 71, 63, 101, 45, 147, 135, 93, 26, 164, 90, 67, 12, 34, 14, 131, 92, 145, 48, 74, 62, 57, 20, 80, 113, 46, 88, 102, 134, 98, 137, 123, 169, 133, 146,
160, 58, 42, 52, 158, 170, 2, 152, 10, 130, 30, 33, 144, 73, 41, 139, 107, 163, 9, 66, 81, 79, 127, 40, 165, 106, 161, 16, 1, 112, 70, 115, 138, 76, 105, 17, 118, 114, 121, 25, 39, 15, 155, 50, 166,
100, 159, 5, 19, 29, 24, 64, 149, 120, 35, 119, 53, 21, 7, 72, 132, 154, 168, 156, 38, 3, 148, 69, 44, 6, 28, 140, 77, 104, 153, 59, 142, 116, 150, 97, 31, 91, 43, 47, 27, 143, 99, 37, 54, 49, 4, 111,
32, 23, 85, 167, 136, 78, 129, 83, 124, 36, 96, 110, 13, 65, 108, 8, 18, 157, 87, 82, 60, 122, 89, 125, 68, 75, 126, 128, 55, 95, 162, 109, 61, 141, 22))
I would like to perform a pairwise T.test combination between all sellers that sell the same products.
I would like to have an output as it is shown (hypotetical p.values for the example).
Desire output:
seller.x seller.y product p.value
A B banana 0.45
A B lemon 0.87
B C banana 0.03
A C banana 0.23
A C orange 0.01

You first need to group by product. Then, in your j parameter, you need to compute the combinations of seller for this product and get the p.value for the t.test of price between seller.x and seller.y:
DT[
, {
sellercomb <- data.table(t(combn(unique(seller), 2)))
names(sellercomb) <- c("seller.x", "seller.y")
sellercomb[
, {
data.table(p.value = t.test(price[seller == seller.x], price[seller == seller.y])$p.value)
}
, by = .(seller.x, seller.y)
]
}
, by = .(product)
]
The result for your data above looks like this:
product seller.x seller.y p.value
1: banana A B 0.9384329
2: banana A C 0.2413946
3: banana B C 0.2154216
4: lemon A B 0.7282811
5: orange A C 0.0354320

Related

Web scraping and reshaping data

I have a problem when tidying a table from website scraping.
I want to get the table (with header V1 to V5) from the link below, but I failed to convert it into the same format in R studio.
This is what I'm doing
url <- "https://www.r-bloggers.com/2018/08/using-control-charts-in-r/"
library(rvest)
library(tidyverse)
h <- read_html(url)
tab <- h %>% html_nodes("table")
tab <- tab[[2]] %>% html_table()
tab <- separate_rows(tab, 1, sep = " ")
tab <- tab[8:132,]
tab <- as.data.frame(tab)
tab1 <- data.frame(c("V1", "V2", "V3", "V4", "V5"))
tab1 <- tab1 %>% setNames("Cat")
tab2 <- cbind(tab1,tab)
tab3 <- tab2 %>% spread(key = Cat, X1)
Here is the result
Error: Each row of output must be identified by a unique combination of keys.
Keys are shared for 125 rows:
* 1, 6, 11, 16, 21, 26, 31, 36, 41, 46, 51, 56, 61, 66, 71, 76, 81, 86, 91, 96, 101, 106, 111, 116, 121
* 2, 7, 12, 17, 22, 27, 32, 37, 42, 47, 52, 57, 62, 67, 72, 77, 82, 87, 92, 97, 102, 107, 112, 117, 122
* 3, 8, 13, 18, 23, 28, 33, 38, 43, 48, 53, 58, 63, 68, 73, 78, 83, 88, 93, 98, 103, 108, 113, 118, 123
* 4, 9, 14, 19, 24, 29, 34, 39, 44, 49, 54, 59, 64, 69, 74, 79, 84, 89, 94, 99, 104, 109, 114, 119, 124
* 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120, 125
So what should I do to get the same table as from the website?
And if you can think of a better way to get the table from this website, please tell me.
P/s: I'm learning R programming on my own, so please teach me!
Cheers.
Here's a way :
library(rvest)
url <- "https://www.r-bloggers.com/2018/08/using-control-charts-in-r/"
url %>%
read_html %>%
html_nodes('table') %>%
.[[2]] %>%
html_table() %>%
dplyr::pull(X1) %>%
stringr::str_extract_all('\\d+\\.\\d+') %>%
.[[1]] %>%
matrix(ncol = 5, byrow = TRUE) %>%
as.data.frame() %>% type.convert() -> tab
tab
# V1 V2 V3 V4 V5
#1 1.45 1.56 1.40 1.45 1.33
#2 1.75 1.53 1.55 1.42 1.42
#3 1.60 1.41 1.35 1.52 1.36
#4 1.53 1.58 1.54 1.71 1.55
#5 1.48 1.34 1.64 1.59 1.46
#6 1.69 1.55 1.49 1.61 1.47
#...
#...

Levene Test for two groups with different length

I have two different groups with different ages in it, and I would like to compare variance by Levene test but it didn't work.
leveneTest(Total_Dem$age~Total_Rep$age)
The output was
Error in model.frame.default(form) : variable lengths differ (found for 'Total_Rep$age'), and I have no idea what to do next.
Here's the data
dput(Total_Dem$age)
c(59, 62, 51, 67, 36, 66, 61, 72, 68, 70, 55, 61, 89, 74, 66,
79, 69, 77, 65, 51, 73, 66, 68, 78, 70, 77, 67, 74, 56, 81, 63,
49, 70, 50, 64, 72, 68, 33, 65, 70, 75, 65, 53, 63, 74, 63, 59,
72, 67, 33, 67, 64, 57, 74, 71, 30, 52, 62, 78, 54, 62, 58, 70,
54, 35, 63, 64, 63, 73, 73, 72, 58, 72, 64, 71, 55, 74, 64, 39,
30, 33, 52, 28, 62, 61, 80, 40, 63, 60, 44, 51, 40, 23, 63, 43,
39, 61, 34, 72, 32, 31, 24, 40, 66, 32, 64, 21, 69, 32, 80, 40,
21, 30, 26, 49, 59, 53, 67, 62, 53, 34, 88, 55, 66, 60, 22, 61,
38, 38, 67, 36, 48, 29, 72, 33, 57, 29, 48, 64, 51, 55, 71, 38,
79, 68, 25, 61, 47, 42, 39, 57, 38, 44, 69, 79, 40, 75, 59, 32,
75, 28, 34, 77, 75, 58, 56, 64, 64, 79, 60, 61, 53, 49, 36, 41,
71, 40, 54, 71, 32, 43, 44, 33, 67, 65, 67, 70, 56, 49, 34, 64,
59, 57, 53, 70, 54, 54, 53, 74, 65, 60, 79, 60, 60, 61, 34, 69,
42, 62, 73, 53, 61, 41, 70, 80, 76, 64, 67, 68, 71, 33, 27, 68,
30, 59, 64, 51, 76, 65, 76, 70, 35, 35, 60, 31, 47, 56, 58, 32,
34, 46, 68, 41, 87, 37, 57, 53, 80, 67, 66, 61, 44, 58, 66, 61,
80, 59, 30, 75, 82, 66, 59, 60, 80, 71, 51, 43, 48, 33, 77, 49,
27, 63, 45, 75, 34, 42, 19, 48, 71, 52, 47, 57, 57, 56, 49, 80,
56, 29, 60, 55, 57, 45, 60, 64, 55, 29, 50, 41, 26, 27, 67, 26,
32, 65, 81, 26, 63, 77, 47, 55, 24, 28, 73, 67, 36, 29, 57, 51,
44, 39, 67, 38, 75, 82, 52, 61, 75, 72, 64, 21, 36, 30, 52, 59,
53, 31, 46, 38, 51, 61, 50, 59, 55, 53, 71, 53, 68, 76, 63, 56,
39, 35, 75, 66, 33, 66, 62, 61, 57, 24, 57, 54, 61, 52, 38, 48,
65, 53, 36, 69, 75, 39, 88, 60, 66, 51, 73, 67, 75, 59, 80, 60,
64, 60, 51, 28, 37, 71, 37, 67, 61, 62, 45, 64, 58, 34, 69, 57,
49, 68, 81, 47, 55, 66, 42, 60, 49, 62, 59, 49, 22, 75, 56, 47,
34, 42, 34, 27, 45, 70, 73, 54, 74, 66, 75, 54, 31, 48, 58, 33,
50)
dput(Total_Rep$age)
c(33, 30, 57, 71, 60, 73, 70, 60, 64, 83, 60, 75, 55, 73, 67,
70, 69, 77, 74, 48, 77, 60, 56, 55, 75, 70, 34, 37, 58, 69, 65,
61, 68, 23, 81, 64, 66, 65, 72, 56, 59, 62, 71, 80, 63, 71, 56,
62, 66, 66, 88, 69, 45, 73, 79, 70, 50, 58, 83, 84, 61, 46, 47,
64, 60, 55, 58, 48, 64, 72, 65, 87, 61, 65, 60, 25, 36, 51, 64,
64, 61, 75, 65, 26, 67, 59, 58, 62, 41, 61, 70, 89, 57, 66, 68,
73, 84, 46, 43, 63, 71, 20, 62, 67, 71, 39, 54, 54, 41, 42, 39,
51, 74, 42, 47, 56, 74, 53, 70, 66, 58, 67, 83, 64, 48, 30, 53,
44, 54, 71, 77, 51, 62, 62, 68, 25, 56, 91, 61, 61, 62, 61, 70,
45, 42, 76, 66, 64, 80, 80, 58, 65, 68, 66, 47, 55, 55, 56, 54,
81, 70, 59, 78, 44, 55, 70, 63, 60, 43, 73, 65, 73, 80, 81, 25,
60, 80, 62, 52, 58, 73, 54, 71, 72, 77, 57, 24, 69, 79, 72, 62,
38, 63, 67, 52, 41, 59, 59, 78, 58, 70, 66, 51, 70, 54, 64, 74,
54, 34, 60, 37, 65, 79, 48, 31, 55, 63, 36, 59, 62, 34, 69, 60,
47, 58, 55, 56, 59, 29, 84, 87, 80, 65, 49, 62, 27, 60, 67, 73,
76, 65, 28, 73, 50, 81, 44, 39, 36, 30, 70, 54, 69, 28, 67, 65,
40, 53, 66, 80, 88, 58, 55, 63, 53, 54, 20, 81, 72, 75, 67, 57,
58, 34, 32, 47, 67, 52, 63, 78, 42, 50, 30, 32, 83, 58, 67, 26,
56, 21, 25, 62, 64, 64, 33, 69, 28, 56, 62, 62, 51, 56, 61, 70,
67, 63, 51, 35, 70, 61, 71, 56, 44, 62, 44, 77, 26)
The length of two groups are different, and I would like to perform Levene's test with this samples.
The leveneTest function requires the following syntax:
leveneTest(response variable ~ grouping variable)
or
leveneTest(response variable, grouping variable)
The syntax you currently have is trying to group the Total_Dem ages by the Total_Rep ages which (1) cannot be done and why you are getting an error and (2) I am sure it is not what you want.
We need to combine the ages into a single dataframe and label the source (Dem or Rep)
newdf <- data.frame(age=c(Total_Dem$age,Total_Rep$age),
source=c(rep("Dem",times=length(Total_Dem$age)),rep("Rep",times=length(Total_Rep$age))))
We can then run the leveneTest
leveneTest(newdf$age~newdf$source)
# or
leveneTest(newdf$age,newdf$source)
This groups our response variable age by Dem and Rep which is what you want.

For loops with a dataframe in R

I got a dataframe (merged_df) with 52 columns (I show here only the first 4):
Row.names node_demand Node 1 Node 2
1 Node 1 3 0 87
2 Node 10 6 58 52
3 Node 11 10 43 70
4 Node 12 18 94 8
5 Node 13 3 44 63
6 Node 14 6 21 98
7 Node 15 20 31 64
8 Node 16 4 35 76
9 Node 17 14 58 52
10 Node 18 11 19 71
11 Node 19 19 62 38
12 Node 2 14 87 0
13 Node 20 15 102 19
14 Node 21 15 16 76
15 Node 22 4 54 51
16 Node 23 13 59 75
17 Node 24 13 73 28
18 Node 25 5 82 33
19 Node 26 16 62 72
20 Node 27 3 59 30
21 Node 28 7 73 32
22 Node 29 14 45 48
23 Node 3 1 43 78
24 Node 30 17 69 44
25 Node 31 3 70 43
26 Node 32 3 15 87
27 Node 33 12 38 72
28 Node 34 14 62 81
29 Node 35 20 104 17
30 Node 36 13 18 77
31 Node 37 10 70 22
32 Node 38 9 65 46
33 Node 39 6 24 64
34 Node 4 14 68 23
35 Node 40 18 85 8
36 Node 41 7 20 95
37 Node 42 20 55 82
38 Node 43 9 94 16
39 Node 44 1 10 79
40 Node 45 8 62 63
41 Node 46 5 50 88
42 Node 47 1 70 50
43 Node 48 7 54 73
44 Node 49 9 52 43
45 Node 5 19 57 48
46 Node 50 2 4 86
47 Node 6 2 76 22
48 Node 7 14 79 60
49 Node 8 6 108 25
50 Node 9 7 101 18
The columns Node 1, Node 2 .....Node 45....Node 46 show the distance from the Node indicated on the column respect all the other nodes.
I want to pick the closest nodes, and then to select all the nodes under which cumsum() node_demand is less than 120, starting from the first row. Since the first value is the distance between the main Node and itself I don't consider the first row.
To do that for Node 1 I would do:
test <- merged_df[,c(1,2,3)] # Columns 1 and 2 are fixed
test <- test[(order(test[3])),][2:50,] # to get the closest distances first
test<- test[cumsum(test$node_demand)< 120,]
I then need to create a new variable for each node with the last value of the cumsum()
node_1 <- tail(cumsum(test$`Node 1`), n=1) # 381
The output for node_1 would be 381
To do the same for node_2:
test <- merged_df[,c(1,2,4)] #c(1,2,**4**) 4 instead of 3 as before
test <- test[(order(test[3])),][2:50,]
test<- test[cumsum(test$node_demand)< 120,]
node_2 <- tail(cumsum(test$`Node 2`), n=1)
The output for node_2 is 178
Since this process is very repetitive I guess a loop could do it but I am not sure how to create the different variables I need
for(i in 3:52){
test <- merged_df[,c(1,2,i)]
test <- merged_df[order(test[3]),][2:50]
test<- test[cumsum(test$node_demand)< 120,]
}
node_1 <- tail(cumsum(test$test$`Node 1`), n=1) # should return 381
#I'm not sure how to create the variables node_1, node_2....node_50
The process to follow would be:
Create a subset of the dataframe using columns 1, 2 and i (representing the number from Node 1 to Node 50.
Sort the subset by the column Node i so the smallest distances are placed first.
I need to select rows until cumsum(node_demand) < 120. (which is what I do using test<- test[cumsum(test$node_demand)< 120,])
Then I need to calculate cumsum(test$Node 1). This will give me the cumulative distance from all the nodes under the condition of cumsum(node_demand) < 120
Anybody could give me a hand?
Many thanks!
The output of dput() is :
structure(list(Row.names = structure(c("Node 1", "Node 10", "Node 11",
"Node 12", "Node 13", "Node 14", "Node 15", "Node 16", "Node 17",
"Node 18", "Node 19", "Node 2", "Node 20", "Node 21", "Node 22",
"Node 23", "Node 24", "Node 25", "Node 26", "Node 27", "Node 28",
"Node 29", "Node 3", "Node 30", "Node 31", "Node 32", "Node 33",
"Node 34", "Node 35", "Node 36", "Node 37", "Node 38", "Node 39",
"Node 4", "Node 40", "Node 41", "Node 42", "Node 43", "Node 44",
"Node 45", "Node 46", "Node 47", "Node 48", "Node 49", "Node 5",
"Node 50", "Node 6", "Node 7", "Node 8", "Node 9"), class = "AsIs"),
node_demand = c(3L, 6L, 10L, 18L, 3L, 6L, 20L, 4L, 14L, 11L,
19L, 14L, 15L, 15L, 4L, 13L, 13L, 5L, 16L, 3L, 7L, 14L, 1L,
17L, 3L, 3L, 12L, 14L, 20L, 13L, 10L, 9L, 6L, 14L, 18L, 7L,
20L, 9L, 1L, 8L, 5L, 1L, 7L, 9L, 19L, 2L, 2L, 14L, 6L, 7L
), `Node 1` = c(0, 58, 43, 94, 44, 21, 31, 35, 58, 19, 62,
87, 102, 16, 54, 59, 73, 82, 62, 59, 73, 45, 43, 69, 70,
15, 38, 62, 104, 18, 70, 65, 24, 68, 85, 20, 55, 94, 10,
62, 50, 70, 54, 52, 57, 4, 76, 79, 108, 101), `Node 2` = c(87,
52, 70, 8, 63, 98, 64, 76, 52, 71, 38, 0, 19, 76, 51, 75,
28, 33, 72, 30, 32, 48, 78, 44, 43, 87, 72, 81, 17, 77, 22,
46, 64, 23, 8, 95, 82, 16, 79, 63, 88, 50, 73, 43, 48, 86,
22, 60, 25, 18), `Node 3` = c(43, 28, 11, 84, 15, 35, 52,
68, 30, 45, 73, 78, 97, 43, 72, 20, 78, 57, 91, 58, 80, 58,
0, 42, 83, 29, 69, 94, 91, 51, 70, 36, 41, 70, 79, 33, 22,
78, 34, 25, 13, 86, 84, 35, 73, 46, 60, 43, 101, 94), `Node 4` = c(68,
50, 62, 30, 56, 82, 43, 53, 49, 51, 16, 23, 34, 57, 29, 71,
10, 44, 50, 15, 15, 26, 70, 46, 25, 71, 49, 58, 39, 57, 5,
47, 45, 0, 19, 79, 76, 37, 62, 61, 81, 31, 50, 36, 25, 67,
30, 65, 41, 35), `Node 5` = c(57, 62, 66, 54, 62, 74, 30,
33, 61, 39, 10, 48, 55, 45, 15, 79, 23, 66, 26, 30, 24, 16,
73, 62, 22, 64, 27, 34, 63, 42, 28, 62, 37, 25, 42, 71, 80,
62, 55, 73, 84, 24, 28, 48, 0, 55, 53, 81, 63, 58), `Node 6` = c(76,
34, 54, 27, 46, 84, 60, 73, 36, 63, 45, 22, 40, 68, 53, 58,
38, 16, 77, 25, 41, 46, 60, 29, 50, 73, 71, 85, 34, 71, 28,
29, 55, 30, 27, 82, 67, 20, 67, 44, 70, 56, 75, 28, 53, 76,
0, 44, 41, 35), `Node 7` = c(79, 24, 38, 62, 36, 75, 72,
93, 23, 72, 75, 60, 79, 72, 84, 28, 75, 30, 105, 58, 79,
69, 43, 20, 88, 69, 91, 112, 66, 80, 66, 20, 66, 65, 64,
73, 38, 53, 70, 18, 45, 93, 102, 35, 81, 82, 44, 0, 81, 75
), `Node 8` = c(108, 75, 94, 23, 86, 120, 85, 91, 76, 91,
54, 25, 11, 97, 64, 98, 40, 52, 83, 49, 41, 67, 101, 67,
51, 109, 88, 91, 21, 98, 39, 69, 84, 41, 27, 117, 106, 28,
101, 85, 111, 56, 85, 66, 63, 107, 41, 81, 0, 7), `Node 9` = c(101,
68, 87, 17, 79, 113, 78, 86, 69, 85, 48, 18, 9, 90, 58, 91,
35, 46, 78, 42, 36, 60, 94, 60, 47, 102, 83, 87, 18, 91,
32, 62, 78, 35, 20, 110, 99, 23, 94, 79, 104, 52, 80, 59,
58, 100, 35, 75, 7, 0), `Node 10` = c(58, 0, 23, 57, 16,
58, 51, 70, 8, 50, 58, 52, 71, 51, 63, 24, 60, 29, 85, 40,
62, 48, 28, 16, 69, 49, 69, 91, 64, 59, 50, 8, 43, 50, 55,
56, 34, 50, 48, 12, 37, 73, 80, 14, 62, 60, 34, 24, 75, 68
), `Node 11` = c(43, 23, 0, 76, 10, 37, 45, 65, 22, 41, 65,
70, 89, 39, 67, 17, 71, 52, 85, 51, 73, 51, 11, 34, 77, 31,
64, 89, 83, 47, 63, 30, 36, 62, 71, 35, 18, 72, 34, 21, 19,
81, 79, 27, 66, 46, 54, 38, 94, 87), `Node 12` = c(94, 57,
76, 0, 69, 104, 71, 83, 56, 78, 44, 8, 17, 83, 58, 79, 34,
36, 78, 38, 38, 55, 84, 47, 50, 94, 79, 87, 9, 85, 29, 50,
71, 30, 12, 101, 86, 14, 87, 67, 93, 56, 80, 49, 54, 94,
27, 62, 23, 17), `Node 13` = c(44, 16, 10, 69, 0, 43, 45,
63, 18, 40, 60, 63, 82, 40, 61, 22, 65, 44, 82, 44, 67, 47,
15, 30, 71, 34, 62, 86, 77, 48, 56, 24, 34, 56, 65, 41, 27,
64, 34, 19, 26, 74, 76, 21, 62, 47, 46, 36, 86, 79), `Node 14` = c(21,
58, 37, 104, 43, 0, 46, 56, 58, 35, 78, 98, 114, 30, 73,
51, 88, 86, 82, 71, 89, 60, 35, 70, 88, 12, 58, 82, 113,
36, 84, 66, 39, 82, 97, 3, 44, 103, 22, 58, 37, 89, 74, 57,
74, 25, 84, 75, 120, 113), `Node 15` = c(31, 51, 45, 71,
45, 46, 0, 27, 49, 12, 35, 64, 77, 16, 34, 61, 47, 68, 42,
39, 49, 19, 52, 56, 48, 37, 22, 45, 80, 16, 46, 55, 16, 43,
60, 43, 58, 74, 29, 59, 61, 49, 38, 40, 30, 30, 60, 72, 85,
78), `Node 16` = c(35, 70, 65, 83, 63, 56, 27, 0, 70, 26,
42, 76, 85, 32, 28, 82, 53, 85, 29, 48, 51, 30, 68, 77, 44,
48, 9, 28, 93, 23, 55, 74, 29, 53, 72, 54, 80, 87, 38, 79,
79, 42, 19, 59, 33, 31, 73, 93, 91, 86), `Node 17` = c(58,
8, 22, 56, 18, 58, 49, 70, 0, 49, 56, 52, 71, 50, 63, 23,
59, 31, 84, 41, 63, 47, 30, 12, 70, 49, 68, 90, 63, 57, 50,
10, 43, 49, 54, 55, 31, 51, 48, 14, 37, 74, 80, 14, 61, 60,
36, 23, 76, 69), `Node 18` = c(19, 50, 41, 78, 40, 35, 12,
26, 49, 0, 44, 71, 85, 8, 39, 58, 55, 71, 48, 43, 56, 26,
45, 58, 54, 27, 25, 50, 87, 8, 53, 55, 9, 51, 68, 33, 55,
80, 17, 57, 55, 55, 41, 41, 39, 18, 63, 72, 91, 85), `Node 19` = c(62,
58, 65, 44, 60, 78, 35, 42, 56, 44, 0, 38, 46, 50, 19, 77,
14, 58, 35, 24, 18, 18, 73, 56, 21, 68, 37, 44, 53, 48, 19,
57, 40, 16, 32, 75, 79, 53, 58, 69, 84, 25, 37, 44, 10, 60,
45, 75, 54, 48), `Node 20` = c(102, 71, 89, 17, 82, 114,
77, 85, 71, 85, 46, 19, 0, 90, 58, 94, 33, 51, 75, 44, 35,
60, 97, 63, 46, 103, 81, 84, 18, 91, 32, 66, 78, 34, 19,
111, 101, 28, 95, 82, 107, 51, 78, 62, 55, 101, 40, 79, 11,
9), `Node 21` = c(16, 51, 39, 83, 40, 30, 16, 32, 50, 8,
50, 76, 90, 0, 46, 55, 62, 74, 54, 49, 63, 33, 43, 59, 61,
22, 30, 56, 92, 10, 59, 57, 15, 57, 73, 27, 51, 84, 16, 57,
51, 62, 48, 44, 45, 17, 68, 72, 97, 90), `Node 22` = c(54,
63, 67, 58, 61, 73, 34, 28, 63, 39, 19, 51, 58, 46, 0, 81,
26, 67, 26, 28, 23, 19, 72, 65, 16, 63, 26, 33, 67, 42, 29,
64, 35, 29, 46, 70, 83, 64, 52, 74, 84, 16, 23, 49, 15, 52,
53, 84, 64, 58), `Node 23` = c(59, 24, 17, 79, 22, 51, 61,
82, 23, 58, 77, 75, 94, 55, 81, 0, 81, 50, 100, 62, 84, 65,
20, 33, 90, 46, 80, 105, 85, 64, 73, 29, 53, 71, 77, 49,
13, 73, 51, 14, 19, 94, 95, 35, 79, 63, 58, 28, 98, 91),
`Node 24` = c(73, 60, 71, 34, 65, 88, 47, 53, 59, 55, 14,
28, 33, 62, 26, 81, 0, 53, 44, 22, 7, 29, 78, 56, 18, 77,
49, 53, 43, 60, 10, 57, 50, 10, 22, 85, 85, 43, 68, 71, 90,
24, 47, 46, 23, 71, 38, 75, 40, 35), `Node 25` = c(82, 29,
52, 36, 44, 86, 68, 85, 31, 71, 58, 33, 51, 74, 67, 50, 53,
0, 91, 39, 56, 58, 57, 22, 66, 76, 82, 99, 40, 79, 43, 22,
62, 44, 39, 84, 61, 25, 72, 36, 65, 72, 89, 30, 66, 83, 16,
30, 52, 46), `Node 26` = c(62, 85, 85, 78, 82, 82, 42, 29,
84, 48, 35, 72, 75, 54, 26, 100, 44, 91, 0, 53, 43, 37, 91,
86, 34, 74, 25, 9, 86, 46, 51, 86, 50, 50, 66, 80, 99, 86,
64, 95, 101, 30, 12, 71, 26, 59, 77, 105, 83, 78), `Node 27` = c(59,
40, 51, 38, 44, 71, 39, 48, 41, 43, 24, 30, 44, 49, 28, 62,
22, 39, 53, 0, 23, 23, 58, 40, 30, 60, 46, 60, 48, 50, 14,
39, 35, 15, 29, 69, 67, 39, 51, 52, 69, 34, 50, 27, 30, 58,
25, 58, 49, 42), `Node 28` = c(73, 62, 73, 38, 67, 89, 49,
51, 63, 56, 18, 32, 35, 63, 23, 84, 7, 56, 43, 23, 0, 31,
80, 60, 12, 78, 48, 51, 46, 61, 14, 60, 51, 15, 27, 86, 89,
46, 68, 74, 92, 18, 44, 49, 24, 71, 41, 79, 41, 36), `Node 29` = c(45,
48, 51, 55, 47, 60, 19, 30, 47, 26, 18, 48, 60, 33, 19, 65,
29, 58, 37, 23, 31, 0, 58, 50, 31, 50, 26, 43, 64, 31, 28,
50, 23, 26, 44, 57, 66, 59, 40, 59, 69, 33, 34, 35, 16, 43,
46, 69, 67, 60), `Node 30` = c(69, 16, 34, 47, 30, 70, 56,
77, 12, 58, 56, 44, 63, 59, 65, 33, 56, 22, 86, 40, 60, 50,
42, 0, 69, 61, 74, 93, 53, 66, 47, 9, 51, 46, 46, 67, 41,
41, 59, 21, 49, 74, 84, 19, 62, 70, 29, 20, 67, 60), `Node 31` = c(70,
69, 77, 50, 71, 88, 48, 44, 70, 54, 21, 43, 46, 61, 16, 90,
18, 66, 34, 30, 12, 31, 83, 69, 0, 77, 42, 42, 58, 58, 24,
68, 50, 25, 39, 86, 94, 57, 67, 81, 95, 6, 34, 55, 22, 68,
50, 88, 51, 47), `Node 32` = c(15, 49, 31, 94, 34, 12, 37,
48, 49, 27, 68, 87, 103, 22, 63, 46, 77, 76, 74, 60, 78,
50, 29, 61, 77, 0, 50, 75, 102, 29, 73, 57, 28, 71, 86, 9,
42, 92, 11, 51, 35, 78, 66, 47, 64, 19, 73, 69, 109, 102),
`Node 33` = c(38, 69, 64, 79, 62, 58, 22, 9, 68, 25, 37,
72, 81, 30, 26, 80, 49, 82, 25, 46, 48, 26, 69, 74, 42, 50,
0, 26, 88, 22, 51, 72, 28, 49, 67, 55, 79, 84, 40, 78, 79,
40, 18, 57, 27, 34, 71, 91, 88, 83), `Node 34` = c(62, 91,
89, 87, 86, 82, 45, 28, 90, 50, 44, 81, 84, 56, 33, 105,
53, 99, 9, 60, 51, 43, 94, 93, 42, 75, 26, 0, 96, 47, 60,
93, 53, 58, 75, 80, 103, 95, 65, 101, 104, 37, 12, 77, 34,
58, 85, 112, 91, 87), `Node 35` = c(104, 64, 83, 9, 77, 113,
80, 93, 63, 87, 53, 17, 18, 92, 67, 85, 43, 40, 86, 48, 46,
64, 91, 53, 58, 102, 88, 96, 0, 94, 38, 57, 80, 39, 21, 110,
93, 17, 96, 73, 100, 64, 89, 57, 63, 103, 34, 66, 21, 18),
`Node 36` = c(18, 59, 47, 85, 48, 36, 16, 23, 57, 8, 48,
77, 91, 10, 42, 64, 60, 79, 46, 50, 61, 31, 51, 66, 58, 29,
22, 47, 94, 0, 59, 64, 18, 57, 74, 34, 61, 87, 21, 65, 60,
57, 40, 49, 42, 16, 71, 80, 98, 91), `Node 37` = c(70, 50,
63, 29, 56, 84, 46, 55, 50, 53, 19, 22, 32, 59, 29, 73, 10,
43, 51, 14, 14, 28, 70, 47, 24, 73, 51, 60, 38, 59, 0, 48,
47, 5, 18, 81, 78, 35, 64, 62, 82, 30, 52, 37, 28, 69, 28,
66, 39, 32), `Node 38` = c(65, 8, 30, 50, 24, 66, 55, 74,
10, 55, 57, 46, 66, 57, 64, 29, 57, 22, 86, 39, 60, 50, 36,
9, 68, 57, 72, 93, 57, 64, 48, 0, 48, 47, 50, 63, 39, 44,
55, 16, 44, 73, 83, 16, 62, 66, 29, 20, 69, 62), `Node 39` = c(24,
43, 36, 71, 34, 39, 16, 29, 43, 9, 40, 64, 78, 15, 35, 53,
50, 62, 50, 35, 51, 23, 41, 51, 50, 28, 28, 53, 80, 18, 47,
48, 0, 45, 62, 36, 53, 72, 18, 51, 52, 51, 43, 33, 37, 23,
55, 66, 84, 78), `Node 40` = c(85, 55, 71, 12, 65, 97, 60,
72, 54, 68, 32, 8, 19, 73, 46, 77, 22, 39, 66, 29, 27, 44,
79, 46, 39, 86, 67, 75, 21, 74, 18, 50, 62, 19, 0, 94, 83,
23, 78, 66, 89, 45, 68, 44, 42, 84, 27, 64, 27, 20), `Node 41` = c(20,
56, 35, 101, 41, 3, 43, 54, 55, 33, 75, 95, 111, 27, 70,
49, 85, 84, 80, 69, 86, 57, 33, 67, 86, 9, 55, 80, 110, 34,
81, 63, 36, 79, 94, 0, 42, 100, 20, 56, 36, 86, 72, 55, 71,
24, 82, 73, 117, 110), `Node 42` = c(55, 34, 18, 86, 27,
44, 58, 80, 31, 55, 79, 82, 101, 51, 83, 13, 85, 61, 99,
67, 89, 66, 22, 41, 94, 42, 79, 103, 93, 61, 78, 39, 53,
76, 83, 42, 0, 82, 48, 26, 16, 97, 95, 42, 80, 59, 67, 38,
106, 99), `Node 43` = c(94, 50, 72, 14, 64, 103, 74, 87,
51, 80, 53, 16, 28, 84, 64, 73, 43, 25, 86, 39, 46, 59, 78,
41, 57, 92, 84, 95, 17, 87, 35, 44, 72, 37, 23, 100, 82,
0, 86, 59, 87, 63, 87, 45, 62, 94, 20, 53, 28, 23), `Node 44` = c(10,
48, 34, 87, 34, 22, 29, 38, 48, 17, 58, 79, 95, 16, 52, 51,
68, 72, 64, 51, 68, 40, 34, 59, 67, 11, 40, 65, 96, 21, 64,
55, 18, 62, 78, 20, 48, 86, 0, 52, 43, 67, 55, 43, 55, 13,
67, 70, 101, 94), `Node 45` = c(62, 12, 21, 67, 19, 58, 59,
79, 14, 57, 69, 63, 82, 57, 74, 14, 71, 36, 95, 52, 74, 59,
25, 21, 81, 51, 78, 101, 73, 65, 62, 16, 51, 61, 66, 56,
26, 59, 52, 0, 30, 85, 91, 26, 73, 65, 44, 18, 85, 79), `Node 46` = c(50,
37, 19, 93, 26, 37, 61, 79, 37, 55, 84, 88, 107, 51, 84,
19, 90, 65, 101, 69, 92, 69, 13, 49, 95, 35, 79, 104, 100,
60, 82, 44, 52, 81, 89, 36, 16, 87, 43, 30, 0, 98, 95, 45,
84, 54, 70, 45, 111, 104), `Node 47` = c(70, 73, 81, 56,
74, 89, 49, 42, 74, 55, 25, 50, 51, 62, 16, 94, 24, 72, 30,
34, 18, 33, 86, 74, 6, 78, 40, 37, 64, 57, 30, 73, 51, 31,
45, 86, 97, 63, 67, 85, 98, 0, 30, 60, 24, 67, 56, 93, 56,
52), `Node 48` = c(54, 80, 79, 80, 76, 74, 38, 19, 80, 41,
37, 73, 78, 48, 23, 95, 47, 89, 12, 50, 44, 34, 84, 84, 34,
66, 18, 12, 89, 40, 52, 83, 43, 50, 68, 72, 95, 87, 55, 91,
95, 30, 0, 67, 28, 50, 75, 102, 85, 80), `Node 49` = c(52,
14, 27, 49, 21, 57, 40, 59, 14, 41, 44, 43, 62, 44, 49, 35,
46, 30, 71, 27, 49, 35, 35, 19, 55, 47, 57, 77, 57, 49, 37,
16, 33, 36, 44, 55, 42, 45, 43, 26, 45, 60, 67, 0, 48, 53,
28, 35, 66, 59), `Node 50` = c(4, 60, 46, 94, 47, 25, 30,
31, 60, 18, 60, 86, 101, 17, 52, 63, 71, 83, 59, 58, 71,
43, 46, 70, 68, 19, 34, 58, 103, 16, 69, 66, 23, 67, 84,
24, 59, 94, 13, 65, 54, 67, 50, 53, 55, 0, 76, 82, 107, 100
)), .Names = c("Row.names", "node_demand", "Node 1", "Node 2",
"Node 3", "Node 4", "Node 5", "Node 6", "Node 7", "Node 8", "Node 9",
"Node 10", "Node 11", "Node 12", "Node 13", "Node 14", "Node 15",
"Node 16", "Node 17", "Node 18", "Node 19", "Node 20", "Node 21",
"Node 22", "Node 23", "Node 24", "Node 25", "Node 26", "Node 27",
"Node 28", "Node 29", "Node 30", "Node 31", "Node 32", "Node 33",
"Node 34", "Node 35", "Node 36", "Node 37", "Node 38", "Node 39",
"Node 40", "Node 41", "Node 42", "Node 43", "Node 44", "Node 45",
"Node 46", "Node 47", "Node 48", "Node 49", "Node 50"), class = "data.frame", row.names = c(NA,
-50L))
You can try a tidyverse
library(tidyverse)
d %>%
as.tibble() %>%
gather(k,v, -node_demand, -Row.names) %>%
arrange(k, v) %>%
group_by(k) %>%
filter(Row.names != k) %>%
filter(cumsum(node_demand)<120) %>%
summarise(sum(v))
# A tibble: 50 x 2
k `sum(v)`
<chr> <dbl>
1 Node 1 381
2 Node 10 202
3 Node 11 332
4 Node 12 186
5 Node 13 262
6 Node 14 419
7 Node 15 282
8 Node 16 279
9 Node 17 272
10 Node 18 302
# ... with 40 more rows
Prove result for Node 1 and 2:
.Last.value %>%
filter(k %in% c("Node 1", "Node 2"))
# A tibble: 2 x 2
k `sum(v)`
<chr> <dbl>
1 Node 1 381
2 Node 2 178
The idea is to transform the data from long to wide. After arranging, we group by Node (column k) and filter 1) "self-nodes" and 2) cumsum<120. Finally calculate the sum for each Node.

How to add element-wise edges between two sets of vertices in Julia

After running the command connected_components on an undirected graph g with LightGraphs in Julia, I obtain the following result:
9-element Array{Array{Int64,1},1}:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10 … 41, 42, 43, 44, 45, 46, 47, 48, 49, 50]
[51, 52, 53, 54, 55, 56, 57, 58, 59, 60 … 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]
[69, 88]
[71, 73, 84, 102, 114, 122, 124, 127, 128, 134, 139, 143, 147, 150]
[101, 104, 105, 111, 112, 113, 116, 117, 121, 125 … 137, 138, 140, 141, 142, 144, 145, 146, 148, 149]
[103, 106, 108, 110, 118, 119, 123, 126, 130, 131, 132, 136]
[107]
[109]
[115]
I want to add an element-wise edge between the vertices 107 109 115 and the vertices 91 113 102. I know I can use the commands add_edge!(g,107,91) add_edge!(g,109,113) and add_edge!(g,115,102) but isn't there a command that can do all that in one shot instead of creating a loop or a function?
Thank you!

groupby + rowMean with dplyr

I have a large data frame, which includes impoTand nlc as key (ignore t all together), and other columns which each hold a number. I want to find, for each impoTand nlc pair, the average of all the other columns, or basically a rowMean. A subset of my data, which only includes one nlc, is given at the end. The last thing I tried was :
avg <- data.frame(a %>% group_by(impoT, nlc) %>% select(-c(1:3)) %>% mutate(r= rowMeans(.) ))
stds = (a %>% group_by(impoT, nlc) %>% select(-c(1:3)) %>% apply( 1, sd)) #wrong
dput(a)
structure(list(impoT = 1:18, nlc = c(669L, 669L, 669L, 669L,
669L, 669L, 669L, 669L, 669L, 669L, 669L, 669L, 669L, 669L, 669L,
669L, 669L, 669L), t = c(102L, 118L, 134L, 150L, 166L, 182L,
198L, 214L, 230L, 246L, 262L, 278L, 294L, 310L, 326L, 342L, 358L,
374L), X11950 = c(6, 14, 40, 53, 59, 70, 118, 119, 111, 114,
103, 220, 278, 94, 28, 13, 5, 8), X11951 = c(4, 18, 41, 64, 78,
87, 140, 112, 113, 129, 112, 245, 322, 102, 52, 20, 15, 7), X11952 = c(8,
13, 30, 42, 52, 86, 126, 118, 52, 87, 116, 251, 262, 101, 35,
21, 15, 21), X11955 = c(9, 11, 47, 38, 39, 70, 95, 82, 80, 77,
77, 142, 192, 78, 13, 13, 5, 0), X11956 = c(14, 13, 44, 65, 65,
72, 125, 138, 117, 111, 104, 175, 282, 93, 28, 14, 8, 4), X11957 = c(10,
7, 45, 42, 50, 83, 123, 102, 104, 82, 102, 234, 265, 101, 23,
13, 7, 6), X11958 = c(10, 13, 42, 60, 68, 69, 106, 125, 104,
103, 112, 233, 310, 128, 50, 22, 10, 5), X11959 = c(7, 11, 32,
45, 63, 74, 119, 87, 121, 108, 104, 229, 266, 111, 46, 26, 22,
11), X11962 = c(8, 12, 38, 35, 49, 58, 96, 66, 73, 109, 82, 161,
192, 75, 22, 4, 2, 3), X11963 = c(8, 9, 39, 40, 56, 50, 142,
98, 102, 78, 79, 220, 229, 87, 25, 5, 7, 2), X11964 = c(10, 9,
42, 60, 53, 52, 105, 114, 96, 94, 95, 180, 268, 114, 23, 10,
7, 10), X11965 = c(9, 9, 41, 40, 61, 81, 150, 102, 102, 121,
125, 222, 347, 116, 37, 18, 3, 4), X11966 = c(10, 9, 34, 43,
49, 73, 112, 123, 102, 92, 107, 207, 239, 115, 60, 18, 15, 5),
X11969 = c(8, 9, 31, 34, 41, 51, 93, 92, 68, 103, 76, 166,
182, 63, 24, 14, 6, 4), X11970 = c(7, 12, 33, 48, 56, 59,
102, 88, 99, 86, 103, 194, 233, 90, 25, 13, 7, 3), X11971 = c(9,
16, 37, 60, 78, 62, 114, 106, 129, 107, 91, 212, 272, 88,
31, 10, 3, 3), X12088 = c(6, 11, 41, 44, 56, 70, 106, 97,
64, 73, 75, 161, 186, 76, 17, 8, 2, 2), X12089 = c(0, 11,
53, 59, 62, 64, 114, 109, 109, 100, 66, 222, 241, 88, 19,
8, 8, 3), X12090 = c(4, 12, 57, 52, 65, 73, 132, 109, 120,
101, 104, 227, 238, 99, 17, 8, 10, 8), X12091 = c(4, 16,
54, 167, 74, 62, 111, 95, 120, 102, 92, 227, 317, 106, 44,
16, 10, 4), X12092 = c(9, 10, 50, 55, 63, 64, 130, 103, 98,
116, 83, 249, 279, 88, 35, 36, 22, 15), X12095 = c(5, 15,
39, 44, 53, 58, 95, 92, 67, 63, 69, 163, 182, 69, 20, 8,
4, 2), X12096 = c(3, 14, 49, 53, 71, 70, 107, 130, 90, 89,
101, 214, 253, 100, 30, 10, 3, 3), X12097 = c(2, 16, 53,
61, 82, 83, 123, 124, 125, 98, 89, 220, 274, 107, 20, 17,
7, 5), X12098 = c(6, 17, 56, 59, 51, 77, 102, 115, 93, 98,
83, 221, 288, 97, 36, 16, 9, 10), X12099 = c(2, 16, 39, 49,
60, 84, 112, 91, 102, 103, 108, 246, 261, 131, 49, 24, 18,
14), X12102 = c(4, 12, 29, 47, 64, 69, 104, 111, 92, 72,
105, 174, 179, 64, 16, 10, 2, 1)), .Names = c("impoT", "nlc",
"t", "X11950", "X11951", "X11952", "X11955", "X11956", "X11957",
"X11958", "X11959", "X11962", "X11963", "X11964", "X11965", "X11966",
"X11969", "X11970", "X11971", "X12088", "X12089", "X12090", "X12091",
"X12092", "X12095", "X12096", "X12097", "X12098", "X12099", "X12102"
), row.names = c(NA, -18L), class = "data.frame")
It's easiest if you split your means into two steps, as you're actually taking the mean of irregular groups: first each row, and second each group. This means you're taking the means of means, but given each of the row means is of the same amount of numbers, they should be fine that way, although you should consider that the grouping means may be means of different amounts of rows.
You also need to nest the select so you don't lose your grouping variables, and use summarise to collapse the groups. All told,
a %>% mutate(r = rowMeans(select(a, -c(1:3)))) %>%
group_by(impoT, nlc) %>% summarise(r = mean(r))
produces
Source: local data frame [18 x 3]
Groups: impoT [?]
impoT nlc r
(int) (int) (dbl)
1 1 669 6.740741
2 2 669 12.407407
3 3 669 42.074074
4 4 669 54.037037
5 5 669 59.925926
6 6 669 69.296296
7 7 669 114.888889
8 8 669 105.481481
9 9 669 98.259259
10 10 669 96.888889
11 11 669 94.925926
12 12 669 207.962963
13 13 669 253.222222
14 14 669 95.592593
15 15 669 30.555556
16 16 669 14.629630
17 17 669 8.592593
18 18 669 6.037037

Resources