I was going through some earlier post-
Quickest way to read a subset of rows of a CSV
One way to select subset of data is
write.csv(iris,"iris.csv")
fread("shuf -n 5 iris.csv")
However I was wondering if I can pass some SQL query instead of top 5 rows e.g. only import those rows that have V6 = versicolor
Is there any way to do this using fread function?
This worked for me in windows (unix alternative is grep)
write.csv(iris,"iris.csv")
fread(cmd = paste('findstr', 'versicolor', 'iris.csv'))
V1 V2 V3 V4 V5 V6
1: 51 7.0 3.2 4.7 1.4 versicolor
2: 52 6.4 3.2 4.5 1.5 versicolor
3: 53 6.9 3.1 4.9 1.5 versicolor
4: 54 5.5 2.3 4.0 1.3 versicolor
5: 55 6.5 2.8 4.6 1.5 versicolor
6: 56 5.7 2.8 4.5 1.3 versicolor
7: 57 6.3 3.3 4.7 1.6 versicolor
8: 58 4.9 2.4 3.3 1.0 versicolor
9: 59 6.6 2.9 4.6 1.3 versicolor
10: 60 5.2 2.7 3.9 1.4 versicolor
11: 61 5.0 2.0 3.5 1.0 versicolor
It outputs only those rows that contain "versicolor" in any field.
Related
This question already has answers here:
Order data frame by the last column with dplyr
(3 answers)
Closed 7 months ago.
I want to automatically sort the data frame based on the last column name.
Since the last column name in my data frame will be dynamic, I cannot specify the column name.
Below is what i want to achieve as an output.
iris %>%
select(-Species) %>%
arrange(desc(Petal.Width))
As suggested in response i tried below option, however it dosen't work. Am i missing something?
iris %>%
select(-Species) %>%
arrange(desc(ncol(.)))
last_col() is only supported in functions that feature tidy selection syntax, which arrange()doesn’t.
ncol() will give you the number of the last column, we can use it to subset the data.frame.
See TarJae‘s answer for another Dplyr option to make tidy selection syntax available inside arrange().
library(dplyr)
iris %>%
select(!Species) %>%
arrange(desc(.[,ncol(.)]))
#> Sepal.Length Sepal.Width Petal.Length Petal.Width
#> 1 6.3 3.3 6.0 2.5
#> 2 7.2 3.6 6.1 2.5
#> 3 6.7 3.3 5.7 2.5
#> 4 5.8 2.8 5.1 2.4
#> 5 6.3 3.4 5.6 2.4
#> 6 6.7 3.1 5.6 2.4
#> 7 6.4 3.2 5.3 2.3
#> 8 7.7 2.6 6.9 2.3
#> 9 6.9 3.2 5.7 2.3
#> 10 7.7 3.0 6.1 2.3
#> 11 6.9 3.1 5.1 2.3
#> 12 6.8 3.2 5.9 2.3
#> 13 6.7 3.0 5.2 2.3
#> 14 6.2 3.4 5.4 2.3
#> 15 6.5 3.0 5.8 2.2
#> 16 7.7 3.8 6.7 2.2
#> 17 6.4 2.8 5.6 2.2
#> 18 7.1 3.0 5.9 2.1
#> 19 7.6 3.0 6.6 2.1
#> 20 6.8 3.0 5.5 2.1
#> 21 6.7 3.3 5.7 2.1
#> 22 6.4 2.8 5.6 2.1
#> 23 6.9 3.1 5.4 2.1
#> 24 6.5 3.2 5.1 2.0
#> 25 5.7 2.5 5.0 2.0
#> 26 5.6 2.8 4.9 2.0
#> 27 7.7 2.8 6.7 2.0
#> 28 7.9 3.8 6.4 2.0
#> 29 6.5 3.0 5.2 2.0
#> 30 5.8 2.7 5.1 1.9
#> 31 6.4 2.7 5.3 1.9
#> 32 7.4 2.8 6.1 1.9
#> 33 5.8 2.7 5.1 1.9
#> 34 6.3 2.5 5.0 1.9
#> 35 5.9 3.2 4.8 1.8
#> 36 6.3 2.9 5.6 1.8
#> 37 7.3 2.9 6.3 1.8
#> 38 6.7 2.5 5.8 1.8
#> 39 6.5 3.0 5.5 1.8
#> 40 6.3 2.7 4.9 1.8
#> 41 7.2 3.2 6.0 1.8
#> 42 6.2 2.8 4.8 1.8
#> 43 6.1 3.0 4.9 1.8
#> 44 6.4 3.1 5.5 1.8
#> 45 6.0 3.0 4.8 1.8
#> 46 5.9 3.0 5.1 1.8
#> 47 6.7 3.0 5.0 1.7
#> 48 4.9 2.5 4.5 1.7
#> 49 6.3 3.3 4.7 1.6
#> 50 6.0 2.7 5.1 1.6
#> 51 6.0 3.4 4.5 1.6
#> 52 7.2 3.0 5.8 1.6
#> 53 6.4 3.2 4.5 1.5
#> 54 6.9 3.1 4.9 1.5
#> 55 6.5 2.8 4.6 1.5
#> 56 5.9 3.0 4.2 1.5
#> 57 5.6 3.0 4.5 1.5
#> 58 6.2 2.2 4.5 1.5
#> 59 6.3 2.5 4.9 1.5
#> 60 6.0 2.9 4.5 1.5
#> 61 5.4 3.0 4.5 1.5
#> 62 6.7 3.1 4.7 1.5
#> 63 6.0 2.2 5.0 1.5
#> 64 6.3 2.8 5.1 1.5
#> 65 7.0 3.2 4.7 1.4
#> 66 5.2 2.7 3.9 1.4
#> 67 6.1 2.9 4.7 1.4
#> 68 6.7 3.1 4.4 1.4
#> 69 6.6 3.0 4.4 1.4
#> 70 6.8 2.8 4.8 1.4
#> 71 6.1 3.0 4.6 1.4
#> 72 6.1 2.6 5.6 1.4
#> 73 5.5 2.3 4.0 1.3
#> 74 5.7 2.8 4.5 1.3
#> 75 6.6 2.9 4.6 1.3
#> 76 5.6 2.9 3.6 1.3
#> 77 6.1 2.8 4.0 1.3
#> 78 6.4 2.9 4.3 1.3
#> 79 6.3 2.3 4.4 1.3
#> 80 5.6 3.0 4.1 1.3
#> 81 5.5 2.5 4.0 1.3
#> 82 5.6 2.7 4.2 1.3
#> 83 5.7 2.9 4.2 1.3
#> 84 6.2 2.9 4.3 1.3
#> 85 5.7 2.8 4.1 1.3
#> 86 6.1 2.8 4.7 1.2
#> 87 5.8 2.7 3.9 1.2
#> 88 5.5 2.6 4.4 1.2
#> 89 5.8 2.6 4.0 1.2
#> 90 5.7 3.0 4.2 1.2
#> 91 5.6 2.5 3.9 1.1
#> 92 5.5 2.4 3.8 1.1
#> 93 5.1 2.5 3.0 1.1
#> 94 4.9 2.4 3.3 1.0
#> 95 5.0 2.0 3.5 1.0
#> 96 6.0 2.2 4.0 1.0
#> 97 5.8 2.7 4.1 1.0
#> 98 5.7 2.6 3.5 1.0
#> 99 5.5 2.4 3.7 1.0
#> 100 5.0 2.3 3.3 1.0
#> 101 5.0 3.5 1.6 0.6
#> 102 5.1 3.3 1.7 0.5
#> 103 5.4 3.9 1.7 0.4
#> 104 5.7 4.4 1.5 0.4
#> 105 5.4 3.9 1.3 0.4
#> 106 5.1 3.7 1.5 0.4
#> 107 5.0 3.4 1.6 0.4
#> 108 5.4 3.4 1.5 0.4
#> 109 5.1 3.8 1.9 0.4
#> 110 4.6 3.4 1.4 0.3
#> 111 5.1 3.5 1.4 0.3
#> 112 5.7 3.8 1.7 0.3
#> 113 5.1 3.8 1.5 0.3
#> 114 5.0 3.5 1.3 0.3
#> 115 4.5 2.3 1.3 0.3
#> 116 4.8 3.0 1.4 0.3
#> 117 5.1 3.5 1.4 0.2
#> 118 4.9 3.0 1.4 0.2
#> 119 4.7 3.2 1.3 0.2
#> 120 4.6 3.1 1.5 0.2
#> 121 5.0 3.6 1.4 0.2
#> 122 5.0 3.4 1.5 0.2
#> 123 4.4 2.9 1.4 0.2
#> 124 5.4 3.7 1.5 0.2
#> 125 4.8 3.4 1.6 0.2
#> 126 5.8 4.0 1.2 0.2
#> 127 5.4 3.4 1.7 0.2
#> 128 4.6 3.6 1.0 0.2
#> 129 4.8 3.4 1.9 0.2
#> 130 5.0 3.0 1.6 0.2
#> 131 5.2 3.5 1.5 0.2
#> 132 5.2 3.4 1.4 0.2
#> 133 4.7 3.2 1.6 0.2
#> 134 4.8 3.1 1.6 0.2
#> 135 5.5 4.2 1.4 0.2
#> 136 4.9 3.1 1.5 0.2
#> 137 5.0 3.2 1.2 0.2
#> 138 5.5 3.5 1.3 0.2
#> 139 4.4 3.0 1.3 0.2
#> 140 5.1 3.4 1.5 0.2
#> 141 4.4 3.2 1.3 0.2
#> 142 5.1 3.8 1.6 0.2
#> 143 4.6 3.2 1.4 0.2
#> 144 5.3 3.7 1.5 0.2
#> 145 5.0 3.3 1.4 0.2
#> 146 4.9 3.1 1.5 0.1
#> 147 4.8 3.0 1.4 0.1
#> 148 4.3 3.0 1.1 0.1
#> 149 5.2 4.1 1.5 0.1
#> 150 4.9 3.6 1.4 0.1
Created on 2022-08-07 by the reprex package (v2.0.1)
We could do it using across:
library(dplyr)
library(psych) # for `headTail()`
iris %>%
select(-Species) %>%
arrange(across(last_col(), desc)) %>%
headTail()
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 6.3 3.3 6 2.5
2 7.2 3.6 6.1 2.5
3 6.7 3.3 5.7 2.5
4 5.8 2.8 5.1 2.4
... ... ... ... ...
147 4.8 3 1.4 0.1
148 4.3 3 1.1 0.1
149 5.2 4.1 1.5 0.1
150 4.9 3.6 1.4 0.1
Using Base R option
subset(iris , select = -Species) |>
(\(x) x[order(- x[ncol(x)]) , ])()
output
Sepal.Length Sepal.Width Petal.Length Petal.Width
101 6.3 3.3 6.0 2.5
110 7.2 3.6 6.1 2.5
145 6.7 3.3 5.7 2.5
115 5.8 2.8 5.1 2.4
137 6.3 3.4 5.6 2.4
141 6.7 3.1 5.6 2.4
116 6.4 3.2 5.3 2.3
119 7.7 2.6 6.9 2.3
......................................................
40 5.1 3.4 1.5 0.2
43 4.4 3.2 1.3 0.2
47 5.1 3.8 1.6 0.2
48 4.6 3.2 1.4 0.2
49 5.3 3.7 1.5 0.2
50 5.0 3.3 1.4 0.2
10 4.9 3.1 1.5 0.1
13 4.8 3.0 1.4 0.1
14 4.3 3.0 1.1 0.1
33 5.2 4.1 1.5 0.1
38 4.9 3.6 1.4 0.1
Let's take the iris dataset for example.
I want to create two dataframes from this one. This first one would have the contiuous variables, and the second one the discrete ones.
What I do first is create a list with the category of the column
iris <- iris
a <- lapply(iris, class)
Then I create two empty lists and store the index of both discrete and continuous columns.
cont <- list()
disc <- list()
for (i in 1:length(a)){
if (a[[i]][1] == "numeric")
cont <- append(cont, i)
else
disc <- append(disc, i)
}
But I do not know how to split based on this lists.
Try the following code:
classes_iris <- sapply(iris, class)
split.default(iris, classes_iris)
Output:
$factor
Species
1 setosa
2 setosa
3 setosa
4 setosa
5 setosa
6 setosa
7 setosa
8 setosa
9 setosa
10 setosa
11 setosa
12 setosa
13 setosa
14 setosa
15 setosa
16 setosa
17 setosa
18 setosa
19 setosa
20 setosa
21 setosa
22 setosa
23 setosa
24 setosa
25 setosa
26 setosa
27 setosa
28 setosa
29 setosa
30 setosa
31 setosa
32 setosa
33 setosa
34 setosa
35 setosa
36 setosa
37 setosa
38 setosa
39 setosa
40 setosa
41 setosa
42 setosa
43 setosa
44 setosa
45 setosa
46 setosa
47 setosa
48 setosa
49 setosa
50 setosa
51 versicolor
52 versicolor
53 versicolor
54 versicolor
55 versicolor
56 versicolor
57 versicolor
58 versicolor
59 versicolor
60 versicolor
61 versicolor
62 versicolor
63 versicolor
64 versicolor
65 versicolor
66 versicolor
67 versicolor
68 versicolor
69 versicolor
70 versicolor
71 versicolor
72 versicolor
73 versicolor
74 versicolor
75 versicolor
76 versicolor
77 versicolor
78 versicolor
79 versicolor
80 versicolor
81 versicolor
82 versicolor
83 versicolor
84 versicolor
85 versicolor
86 versicolor
87 versicolor
88 versicolor
89 versicolor
90 versicolor
91 versicolor
92 versicolor
93 versicolor
94 versicolor
95 versicolor
96 versicolor
97 versicolor
98 versicolor
99 versicolor
100 versicolor
101 virginica
102 virginica
103 virginica
104 virginica
105 virginica
106 virginica
107 virginica
108 virginica
109 virginica
110 virginica
111 virginica
112 virginica
113 virginica
114 virginica
115 virginica
116 virginica
117 virginica
118 virginica
119 virginica
120 virginica
121 virginica
122 virginica
123 virginica
124 virginica
125 virginica
126 virginica
127 virginica
128 virginica
129 virginica
130 virginica
131 virginica
132 virginica
133 virginica
134 virginica
135 virginica
136 virginica
137 virginica
138 virginica
139 virginica
140 virginica
141 virginica
142 virginica
143 virginica
144 virginica
145 virginica
146 virginica
147 virginica
148 virginica
149 virginica
150 virginica
$numeric
Sepal.Length Sepal.Width Petal.Length Petal.Width
1 5.1 3.5 1.4 0.2
2 4.9 3.0 1.4 0.2
3 4.7 3.2 1.3 0.2
4 4.6 3.1 1.5 0.2
5 5.0 3.6 1.4 0.2
6 5.4 3.9 1.7 0.4
7 4.6 3.4 1.4 0.3
8 5.0 3.4 1.5 0.2
9 4.4 2.9 1.4 0.2
10 4.9 3.1 1.5 0.1
11 5.4 3.7 1.5 0.2
12 4.8 3.4 1.6 0.2
13 4.8 3.0 1.4 0.1
14 4.3 3.0 1.1 0.1
15 5.8 4.0 1.2 0.2
16 5.7 4.4 1.5 0.4
17 5.4 3.9 1.3 0.4
18 5.1 3.5 1.4 0.3
19 5.7 3.8 1.7 0.3
20 5.1 3.8 1.5 0.3
21 5.4 3.4 1.7 0.2
22 5.1 3.7 1.5 0.4
23 4.6 3.6 1.0 0.2
24 5.1 3.3 1.7 0.5
25 4.8 3.4 1.9 0.2
26 5.0 3.0 1.6 0.2
27 5.0 3.4 1.6 0.4
28 5.2 3.5 1.5 0.2
29 5.2 3.4 1.4 0.2
30 4.7 3.2 1.6 0.2
31 4.8 3.1 1.6 0.2
32 5.4 3.4 1.5 0.4
33 5.2 4.1 1.5 0.1
34 5.5 4.2 1.4 0.2
35 4.9 3.1 1.5 0.2
36 5.0 3.2 1.2 0.2
37 5.5 3.5 1.3 0.2
38 4.9 3.6 1.4 0.1
39 4.4 3.0 1.3 0.2
40 5.1 3.4 1.5 0.2
41 5.0 3.5 1.3 0.3
42 4.5 2.3 1.3 0.3
43 4.4 3.2 1.3 0.2
44 5.0 3.5 1.6 0.6
45 5.1 3.8 1.9 0.4
46 4.8 3.0 1.4 0.3
47 5.1 3.8 1.6 0.2
48 4.6 3.2 1.4 0.2
49 5.3 3.7 1.5 0.2
50 5.0 3.3 1.4 0.2
51 7.0 3.2 4.7 1.4
52 6.4 3.2 4.5 1.5
53 6.9 3.1 4.9 1.5
54 5.5 2.3 4.0 1.3
55 6.5 2.8 4.6 1.5
56 5.7 2.8 4.5 1.3
57 6.3 3.3 4.7 1.6
58 4.9 2.4 3.3 1.0
59 6.6 2.9 4.6 1.3
60 5.2 2.7 3.9 1.4
61 5.0 2.0 3.5 1.0
62 5.9 3.0 4.2 1.5
63 6.0 2.2 4.0 1.0
64 6.1 2.9 4.7 1.4
65 5.6 2.9 3.6 1.3
66 6.7 3.1 4.4 1.4
67 5.6 3.0 4.5 1.5
68 5.8 2.7 4.1 1.0
69 6.2 2.2 4.5 1.5
70 5.6 2.5 3.9 1.1
71 5.9 3.2 4.8 1.8
72 6.1 2.8 4.0 1.3
73 6.3 2.5 4.9 1.5
74 6.1 2.8 4.7 1.2
75 6.4 2.9 4.3 1.3
76 6.6 3.0 4.4 1.4
77 6.8 2.8 4.8 1.4
78 6.7 3.0 5.0 1.7
79 6.0 2.9 4.5 1.5
80 5.7 2.6 3.5 1.0
81 5.5 2.4 3.8 1.1
82 5.5 2.4 3.7 1.0
83 5.8 2.7 3.9 1.2
84 6.0 2.7 5.1 1.6
85 5.4 3.0 4.5 1.5
86 6.0 3.4 4.5 1.6
87 6.7 3.1 4.7 1.5
88 6.3 2.3 4.4 1.3
89 5.6 3.0 4.1 1.3
90 5.5 2.5 4.0 1.3
91 5.5 2.6 4.4 1.2
92 6.1 3.0 4.6 1.4
93 5.8 2.6 4.0 1.2
94 5.0 2.3 3.3 1.0
95 5.6 2.7 4.2 1.3
96 5.7 3.0 4.2 1.2
97 5.7 2.9 4.2 1.3
98 6.2 2.9 4.3 1.3
99 5.1 2.5 3.0 1.1
100 5.7 2.8 4.1 1.3
101 6.3 3.3 6.0 2.5
102 5.8 2.7 5.1 1.9
103 7.1 3.0 5.9 2.1
104 6.3 2.9 5.6 1.8
105 6.5 3.0 5.8 2.2
106 7.6 3.0 6.6 2.1
107 4.9 2.5 4.5 1.7
108 7.3 2.9 6.3 1.8
109 6.7 2.5 5.8 1.8
110 7.2 3.6 6.1 2.5
111 6.5 3.2 5.1 2.0
112 6.4 2.7 5.3 1.9
113 6.8 3.0 5.5 2.1
114 5.7 2.5 5.0 2.0
115 5.8 2.8 5.1 2.4
116 6.4 3.2 5.3 2.3
117 6.5 3.0 5.5 1.8
118 7.7 3.8 6.7 2.2
119 7.7 2.6 6.9 2.3
120 6.0 2.2 5.0 1.5
121 6.9 3.2 5.7 2.3
122 5.6 2.8 4.9 2.0
123 7.7 2.8 6.7 2.0
124 6.3 2.7 4.9 1.8
125 6.7 3.3 5.7 2.1
126 7.2 3.2 6.0 1.8
127 6.2 2.8 4.8 1.8
128 6.1 3.0 4.9 1.8
129 6.4 2.8 5.6 2.1
130 7.2 3.0 5.8 1.6
131 7.4 2.8 6.1 1.9
132 7.9 3.8 6.4 2.0
133 6.4 2.8 5.6 2.2
134 6.3 2.8 5.1 1.5
135 6.1 2.6 5.6 1.4
136 7.7 3.0 6.1 2.3
137 6.3 3.4 5.6 2.4
138 6.4 3.1 5.5 1.8
139 6.0 3.0 4.8 1.8
140 6.9 3.1 5.4 2.1
141 6.7 3.1 5.6 2.4
142 6.9 3.1 5.1 2.3
143 5.8 2.7 5.1 1.9
144 6.8 3.2 5.9 2.3
145 6.7 3.3 5.7 2.5
146 6.7 3.0 5.2 2.3
147 6.3 2.5 5.0 1.9
148 6.5 3.0 5.2 2.0
149 6.2 3.4 5.4 2.3
150 5.9 3.0 5.1 1.8
As you can see it is split based on the classes factor and numeric.
I'm trying to flag rows that have values that match elements in 2 external vectors. I need these values to match, meaning it can't be any case, but something where both conditions are met (i.e. not using an %in% operator).
Here's what I've tried:
widths <- c(1.2, 1.7, 1.8, 1.8, 1.9)
species <- c(rep("versicolor", 3), rep("virginica", 2))
iris_flagged <- iris %>%
filter(Species != "setosa") %>%
mutate(flag = ifelse((Petal.Width == widths & Species == species), "check", "")) %>%
arrange(Species, Petal.Width) #to help with visualization
But when I run this some flags are missed. If you look at the data you'll see that rows 14, 50, 57, 59, 61:64, and 66:71 are missing flags.
A clunky, non-robust method could look like this, but I really want to make it robust and use these vectors to my advantage.
iris_flagged2 <- iris %>%
filter(Species != "setosa") %>%
mutate(flag = ifelse((Species == "versicolor" & Petal.Width == 1.2 |
Species == "versicolor" & Petal.Width == 1.7 |
Species == "versicolor" & Petal.Width == 1.8 |
Species == "virginica" & Petal.Width == 1.8 |
Species == "virginica" & Petal.Width == 1.9),
"check", "")) %>%
arrange(Species, Petal.Width)
Thanks for your help. I tried to search but wasn't sure how to phrase my question.
Here is one way using map2 to pass each pair of condition and reduce to combine them into one.
library(tidyverse)
iris2 <- iris %>% filter(Species != "setosa")
iris2 <- iris2 %>%
mutate(flag = ifelse(map2(species, widths,
~Species == .x & Petal.Width == .y) %>% reduce(`|`), 'check', ''))
iris2
Additional to the comment of #akrun, here is a possible solution for your task:
library(dplyr)
iris %>%
mutate(across(-Species, ~round(.,1)),
flag = ifelse(Petal.Width %in% widths &
Species %in% species, "check",""))
output:
Sepal.Length Sepal.Width Petal.Length Petal.Width Species flag
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
21 5.4 3.4 1.7 0.2 setosa
22 5.1 3.7 1.5 0.4 setosa
23 4.6 3.6 1.0 0.2 setosa
24 5.1 3.3 1.7 0.5 setosa
25 4.8 3.4 1.9 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
33 5.2 4.1 1.5 0.1 setosa
34 5.5 4.2 1.4 0.2 setosa
35 4.9 3.1 1.5 0.2 setosa
36 5.0 3.2 1.2 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
39 4.4 3.0 1.3 0.2 setosa
40 5.1 3.4 1.5 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
42 4.5 2.3 1.3 0.3 setosa
43 4.4 3.2 1.3 0.2 setosa
44 5.0 3.5 1.6 0.6 setosa
45 5.1 3.8 1.9 0.4 setosa
46 4.8 3.0 1.4 0.3 setosa
47 5.1 3.8 1.6 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
49 5.3 3.7 1.5 0.2 setosa
50 5.0 3.3 1.4 0.2 setosa
51 7.0 3.2 4.7 1.4 versicolor
52 6.4 3.2 4.5 1.5 versicolor
53 6.9 3.1 4.9 1.5 versicolor
54 5.5 2.3 4.0 1.3 versicolor
55 6.5 2.8 4.6 1.5 versicolor
56 5.7 2.8 4.5 1.3 versicolor
57 6.3 3.3 4.7 1.6 versicolor
58 4.9 2.4 3.3 1.0 versicolor
59 6.6 2.9 4.6 1.3 versicolor
60 5.2 2.7 3.9 1.4 versicolor
61 5.0 2.0 3.5 1.0 versicolor
62 5.9 3.0 4.2 1.5 versicolor
63 6.0 2.2 4.0 1.0 versicolor
64 6.1 2.9 4.7 1.4 versicolor
65 5.6 2.9 3.6 1.3 versicolor
66 6.7 3.1 4.4 1.4 versicolor
67 5.6 3.0 4.5 1.5 versicolor
68 5.8 2.7 4.1 1.0 versicolor
69 6.2 2.2 4.5 1.5 versicolor
70 5.6 2.5 3.9 1.1 versicolor
71 5.9 3.2 4.8 1.8 versicolor check
72 6.1 2.8 4.0 1.3 versicolor
73 6.3 2.5 4.9 1.5 versicolor
74 6.1 2.8 4.7 1.2 versicolor check
75 6.4 2.9 4.3 1.3 versicolor
76 6.6 3.0 4.4 1.4 versicolor
77 6.8 2.8 4.8 1.4 versicolor
78 6.7 3.0 5.0 1.7 versicolor check
79 6.0 2.9 4.5 1.5 versicolor
80 5.7 2.6 3.5 1.0 versicolor
81 5.5 2.4 3.8 1.1 versicolor
82 5.5 2.4 3.7 1.0 versicolor
83 5.8 2.7 3.9 1.2 versicolor check
84 6.0 2.7 5.1 1.6 versicolor
85 5.4 3.0 4.5 1.5 versicolor
86 6.0 3.4 4.5 1.6 versicolor
87 6.7 3.1 4.7 1.5 versicolor
88 6.3 2.3 4.4 1.3 versicolor
89 5.6 3.0 4.1 1.3 versicolor
90 5.5 2.5 4.0 1.3 versicolor
91 5.5 2.6 4.4 1.2 versicolor check
92 6.1 3.0 4.6 1.4 versicolor
93 5.8 2.6 4.0 1.2 versicolor check
94 5.0 2.3 3.3 1.0 versicolor
95 5.6 2.7 4.2 1.3 versicolor
96 5.7 3.0 4.2 1.2 versicolor check
97 5.7 2.9 4.2 1.3 versicolor
98 6.2 2.9 4.3 1.3 versicolor
99 5.1 2.5 3.0 1.1 versicolor
100 5.7 2.8 4.1 1.3 versicolor
101 6.3 3.3 6.0 2.5 virginica
102 5.8 2.7 5.1 1.9 virginica check
103 7.1 3.0 5.9 2.1 virginica
104 6.3 2.9 5.6 1.8 virginica check
105 6.5 3.0 5.8 2.2 virginica
106 7.6 3.0 6.6 2.1 virginica
107 4.9 2.5 4.5 1.7 virginica check
108 7.3 2.9 6.3 1.8 virginica check
109 6.7 2.5 5.8 1.8 virginica check
110 7.2 3.6 6.1 2.5 virginica
111 6.5 3.2 5.1 2.0 virginica
112 6.4 2.7 5.3 1.9 virginica check
113 6.8 3.0 5.5 2.1 virginica
114 5.7 2.5 5.0 2.0 virginica
115 5.8 2.8 5.1 2.4 virginica
116 6.4 3.2 5.3 2.3 virginica
117 6.5 3.0 5.5 1.8 virginica check
118 7.7 3.8 6.7 2.2 virginica
119 7.7 2.6 6.9 2.3 virginica
120 6.0 2.2 5.0 1.5 virginica
121 6.9 3.2 5.7 2.3 virginica
122 5.6 2.8 4.9 2.0 virginica
123 7.7 2.8 6.7 2.0 virginica
124 6.3 2.7 4.9 1.8 virginica check
125 6.7 3.3 5.7 2.1 virginica
126 7.2 3.2 6.0 1.8 virginica check
127 6.2 2.8 4.8 1.8 virginica check
128 6.1 3.0 4.9 1.8 virginica check
129 6.4 2.8 5.6 2.1 virginica
130 7.2 3.0 5.8 1.6 virginica
131 7.4 2.8 6.1 1.9 virginica check
132 7.9 3.8 6.4 2.0 virginica
133 6.4 2.8 5.6 2.2 virginica
134 6.3 2.8 5.1 1.5 virginica
135 6.1 2.6 5.6 1.4 virginica
136 7.7 3.0 6.1 2.3 virginica
137 6.3 3.4 5.6 2.4 virginica
138 6.4 3.1 5.5 1.8 virginica check
139 6.0 3.0 4.8 1.8 virginica check
140 6.9 3.1 5.4 2.1 virginica
141 6.7 3.1 5.6 2.4 virginica
142 6.9 3.1 5.1 2.3 virginica
143 5.8 2.7 5.1 1.9 virginica check
144 6.8 3.2 5.9 2.3 virginica
145 6.7 3.3 5.7 2.5 virginica
146 6.7 3.0 5.2 2.3 virginica
147 6.3 2.5 5.0 1.9 virginica check
148 6.5 3.0 5.2 2.0 virginica
149 6.2 3.4 5.4 2.3 virginica
150 5.9 3.0 5.1 1.8 virginica check
This question already has an answer here:
Subset rows corresponding to max value by group using data.table
(1 answer)
Closed 7 years ago.
I understand that data.table allows you to do computations based on groups within a column. For example.
Reproducible example
iris[,.SD[which.min(Petal.Width)], by=Species]
generating
Species Sepal.Length Sepal.Width Petal.Length Petal.Width
1: setosa 4.9 3.1 1.5 0.1
2: versicolor 4.9 2.4 3.3 1.0
3: virginica 6.1 2.6 5.6 1.4
I want every row where the minimum is met; not just the first, something that is easily achieved in a DF:
for example this:
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
10 4.9 3.1 1.5 0.1 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
33 5.2 4.1 1.5 0.1 setosa
38 4.9 3.6 1.4 0.1 setosa
58 4.9 2.4 3.3 1.0 versicolor
61 5.0 2.0 3.5 1.0 versicolor
63 6.0 2.2 4.0 1.0 versicolor
68 5.8 2.7 4.1 1.0 versicolor
80 5.7 2.6 3.5 1.0 versicolor
82 5.5 2.4 3.7 1.0 versicolor
94 5.0 2.3 3.3 1.0 versicolor
135 6.1 2.6 5.6 1.4 virginica
What I don't want is just the first instance of where the minima is met:
This would be equivalent to doing something like this using a data.frame
iris
iris <- as.data.frame(iris) #in case reader does not start new R session
f.min <- function(spec) {
spec.sub <- iris[iris$Species==spec,]
min.rows <- spec.sub[spec.sub$Petal.Width == min(spec.sub$Petal.Width),]
}
do.call(rbind, lapply(levels(iris$Species), f.min ))
There are some powerful features in data.table which are worth learning. Hence why I would like to know the equivalent in data.table.
Try:
iris[,.SD[which.min(Petal.Width)], by=Species]
This will give you the minimas but does not show ties.
Species Sepal.Length Sepal.Width Petal.Length Petal.Width
1: setosa 4.9 3.1 1.5 0.1
2: versicolor 4.9 2.4 3.3 1.0
3: virginica 6.1 2.6 5.6 1.4
A dplyr solution showing the ties as well would be:
require(dplyr)
require(magrittr)
iris %>%
group_by(Species) %>%
filter(rank(Petal.Width, ties.method= "min") == 1)
Source: local data table [13 x 5]
Groups: Species
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 4.9 3.1 1.5 0.1 setosa
2 4.8 3.0 1.4 0.1 setosa
3 4.3 3.0 1.1 0.1 setosa
4 5.2 4.1 1.5 0.1 setosa
5 4.9 3.6 1.4 0.1 setosa
6 4.9 2.4 3.3 1.0 versicolor
7 5.0 2.0 3.5 1.0 versicolor
8 6.0 2.2 4.0 1.0 versicolor
9 5.8 2.7 4.1 1.0 versicolor
10 5.7 2.6 3.5 1.0 versicolor
11 5.5 2.4 3.7 1.0 versicolor
12 5.0 2.3 3.3 1.0 versicolor
13 6.1 2.6 5.6 1.4 virginica
The 'ties.method' parameter is where you can select what should be displayed.
Hope this helps.
Data
Please, I need to calculate the mean of the column "Sepal.Length" for the specie virginica in this data.frame:
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
21 5.4 3.4 1.7 0.2 setosa
22 5.1 3.7 1.5 0.4 setosa
23 4.6 3.6 1.0 0.2 setosa
24 5.1 3.3 1.7 0.5 setosa
25 4.8 3.4 1.9 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
33 5.2 4.1 1.5 0.1 setosa
34 5.5 4.2 1.4 0.2 setosa
35 4.9 3.1 1.5 0.2 setosa
36 5.0 3.2 1.2 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
39 4.4 3.0 1.3 0.2 setosa
40 5.1 3.4 1.5 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
42 4.5 2.3 1.3 0.3 setosa
43 4.4 3.2 1.3 0.2 setosa
44 5.0 3.5 1.6 0.6 setosa
45 5.1 3.8 1.9 0.4 setosa
46 4.8 3.0 1.4 0.3 setosa
47 5.1 3.8 1.6 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
49 5.3 3.7 1.5 0.2 setosa
50 5.0 3.3 1.4 0.2 setosa
51 7.0 3.2 4.7 1.4 versicolor
52 6.4 3.2 4.5 1.5 versicolor
53 6.9 3.1 4.9 1.5 versicolor
54 5.5 2.3 4.0 1.3 versicolor
55 6.5 2.8 4.6 1.5 versicolor
56 5.7 2.8 4.5 1.3 versicolor
57 6.3 3.3 4.7 1.6 versicolor
58 4.9 2.4 3.3 1.0 versicolor
59 6.6 2.9 4.6 1.3 versicolor
60 5.2 2.7 3.9 1.4 versicolor
61 5.0 2.0 3.5 1.0 versicolor
62 5.9 3.0 4.2 1.5 versicolor
63 6.0 2.2 4.0 1.0 versicolor
64 6.1 2.9 4.7 1.4 versicolor
65 5.6 2.9 3.6 1.3 versicolor
66 6.7 3.1 4.4 1.4 versicolor
67 5.6 3.0 4.5 1.5 versicolor
68 5.8 2.7 4.1 1.0 versicolor
69 6.2 2.2 4.5 1.5 versicolor
70 5.6 2.5 3.9 1.1 versicolor
71 5.9 3.2 4.8 1.8 versicolor
72 6.1 2.8 4.0 1.3 versicolor
73 6.3 2.5 4.9 1.5 versicolor
74 6.1 2.8 4.7 1.2 versicolor
75 6.4 2.9 4.3 1.3 versicolor
76 6.6 3.0 4.4 1.4 versicolor
77 6.8 2.8 4.8 1.4 versicolor
78 6.7 3.0 5.0 1.7 versicolor
79 6.0 2.9 4.5 1.5 versicolor
80 5.7 2.6 3.5 1.0 versicolor
81 5.5 2.4 3.8 1.1 versicolor
82 5.5 2.4 3.7 1.0 versicolor
83 5.8 2.7 3.9 1.2 versicolor
84 6.0 2.7 5.1 1.6 versicolor
85 5.4 3.0 4.5 1.5 versicolor
86 6.0 3.4 4.5 1.6 versicolor
87 6.7 3.1 4.7 1.5 versicolor
88 6.3 2.3 4.4 1.3 versicolor
89 5.6 3.0 4.1 1.3 versicolor
90 5.5 2.5 4.0 1.3 versicolor
91 5.5 2.6 4.4 1.2 versicolor
92 6.1 3.0 4.6 1.4 versicolor
93 5.8 2.6 4.0 1.2 versicolor
94 5.0 2.3 3.3 1.0 versicolor
95 5.6 2.7 4.2 1.3 versicolor
96 5.7 3.0 4.2 1.2 versicolor
97 5.7 2.9 4.2 1.3 versicolor
98 6.2 2.9 4.3 1.3 versicolor
99 5.1 2.5 3.0 1.1 versicolor
100 5.7 2.8 4.1 1.3 versicolor
101 6.3 3.3 6.0 2.5 virginica
102 5.8 2.7 5.1 1.9 virginica
103 7.1 3.0 5.9 2.1 virginica
104 6.3 2.9 5.6 1.8 virginica
105 6.5 3.0 5.8 2.2 virginica
106 7.6 3.0 6.6 2.1 virginica
107 4.9 2.5 4.5 1.7 virginica
108 7.3 2.9 6.3 1.8 virginica
109 6.7 2.5 5.8 1.8 virginica
110 7.2 3.6 6.1 2.5 virginica
111 6.5 3.2 5.1 2.0 virginica
112 6.4 2.7 5.3 1.9 virginica
113 6.8 3.0 5.5 2.1 virginica
114 5.7 2.5 5.0 2.0 virginica
115 5.8 2.8 5.1 2.4 virginica
116 6.4 3.2 5.3 2.3 virginica
117 6.5 3.0 5.5 1.8 virginica
118 7.7 3.8 6.7 2.2 virginica
119 7.7 2.6 6.9 2.3 virginica
120 6.0 2.2 5.0 1.5 virginica
121 6.9 3.2 5.7 2.3 virginica
122 5.6 2.8 4.9 2.0 virginica
123 7.7 2.8 6.7 2.0 virginica
124 6.3 2.7 4.9 1.8 virginica
125 6.7 3.3 5.7 2.1 virginica
126 7.2 3.2 6.0 1.8 virginica
127 6.2 2.8 4.8 1.8 virginica
128 6.1 3.0 4.9 1.8 virginica
129 6.4 2.8 5.6 2.1 virginica
130 7.2 3.0 5.8 1.6 virginica
131 7.4 2.8 6.1 1.9 virginica
132 7.9 3.8 6.4 2.0 virginica
133 6.4 2.8 5.6 2.2 virginica
134 6.3 2.8 5.1 1.5 virginica
135 6.1 2.6 5.6 1.4 virginica
136 7.7 3.0 6.1 2.3 virginica
137 6.3 3.4 5.6 2.4 virginica
138 6.4 3.1 5.5 1.8 virginica
139 6.0 3.0 4.8 1.8 virginica
140 6.9 3.1 5.4 2.1 virginica
141 6.7 3.1 5.6 2.4 virginica
142 6.9 3.1 5.1 2.3 virginica
143 5.8 2.7 5.1 1.9 virginica
144 6.8 3.2 5.9 2.3 virginica
145 6.7 3.3 5.7 2.5 virginica
146 6.7 3.0 5.2 2.3 virginica
147 6.3 2.5 5.0 1.9 virginica
148 6.5 3.0 5.2 2.0 virginica
149 6.2 3.4 5.4 2.3 virginica
150 5.9 3.0 5.1 1.8 virginica
When I used sapply(s, function(x) colMeans(x[, c("virginica")], na.rm =TRUE))
I get this error:
Error in `[.data.frame`(x, , c("virginica")) : undefined columns selected
When I use sapply(split(iris[1:2],iris$Specie),mean, na.rm = TRUE)
I get this response:
setosa versicolor virginica
NA NA NA
Mensajes de aviso perdidos
1: In mean.default(X[[1L]], ...) :
argument is not numeric or logical: returning NA
2: In mean.default(X[[2L]], ...) :
argument is not numeric or logical: returning NA
3: In mean.default(X[[3L]], ...) :
argument is not numeric or logical: returning NA
AND
When I use apply(iris,2,mean)
I get this:
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
NA NA NA NA NA
Mensajes de aviso perdidos
1: In mean.default(newX[, i], ...) :
argument is not numeric or logical: returning NA
2: In mean.default(newX[, i], ...) :
argument is not numeric or logical: returning NA
3: In mean.default(newX[, i], ...) :
argument is not numeric or logical: returning NA
4: In mean.default(newX[, i], ...) :
argument is not numeric or logical: returning NA
5: In mean.default(newX[, i], ...) :
argument is not numeric or logical: returning NA
I am new in programming.
Thanks in advance!!
In order to correct the errors, you could use:
colMeans(subset(s, Species=="virginica", select=-Species), na.rm=TRUE)
#Sepal.Length Sepal.Width Petal.Length Petal.Width
# 6.588 2.974 5.552 2.026
For the second case:
sapply(split(iris[1:2], iris$Species), colMeans, na.rm=TRUE)
# setosa versicolor virginica
#Sepal.Length 5.006 5.936 6.588
#Sepal.Width 3.428 2.770 2.974
mean would not work the same way as min, max in a data.frame
dat <- data.frame(col1=1:5, col2=6:10)
mean(dat)
#[1] NA
Also, your title indicates you need column means, So, it is better to use colMeans. But, if you are looking for a single value from all the columns just like min
min(dat)
#[1] 1
mean(colMeans(dat))
#[1] 5.5
There are several methods to get the mean by group. If you have several columns,
library(dplyr)
s%>%
group_by(Species) %>%
summarise_each(funs(mean=mean(., na.rm=TRUE)))
#Source: local data frame [3 x 5]
# Species Sepal.Length Sepal.Width Petal.Length Petal.Width
#1 setosa 5.006 3.428 1.462 0.246
#2 versicolor 5.936 2.770 4.260 1.326
#3 virginica 6.588 2.974 5.552 2.026
Or
library(data.table)
setDT(s)[, lapply(.SD, mean, na.rm=TRUE),by='Species']
# Species Sepal.Length Sepal.Width Petal.Length Petal.Width
#1: setosa 5.006 3.428 1.462 0.246
#2: versicolor 5.936 2.770 4.260 1.326
#3: virginica 6.588 2.974 5.552 2.026
Try:
mean( iris[iris$Species=='virginica', ]$Sepal.Length )
[1] 6.588
or:
mean(iris[iris$Species=='virginica',1])
[1] 6.588
For mean of all species:
with(iris, tapply(Sepal.Length, Species, mean))
setosa versicolor virginica
5.006 5.936 6.588
For data.frame output:
with(iris, aggregate(Sepal.Length~Species, FUN=mean))
Species Sepal.Length
1 setosa 5.006
2 versicolor 5.936
3 virginica 6.588
For this species vs others:
by(iris[,1], iris$Species=='virginica', mean)
iris$Species == "virginica": FALSE
[1] 5.471
-------------------------------------------------------------------------------------------------
iris$Species == "virginica": TRUE
[1] 6.588
How about using by()?
### Get mean for the 1st-4th column for each level of Species
by(iris[,1:4], iris$Species, colMeans)