Suppose I have a data.table called mysample. It has multiple columns, two of them being weight and height. I can access the weight column by typing:
mysample[,weight]
But when I try to write mysample[,colnames(mysample)[1]] I cannot see the elements of weight. Is there something wrong with my code?
Please refer to section 1.1 of data.table FAQ: http://cran.r-project.org/web/packages/data.table/vignettes/datatable-faq.pdf
colnames(mysample)[1] evaluates to character vector "weight", and the 2nd argument J in data.table is an expression which is evaluated within the scope of DT. Thus, "weight" evaluates to character vector "weight" itself and you can't see the elements of "weight" column. To actually subset "weight" column you should try:
mysample[,colnames(mysample)[1], with = F]
Your syntax should work for data frames. data.table has its unique rules.
df <- data.frame(a=1:3, b=4:6)
df
a b
1 1 4
2 2 5
3 3 6
df[,"a"]
[1] 1 2 3
df$a
[1] 1 2 3
df[,1]
[1] 1 2 3
df[,colnames(df)[1]]
[1] 1 2 3
Related
I'm having trouble understanding both the reason for use and behavior of the $ symbol in subsetting a data.frame in R. The following example was presented in a beginner's class I'm taking (not with a live professor so can't ask there):
temp_mat <- matrix(1:9, nrow=3)
colnames(temp_mat) <- c('a', 'b', 'c')
temp_df <- data.frame(temp_mat)
Calling temp_df obviously outputs:
a b c
1 1 4 7
2 2 5 8
3 3 6 9
The example given in the course is then:
temp_df[temp_df$c < 10]
Which outputs:
a b c
1 1 4 7
2 2 5 8
3 3 6 9
Reason for use question: The course indicates that $ is used for partial matching, and that x$y is an exact substitute for x[["y", exact=FALSE]]. Why would we want to use a partial matching operator here? Do we use it because we know for sure that in our temp_df there is no other column similar to "c" that could be mistakenly picked up? Additionally, how is partial match measured? A minimum % of characters matching or something? It appears there is a getElement function that would be much more appropriate if working with datasets with unknown or similar column names (e.g. Home Phone versus Cell Phone, would these be seen as a valid partial match?)
Behavior question: it appears the above example temp_df[temp_df$c < 10] is saying "return the subset of elements from temp_df where column c is less than 10" and because all column c elements meet the criteria, the entire dataframe is returned. My interpretation is obviously wrong because temp_df[temp_df$c < 9] returns:
a b
1 1 4
2 2 5
3 3 6
Although the row 1 and 2 elements in column c do meet the criteria of being less than 9, the entire column is omitted. My question then becomes twofold: what is that logical vector actually saying/doing? And how would I write my interpretation of "return the subset of elements from temp_df where column c is less than 9" and have it return:
a b c
1 1 4 7
2 2 5 8
Because in my mind, elements 1 and 2 (rows 1 and 2) met that criteria as their column c values are less than 9 and thus should be returned.
Try breaking down the operation in steps.
temp_df$c < 9
gives a vector as follows:
[1] TRUE TRUE FALSE
When you pass this vector in the manner you have shown:
temp_df[c(TRUE, TRUE, FALSE)] has the effect of operating on columns.
Think about a data.frame as a list, with column names as the keys and the column contents as vector values. The operation preserves the TRUE keys (i.e. columns) and drops the FALSE.
The comma serves to mark the vector as row index. The first two rows are retained and the last one is dropped. Thus, temp_df[c(TRUE, TRUE, FALSE), ] gives:
a b c
1 1 4 7
2 2 5 8
Both the $ and [[ are extract operator which allows to extract elements by name.
OP has raised one query about behavior of exact argument. The exact argument of the [[ operator has been documented in RStudio as:
Controls possible partial matching of [[ when extracting by a
character vector (for most objects, but see under ‘Environments’). The
default is no partial matching. Value NA allows partial matching but
issues a warning when it occurs. Value FALSE allows partial matching
without any warning.
What does it mean? To understand its behavior lets change the column names of data.frame used by OP as:
names(temp_df) <- c("aa","bb","cc")
#partial name of column will work with exact = FALSE
temp_df[["a", exact = FALSE]]
#[1] 1 2 3
#partial name of column will not work with exact = TRUE
temp_df[["a", exact = TRUE]]
#NULL
temp_df[["a", exact = NA]]
#[1] 1 2 3
#Warning message:
#In .subset2(x, i, exact = exact) : partial match of 'a' to 'aa'
Edit
This question seems to be a duplicate of the question How to group a vector into a list of vectors?, and the answer split(df$b, df$id) was suggested. First happy with the solution, I realized that the given answers do not fully address my question. In the below question, I would like to obtain a list in which the vector elements are assigned to the value of a third column (in my example df$a). This is important, as otherwise the order of df$b plays a role. I mean obviously I can arrange by df$a and then call split(), but maybe there is another way of doing that.
My sample df:
df <- data_frame(id = paste0('id',rep(1:2, each = 5)), a = rep(letters[1:5],2),b=c(1:5,5:1))
Df should be grouped by ID (in df$id). I would like to create a list of vectors for each group (id) element that contains the values of df$b. My approach
require(tidyr)
spread_df <- df %>% spread(id,b) #makes new columns for each id
#loop over spread_df
for (i in 1:length(spread_df)) {
list_group_elements [i]<- list(spread_df[[i]])
#I want each vector to be identified by the identifier of column df$a
#therefore:
names(list_group_elements[[i]]) <- list_group_elements[[1]]
}
This results in :
list_group_elements
[[1]]
a b c d e
"a" "b" "c" "d" "e"
[[2]]
a b c d e
1 2 3 4 5
[[3]]
a b c d e
5 4 3 2 1
I don't need the first element of the list, but the rest is basically what I need. I have the peculiar impression that my approach is somewhat not ideal and if someone has an idea to improve this, (e.g., with dplyr?) this would be highly appreciated. Why do I want this: I made a function that uses vectors as arguments and I would like to run this function over certain columns from dataframes - but only using the grouped values as arguments and not the entire column.
You may make df$b a named vector using setNames, and then split it into a list:
split(setNames(df$b, df$a), df$id)
# $id1
# a b c d e
# 1 2 3 4 5
#
# $id2
# a b c d e
# 5 4 3 2 1
One way is
lapply(levels(df$id), function(L) df$b[df$id == L])
[[1]]
[1] 1 2 3 4 5
[[2]]
[1] 5 4 3 2 1
Consider by, object-oriented wrapper of tapply, designed to split dataframe by factor(s):
by(df, df$id, FUN=function(i) i$b)
I have a dataframe
names2 <- c('AdagioBarber','AdagioBarber', 'Beethovan','Beethovan')
Value <- c(33,55,21,54)
song.data <- data.frame(names2,Value)
I would like to arrange it according to this character vector
names <- c('Beethovan','Beethovan','AdagioBarber','AdagioBarber')
I am using match() to achieve this
data.frame(song.data[match((names), (song.data$names2)),])
The problem is that match returns only first occurences
names2 Value
3 Beethovan 21
3.1 Beethovan 21
1 AdagioBarber 33
1.1 AdagioBarber 33
You can use order, as #zx8754 and #Evan Friedland have pointed out.
> name.order <- c('Beethovan','AdagioBarber')
> song.data$names2 <- factor(song.data$names2, levels= name.order)
> song.data[order(song.data$names2), ]
names2 Value
3 Beethovan 21
4 Beethovan 54
1 AdagioBarber 33
2 AdagioBarber 55
Basically, factor turns the strings into integers and creates a lookup table of what integers correspond to what strings. The levels argument specifies what you want that lookup table to be. Without that argument, it would just go by order of appearance.
So for example:
> as.numeric(factor(letters[1:5]))
[1] 1 2 3 4 5
> as.numeric(factor(letters[1:5], levels=c("d","b","e","a","c")))
[1] 4 2 5 1 3
Note: You'll need to be absolutely sure you get all your (correctly spelled) levels in that name.order vector, otherwise you'll end up with NA's in the output from order.
(I'm not sure why sort doesn't have the ability to sort factors, but it is what it is.)
This question already has answers here:
How to calculate the number of occurrence of a given character in each row of a column of strings?
(14 answers)
Closed 6 years ago.
suppose I have a long string such like:
c<-"abcabcdabcdeabcdefghijkabcdabcaba"
My question is how to quickly count the number of exact "abcd" in c.
1) gregexpr First paste "abcd" onto c so that there is at least 1 match. (This is needed because gregexpr returns -1 for any component of c having no matches rather than a zero length numeric vector.) Now, gregexpr returns a list whose components are numeric vectors of the starting positions of the matches one component per component of c -- in this case c only has one component but the code below works more generally. Now find the lengths of the components of the result of gregexpr and subtract 1 to take into account the extra abcd we added. No packages are used.
Example 1
lengths(gregexpr("abcd", paste(c, "abcd"))) - 1
## [1] 4
Note: If we knew that there was at least one match it could be slightly simplified to: lengths(gregexpr("abcd", c)) .
Example 2
Here is another example. Here DF has 3 rows and the corresponding components of c have 4, 4, and 0 occurrences of "abcd".
DF <- data.frame(c = c(c, c, "X")) # test input
lengths(gregexpr("abcd", paste(DF$c, "abcd"))) - 1
## [1] 4 4 0
2) regmatches
Here is an alternative approach. This approach has the advantage that no special code is needed for the no-match case. Again, no packages are used.
Here are the same two examples:
lengths(regmatches(c, gregexpr("abcd", c)))
## [1] 4
lengths(regmatches(DF$c, gregexpr("abcd", DF$c)))
## [1] 4 4 0
Using library stringr, you can do it as follows (on larger set, it will be fairly fast and efficient):
library(stringr)
c <- "abcabcdabcdeabcdefghijkabcdabcaba"
c
[1] "abcabcdabcdeabcdefghijkabcdabcaba"
str_count(c, 'abcd')
[1] 4
This will work on a column of a data frame as follows:
df <- data.frame(txt = rep(c, 10))
df$abcd_count <- str_count(df$txt, 'abcd')
df
txt abcd_count
1 abcabcdabcdeabcdefghijkabcdabcaba 4
2 abcabcdabcdeabcdefghijkabcdabcaba 4
3 abcabcdabcdeabcdefghijkabcdabcaba 4
4 abcabcdabcdeabcdefghijkabcdabcaba 4
5 abcabcdabcdeabcdefghijkabcdabcaba 4
6 abcabcdabcdeabcdefghijkabcdabcaba 4
7 abcabcdabcdeabcdefghijkabcdabcaba 4
8 abcabcdabcdeabcdefghijkabcdabcaba 4
9 abcabcdabcdeabcdefghijkabcdabcaba 4
10 abcabcdabcdeabcdefghijkabcdabcaba 4
Here is one method using base Rs gsub and strsplit:
# example
temp <- "abcabcdabcdeabcdefghijkabcdabcaba"
# substitute pattern for character not in string, here 9
temp2 <- gsub("abcd", "9", temp)
# split on 9, and count number of elements
length(strsplit(temp2, split="9")[[1]]) - 1
You need the [[1]] because strsplit is designed to operate over vectors of strings, here the vector is of length 1. An alternative to [[1]] in this case is unlist.
Also, 1 is subtracted because the number of elements are one larger than the number of abcd patterns by 1.
My data frame contains the output of a survey with a select multiple question type. Some cells have multiple values.
df <- data.frame(a=1:3,b=I(list(1,1:2,1:3)))
df
a b
1 1 1
2 2 1, 2
3 3 1, 2, 3
I would like to flatten out the list to obtain the following output:
df
a b
1 1 1
2 2 1
3 2 2
4 3 1
5 3 2
6 3 3
should be easy but somehow I can't find the search terms. thanks.
You can just use unnest from "tidyr":
library(tidyr)
unnest(df, b)
# a b
# 1 1 1
# 2 2 1
# 3 2 2
# 4 3 1
# 5 3 2
# 6 3 3
Using base R, one option is stack after naming the list elements of 'b' column with that of the elements of 'a'. We can use setNames to change the names.
stack(setNames(df$b, df$a))
Or another option would be to use unstack to automatically name the list element of 'b' with 'a' elements and then do the stack to get a data.frame output.
stack(unstack(df, b~a))
Or we can use a convenient function listCol_l from splitstackshape to convert the list to data.frame.
library(splitstackshape)
listCol_l(df, 'b')
Here's one way, with data.table:
require(data.table)
data.table(df)[,as.integer(unlist(b)),by=a]
If b is stored consistently, as.integer can be skipped. You can check with
unique(sapply(df$b,class))
# [1] "numeric" "integer"
Here's another base solution, far less elegant than any other solution posted thus far. Posting for the sake of completeness, though personally I would recommend akrun's base solution.
with(df, cbind(a = rep(a, sapply(b, length)), b = do.call(c, b)))
This constructs the first column as the elements of a, where each is repeated to match the length of the corresponding list item from b. The second column is b "flattened" using do.call() with c().
As Ananda Mahto pointed out in a comment, sapply(b, length) can be replaced with lengths(b) in the most recent version of R (3.2, if I'm not mistaken).
A base R approach might also be to create a new data.frame for each row and rbind it afterwards:
df <- data.frame(a=1:3,b=I(list(1,1:2,1:3)))
df
df <- lapply(seq_along(df$a), function(x){data.frame(a = df$a[[x]], b = df$b[[x]])})
df <- do.call("rbind", df)
df