I'm renaming the majority of the variables in a data frame and I'm not really impressed with my method.
Therefore, does anyone on SO have a smarter or faster way then the one presented below using only base?
data(mtcars)
# head(mtcars)
temp.mtcars <- mtcars
names(temp.mtcars) <- c((x <- c("mpg", "cyl", "disp")),
gsub('^', "baR.", setdiff(names (mtcars),x)))
str(temp.mtcars)
'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp : num 160 160 108 258 360 ...
$ baR.hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ baR.drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ baR.wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ baR.qsec: num 16.5 17 18.6 19.4 17 ...
$ baR.vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ baR.am : num 1 1 1 0 0 0 0 0 0 0 ...
$ baR.gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ baR.carb: num 4 4 1 1 2 1 4 2 2 4 ...
Edited for answer using base R only
The package plyr has a convenient function rename() that does what you ask. Your modified question specifies using base R only. One easy way of doing this is to simply copy the code from plyr::rename and create your own function.
rename <- function (x, replace) {
old_names <- names(x)
new_names <- unname(replace)[match(old_names, names(replace))]
setNames(x, ifelse(is.na(new_names), old_names, new_names))
}
The function rename takes an argument that is a named vector, where the elements of the vectors are the new names, and the names of the vector are the existing names. There are many ways to construct such a named vector. In the example below I simply use structure.
x <- c("mpg", "disp", "wt")
some.names <- structure(paste0("baR.", x), names=x)
some.names
mpg disp wt
"baR.mpg" "baR.disp" "baR.wt"
Now you are ready to rename:
mtcars <- rename(mtcars, replace=some.names)
The results:
'data.frame': 32 obs. of 11 variables:
$ baR.mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ baR.disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat : num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ baR.wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec : num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear : num 4 4 4 3 3 3 3 4 4 4 ...
$ carb : num 4 4 1 1 2 1 4 2 2 4 ...
I would use ifelse:
names(temp.mtcars) <- ifelse(names(mtcars) %in% c("mpg", "cyl", "disp"),
names(mtcars),
paste("bar", names(mtcars), sep = "."))
Nearly the same but without plyr:
data(mtcars)
temp.mtcars <- mtcars
carNames <- names(temp.mtcars)
modifyNames <- !(carNames %in% c("mpg", "cyl", "disp"))
names(temp.mtcars)[modifyNames] <- paste("baR.", carNames[modifyNames], sep="")
Output:
str(temp.mtcars)
'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp : num 160 160 108 258 360 ...
$ baR.hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ baR.drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ baR.wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ baR.qsec: num 16.5 17 18.6 19.4 17 ...
$ baR.vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ baR.am : num 1 1 1 0 0 0 0 0 0 0 ...
$ baR.gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ baR.carb: num 4 4 1 1 2 1 4 2 2 4 ...
You could use the rename.vars function in the gdata package.
It works well when you only want to replace a subset of variable names and where the order of your vector of names is not the same as the order of names in the data.frame.
Adapted from the help file:
library(gdata)
data <- data.frame(x=1:10,y=1:10,z=1:10)
names(data)
data <- rename.vars(data, from=c("z","y"), to=c("Z","Y"))
names(data)
Converts data.frame names:
[1] "x" "y" "z"
to
[1] "x" "Y" "Z"
I.e., Note how this handles the subsetting and the fact that string of names are not in the same order as the names in the data.frame.
names(df)[match(
c('old_var1','old_var2'),
names(df)
)]=c('new_var1', 'new_var2')
Related
I received a script that generates a bunch of objects. I want to combine multiple dataframes using bind_rows. I am able to choose the correct objects using grep but I am not able to pass those object names as argument to bind_rows.
For example, I want to select the objects that start with df and pass those to bind_rows. In the example below I expect to have a dataframe named data which have the dataframe mtcars 3 times.
df1 <- mtcars
df2 <- mtcars
df3 <- mtcars
notdf4 <- mtcars
dfx <- ls()[grep("^df", ls())]
data <- bind_rows(eval(parse(text = dfx)))
The suggestion to use mget makes sense, although it returns a list so you would need to use do.call to execute an `rbind operation.
str( do.call( rbind, mget(ls( patt="^df.") ) ) )
'data.frame': 96 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
I think using mget and do.call (rather than will have a lower chance of offending people like me who might be called R purists. I chose to use the "pattern" argument to ls as cleaner than first getting all the workspace names and then selecting from them with grep.
names <- names(mtcars)
str(mtcars[names[1]]) # shows the str for mpg data frame
I would like to select everything EXCEPT names[1] which in this example is mpg.
Tried:
str(mtcars[!names[1]])
Error in !names[1] : invalid argument type
Also tried
str(mtcars[-names[1]])
Error in -names[1] : invalid argument to unary operator
How can I select mtcars minus names[1] feature using square braces syntax?
str(mtcars[!names %in% names[1]])
'data.frame': 32 obs. of 10 variables:
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
If you want to use numerical indexing for selection, you can just use a - in front of that to do the reverse.
str(mtcars[names[1]]) # shows the str for mpg data frame
'data.frame': 32 obs. of 1 variable:
$ mpg: num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
str(mtcars[names[-1]])
'data.frame': 32 obs. of 10 variables:
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
I'm working on a package that relies on adding a class and attributes to a data frame, and would like to be able to use dplyr verbs with it.
The only trouble is they seem to strip away the classes and attributes that I've added to my data frames.
Example
class(mtcars) <- c("new_class", class(mtcars))
attr(mtcars, "foo") <- "bar"
Examining the structure shows that mtcars now includes the new class and attributes
> mtcars %>% str
Classes ‘new_class’, ‘new_class’ and 'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
- attr(*, "foo")= chr "bar"
But when I use filter, it seems to lose the classes and attributes.
> mtcars %>% filter(cyl == 8) %>% str
'data.frame': 14 obs. of 11 variables:
$ mpg : num 18.7 14.3 16.4 17.3 15.2 10.4 10.4 14.7 15.5 15.2 ...
$ cyl : num 8 8 8 8 8 8 8 8 8 8 ...
$ disp: num 360 360 276 276 276 ...
$ hp : num 175 245 180 180 180 205 215 230 150 150 ...
$ drat: num 3.15 3.21 3.07 3.07 3.07 2.93 3 3.23 2.76 3.15 ...
$ wt : num 3.44 3.57 4.07 3.73 3.78 ...
$ qsec: num 17 15.8 17.4 17.6 18 ...
$ vs : num 0 0 0 0 0 0 0 0 0 0 ...
$ am : num 0 0 0 0 0 0 0 0 0 0 ...
$ gear: num 3 3 3 3 3 3 3 3 3 3 ...
$ carb: num 2 4 3 3 3 4 4 4 2 2 ...
Is that behaviour expected?
What can I do to work around it?
filter calls filter_, which uses the method for a data.frame (because there is no filter method for new_class. filter_.data.frame then uses calls the filter method for a tbl_df and uses as.data.frame to return a data.frame.
dplyr:::filter_.data.frame
## function (.data, ..., .dots)
## {
## dots <- lazyeval::all_dots(.dots, ..., all_named = TRUE)
## as.data.frame(filter_(tbl_df(.data), .dots = dots))
## }
## <environment: namespace:dplyr>
The coercion to tbl_df removes the extra class, but keeps the attribute foo.
mtcars %>% tbl_df %>% str
The filtering seems to loose the attribute foo.
mtcars %>% tbl_df %>% filter(cyl == 8) %>% str
I have a big ol' data frame with two ID columns for courses and users, and I needed to split it into one dataframe per course to do some further analysis/subsetting. After eliminating quite a few rows from each of the individual course dataframes, I'll need to stick them back together.
I split it up using, you guessed it, split, and that worked exactly as I needed it to. However, unsplitting was harder than I thought. The R documentation says that "unsplit reverses the effect of split," but my reading on the web so far is suggesting that that is not the case when the elements of the split-out list are themselves dataframes.
What can I do to rejoin my modified dfs?
This is a place for do.call. Simply calling df <- rbind(split.df) will result in a weird and useless list object, but do.call("rbind", split.df) should give you the result you're looking for.
unsplit() will work / does seem to work in the general situation that you describe, but not the particular situation of removing rows from the thus split data frame.
Consider
> spl <- split(mtcars, mtcars$cyl)
> str(spl, max = 1)
List of 3
$ 4:'data.frame': 11 obs. of 11 variables:
$ 6:'data.frame': 7 obs. of 11 variables:
$ 8:'data.frame': 14 obs. of 11 variables:
> str(unsplit(spl, f = mtcars$cyl))
'data.frame': 32 obs. of 11 variables:
$ mpg : num 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
$ cyl : num 6 6 4 6 8 6 8 4 4 6 ...
$ disp: num 160 160 108 258 360 ...
$ hp : num 110 110 93 110 175 105 245 62 95 123 ...
$ drat: num 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
$ wt : num 2.62 2.88 2.32 3.21 3.44 ...
$ qsec: num 16.5 17 18.6 19.4 17 ...
$ vs : num 0 0 1 1 0 1 0 1 1 1 ...
$ am : num 1 1 1 0 0 0 0 0 0 0 ...
$ gear: num 4 4 4 3 3 3 3 4 4 4 ...
$ carb: num 4 4 1 1 2 1 4 2 2 4 ...
As we can see, unsplit() can undo a split. However, in the case where the split data frame is further worked upon and altered to remove rows, there will be a mismatch between the total number of rows in the data frames in the split list and the variable used to split the original data frame.
If you know or can compute the changes required to make the variable used to split the original data frame then unsplit() can be deployed. Though it is more than likely that this will not be trivial.
The general solution is, as #Andrew Sannier mentions is the do.call(rbind, ...) idiom:
> spl <- split(mtcars, mtcars$cyl)
> str(do.call(rbind, spl))
'data.frame': 32 obs. of 11 variables:
$ mpg : num 22.8 24.4 22.8 32.4 30.4 33.9 21.5 27.3 26 30.4 ...
$ cyl : num 4 4 4 4 4 4 4 4 4 4 ...
$ disp: num 108 146.7 140.8 78.7 75.7 ...
$ hp : num 93 62 95 66 52 65 97 66 91 113 ...
$ drat: num 3.85 3.69 3.92 4.08 4.93 4.22 3.7 4.08 4.43 3.77 ...
$ wt : num 2.32 3.19 3.15 2.2 1.61 ...
$ qsec: num 18.6 20 22.9 19.5 18.5 ...
$ vs : num 1 1 1 1 1 1 1 1 0 1 ...
$ am : num 1 0 0 1 1 1 0 1 1 1 ...
$ gear: num 4 4 4 4 4 4 3 4 5 5 ...
$ carb: num 1 2 2 1 2 1 1 1 2 2 ...
Outside of base R, also consider:
data.table::rbindlist() with the side effect of the result being a data.table
dplyr::bind_rows() which despite its somewhat confusing name will bind rows across lists
The answer by Andrew Sannier works but has the side-effect that the rownames get changed. rbind adds the list names to them, so e.g. "Datsun 710" becomes "4.Datsun 710". One can use unname in between to avoid this problem.
Complete example:
mtcars_reorder = mtcars[order(mtcars$cyl), ] #reorder based on cyl first
l1 = split(mtcars_reorder, mtcars_reorder$cyl) #split by cyl
l1 = unname(l1) #remove list names
l2 = do.call(what = "rbind", l1) #unsplit
all(l2 == mtcars_reorder) #check if matches
#> TRUE
I'm trying to learn about loops and I currently have a long list of data frames and I need to go inside a bunch of these data frames and rename some variables. I have a function, but I’m struggling to construct a smart way to loop thru my list (the real list is much longer than in the example below) and at the same time apply varying suffixes prefixes hen renaming.
Hopefully my working example below will illustrate the situation. I imagine I can build the last part into two loops, but I can't seem to figure out how I write to the data frame inside the list inside a loop.
Any help would be appreciated!
data(mtcars)
mtcarsList <- list(mtcars1 = mtcars, mtcars2 = mtcars,
mtcarsA = mtcars, mtcars = mtcars )
# function I use to renames a specific number of variables
baRadd <- function(df, vector, suffix){
names(df) <- ifelse(names(df) %in% vector,names(df),
paste(suffix, names(df), sep = "."))
return(df)}
foo <- c("mpg", "cyl", "disp")
suffix1 <- "bar"
suffix2 <- "barBAR"
suffix3 <- "barBARbar"
mtcarsList$mtcars1 <- baRadd(mtcarsList$mtcars1, foo, suffix1)
mtcarsList$mtcars2 <- baRadd(mtcarsList$mtcars2, foo, suffix2)
mtcarsList$mtcarsA <- baRadd(mtcarsList$mtcarsA, foo, suffix3)
names(mtcarsList$mtcars1)
# [1] "mpg" "cyl" "disp" "bar.hp" "bar.drat" "bar.wt"
# [7] "bar.qsec" "bar.vs" "bar.am" "bar.gear" "bar.carb"
names(mtcarsList$mtcars2)
# [1] "mpg" "cyl" "disp" "barBAR.hp" "barBAR.drat"
# [6] "barBAR.wt" "barBAR.qsec" "barBAR.vs" "barBAR.am" "barBAR.gear"
# [11] "barBAR.carb"
names(mtcarsList$mtcarsA)
# [1] "mpg" "cyl" "disp" "barBARbar.hp"
# [5] "barBARbar.drat" "barBARbar.wt" "barBARbar.qsec" "barBARbar.vs"
# [9] "barBARbar.am" "barBARbar.gear" "barBARbar.carb"
names(mtcarsList$mtcars)
# [1] "mpg" "cyl" "disp" "hp" "drat" "wt" "qsec" "vs" "am" "gear"
# [11] "carb"
Update,
Based on DWin's response below I write this scrip that solves my issue,
# rm(list = ls(all = TRUE)) ## Clear workspace
data(mtcars)
mtcarsList <- list(mtcars1 = mtcars, mtcars2 = mtcars,
mtcarsA = mtcars, mtcars = mtcars)
## function I use to renames a specific number of variables
baRadd <- function(df, vector, suffix){
names(df) <- ifelse(names(df) %in% vector,names(df),
paste(suffix, names(df), sep = "."))
return(df)}
suffixes <- c('A', 'B', 'C') # suffixes to be added to the three dfTO
whatNOTtoRename <- c("mpg", "cyl", "disp")
# variables within the data frame I do not want to renames
dfTO <- c('mtcars1','mtcars2','mtcarsA')
# the specific data frames I need to rename
# str(mtcarsList)
mtcarsList[ names( mtcarsList[dfTO]) ] <-
mapply(baRadd, df=mtcarsList[dfTO],
suffix= suffixes,
MoreArgs=list(vector=whatNOTtoRename) , SIMPLIFY=FALSE)
str(mtcarsList)
Looks as though mapply can do this task:
> newList <- mapply(baRadd, df=mtcarsList[1:3], suffix= c(suffix1, suffix2, suffix3), MoreArgs=list(vector=foo) , SIMPLIFY=FALSE)
> str(newList)
List of 3
$ mtcars1:'data.frame': 32 obs. of 11 variables:
..$ mpg : num [1:32] 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
..$ cyl : num [1:32] 6 6 4 6 8 6 8 4 4 6 ...
..$ disp : num [1:32] 160 160 108 258 360 ...
..$ bar.hp : num [1:32] 110 110 93 110 175 105 245 62 95 123 ...
..$ bar.drat: num [1:32] 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
..$ bar.wt : num [1:32] 2.62 2.88 2.32 3.21 3.44 ...
..$ bar.qsec: num [1:32] 16.5 17 18.6 19.4 17 ...
..$ bar.vs : num [1:32] 0 0 1 1 0 1 0 1 1 1 ...
..$ bar.am : num [1:32] 1 1 1 0 0 0 0 0 0 0 ...
..$ bar.gear: num [1:32] 4 4 4 3 3 3 3 4 4 4 ...
..$ bar.carb: num [1:32] 4 4 1 1 2 1 4 2 2 4 ...
$ mtcars2:'data.frame': 32 obs. of 11 variables:
..$ mpg : num [1:32] 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
..$ cyl : num [1:32] 6 6 4 6 8 6 8 4 4 6 ...
..$ disp : num [1:32] 160 160 108 258 360 ...
..$ barBAR.hp : num [1:32] 110 110 93 110 175 105 245 62 95 123 ...
..$ barBAR.drat: num [1:32] 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
..$ barBAR.wt : num [1:32] 2.62 2.88 2.32 3.21 3.44 ...
..$ barBAR.qsec: num [1:32] 16.5 17 18.6 19.4 17 ...
..$ barBAR.vs : num [1:32] 0 0 1 1 0 1 0 1 1 1 ...
..$ barBAR.am : num [1:32] 1 1 1 0 0 0 0 0 0 0 ...
..$ barBAR.gear: num [1:32] 4 4 4 3 3 3 3 4 4 4 ...
..$ barBAR.carb: num [1:32] 4 4 1 1 2 1 4 2 2 4 ...
$ mtcarsA:'data.frame': 32 obs. of 11 variables:
..$ mpg : num [1:32] 21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
..$ cyl : num [1:32] 6 6 4 6 8 6 8 4 4 6 ...
..$ disp : num [1:32] 160 160 108 258 360 ...
..$ barBARbar.hp : num [1:32] 110 110 93 110 175 105 245 62 95 123 ...
..$ barBARbar.drat: num [1:32] 3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
..$ barBARbar.wt : num [1:32] 2.62 2.88 2.32 3.21 3.44 ...
..$ barBARbar.qsec: num [1:32] 16.5 17 18.6 19.4 17 ...
..$ barBARbar.vs : num [1:32] 0 0 1 1 0 1 0 1 1 1 ...
..$ barBARbar.am : num [1:32] 1 1 1 0 0 0 0 0 0 0 ...
..$ barBARbar.gear: num [1:32] 4 4 4 3 3 3 3 4 4 4 ...
..$ barBARbar.carb: num [1:32] 4 4 1 1 2 1 4 2 2 4 ...
If you wanted to assign that result to mtcarsList[1:3], that too should be possible.
To your comment: this succeeds ....
mtcarsList[ names( mtcarsList[1:3]) ] <-
mapply(baRadd, df=mtcarsList[1:3],
suffix= c(suffix1, suffix2, suffix3),
MoreArgs=list(vector=foo) , SIMPLIFY=FALSE)
# omitted output of str(mtcarsList) ....