R rearrange factor levels [duplicate] - r

I have data frame with some numerical variables and some categorical factor variables. The order of levels for those factors is not the way I want them to be.
numbers <- 1:4
letters <- factor(c("a", "b", "c", "d"))
df <- data.frame(numbers, letters)
df
# numbers letters
# 1 1 a
# 2 2 b
# 3 3 c
# 4 4 d
If I change the order of the levels, the letters no longer are with their corresponding numbers (my data is total nonsense from this point on).
levels(df$letters) <- c("d", "c", "b", "a")
df
# numbers letters
# 1 1 d
# 2 2 c
# 3 3 b
# 4 4 a
I simply want to change the level order, so when plotting, the bars are shown in the desired order - which may differ from default alphabetical order.

Use the levels argument of factor:
df <- data.frame(f = 1:4, g = letters[1:4])
df
# f g
# 1 1 a
# 2 2 b
# 3 3 c
# 4 4 d
levels(df$g)
# [1] "a" "b" "c" "d"
df$g <- factor(df$g, levels = letters[4:1])
# levels(df$g)
# [1] "d" "c" "b" "a"
df
# f g
# 1 1 a
# 2 2 b
# 3 3 c
# 4 4 d

some more, just for the record
## reorder is a base function
df$letters <- reorder(df$letters, new.order=letters[4:1])
library(gdata)
df$letters <- reorder.factor(df$letters, letters[4:1])
You may also find useful Relevel and combine_factor.

Since this question was last active Hadley has released his new forcats package for manipulating factors and I'm finding it outrageously useful. Examples from the OP's data frame:
levels(df$letters)
# [1] "a" "b" "c" "d"
To reverse levels:
library(forcats)
fct_rev(df$letters) %>% levels
# [1] "d" "c" "b" "a"
To add more levels:
fct_expand(df$letters, "e") %>% levels
# [1] "a" "b" "c" "d" "e"
And many more useful fct_xxx() functions.

so what you want, in R lexicon, is to change only the labels for a given factor variable (ie, leave the data as well as the factor levels, unchanged).
df$letters = factor(df$letters, labels=c("d", "c", "b", "a"))
given that you want to change only the datapoint-to-label mapping and not the data or the factor schema (how the datapoints are binned into individual bins or factor values, it might help to know how the mapping is originally set when you initially create the factor.
the rules are simple:
labels are mapped to levels by index value (ie, the value
at levels[2] is given the label, label[2]);
factor levels can be set explicitly by passing them in via the the
levels argument; or
if no value is supplied for the levels argument, the default
value is used which is the result calling unique on the data vector
passed in (for the data argument);
labels can be set explicitly via the labels argument; or
if no value is supplied for the labels argument, the default value is
used which is just the levels vector

Dealing with factors in R is quite peculiar job, I must admit... While reordering the factor levels, you're not reordering underlying numerical values. Here's a little demonstration:
> numbers = 1:4
> letters = factor(letters[1:4])
> dtf <- data.frame(numbers, letters)
> dtf
numbers letters
1 1 a
2 2 b
3 3 c
4 4 d
> sapply(dtf, class)
numbers letters
"integer" "factor"
Now, if you convert this factor to numeric, you'll get:
# return underlying numerical values
1> with(dtf, as.numeric(letters))
[1] 1 2 3 4
# change levels
1> levels(dtf$letters) <- letters[4:1]
1> dtf
numbers letters
1 1 d
2 2 c
3 3 b
4 4 a
# return numerical values once again
1> with(dtf, as.numeric(letters))
[1] 1 2 3 4
As you can see... by changing levels, you change levels only (who would tell, eh?), not the numerical values! But, when you use factor function as #Jonathan Chang suggested, something different happens: you change numerical values themselves.
You're getting error once again 'cause you do levels and then try to relevel it with factor. Don't do it!!! Do not use levels or you'll mess things up (unless you know exactly what you're doing).
One lil' suggestion: avoid naming your objects with an identical name as R's objects (df is density function for F distribution, letters gives lowercase alphabet letters). In this particular case, your code would not be faulty, but sometimes it can be... but this can create confusion, and we don't want that, do we?!? =)
Instead, use something like this (I'll go from the beginning once again):
> dtf <- data.frame(f = 1:4, g = factor(letters[1:4]))
> dtf
f g
1 1 a
2 2 b
3 3 c
4 4 d
> with(dtf, as.numeric(g))
[1] 1 2 3 4
> dtf$g <- factor(dtf$g, levels = letters[4:1])
> dtf
f g
1 1 a
2 2 b
3 3 c
4 4 d
> with(dtf, as.numeric(g))
[1] 4 3 2 1
Note that you can also name you data.frame with df and letters instead of g, and the result will be OK. Actually, this code is identical with the one you posted, only the names are changed. This part factor(dtf$letter, levels = letters[4:1]) wouldn't throw an error, but it can be confounding!
Read the ?factor manual thoroughly! What's the difference between factor(g, levels = letters[4:1]) and factor(g, labels = letters[4:1])? What's similar in levels(g) <- letters[4:1] and g <- factor(g, labels = letters[4:1])?
You can put ggplot syntax, so we can help you more on this one!
Cheers!!!
Edit:
ggplot2 actually requires to change both levels and values? Hm... I'll dig this one out...

I wish to add another case where the levels could be strings carrying numbers alongwith some special characters : like below example
df <- data.frame(x = c("15-25", "0-4", "5-10", "11-14", "100+"))
The default levels of x is :
df$x
# [1] 15-25 0-4 5-10 11-14 100+
# Levels: 0-4 100+ 11-14 15-25 5-10
Here if we want to reorder the factor levels according to the numeric value, without explicitly writing out the levels, what we could do is
library(gtools)
df$x <- factor(df$x, levels = mixedsort(df$x))
df$x
# [1] 15-25 0-4 5-10 11-14 100+
# Levels: 0-4 5-10 11-14 15-25 100+
as.numeric(df$x)
# [1] 4 1 2 3 5
I hope this can be considered as useful information for future readers.

Here's my function to reorder factors of a given dataframe:
reorderFactors <- function(df, column = "my_column_name",
desired_level_order = c("fac1", "fac2", "fac3")) {
x = df[[column]]
lvls_src = levels(x)
idxs_target <- vector(mode="numeric", length=0)
for (target in desired_level_order) {
idxs_target <- c(idxs_target, which(lvls_src == target))
}
x_new <- factor(x,levels(x)[idxs_target])
df[[column]] <- x_new
return (df)
}
Usage: reorderFactors(df, "my_col", desired_level_order = c("how","I","want"))

I would simply use the levels argument:
levels(df$letters) <- levels(df$letters)[c(4:1)]

To add yet another approach that is quite useful as it frees us from remembering functions from differents packages. The levels of a factor are just attributes, so one can do the following:
numbers <- 1:4
letters <- factor(c("a", "b", "c", "d"))
df <- data.frame(numbers, letters)
# Original attributes
> attributes(df$letters)
$levels
[1] "a" "b" "c" "d"
$class
[1] "factor"
# Modify attributes
attr(df$letters,"levels") <- c("d", "c", "b", "a")
> df$letters
[1] d c b a
Levels: d c b a
# New attributes
> attributes(df$letters)
$levels
[1] "d" "c" "b" "a"
$class
[1] "factor"

Related

Gather all the unique values from 2 columns into a new column

I have a dataframe which includes all the nodes' connections in a network and I want to create a new dataframe named 'nodes' with all the unique nodes. Im trying to do something like
eids<-as.factor(d$from)
mids<-as.factor(d$to)
nodes<-data.frame(c(eids,mids))
nodes<-unique(nodes)
but when I try to create the graph I get:Some vertex names in edge list are not listed in vertex data frame which means that part of my data is missed with this method. My dataset is quite large so I put a toy dataset here.
from<-c(2,3,4,3,1,2)
to<-c(8,8,7,5,6,5)
d<-data.frame(from,to)
First, to solve your question, you can use unique(stack(d)[1]) to get a data frame with one column with values 1 to 8.
Here I explain why your code doesn't work. Using c() to combine objects of factor class is dangerous. You can try the following example:
(x <- factor(c("A", "B", "C", "D")))
# [1] A B C D
# Levels: A B C D
(y <- factor(c("E", "F", "G", "H")))
# [1] E F G H
# Levels: E F G H
c(x, y)
# [1] 1 2 3 4 1 2 3 4
Actually, the factor object is based on numeric data, not character. You can strip away its class and find that it belongs to a numeric vector with an attribute named levels:
unclass(x)
# [1] 1 2 3 4
# attr(,"levels")
# [1] "A" "B" "C" "D"
The numeric part means the indices of levels. A factor object actually works like recording the indices of its levels.

Why are empty levels in my factor tabulated after I assign NAs to missing values?

I have a dataframe df with a column foo containing data of type factor:
df <- data.frame("bar" = c(1:4), "foo" = c("M", "F", "F", "M"))
When I inspect the structure with str(df$foo), I get this:
Factor w/ 3 levels "","F",..: 2 2 2 2 2 2 2 2 2 2 ..
Why does it report 3 levels when there are only 2 in my data?
Edit:
There seems to be a missing value "" that I clean up by assigning it NA.
When I call table(df$foo), it seems to still count the "missing value" level, but finds no occurences:
F M
0 2 2
However, when I call df$foo I find it reports only two levels:
Levels: F M
How is it possible that table still counts the empty level, and how can I fix that behaviour?
Check whether your dataframe indeed has no missing values, because it does look to be that way. Try this:
# works because factor-levels are integers, internally; "" seems to be level 1
which(as.integer(df$MF) == 1)
# works if your missing value is just ""
which(df$MF == "")
You should then clean up your dataframe to properly refeclet missing values. A factor will handle NA:
df <- data.frame("rest" = c(1:5), "sex" = c("M", "F", "F", "M", ""))
df$sex[which(as.integer(df$sex) == 1)] <- NA
Once you have cleaned your data, you will have to drop unused levels to avoid tabulations such as table counting occurences of the empty level.
Observe this sequence of steps and its outputs:
# Build a dataframe to reproduce your behaviour
> df <- data.frame("Restaurant" = c(1:5), "MF" = c("M", "F", "F", "M", ""))
# notice the empty level "" for the missing value
> levels(df$MF)
[1] "" "F" "M"
# notice how a tabulation counts the empty level;
# this is the first column with a 1 (it has no label because
# there is no label, it is "")
> table(df$MF)
F M
1 2 2
# find the culprit and change it to NA
> df$MF[which(as.integer(df$MF) == 1)] <- as.factor(NA)
# AHA! So despite us changing the value, the original factor
# was not updated! I wonder what happens if we tabulate the column...
> levels(df$MF)
[1] "" "F" "M"
# Indeed, the empty level is present in the factor, but there are
# no occurences!
> table(df$MF)
F M
0 2 2
# droplevels to the rescue:
# it is used to drop unused levels from a factor or, more commonly,
# from factors in a data frame.
> df$MF <- droplevels(df$MF)
# factors fixed
> levels(df$MF)
[1] "F" "M"
# tabulation fixed
> table(df$MF)
F M
2 2

Removing a row does not change change output of length() and levels()

Using below code I import a dataset, explore it and remove a row.
After removing the row the output of my length and levels command is unchanged. Why?
MT <- read_csv("Q:/PhD/PhD courses/Data Doc and Man/day3-day4/bromraw.txt",
col_names = FALSE)
names(MT) <- c("id","pnr","age","sex", "runtime")
MT$sex <- as.factor(MT$sex)
length(levels(MT$sex))
levels(MT$sex)
This is the output:
[1] 3
[1] "33529" "K" "M"
Something is wrong. I investigate the row where sex has the value 33529
filter(MT, sex == 33529)
After examining the row I decide to drop it, and recheck the sex variable again.
MT <- subset(MT, sex !=33529)
length(levels(MT$sex))
levels(MT$sex)
[1] 3
[1] "33529" "K" "M"
The row is not there when I browse the data, but the output of the length and levels command is the same as before. What am I doing wrong?
I feel the question deserves a better explanation than just a piece of code.
Factor levels can exist independent of the data, e.g.
x <- factor(character(0), levels = LETTERS[1:3])
creates a vector of length 0 which has 3 factor levels
x
factor(0)
Levels: A B C
The length of the vector length(x) is zero but x has 3 levels
levels(x)
[1] "A" "B" "C"
(and length(levels(x)) is 3, accordingly).
The benefit is that we can add data later on which is checked if it is compatible with the defined factor levels:
x[1:4] <- LETTERS[1:4]
Warning message: In [<-.factor(*tmp*, 1:4, value = c("A", "B",
"C", "D")) : invalid factor level, NA generated
x
[1] A B C <NA>
Levels: A B C
Now, the vector consists of 4 elements (length(x)) but there are still only 3 factor levels. Note that "D" has not become an additional factor level automatically but was replaced by NA instead.
If elements of the vector are removed, e.g.
y <- x[-c(1L, 4L)]
y
[1] B C
Levels: A B C
the factor levels remain unchanged while length(y) is 2 now.
However, if you want to remove unused factor levels you can do so by explicitely using the droplevels() function as pointed out by akrun:
y <- droplevels(y)
y
[1] B C
Levels: B C
Now, factor level "A" has been dropped as it is unused.
While the levels() function shows the factor levels which are defined it does not tell which of the boxes (credit to Acccumulation for the picture) are filled or not. The unique() function returns a vector of distinct values while the table() function counts the number of occurrences:
set.seed(1L)
z <- sample(LETTERS[1:8], 10, replace = TRUE)
z
[1] "C" "B" "E" "H" "A" "B" "D" "A" "D" "C"
unique(z)
[1] "C" "B" "E" "H" "A" "D"
table(z)
z
A B C D E H
2 2 2 2 1 1
This could be a case of unused levels. We can resolve it by dropping the levels
MT <- droplevels(subset(MT, sex != 33529))

Extract the factor's values positions in level

I'm returning to R after some time, and the following has me stumped:
I'd like to build a list of the positions factor values have in the facor levels list.
Example:
> data = c("a", "b", "a","a","c")
> fdata = factor(data)
> fdata
[1] a b a a c
Levels: a b c
> fdata$lvl_idx <- ????
Such that:
> fdata$lvl_idx
[1] 1 2 1 1 3
Appreciate any hints or tips.
If you convert a factor to integer, you get the position in the levels:
as.integer(fdata)
## [1] 1 2 1 1 3
In certain situations, this is counter-intuitive:
f <- factor(2:4)
f
## [1] 2 3 4
## Levels: 2 3 4
as.integer(f)
## [1] 1 2 3
Also if you silently coerce to integer, for example by using a factor as a vector index:
LETTERS[2:4]
## [1] "B" "C" "D"
LETTERS[f]
## [1] "A" "B" "C"
Converting to character before converting to integer gives the expected values. See ?factor for details.
The solution provided years ago by Matthew Lundberg is not robust. It could be that the as.integer() function was defined for a specific S3 type of factors. Imagine someone would create a new factor class to keep operators like >=.
as.myfactor <- function(x, ...) {
structure(as.factor(x), class = c("myfactor", "factor"))
}
# and that someone would create an S3 method for integers - it should
# only remove the operators, which makes sense...
as.integer.myfactor <- function(x, ...) {
as.integer(gsub("(<|=|>)+", "", as.character(x)))
}
Now this is not working anymore, - it just removes operators:
f <- as.myfactor(">=2")
as.integer(f)
#> [1] 2
But this is robust with any factor you want to know the index of the level of, using which():
f <- factor(2:4)
which(levels(f) == 2)
#> [1] 1

How does one change the levels of a factor column in a data.table

What is the correct way to change the levels of a factor column in a data.table (note: not data frame)
library(data.table)
mydt <- data.table(id=1:6, value=as.factor(c("A", "A", "B", "B", "B", "C")), key="id")
mydt[, levels(value)]
[1] "A" "B" "C"
I am looking for something like:
mydt[, levels(value) <- c("X", "Y", "Z")]
But of course, the above line does not work.
# Actual # Expected result
> mydt > mydt
id value id value
1: 1 A 1: 1 X
2: 2 A 2: 2 X
3: 3 B 3: 3 Y
4: 4 B 4: 4 Y
5: 5 B 5: 5 Y
6: 6 C 6: 6 Z
You can still set them the traditional way:
levels(mydt$value) <- c(...)
This should be plenty fast unless mydt is very large since that traditional syntax copies the entire object. You could also play the un-factoring and refactoring game... but no one likes that game anyway.
To change the levels by reference with no copy of mydt :
setattr(mydt$value,"levels",c(...))
but be sure to assign a valid levels vector (type character of sufficient length) otherwise you'll end up with an invalid factor (levels<- does some checking as well as copying).
I would rather go the traditional way of re-assignment to the factors
> mydt$value # This we what we had originally
[1] A A B B B C
Levels: A B C
> levels(mydt$value) # just checking the levels
[1] "A" "B" "C"
**# Meat of the re-assignment**
> levels(mydt$value)[levels(mydt$value)=="A"] <- "X"
> levels(mydt$value)[levels(mydt$value)=="B"] <- "Y"
> levels(mydt$value)[levels(mydt$value)=="C"] <- "Z"
> levels(mydt$value)
[1] "X" "Y" "Z"
> mydt # This is what we wanted
id value
1: 1 X
2: 2 X
3: 3 Y
4: 4 Y
5: 5 Y
6: 6 Z
As you probably notices, the meat of the re-assignment is very intuitive, it checks for the exact level(use grepl in case there's a fuzzy math, regular expressions or likewise)
levels(mydt$value)[levels(mydt$value)=="A"] <- "X"
This explicitly checks the value in the levels of the variable under consideration and then reassigns X (and so on) to it - The advantage- you explicitly KNOW what labeled what.
I find renaming levels as here levels(mydt$value) <- c("X","Y","Z") very non-intuitive, since it just assigns X to the 1st level it SEES in the data (so the order really matters)
PPS : In case of too many levels, use looping constructs.
You can also rename and add to your levels using a related approach, which can be very handy, especially if you are making a plot that needs more informative labels in a particular order (as opposed to the default):
f <- factor(c("a","b"))
levels(f) <- list(C = "C", D = "a", B = "b")
(modified from ?levels)
This is safer than Matt Dowle's suggestion (because it uses the checks skipped by setattr) but won't copy the entire data.table. It will replace the entire column vector (whereas Matt's solution only replaces the attributes of the column vector) , but that seems like an acceptable trade-off in order to reduce the risk of messing up the factor object.
mydt[, value:=`levels<-`(value, c("X", "Y", "Z"))]
Simplest way to change a column's levels:
dat$colname <- as.factor(as.vector(dat$colname));

Resources