I have data in a form much like the output from aggregate, except that I do not have the original non-aggregated data.
Example:
data <- data.frame(grade=letters[1:4], count=c(3,9,4,1))
grade count
1 a 3
2 b 9
3 c 4
4 d 1
I would like to sample from this population of grades, e.g. using sample. What is the easiest way of taking a sample (without replacement) from summarized counts like this?
Do you expect something like this?
> sample(with(data, rep(as.character(grade), count)), 10)
[1] "b" "b" "d" "a" "c" "c" "b" "b" "c" "b"
Related
I am a developing beginner in r. I have a simple question about r language.
Thanks to many experts in this site, I am improving a lot.
I am always grateful for that, and anyone who's giving hand with this question, thank you in advance.
This is the code.
Data=sample(1:5,size=25,replace=T)
names(Data)=c("a","b","c","d","e")
I want to name each of 1,2,3,4,5 to a,b,c,d,e.
so I thought I could accomplish this by using the upper code.
I know that the right code is
Data=c("a","b","c","d","e")[Data]
But I can't understand why this is the right code and why I need the last [Data].
Any help would be really appreciated!! Thank you so much in advance!!:)
The last Data provides an index to subset values from c("a","b","c","d","e").
Let's take a simple example :
Consider,
a <- 1:10
Now to get the first value in a you can do
a[1]
#[1] 1
To get 3rd value in a you can do
a[3]
#[1] 3
To get 6th and 8th value in a you can do
a[c(6, 8)]
#[1] 6 8
What will happen if you repeat a certain index? Say you select 1 twice and 3 once.
a[c(1, 1, 3)]
#[1] 1 1 3
As you can see the first value is selected two times and third one time.
Now ,Data that you have serves as that index to subset whereas a becomes c("a","b","c","d","e")
a <- c("a","b","c","d","e")
set.seed(123)
Data=sample(1:5,size=25,replace=T)
Data
#[1] 3 3 2 2 3 5 4 1 2 3 5 3 3 1 4 1 1 5 3 2 2 1 3 4 1
Now you use this Data values to subset from a giving
a[Data]
#[1] "c" "c" "b" "b" "c" "e" "d" "a" "b" "c" "e" "c" "c" "a" "d" "a" "a" "e" "c" "b" "b" "a" "c" "d" "a"
A side note, there is an inbuilt constant letters and LETTERS which gives 26 lower and upper case alphabets.
letters
#[1] "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m" "n" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z"
LETTERS
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"
Here is a way that takes advantage of how objects of class "factor" are coded internally in R. In R, factors are coded as consecutive integers starting at 1, and what the user sees is their labels and levels, not the integer values. But the integer values do not go away, they are still there.
First, create a vector of integers like in the question but setting the RNG seed in order to make the results reproducible. This vector is saved for later.
set.seed(123)
Data <- sample(1:5, size = 25, replace = TRUE)
Saved <- Data
Now create the factor. Note the labels atribute is set to the letters "a" to"e".
Data <- factor(Data, labels = c("a","b","c","d","e"))
Data
# [1] c c b b c e d a b c e c c a d a a e c b b a c d a
#Levels: a b c d e
See the internal representation.
as.integer(Data)
# [1] 3 3 2 2 3 5 4 1 2 3 5 3 3 1 4 1 1 5 3 2 2 1 3 4 1
And compare with the initial values.
identical(Saved, as.integer(Data))
# [1] TRUE
This is because Data contains the numbers you want to name in the order you want to name them. By adding [Data] to the end you are selecting the letters in the order of Data. To understand this, try what c("a","b","c","d","e")[c(1, 2)] does; it selects just the two first letters. If you instead type c("a","b","c","d","e")[c(5, 4)] it will select the two last letters, but in reverse order. Then if you print just Data, you'll see that it contains the numbers from 1 to 5, which is the amount of unique letters. So it will select the letters according to that order. You can see that all the numbers correspond to the letters in order by printing the correctly named Data.
Using names(Data)=c("a","b","c","d","e") does not work correctly since you aren't naming all 25 of the numbers, but rather just the first five of them.
Im trying to convert a data set in a long format panel structure to an adjacency matrix or edge list to make network graphs. The data set contains articles each identified by an ID-number. Each article can appear several times under a number of categories. Hence I have a long format structure at the moment:
ID <- c(1,1,1,2,2,2,3,3)
Category <- c("A","B","C","B","E","H","C","E")
dat <- data.frame(ID,Category)
I want to convert this into an adjacency matrix or edge list. Where the edge list such look something like this
A B
A C
B C
B E
B H
E H
C E
Edit: I have tried dat <- merge(ID, Category, by="Category") but it returns the error message Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column
Thanks in advance
Update: I ended up using the crossprod(table(dat)) from the comments, but the solution suggested by Navy Cheng below works just as well
This code will work
do.call(rbind,lapply(split(dat, dat$ID), function(x){
t(combn(as.vector(x$Category), 2))
}))
Update
As #Parfait 's suggestion, you can have by instead of split+lapply.
1) Use by to group nodes ("A", "B", "C" ...) by Category;
2) Use combn to create edge between nodes in each group, and t to transform the matrix for further rbind
> edge.list <- by(dat, dat$ID, function(x) t(combn(as.vector(x$Category), 2)))
dat$ID: 1
[,1] [,2]
[1,] "A" "B"
[2,] "A" "C"
[3,] "B" "C"
------------------------------------------------------------
dat$ID: 2
[,1] [,2]
[1,] "B" "E"
[2,] "B" "H"
[3,] "E" "H"
------------------------------------------------------------
dat$ID: 3
[,1] [,2]
[1,] "C" "E"
3) Then merge the list
> do.call(rbind, edge.list)
[,1] [,2]
[1,] "A" "B"
[2,] "A" "C"
[3,] "B" "C"
[4,] "B" "E"
[5,] "B" "H"
[6,] "E" "H"
[7,] "C" "E"
So if you are willing to convert your data.frame to a data.table this problem can be solved pretty efficiently and cleanly and if you have many rows will be much faster.
library(data.table)
dat<-data.table(dat)
Basically you can apply functions to columns of the data.table in the j cell and group in the k cell. So you want all the combinations of categories taken two at a time for each ID which looks like this:
dat[,combn(Categories,2),by=ID]
However stopping at this point will keep the ID column and by default create a column called V1 that basically concatenates the array returned by combn into a vector of the categories and not the two-column adjacency matrix that you need. But by chaining another call to this you can create the matrix easily as you would with any single vector. In one line of code this will look like:
dat[,combn(Category,2),by=ID][,matrix(V1,ncol=2,byrow = T)]
Remember that the vector column we wish to convert to a matrix is called V1 by default and also we want the 2-column matrix to be created by row instead of the default which is by column. Hope that helps and let me know if I need to add anything to my explanation. Good luck!
I would like to sort the data x (here: 1:12) according to the sectors sec and subsectors ssec. Below is an example showing how this can be done. The questions is whether this can be done more elegantly (maybe with a base-R function/not using additional packages)?
## Data
set.seed(17)
(sec <- sample(rep(LETTERS[1:3], each = 4))) # 3 sectors
(ssec <- rep(sample(1:4, 12, replace = TRUE))) # 4 subsectors
x <- 1:12 # data to sort according to increasing sectors and subsectors
## Sort according to sectors
ord <- order(sec)
x. <- x[ord]
sec. <- sec[ord]
ssec. <- ssec[ord]
## Sort according to subsectors
usec. <- unique(sec.)
x.. <- x.
ssec.. <- ssec.
for(grp in usec.) {
ii <- sec. == grp # indices of components in that sector
ord. <- order(ssec.[ii])
x..[ii] <- x.[ii][ord.]
ssec..[ii] <- ssec.[ii][ord.]
}
## Result
x..
sec.
ssec..
The order function from base R also accepts multiple arguments. From ?order:
order returns a permutation which rearranges its first argument into
ascending or descending order, breaking ties by further arguments.
To demonstrate, we can check how order(sec, ssec) sort the sector and subsector
Here is the original sec and ssec:
sec
[1] "B" "C" "A" "B" "A" "B" "C" "C" "C" "A" "B" "A"
ssec
[1] 3 1 3 2 1 2 4 1 3 2 1 4
After applying the ordered index, sec is sorted alphabetically and ssec is sorted within each sec, which means the index order(sec, ssec) is the sorting index expected:
sec[order(sec, ssec)]
[1] "A" "A" "A" "A" "B" "B" "B" "B" "C" "C" "C" "C"
ssec[order(sec, ssec)]
[1] 1 2 3 4 1 2 2 3 1 1 3 4
I have a dataframe that consists of blocks of X rows, each corresponding to a single individual (where X can be different for each individual). I'd like to randomly distribute these individuals into train, test and validation samples but so far I haven't been able to get the syntax correct to ensure that each of a user's X rows are always collected into the same subsample.
For example, the data can be simplified to look like:
user feature1 feature2
1 "A" "B"
1 "L" "L"
1 "Q" "B"
1 "D" "M"
1 "D" "M"
1 "P" "E"
2 "A" "B"
2 "R" "P"
2 "A" "F"
3 "X" "U"
... ... ...
and then if I ended up randomly assigning the users to a train, test or validation set all of the rows for that user (the user number is unique) would be in the same set, and grouped together so that if user 1 was in the traininng set, for example, then the format would still be:
user feature1 feature2
1 "A" "B"
1 "L" "L"
1 "Q" "B"
1 "D" "M"
1 "D" "M"
1 "P" "E"
As a bonus I'd love to know if the solution to this could be extended to do k-folds cross validation, but so far I haven't even figured out this more simple first step.
Thanks in advance.
you can use sample():
# 60 % for training, 20% for testing & validation
indeces <- sample(1:nrow(df),nrow(df)*0.6)
df.train <- df[indeces,]
df <- df[-indeces,]
indeces <- sample(1:nrow(df),nrow(df)*0.5)
df.test <- df[indeces,]
df.validate <- df[-indeces,]
for k-fold cross validation :
library(caret)
library(mlbench)
fld <- createFolds(df$your_dependent_variable, k= 10,list = TRUE, returnTrain = FALSE)
The above code splits the data into 10 folds. Run your model on each samples and validate them.
Edited:
user.df <- split( df , f = df$user )
this produces a separate data frame containing data for a particular user. use user.df[[1]] to access them individually.
in R I have produced the following list L:
>L
[[1]]
[1] "A" "B" "C"
[[2]]
[1] "D"
[[3]]
[1] NULL
I would like to manipulate the list L arriving at a database df like
>df
df[,1] df[,2]
"A" 1
"B" 1
"C" 1
"D" 2
where the 2nd column gives the position in the list L of the corresponding element in column 1.
My question is: is(are) there a() built-in R function(s) which can do this manipulation quickly? I can do it using "brute force", but my solution does not scale well when I consider much bigger lists.
I thank you all!
You'll get a warning because of your NULL value, but you can use stack if you give your list items names:
L <- list(c("A", "B", "C"), "D", NULL)
stack(setNames(L, seq_along(L)))
# values ind
# 1 A 1
# 2 B 1
# 3 C 1
# 4 D 2
# Warning message:
# In stack.default(setNames(L, seq_along(L))) :
# non-vector elements will be ignored
If the warning displeases you, you can, of course, run stack on the non-NULL elements, but do it after you name your list elements so that the "ind" column reflects the correct value.
I'll show in 2 steps just for clarity:
names(L) <- seq_along(L)
stack(L[!sapply(L, is.null)])
Similarly, if you've gotten rid of the NULL list elements, you can use melt from "reshape2". You don't gain anything in brevity, and I'm not sure that you gain anything in efficiency either, but I thought I'd share it as an option.
library(reshape2)
names(L) <- seq_along(L)
melt(L[!sapply(L, is.null)])
Ananda's answer is seemingly better than this, but I'll put it up anyway:
> cbind(unlist(L), rep(1:length(L), sapply(L, length)))
[,1] [,2]
[1,] "A" "1"
[2,] "B" "1"
[3,] "C" "1"
[4,] "D" "2"