looping over the name of the columns in R for creating new columns - r

I am trying to use the loop over the column names of the existing dataframe and then create new columns based on one of the old column.Here is my sample data:
sample<-list(c(10,12,17,7,9,10),c(NA,NA,NA,10,12,13),c(1,1,1,0,0,0))
sample<-as.data.frame(sample)
colnames(sample)<-c("x1","x2","D")
>sample
x1 x2 D
10 NA 1
12 NA 1
17 NA 1
7 10 0
9 20 0
10 13 0
Now, I am trying to use for loop to generate two variables x1.imp and x2.imp that have values related to D=0 when D=1 and values related to D=1 when D=0(Here I actually don't need for loop but for my original dataset with large cols (variables), I really need the loop) based on the following condition:
for (i in names(sample[,1:2])){
sample$i.imp<-with (sample, ifelse (D==1, i[D==0],i[D==1]))
i=i+1
return(sample)
}
Error in i + 1 : non-numeric argument to binary operator
However, the following works, but it doesn't give the names of new cols as imp.x2 and imp.x3
for(i in sample[,1:2]){
impt.i<-with(sample,ifelse(D==1,i[D==0],i[D==1]))
i=i+1
print(as.data.frame(impt.i))
}
impt.i
1 7
2 9
3 10
4 10
5 12
6 17
impt.i
1 10
2 12
3 13
4 NA
5 NA
6 NA
Note that I already know the solution without loop [here]. I want with loop.
Expected output:
x1 x2 D x1.impt x2.imp
10 NA 1 7 10
12 NA 1 9 20
17 NA 1 10 13
7 10 0 10 NA
9 20 0 12 NA
10 13 0 17 NA
I would greatly appreciate your valuable input in this regard.

This is nuts, but since you are asking for it... Your code with minimum changes would be:
for (i in colnames(sample)[1:2]){
sample[[paste0(i, '.impt')]] <- with(sample, ifelse(D==1, get(i)[D==0],get(i)[D==1]))
}
A few comments:
replaced names(sample[,1:2]) with the more elegant colnames(sample)[1:2]
the $ is for interactive usage. Instead, when programming, i.e. when the column name is to be interpreted, you need to use [ or [[, hence I replaced sample$i.imp with sample[[paste0(i, '.impt')]]
inside with, i[D==0] will not give you x1[D==0] when i is "x1", hence the need to dereference it using get.
you should not name your data.frame sample as it is also the name of a pretty common function

This should work,
test <- sample[,"D"] == 1
for (.name in names(sample)[1:2]){
newvar <- paste(.name, "impt", sep=".")
sample[[newvar]] <- ifelse(test, sample[!test, .name],
sample[test, .name])
}
sample

Related

word_stats function from qdap package application on a dataframe

I have a dataframe, where one column contains strings.
q = data.frame(number=1:2,text=c("The surcingle hung in ribands from my body.", "But a glance will show the fallacy of this idea."))
I want to use the word_stats function for each individual record.
is it possible?
text_statistic <- apply(q,1,word_stats)
this will apply word_stats() row-by-row and return a list with the results of word_stats() for every row
you can do it many ways, lapply or sapply apply a Function over a List or Vector.
word_stats <- function(x) {length(unlist(strsplit(x, ' ')))}
sapply(q$text, word_stats)
Sure have a look at the grouping.var argument:
dat = data.frame(number=1:2,text=c("The surcingle hung in ribands from my body.", "But a glance will show the fallacy of this idea."))
with(dat, qdap::word_stats(text, number))
## number n.sent n.words n.char n.syl n.poly wps cps sps psps cpw spw pspw n.state p.state n.hapax grow.rate
## 1 2 1 10 38 14 2 10 38 14 2 3.800 1.400 .200 1 1 10 1
## 2 1 1 8 35 12 1 8 35 12 1 4.375 1.500 .125 1 1 8 1

R creating variable with satisfying condition

Help sought from anyone.
I have a household survey data set named h2004 and would like to create a variable equals to another variable that satisfy certain condition. Here I have put a sample of observations.
cq15 expen
10 0.4616136
10 1.538712
11 2.308068
11 0.384678
12 2.576797822
12 5.5393632
13 5.4624276
14 2.6158104
14 20.157127
and I tried the following command:
h2004$crops[h2004$cq15>=12 & h2004$cq15<=14]=h2004$expen
and this produces wrong results in R as I know the correct result from using Stata. In the original data set, the above command takes values of 'expen' even when cq15<12 and replaces the observations against cq15>=12 & cq15<=14.
I also tried with filter option of dplyr that correctly subset the data frame but don't know how to apply it to specific variable.
fil<- filter(h2004, cq15>=12 & cq15<=14)
I think my subsetting (cq15>=12 & cq15<=14) is wrong. Please advice. Thanks
The problem is in the command. When the command is executed, the following warning message is issued:
Warning message:
In h2004$crops[h2004$cq15 >= 12 & h2004$cq15 <= 14] = h2004$expen :
number of items to replace is not a multiple of replacement length
The reason for this is that the LHS of this command selects elements satisfying condition h2004$cq15 >= 12 & h2004$cq15 <= 14 while on the RHS, the complete vector h2004$expen is given causing mismatch in length.
Solution:
> h2004$crops[h2004$cq15>=12 & h2004$cq15<=14]=h2004$expen[h2004$cq15>=12 & h2004$cq15<=14]
> h2004
cq15 expen crops
1 10 0.4616136 NA
2 10 1.5387120 NA
3 11 2.3080680 NA
4 11 0.3846780 NA
5 12 2.5767978 2.576798
6 12 5.5393632 5.539363
7 13 5.4624276 5.462428
8 14 2.6158104 2.615810
9 14 20.1571270 20.157127
or Alternatively:
> indices <- which(h2004$cq15>=12 & h2004$cq15<=14)
> h2004$crops[indices] = h2004$expen[indices]
> h2004
cq15 expen crops
1 10 0.4616136 NA
2 10 1.5387120 NA
3 11 2.3080680 NA
4 11 0.3846780 NA
5 12 2.5767978 2.576798
6 12 5.5393632 5.539363
7 13 5.4624276 5.462428
8 14 2.6158104 2.615810
9 14 20.1571270 20.157127

Creating a subset of unique entries for a recursive list in R

I have the following data set df
name draught nav_status date
A 22 0 24/12/2014
A 22 0 25/12/2014
A 11 5 26/12/2014
A 11 1 27/12/2014
B 22 0 24/12/2014
B 22 0 25/12/2014
B 22 0 26/12/2014
B 22 5 27/12/2014
B 9 0 28/12/2014
B 22 0 29/12/2014
from this data set, I need to extract the unique draught values for each object of the list.
I am fairly new to R and have made the following attempts
y <- subset(df,!duplicated(df[,draught]),)
and
Dup <- function(x){
x <- x[!duplicated[x$draught],]
y <- lapply(df, Dup)
But this deletes the draught entries for the entire data. I went through some literature regarding split-apply and combine techniques and also tries those options.
Please provide some guidance, literature so as to solve this problem.
The result should be
name draught nav_status date
A 22 0 24/12/2014
A 11 5 26/12/2014
A 11 1 27/12/2014
B 22 0 25/12/2014
B 9 0 28/12/2014
I even tried to subsetthe data based on first and last entries by arranging them sequentially and deleting the duplicate entries, but there was loss of data.Thank you!!
Using data.table library you can arrive at the result by:
library(data.table)
dt <- as.data.table(df)
unique(dt, by = c('name', 'draught'))
One thing though. Why you have two entries of a pair A 11 in your desired result?

chaining together sequential observations with only current and immediately prior ID values in R

Say I have some data on traits of individuals measured over time, that looks like this:
present <- c(1:4)
pre.1 <- c(5:8)
pre.2 <- c(9:12)
present2 <- c(13:16)
id <- c(present,pre.1,pre.2,present2)
prev.id <- c(pre.1,pre.2,rep(NA,8))
trait <- rnorm(16,10,3)
d <- data.frame(id,prev.id,trait)
print d:
id prev.id trait
1 1 5 10.693266
2 2 6 12.059654
3 3 7 3.594182
4 4 8 14.411477
5 5 9 10.840814
6 6 10 13.712924
7 7 11 11.258689
8 8 12 10.920899
9 9 NA 14.663039
10 10 NA 5.117289
11 11 NA 8.866973
12 12 NA 15.508879
13 13 NA 14.307738
14 14 NA 15.616640
15 15 NA 10.275843
16 16 NA 12.443139
Every observations has a unique value of id. However, some individuals have been observed in the past, and so I also have an observation of prev.id. This allows me to connect an individual with its current and past values of trait. However, some individuals have been remeasured multiple times. Observations 1-4 have previous IDs of 5-8, and observations of 5-8 have previous IDs of 9-12. Observations 9-12 have no previous ID because this is the first time these were measured. Furthermore, observations 13-16 have never been measured before. So, observations 1:4 are unique individuals, observations 5-12 are prior observations of individuals 1-4, and observations 13-16 are another set of unqiue individuals, distinct from 1-4. I would like to write code to generate a table that has every unique individual, as well as every past observation of that individuals trait. The final output would look like:
id <- c(1:4,13:16)
prev.id <- c(5:8, rep(NA,4))
trait <- d$trait[c(1:4,13:16)]
prev.trait.1 <- d$trait[c(5:8 ,rep(NA,4))]
prev.trait.2 <- d$trait[c(9:12,rep(NA,4))]
output<- data.frame(id,prev.id,trait,prev.trait.1,prev.trait.2)
> output
id prev.id trait prev.trait.1 prev.trait.2
1 1 5 10.693266 10.84081 14.663039
2 2 6 12.059654 13.71292 5.117289
3 3 7 3.594182 11.25869 8.866973
4 4 8 14.411477 10.92090 15.508879
5 13 NA 14.307738 NA NA
6 14 NA 15.616640 NA NA
7 15 NA 10.275843 NA NA
8 16 NA 12.443139 NA NA
I can accomplish this in a straightforward manner, but it requires me coding an additional pairing for each previous observation, such that the number of code groups I need to write is the number of times any individual has been recorded. This is a pain, as in the data set I am applying this problem to, there may be anywhere from 0-100 previous observations of an individual.
#first pairing
d.prev <- data.frame(d$id,d$trait,d$prev.id)
colnames(d.prev) <- c('prev.id','prev.trait.1','prev.id.2')
d <- merge(d,d.prev, by = 'prev.id',all.x=T)
#second pairing
d.prev2 <- data.frame(d$id,d$trait,d$prev.id)
colnames(d.prev2) <- c('prev.id.2','prev.trait.2','prev.id.3')
d<- merge(d,d.prev2,by='prev.id.2',all.x=T)
#remove observations that are another individuals previous observation
d <- d[!(d$id %in% d$prev.id),]
How can I go about doing this in fewer lines, so I don't need 100 code chunks to cover individuals that have been remeasured 100 times?
What you have is a forest of linear lists. We'll start at the terminal ends
roots<-d$id[is.na(d$prev.id)]
And determine the paths backwards
path <- function(node) {
a <- integer(nrow(d))
i <- 0
while(!is.na(node)) {
i <- i+1
a[i] <- node
node <- d$id[match(node,d$prev.id)]
}
return(rev(a[1:i]))
}
Then we can get a 'stacked' representation of your desired output with
x<-do.call(rbind,lapply(roots,
function(r) {p<-path(r); data.frame(id=p[[1]],seq=seq_along(p),traits=d$trait[p])}))
And then use reshape2::dcast to get it in the desired shape
library(reshape2)
dcast(x,id~seq,fill=NA,value.var='traits')
id 1 2 3
1 1 10.693266 10.84081 14.663039
2 2 12.059654 13.71292 5.117289
3 3 3.594182 11.25869 8.866973
4 4 14.411477 10.92090 15.508879
5 13 14.307738 NA NA
6 14 15.616640 NA NA
7 15 10.275843 NA NA
8 16 12.443139 NA NA
I leave it to you to adapt column names.

Merging columns from different data frames

I have a problem....
I have two data frames
>anna1
name from to result
11 66607 66841 0
11 66846 67048 0
11 67053 67404 0
11 67409 68216 0
11 68221 68786 0
11 68791 69020 0
11 69025 69289 0
11 69294 70167 0
11 70172 70560 0
and the second data frame is
>anna2
name from to result
11 66607 66841 5
11 66846 67048 6
11 67409 68216 7
11 69025 69289 12
11 70172 70560 45
What I want is to create a new data frame similar with the anna1 where all the 0 values will be replaced by the correct results in the correct row from the anna2
you are going to notice that in the anna2 data frame, in the from and to columns have only some same values with the respective in the anna1 data frame
....the intermediate are missing
So i need somehow to take the numbers from the result column in the anna2 and put them in the correct row in the anna1
thank you in advance
Best regards
Anna
A simpler merge:
anna3 <-merge(anna2,anna1[,1:3], all.y=TRUE)
anna3[is.na(anna3)] <- 0
Gives:
> anna3
name from to result
1 11 66607 66841 5
2 11 66846 67048 6
3 11 67053 67404 0
4 11 67409 68216 7
5 11 68221 68786 0
6 11 68791 69020 0
7 11 69025 69289 12
8 11 69294 70167 0
9 11 70172 70560 45
If the "from" column is guaranteed to be unique in both anna1 and anna2, AND every row in anna2 has a matching row in anna1 (though not vice versa), a simple solution is
row.index = function(d) which(anna1$from == d)[1]
indices = sapply(anna2$from, row.index)
anna1$result[indices] = anna2$result
Another approach
require(plyr)
anna <- rbind(anna1, anna2)
ddply(anna, .(name, from, to), summarize, result = sum(result))
EDIT. If the data frames are large, and speed is an issue, think of using data.table
require(data.table)
data.table(anna)[,list(result = sum(result)),'name, from, to']
You can use merge, but you have to explicitly specify what should be done with the two result columns.
d <- merge(anna1, anna2, by=c("name", "from", "to"), all=TRUE)
d$result <- ifelse(d$result.x == 0 & !is.na( d$result.y ), d$result.y, d$result.x)
d <- d[,c("name", "from", "to", "result")]

Resources