I have ordered a set of rows to get this:
2 1983 TRI-COUNTY TRAUTH 0.1495 0.1395 NA 452 0.0764 4 0 06/02/83
4 1983 TRI-COUNTY TRAUTH 0.1193 0.1113 NA 32 0.0764 4 2 07/20/83
14 1983 TRI-COUNTY TRAUTH 0.1064 0.1064 NA 26 0.0763 6 2 08/03/83
17 1983 TRI-COUNTY TRAUTH 0.1110 0.1010 0.1010 176 0.0763 7 4 08/08/83
24 1983 TRI-COUNTY TRAUTH 0.1293 0.1215 NA 452 0.0763 4 0 09/12/83
41 1984 TRI-COUNTY TRAUTH 0.1325 0.1225 NA 452 0.0740 4 0 06/20/84
45 1984 TRI-COUNTY TRAUTH 0.1425 0.1325 NA 32 0.0741 4 2 07/17/84
47 1984 TRI-COUNTY TRAUTH 0.1395 0.1395 0.1250 91 0.0741 14 11 07/16/84
But I want to renumber these such that its 1,2,3,4,etc...
Can someone please help?
Are you just looking for something like this?:
row.names(datasetname) <- 1:nrow(datasetname)
Alternatively, if the first column in your example data is a variable (say V1) in a dataframe and not the row.names, this will work:
datasetname$V1 <- 1:nrow(datasetname)
This is the easiest way to do it:
rownames(dataset) = NULL
Another solution, normally used when binding rows:
dataset <- rbind( dataset , make.rows.names=FALSE )
Related
This question already has answers here:
Replacing NAs with latest non-NA value
(21 answers)
Closed 3 years ago.
I have a dataframe like this:
ID year fcmstat secmstat mstat
138 4 1998 NA NA 1
139 4 1999 NA NA 1
140 4 2000 NA NA 1
141 4 2001 NA NA 1
142 4 2002 NA NA 1
143 4 2003 2 NA 2
144 4 2004 NA NA NA
145 4 2005 NA NA NA
146 4 2006 NA 3 3
147 4 2007 NA NA NA
375 19 2001 NA NA 2
376 19 2002 6 NA 6
377 19 2003 NA NA NA
378 19 2004 NA 5 5
379 19 2005 NA NA NA
380 19 2006 NA NA 1
fcmstat: type of first marital status change
secmstat: type of second marital status change
first marital status, for ID 4(19), fsmstat was changed in 2003(2002) and second marital status secmstat was changed in 2006(2004). So, for ID 4, in 2004 and 2005 marital status was same as fcmstat of 2003 and for ID 19, 2003's mstat should be same as fcmstat of 2002.
I want to fill in t he last column as follows:
ID year fcmstat secmstat mstat
138 4 1998 NA NA 1
139 4 1999 NA NA 1
140 4 2000 NA NA 1
141 4 2001 NA NA 1
142 4 2002 NA NA 1
143 4 2003 2 NA 2
144 4 2004 NA NA 2
145 4 2005 NA NA 2
146 4 2006 NA 3 3
147 4 2007 NA NA NA
375 19 2001 NA NA 2
376 19 2002 6 NA 6
377 19 2003 NA NA 6
378 19 2004 NA 5 5
379 19 2005 NA NA NA
380 19 2006 NA NA 1
Also, before any first change, the mstatshould be same as before. Consider the following case.
ID year fcmstat secmstat mstat
1171 61 1978 NA NA 0
1172 61 1979 NA NA 0
1173 61 1980 NA NA 0
1174 61 1981 NA NA 0
1175 61 1982 NA NA 0
1176 61 1983 NA NA NA
1177 61 1984 NA NA NA
1178 61 1985 1 NA 1
1179 61 1986 NA NA 1
1180 61 1987 NA NA 1
the first change was in 1985. So, the missing mstat in 1984 and 1983 should be same as mstat of 1982. SO for this case, my desired output is:
ID year fcmstat secmstat mstat
1171 61 1978 NA NA 0
1172 61 1979 NA NA 0
1173 61 1980 NA NA 0
1174 61 1981 NA NA 0
1175 61 1982 NA NA 0
1176 61 1983 NA NA 0
1177 61 1984 NA NA 0
1178 61 1985 1 NA 1
1179 61 1986 NA NA 1
1180 61 1987 NA NA 1
As suggested by Schilker the code df$mstat_updated<-na.locf(df$mstat) gives the following:
ID year fcmstat secmstat mstat mstat_updated
138 4 1998 NA NA 1 1
139 4 1999 NA NA 1 1
140 4 2000 NA NA 1 1
141 4 2001 NA NA 1 1
142 4 2002 NA NA 1 1
143 4 2003 2 NA 2 2
144 4 2004 NA NA NA 2
145 4 2005 NA NA NA 2
146 4 2006 NA 3 3 3
147 4 2007 NA NA NA 3
148 4 2008 NA NA NA 3
However, I do want to fill in mstat for 2004 and 2005 but not in 2007 and 2008. I want to fill in NA's only between first marstat change, fcmstat and second marstat, secmstat change.
As I mentioned in my comment this a duplicate of here
library(zoo)
df<-data.frame(ID=c('4','4','4','4'),
year=c(2003,2004,2005,2006),
mstat=c(2,NA,NA,3))
df$mstat<-na.locf(df$mstat)
A sample of my data is available here.
I am trying to calculate the growth rate (change in weight (wt) over time) for each squirrel.
When I have my data in wide format:
squirrel fieldBirthDate date1 date2 date3 date4 date5 date6 age1 age2 age3 age4 age5 age6 wt1 wt2 wt3 wt4 wt5 wt6 litterid
22922 2017-05-13 2017-05-14 2017-06-07 NA NA NA NA 1 25 NA NA NA NA 12 52.9 NA NA NA NA 7684
22976 2017-05-13 2017-05-16 2017-06-07 NA NA NA NA 3 25 NA NA NA NA 15.5 50.9 NA NA NA NA 7692
22926 2017-05-13 2017-05-16 2017-06-07 NA NA NA NA 0 25 NA NA NA NA 10.1 48 NA NA NA NA 7719
I am able to calculate growth rate with the following code:
library(dplyr)
#growth rate between weight 1 and weight 3, divided by age when weight 3 is recorded
growth <- growth %>%
mutate (g.rate=((wt3-wt1)/age3))
#growth rate between weight 1 and weight 2, divided by age when weight 2 is recorded
merge.growth <- merge.growth %>%
mutate (g.rate=((wt2-wt1)/age2))
However, when the data is in long format (a format needed for the analysis I am running afterwards):
squirrel litterid date age wt
22922 7684 2017-05-13 0 NA
22922 7684 2017-05-14 1 12
22922 7684 2017-06-07 25 52.9
22976 7692 2017-05-13 1 NA
22976 7692 2017-05-16 3 15.5
22976 7692 2017-06-07 25 50.9
22926 7719 2017-05-14 0 10.1
22926 7719 2017-06-08 25 48
I cannot use the mutate function I used above. I am hoping to create a new column that includes growth rate as follows:
squirrel litterid date age wt g.rate
22922 7684 2017-05-13 0 NA NA
22922 7684 2017-05-14 1 12 NA
22922 7684 2017-06-07 25 52.9 1.704
22976 7692 2017-05-13 1 NA NA
22976 7692 2017-05-16 3 15.5 NA
22976 7692 2017-06-07 25 50.9 1.609
22926 7719 2017-05-14 0 10.1 NA
22926 7719 2017-06-08 25 48 1.516
22758 7736 2017-05-03 0 8.8 NA
22758 7736 2017-05-28 25 43 1.368
22758 7736 2017-07-05 63 126 1.860
22758 7736 2017-07-23 81 161 1.879
22758 7736 2017-07-26 84 171 1.930
I have been calculating the growth rates (growth between each wt and the first time it was weighed) in excel, however I would like to do the calculations in R instead since I have a large number of squirrels to work with. I suspect if else loops might be the way to go here, but I am not well versed in that sort of coding. Any suggestions or ideas are welcome!
You can use group_by to calculate this for each squirrel:
group_by(df, squirrel) %>%
mutate(g.rate = (wt - nth(wt, which.min(is.na(wt)))) /
(age - nth(age, which.min(is.na(wt)))))
That leaves NaNs where the age term is zero, but you can change those to NAs if you want with df$g.rate[is.nan(df$g.rate)] <- NA.
alternative using data.table and its function "shift" that takes the previous row
library(data.table)
df= data.table(df)
df[,"growth":=(wt-shift(wt,1))/age,by=.(squirrel)]
I have the following data.table:
Month Day Lat Long Temperature
1: 10 01 80.0 180 -6.383330333333309
2: 10 01 77.5 180 -6.193327999999976
3: 10 01 75.0 180 -6.263328333333312
4: 10 01 72.5 180 -5.759997333333306
5: 10 01 70.0 180 -4.838330999999976
---
117020: 12 31 32.5 310 11.840003833333355
117021: 12 31 30.0 310 13.065001833333357
117022: 12 31 27.5 310 14.685003333333356
117023: 12 31 25.0 310 15.946669666666690
117024: 12 31 22.5 310 16.578336333333358
For every location (given by Lat and Long), I have a temperature for each day from 1 October to 31 December.
There are 1,272 locations consisting of each pairwise combination of Lat:
Lat
1 80.0
2 77.5
3 75.0
4 72.5
5 70.0
--------
21 30.0
22 27.5
23 25.0
24 22.5
and Long:
Long
1 180.0
2 182.5
3 185.0
4 187.5
5 190.0
---------
49 300.0
50 302.5
51 305.0
52 307.5
53 310.0
I'm trying to create a data.table that consists of 1,272 rows (one per location) and 92 columns (one per day). Each element of that data.table will then contain the temperature at that location on that day.
Any advice about how to accomplish that goal without using a for loop?
Here we use ChickWeights as the data, where we use "Chick-Diet" as the equivalent of your "lat-lon", and "Time" as your "Date":
dcast.data.table(data.table(ChickWeight), Chick + Diet ~ Time)
Produces:
Chick Diet 0 2 4 6 8 10 12 14 16 18 20 21
1: 18 1 1 1 NA NA NA NA NA NA NA NA NA NA
2: 16 1 1 1 1 1 1 1 1 NA NA NA NA NA
3: 15 1 1 1 1 1 1 1 1 1 NA NA NA NA
4: 13 1 1 1 1 1 1 1 1 1 1 1 1 1
5: ... 46 rows omitted
You will likely need to lat + lon ~ Month + Day or some such for your formula.
In the future, please make your question reproducible as I did here by using a built-in data set.
First create a date value using the lubridate package (I assumed year = 2014, adjust as necessary):
library(lubridate)
df$datetext <- paste(df$Month,df$Day,"2014",sep="-")
df$date <- mdy(df$datetext)
Then one option is to use the tidyr package to spread the columns:
library(tidyr)
spread(df[,-c(1:2,6)],date,Temperature)
Lat Long 2014-10-01 2014-12-31
1 22.5 310 NA 16.57834
2 25.0 310 NA 15.94667
3 27.5 310 NA 14.68500
4 30.0 310 NA 13.06500
5 32.5 310 NA 11.84000
6 70.0 180 -4.838331 NA
7 72.5 180 -5.759997 NA
8 75.0 180 -6.263328 NA
9 77.5 180 -6.193328 NA
10 80.0 180 -6.383330 NA
I would like to plot things like (where C is column):
C4 vs C2 for all similar C1 and
C1 vs C4 for all similar C2
The data frame in question is:
C1 C2 C3 C4
1 2012-12-28 0 NA 10773
2 2012-12-28 5 NA 34112
3 2012-12-28 10 NA 30901
4 2012-12-28 0 NA 12421
5 2012-12-30 0 NA 3925
6 2012-12-30 5 NA 17436
7 2012-12-30 10 NA 13717
8 2012-12-30 15 NA 36708
9 2012-12-30 20 NA 28408
10 2012-12-30 NA NA 2880
11 2013-01-02 0 -13.89 9972
12 2013-01-02 5 -13.89 10576
13 2013-01-02 10 -13.89 33280
14 2013-01-02 15 -13.89 28667
15 2013-01-02 20 -13.89 21104
16 2013-01-02 25 -13.89 24771
17 2013-01-02 NA NA 22
18 2013-01-05 0 -3.80 20727
19 2013-01-05 5 -3.80 2033
20 2013-01-05 10 -3.80 16045
21 2013-01-05 15 -3.80 12074
22 2013-01-05 20 -3.80 10095
23 2013-01-05 NA NA 32693
24 2013-01-08 0 -1.70 19579
25 2013-01-08 5 -1.70 20200
26 2013-01-08 10 -1.70 12263
27 2013-01-08 15 -1.70 28797
28 2013-01-08 20 -1.70 23963
29 2013-01-11 0 -2.30 26525
30 2013-01-11 5 -2.30 21472
31 2013-01-11 10 -2.30 9633
32 2013-01-11 15 -2.30 27849
33 2013-01-11 20 -2.30 23950
34 2013-01-17 0 1.40 16271
35 2013-01-17 5 1.40 18581
36 2013-01-19 0 0.10 5910
37 2013-01-19 5 0.10 16890
38 2013-01-19 10 0.10 13078
39 2013-01-19 NA NA 55
40 2013-01-23 0 -9.20 15048
41 2013-01-23 6 -9.20 20792
42 2013-01-26 0 NA 21649
43 2013-01-26 6 NA 24655
44 2013-01-29 0 0.10 9100
45 2013-01-29 5 0.10 27514
46 2013-01-29 10 0.10 19392
47 2013-01-29 15 0.10 21720
48 2013-01-29 NA 0.10 112
49 2013-02-11 0 0.40 13619
50 2013-02-11 5 0.40 2748
51 2013-02-11 10 0.40 1290
52 2013-02-11 15 0.40 762
53 2013-02-11 20 0.40 1125
54 2013-02-11 25 0.40 1709
55 2013-02-11 30 0.40 29459
56 2013-02-11 35 0.40 106474
57 2013-02-13 0 1.30 3355
58 2013-02-13 5 1.30 970
59 2013-02-13 10 1.30 2240
60 2013-02-13 15 1.30 35871
61 2013-02-18 0 -0.60 8564
62 2013-02-20 0 -1.20 12399
63 2013-02-26 0 0.30 2985
64 2013-02-26 5 0.30 9891
65 2013-03-01 0 0.90 5221
66 2013-03-01 5 0.90 9736
67 2013-03-05 0 0.60 3192
68 2013-03-05 5 0.60 4243
69 2013-03-09 0 0.10 45138
70 2013-03-09 5 0.10 55534
71 2013-03-12 0 1.40 7278
72 2013-03-12 NA NA 45
73 2013-03-15 0 0.30 2447
74 2013-03-15 5 0.30 2690
75 2013-03-18 0 -2.30 3008
76 2013-03-22 0 -0.90 11411
77 2013-03-22 5 -0.90 NA
78 2013-03-22 10 -0.90 17675
79 2013-03-22 NA NA 47
80 2013-03-25 0 1.20 9802
81 2013-03-25 5 1.20 15790
There are other posts here about time series subseting and merging/matching/pasting subseting, but I think I miss the point when I'm trying to follow those instructions.
The end goal is to have a plot of C1 vs C4 for every C2 = 0 C2 = 5 and so on. Same thing for C4 vs C2 for every same C1. I know there are some duplicate C1 and C2, but the C4 for those values can be averaged. I can figure these plots out, I just need to know how to subset the data in this way. Perhaps creating a new data.frame() with these subsets could be the easiest?
Thanks in advance,
It's relatively easy to plot subsets using ggplot2. First you need to reshape your data from "wide" to "long" format, creating a new categorical variable with possible values C4 and C5.
library(reshape2)
library(ggplot2)
# Starting with the data you posted in a data frame called "dat":
# Convert C2 to date format
dat$C2 = as.Date(dat$C2)
# Reshape data to long format
dat.m = melt(dat, id.var=c("C1","C2","C3"))
# Plot values of C4 and C5 vs. C2 with separate lines for each level of C3
ggplot(dat.m, aes(x=C2, y=value, group=C3, colour=as.factor(C3))) +
geom_line() + geom_point() +
facet_grid(variable ~ ., scales="free_y")
The C4 lines are the same for every level of C3, so they all overlap each other.
You can also have a separate panel for each level of C3:
ggplot(dat.m, aes(x=C2, y=value, group=variable, colour=variable)) +
geom_line() + geom_point() +
facet_grid(variable ~ C3, scales="free_y") +
theme(axis.text.x=element_text(angle=-90)) +
guides(colour=FALSE)
Here's a base graphics method to getting separate plots. I'm using your new column names below:
# Use lapply to create a separate plot for each level of C2
lapply(na.omit(unique(dat$C2)), function(x) {
# The next line of code removes NA values so that there will be a line through
# every point. You can remove this line if you don't care whether all points
# are connected or not.
dat = dat[complete.cases(dat[,c("C1","C2","C4")]),]
# Create a plot of C4 vs. C1 for the current value of C2
plot(dat$C1[dat$C2==x], dat$C4[dat$C2==x],
type="o", pch=16,
xlab=paste0("C2=",x), ylab="C4")
})
I am struggling with the following problem:
The dataframe below contains the development of a value over time for various ids. What i try to get is the increase/decrease of these values based on a the value in a year when event occurred. Several events can occur within one id, so a new event becomes the new baseline year for the id.
To make things clearer, I also add the outcome I want below
What i have
id value year event
a 100 1950 NA
a 101 1951 NA
a 102 1952 NA
a 103 1953 NA
a 104 1954 NA
a 105 1955 X
a 106 1956 NA
a 107 1957 NA
a 108 1958 NA
a 107 1959 Y
a 106 1960 NA
a 105 1961 NA
a 104.8 1962 NA
a 104.2 1963 NA
b 70 1970 NA
b 75 1971 NA
b 80 1972 NA
b 85 1973 NA
b 90 1974 NA
b 60 1975 Z
b 59 1976 NA
b 58 1977 NA
b 57 1978 NA
b 56 1979 NA
b 55 1980 W
b 54 1981 NA
b 53 1982 NA
b 52 1983 NA
b 51 1984 NA
What I am looking for
id value year event index growth
a 100 1950 NA 0
a 101 1951 NA 0
a 102 1952 NA 0
a 103 1953 NA 0
a 104 1954 NA 0
a 105 1955 X 1 1
a 106 1956 NA 2 1.00952381
a 107 1957 NA 3 1.019047619
a 108 1958 NA 4 1.028571429
a 107 1959 Y 1 1 #new baseline year
a 106 1960 NA 2 0.990654206
a 105 1961 NA 3 0.981308411
a 104.8 1962 NA 4 0.979439252
a 104.2 1963 NA 5 0.973831776
b 70 1970 NA 6
b 75 1971 NA 7
b 80 1972 NA 8
b 85 1973 NA 9
b 90 1974 NA 10
b 60 1975 Z 1 1
b 59 1976 NA 2 0.983333333
b 58 1977 NA 3 0.966666667
b 57 1978 NA 4 0.95
b 56 1979 NA 5 0.933333333
b 55 1980 W 1 1 #new baseline year
b 54 1981 NA 2 0.981818182
b 53 1982 NA 3 0.963636364
b 52 1983 NA 4 0.945454545
b 51 1984 NA 5 0.927272727
What I tried
This and this post were quite helpful and I managed to create differences between the years, however, I fail to reset the base year (index) when there is a new event. Furthermore, I am doubtful whether my approach is indeed the most efficient/elegant one. Seems a bit clumsy to me...
x <- ddply(x, .(id), transform, year.min=min(year[!is.na(event)])) #identifies first event year
x1 <- ddply(x[x$year>=x$year.min,], .(id), transform, index=seq_along(id)) #creates counter years following first event; prior years are removed
x1 <- x1[order(x1$id, x1$year),] #sort
x1 <- ddply(x1, .(id), transform, growth=100*(value/value[1])) #calculate difference, however, based on first event year; this is wrong.
library(Interact) #i then merge the df with the years prior to first event which have been removed in the begining
x$id.year <- interaction(x$id,x$year)
x1$id.year <- interaction(x1$id,x1$year)
x$index <- x$growth <- NA
y <- rbind(x[x$year<x$year.min,],x1)
y <- y[order(y$id,y$year),]
Many thanks for any advice.
# Create a tag to indicate the start of each new event by id or
# when id changes
dat$tag <- with(dat, ave(as.character(event), as.character(id),
FUN=function(i) cumsum(!is.na(i))))
# Calculate the growth by id and tag
# this will also produce results for each id before an event has happened
dat$growth <- with(dat, ave(value, tag, id, FUN=function(i) i/i[1] ))
# remove growth prior to an event (this will be when tag equals zero as no
# event have occurred)
dat$growth[dat$tag==0] <- NA
Here is a solution with dplyr.
ana <- group_by(mydf, id) %>%
do(na.locf(., na.rm = FALSE)) %>%
mutate(value = as.numeric(value)) %>%
group_by(id, event) %>%
mutate(growth = value/value[1]) %>%
mutate(index = row_number(event))
ana$growth[is.na(ana$event)] <- 0
id value year event growth index
1 a 100.0 1950 NA 0.0000000 1
2 a 101.0 1951 NA 0.0000000 2
3 a 102.0 1952 NA 0.0000000 3
4 a 103.0 1953 NA 0.0000000 4
5 a 104.0 1954 NA 0.0000000 5
6 a 105.0 1955 X 1.0000000 1
7 a 106.0 1956 X 1.0095238 2
8 a 107.0 1957 X 1.0190476 3
9 a 108.0 1958 X 1.0285714 4
10 a 107.0 1959 Y 1.0000000 1
11 a 106.0 1960 Y 0.9906542 2
12 a 105.0 1961 Y 0.9813084 3
13 a 104.8 1962 Y 0.9794393 4
14 a 104.2 1963 Y 0.9738318 5
15 b 70.0 1970 NA 0.0000000 1
16 b 75.0 1971 NA 0.0000000 2
17 b 80.0 1972 NA 0.0000000 3
18 b 85.0 1973 NA 0.0000000 4
19 b 90.0 1974 NA 0.0000000 5
20 b 60.0 1975 Z 1.0000000 1
21 b 59.0 1976 Z 0.9833333 2
22 b 58.0 1977 Z 0.9666667 3
23 b 57.0 1978 Z 0.9500000 4
24 b 56.0 1979 Z 0.9333333 5
25 b 55.0 1980 W 1.0000000 1
26 b 54.0 1981 W 0.9818182 2
27 b 53.0 1982 W 0.9636364 3
28 b 52.0 1983 W 0.9454545 4
Try:
ddf$index=0
ddf$growth=0
baseline =0
r=1; start=FALSE
for(r in 1:nrow(ddf)){
if(is.na(ddf$event[r])){
if(start) {
ddf$index[r] = ddf$index[r-1]+1
ddf$growth[r] = ddf$value[r]/baseline
}
else {ddf$index[r] = 0;
}
}
else{
start=T
ddf$index[r] = 1
ddf$growth[r]=1
baseline = ddf$value[r]
}
}
ddf
id value year event index growth
1 a 100.0 1950 <NA> 0 0.0000000
2 a 101.0 1951 <NA> 0 0.0000000
3 a 102.0 1952 <NA> 0 0.0000000
4 a 103.0 1953 <NA> 0 0.0000000
5 a 104.0 1954 <NA> 0 0.0000000
6 a 105.0 1955 X 1 1.0000000
7 a 106.0 1956 <NA> 2 1.0095238
8 a 107.0 1957 <NA> 3 1.0190476
9 a 108.0 1958 <NA> 4 1.0285714
10 a 107.0 1959 Y 1 1.0000000
11 a 106.0 1960 <NA> 2 0.9906542
12 a 105.0 1961 <NA> 3 0.9813084
13 a 104.8 1962 <NA> 4 0.9794393
14 a 104.2 1963 <NA> 5 0.9738318
15 b 70.0 1970 <NA> 6 0.6542056
16 b 75.0 1971 <NA> 7 0.7009346
17 b 80.0 1972 <NA> 8 0.7476636
18 b 85.0 1973 <NA> 9 0.7943925
19 b 90.0 1974 <NA> 10 0.8411215
20 b 60.0 1975 Z 1 1.0000000
21 b 59.0 1976 <NA> 2 0.9833333
22 b 58.0 1977 <NA> 3 0.9666667
23 b 57.0 1978 <NA> 4 0.9500000
24 b 56.0 1979 <NA> 5 0.9333333
25 b 55.0 1980 W 1 1.0000000
26 b 54.0 1981 <NA> 2 0.9818182
27 b 53.0 1982 <NA> 3 0.9636364
28 b 52.0 1983 <NA> 4 0.9454545
29 b 51.0 1984 <NA> 5 0.9272727