Creating an expanded matrix with interpolated data in R - r

Question

I have a data frame that has annual data for population by MSA. They are organized as follows:
MSA FIPS x1969 x1970 x1971 .... x2012
Akron 123 12 14 17 .... 22
Miami 234 23 20 24 .... 29
etc.
I need to reshape the data into
MSA FIPS Year Data
Akron 123 1969 12
Akron 123 1970 14
Akron 123 1971 17
...
I can do this using "melt", but I also want to interpolate these annual data to include quarterly data points for the full time series. So, how best to create the quarterly (interpolated) matrix on the fly?
I can do this using a loop over the rows of the first matrix above and then use melt to reshape the new data, but I've been asked to slap myself anytime I catch myself building explicitly coded loops.
I've been tinkering with "apply", but it creates a list of lists -- which would then require assembling the final data frame.
I can feel a simple solution must be out there.
Thanks, Chris.

Answer 1

May be you could try td from tempdisagg
library(tempdisagg)
library(reshape2)
library(zoo)
dM <- transform(melt(df, id.var=c('MSA', 'FIPS')),
variable=as.numeric(gsub('^x', '', variable)))
res <- lapply(split(dM, dM$MSA), function(x) {
val <- ts(x$value, start=x$variable[1], end=x$variable[nrow(x)])
val2 <-predict(td(val~1, to='quarterly', method='uniform'))
#change the options as needed
data.frame(yearQtr= as.yearqtr(time(val2)), val=val2)})
data
df <- structure(list(MSA = c("Akron", "Miami"), FIPS = c(123L, 234L
), x1969 = c(12L, 23L), x1970 = c(14L, 20L), x1971 = c(17L, 24L
)), .Names = c("MSA", "FIPS", "x1969", "x1970", "x1971"), class = "data.frame",
row.names = c(NA, -2L))

Answer 2

This builds on #akrun earlier:
#His data frame build:
df <- structure(list(MSA = c("Akron", "Miami"), FIPS = c(123L, 234L),
x1969 = c(12L, 23L), x1970 = c(14L, 20L), x1971 = c(17L, 24L)),
.Names = c("MSA", "FIPS", "x1969", "x1970", "x1971"), class = "data.frame",
row.names = c(NA, -2L))
#His set up:
dM <- transform(melt(df, id.var=c('MSA', 'FIPS')),
variable=as.numeric(gsub('^x', '', variable)))
#My variation on his lapply:
res <- lapply(split(dM, dM$MSA), function(x) {
xseq=seq(min(x$variable),max(x$variable),by=.25)
val <- approx(x$variable,x$value,xout=xseq)
data.frame(yearQtr=xseq,val=val$y)})
df.new <- do.call(rbind.data.frame,res)
It's not quite perfect, but I'll get back to it later. We're close. Thank you #akrun