I have one main XTS object "Data" with ~1M rows spanning 22 days. I have another XTS object "Set" with 22 rows, with 1 entry per day. I would like to combine this smaller XTS object into the larger one, such that it would have an additional column containing the value in Set for that day.
First I tried:
> Data=cbind(Data,as.numeric(Set[as.Date(index(Data[]))]))
Error in error(x, ...) :
improper length of one or more arguments to merge.xts
Then I tried:
> Data=cbind(Data,1)
> Data[,6]=as.numeric(Set[as.Date(index(Data[,6]))])
Error in NextMethod(.Generic) :
number of items to replace is not a multiple of replacement length
I also tried without the as.numeric but received the same error. I tried turning Data into a data.frame and got the error:
Error in `[<-.data.frame`(`*tmp*`, , 6, value = c(1, 397.16, 397.115, :
replacement has 22 rows, data has 835771
What am I doing wrong and how do I make this happen? I've only been using R the past two weeks.
Thanks!
> str(Data)
An ‘xts’ object from 2012-01-03 05:01:05 to 2012-01-31 14:59:59 containing:
Data: num [1:835771, 1:5] 397 397 397 397 397 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "SYN" "\"WhitePack.BID_SIZE\"" "\"WhitePack.BID_PRICE\"" "\"WhitePack.ASK_PRICE\"" ...
Indexed by objects of class: [POSIXct,POSIXt] TZ:
xts Attributes:
NULL
> str(Set)
An ‘xts’ object from 2012-01-02 to 2012-01-31 containing:
Data: chr [1:22, 1] " 1.000" "397.160" "397.115" "397.175" "397.200" "397.390" "397.560" "397.580" "397.715" ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr "Settle"
Indexed by objects of class: [POSIXct,POSIXt] TZ:
xts Attributes:
NULL
Do you get success with :
df3 <- merge(Data, Set)
To address my lack of full understanding of the original problem, I think the only additional step would be:
df3[, 6] <- na.locf( df3[, 6] )
Related
I'm trying to operate on a specific column in an xts object by name within a function but I keep getting an error:
Error in if (length(c(year, month, day, hour, min, sec)) == 6 && all(c(year, :
missing value where TRUE/FALSE needed
In addition: Warning messages:
1: In as_numeric(YYYY) : NAs introduced by coercion
2: In as_numeric(YYYY) : NAs introduced by coercion
If I have an xts object:
xts1 <- xts(x=1:10, order.by=Sys.Date()-1:10)
xts2 <- xts(x=1:10, order.by=Sys.Date()+1:10)
xts3 <- merge(xts1, xts2)
Then I can select a specific column with:
xts3$xts1
With a dataframe I can pass xts3 to another function and then select a specific column with:
xts3['xts1']
But if I try to do the same thing with an xts object I get the error above. e.g.
testfun <- function(xts_data){
print(xts_data['xts1'])
}
Called with:
testfun(xts3)
This works:
testfun <- function(xts_data){
print(xts_data[,1])
}
But I'd really like to select by name as I can't be certain of the column order.
Can anyone suggest how to solve this?
Thanks!
xts-objects have class c("xts", "zoo"), which means they are matrices with special attributes that are assigned by their creation functions. Although $ will not succeed with a matrix, it works with xts and zoo objects thanks to the $.zoo method. (It's also not recommended to use $ inside functions because of the potential for name-evaluation-confusion and partial name matching.) See: ?xts and examine the sample.xts object created with the first example with str:
> ?xts
starting httpd help server ... done
> data(sample_matrix)
> sample.xts <- as.xts(sample_matrix, descr='my new xts object')
>
> str(sample.xts)
An ‘xts’ object on 2007-01-02/2007-06-30 containing:
Data: num [1:180, 1:4] 50 50.2 50.4 50.4 50.2 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:4] "Open" "High" "Low" "Close"
Indexed by objects of class: [POSIXct,POSIXt] TZ:
xts Attributes:
List of 1
$ descr: chr "my new xts object"
class(sample.xts)
# [1] "xts" "zoo"
This explains why the earlier answer advising the use of xts3[ , "x"] or equivalently xts3[ , 1] should succeed. The [.xts function extracts the "Data" element first and then returns the either named or numbered column specified by the j-argument.
str(xts3)
An ‘xts’ object on 2018-05-24/2018-06-13 containing:
Data: int [1:20, 1:2] 10 9 8 7 6 5 4 3 2 1 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:2] "xts1" "xts2"
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
NULL
> xts3[ , "xts1"]
xts1
2018-05-24 10
2018-05-25 9
2018-05-26 8
2018-05-27 7
2018-05-28 6
2018-05-29 5
2018-05-30 4
2018-05-31 3
2018-06-01 2
2018-06-02 1
2018-06-04 NA
2018-06-05 NA
2018-06-06 NA
2018-06-07 NA
2018-06-08 NA
2018-06-09 NA
2018-06-10 NA
2018-06-11 NA
2018-06-12 NA
2018-06-13 NA
The merge.xts operation might not have delivered what you expected since the date ranges didn't overlap. It seems possible that you wanted:
> xts4 <- rbind(xts1, xts2)
> str(xts4)
An ‘xts’ object on 2018-05-24/2018-06-13 containing:
Data: int [1:20, 1] 10 9 8 7 6 5 4 3 2 1 ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
NULL
Note that the rbind.xts-operation failed to deliver an object with the shared column name so numeric access would be needed. (I would have expected a named "Data" element, but you/we also need to read ?rbind.xts.)
Type ?`[.xts` and you'll see that the function has a i and a j argument (among others).
i - the rows to extract. Numeric, timeBased or ISO-8601 style (see details)
j - the columns to extract, numeric or by name
You passed 'xts1' as the i argument, while it should be j. So your function should be
testfun <- function(xts_data){
print(xts_data[, 'xts1']) # or xts3[j = 'xts1']
}
I enter a headed Excel CSV and examine with str(returns.xts). The following code generates character values within the xts.
file <- "~/GCS/returns_Q216.csv"
returns_Q216_ <- read.csv(file=file)
returns <- read.zoo(data.frame(returns_Q216_), FUN = as.Date, format='%d/%m/%Y')
returns.xts <- as.xts(returns)
What is the best way to convert the xts contents to numeric from character whilst preserving xts (and date column)?
> `str(returns)`
An ‘xts’ object on 2007-01-31/2015-05-31 containing:
Data: `chr` [1:101, 1:18] "-0.002535663" "-0.001687755" "0.032882512" "0.024199512" "0.027812955" ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:18] "UK.EQUITY" "EUR.EQUITY" "NA.EQUITY" "ASIA.EQUITY" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
NULL
> returns[8,9]
PROPERTY
2007-08-31 "-4.25063E-05"
When I try as.numeric(returns.xts) I get a structure 1x1 cell without the date as row.
> str(as.numeric(returns))
num [1:1818] -0.00254 -0.00169 0.03288 0.0242 0.02781 ...
You should use the na.strings argument to read.csv (which can be passed via read.zoo), as I said in my answer to your previous question.
file <- "~/GCS/returns_Q216.csv"
returns <- read.zoo(file, FUN=as.Date, format='%d/%m/%Y', na.strings="#N/A")
returns.xts <- as.xts(returns)
I looked many entries on merging R data frames, however they are not clear to me, they talk about merging/joining using a common column, but in my case its missed or may I don't know how to extract. Here is what I am doing.
library(quantmod)
library(xts)
start = '2001-01-01'
end = '2015-08-14'
ticker = 'AAPL'
f = getSymbols(ticker, src = 'yahoo', from = start, to = end, auto.assign=F)
rsi14 <- RSI(f$AAPL.Adjusted,14)
The output I am expecting is all the columns of f and rsi14 match by date, however 'date' is not available as column, so not sure how do I join. I have to join few Moving Average columns as well.
The premise of your question is wrong. getSymbols returns an xts object, not a data.frame:
R> library(quantmod)
R> f <- getSymbols("AAPL", auto.assign=FALSE)
R> str(f)
An ‘xts’ object on 2007-01-03/2015-08-14 containing:
Data: num [1:2170, 1:6] 86.3 84 85.8 86 86.5 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
$ src : chr "yahoo"
$ updated: POSIXct[1:1], format: "2015-08-15 00:46:49"
xts objects do not have a "Date" column. They have an index attribute that holds the datetime. xts extends zoo, so please see the zoo vignettes as well as the xts vignette and FAQ for information about how to use the classes.
Merging xts objects is as simple as:
R> f <- merge(f, rsi14=RSI(Ad(f), 14))
Or you could just use $<- to add/merge a column to an existing xts object:
R> f$rsi14 <- RSI(Ad(f), 14)
very new to programming in R - but I am stumped on this one:
I'd like to only have to enter stock symbol data once in the script, but can't figure out how to reference ie adjusted close later on using Ad(x) without having to type the stock name again. I've tried passing a variable in like below but get error messages:
#get stock series data
stockPair <- c("SPY","DIA")
look_per <- "2015-01-01"
stckA <- suppressWarnings(getSymbols(stockPair[1], from = look_per))
stckB <- suppressWarnings(getSymbols(stockPair[2], from = look_per))
#get Adjusted close data
adA <- Ad(stckA )
adB <- Ad(stckB )
Error in Ad(stckA) :
subscript out of bounds: no column name containing "Adjusted"
The first thing you should do when you get an error is to look at your data. In this case, stckA and stckB are not what you think they are.
R> stckA <- suppressWarnings(getSymbols(stockPair[1], from = look_per))
R> stckB <- suppressWarnings(getSymbols(stockPair[2], from = look_per))
R> str(stckA)
chr "SPY"
R> str(stckB)
chr "DIA"
As you can see, those two objects are only character strings of the symbols returned by getSymbols, not the data. You need to set auto.assign=FALSE if you want to assign the output of getSymbols to an object.
R> stckA <- getSymbols(stockPair[1], from = look_per, auto.assign = FALSE)
R> str(Ad(stckA)) # now stckA contains data
An ‘xts’ object on 2015-01-02/2015-08-05 containing:
Data: num [1:149, 1] 204 200 198 200 204 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr "SPY.Adjusted"
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
$ src : chr "yahoo"
$ updated: POSIXct[1:1], format: "2015-08-05 20:02:30"
I have two data files with stock returns. I'm trying to apply the same function to both but I get an error for one of them. I wanted to find out what's causing the error, so I compared the output of str for both xts objects and the only line that differs is:
Indexed by objects of class: [POSIXct,POSIXt] TZ: # this object errors
Indexed by objects of class: [Date] TZ: GMT # this object works
Is there a way to change the indexing of the dates in an xts object so that the output of str returns: Indexed by objects of class: [Date] TZ: GMT?
I generated the dates using: seq(as.Date("1963/07/01"), as.Date("2004/12/01"), by = "1 month",tzone="GMT").
A reproducible example:
library(xts)
library("PerformanceAnalytics")
load("https://dl.dropboxusercontent.com/u/22681355/data.Rdata")
data(edhec)
data2 <- as.xts(french1)
The function I want to call is Return.portfolio() with the argument rebalance_on="months"
Return.portfolio(edhec["1997",1:10],rebalance_on="months") #this works
Return.portfolio(data2["1976",1:10],rebalance_on="months") #this does not work
xts:::as.xts.data.frame by default assumes that the rownames of your data.frame should be coerced to a POSIXct object/index. If you want to use a different class, specify it via the dateFormat= argument to as.xts.
> data2 <- as.xts(french1, dateFormat="Date")
> str(data2)
An ‘xts’ object on 1963-06-30/2004-11-30 containing:
Data: num [1:498, 1:10] -0.47 4.87 -1.68 2.66 -1.13 2.83 0.79 1.85 3.08 -0.45 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:10] "NoDur" "Durbl" "Manuf" "Enrgy" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
NULL
Though I'm not convinced this is the cause of whatever error you encounter, because I do not get an error without specifying dateFormat="Date".
> data2 <- as.xts(french1)
> Return.portfolio(data2["1976",1:10],rebalance_on="months")
portfolio.returns
1976-01-31 0.3980000
1976-02-29 0.1017811
1976-03-31 1.3408273
1976-04-30 -11.7395151
1976-05-31 8.0197492
1976-06-30 -0.2550812
1976-07-31 2.5732207
1976-08-31 1.3784635
1976-09-30 -1.6859705
1976-10-31 -21.4958124
1976-11-30 5.6863828
1976-12-31 -7.8071966
Warning message:
In Return.portfolio(data2["1976", 1:10], rebalance_on = "months") :
weighting vector is null, calulating an equal weighted portfolio