I have two zoo objects and I am wondering, why the first zoo object shows
..$ : NULL
when the second one shows
..$ : chr [1:2266] "1" "2" "3" "4" ...
instead.
What does this mean in this case? And how could I change
"..$ : chr [1:2266] "1" "2" "3" "4" ..." to "..$ : NULL" in the second object?
Background: I am conducting an eventstudy (package eventstudies) and I get the error:
Error in rval[i, j, drop = drop., ...] : subscript out of bounds
One of the possible problems could be the formatting of the data. The object "OtherReturns" is used as example data in the package. I am hoping to resolve the error by understanding more about zoo objects. Yet, I could not understand the formatting difference between the two following objects:
> str(OtherReturns)
‘zoo’ series from 2010-07-01 to 2013-03-28
Data: num [1:720, 1:4] -1.1568 -0.2727 -0.0229 1.01 -0.9107 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:4] "NiftyIndex" "CallMoneyRate" "SP500" "USDINR"
Index: Date[1:720], format: "2010-07-01" "2010-07-02" "2010-07-05" "2010-07-06" "2010-07-07" "2010-07-08" "2010-07-09" "2010-07-12" ...
> str(zoo_SP500)
‘zoo’ series from 2007-01-03 to 2015-12-31
Data: num [1:2266, 1:2] 1417 1418 1410 1413 1412 ...
- attr(*, "dimnames")=List of 2
..$ : chr [1:2266] "1" "2" "3" "4" ...
..$ : chr [1:2] "SP500_Closing" "Index_return"
Index: Date[1:2266], format: "2007-01-03" "2007-01-04" "2007-01-05" "2007-01-08" "2007-01-09" "2007-01-10" "2007-01-11" "2007-01-12" ...
At the first glance, both objects look the same in this regard:
> head(OtherReturns,3)
NiftyIndex CallMoneyRate SP500 USDINR
2010-07-01 -1.15678265 NA NA 0.3009785
2010-07-02 -0.27267977 NA -1.341017 0.0000000
2010-07-05 -0.02291607 NA NA 0.4070272
> head(zoo_SP500,3)
SP500_Closing Index_return
2007-01-03 1416.60 -0.1198618
2007-01-04 1418.34 0.1228293
2007-01-05 1409.71 -0.6084578
Any help and explanations much appreciated!
Thank you!
given a data.table created from a JSON array :
library(jsonlite)
library(data.table)
library(reprex)
DT<- fromJSON("http://data.sensor.community/airrohr/v1/sensor/21523/",flatten=T)
setDT(DT)
str(DT[,.(timestamp,sensordatavalues),])
#> Classes 'data.table' and 'data.frame': 2 obs. of 2 variables:
#> $ timestamp : chr "2019-10-25 18:21:00" "2019-10-25 18:18:33"
#> $ sensordatavalues:List of 2
#> ..$ :'data.frame': 2 obs. of 3 variables:
#> .. ..$ id : num 1.12e+10 1.12e+10
#> .. ..$ value_type: chr "P1" "P2"
#> .. ..$ value : chr "10.78" "5.90"
#> ..$ :'data.frame': 2 obs. of 3 variables:
#> .. ..$ id : num 1.12e+10 1.12e+10
#> .. ..$ value_type: chr "P1" "P2"
#> .. ..$ value : chr "7.10" "5.33"
#> - attr(*, ".internal.selfref")=<externalptr>
i manage to unested the column containing a list of data.frame's :
DTdata <-rbindlist(DT$sensordatavalues,use.names=TRUE)
DTdata
#> id value_type value
#> 1: 11232984769 P1 10.78
#> 2: 11232984770 P2 5.90
#> 3: 11232945708 P1 7.10
#> 4: 11232945709 P2 5.33
but i'm stuck how to add the timestamp column from the original DT within the rbindlist function. Does anyone could enlight me ?
We can set the names of the list column with the 'timestamp' and make use of the idcol argument from rbindlist
DT[, rbindlist(setNames(sensordatavalues, timestamp), idcol = 'timestamp')]
# timestamp id value_type value
#1: 2019-10-25 18:28:25 11233103168 P1 7.95
#2: 2019-10-25 18:28:25 11233103169 P2 4.95
#3: 2019-10-25 18:25:58 11233063825 P1 9.55
#4: 2019-10-25 18:25:58 11233063829 P2 5.62
Naming the list elements with 'timestamp' can change the 'type' to character class (though in this case the input column 'timestamp' is character). If we need to preserve the type
rbindlist(Map(cbind, timestamp = DT$timestamp, DT$sensordatavalues))
How can I convert the output from getSymbols to have dates as rownames? Here is my code so far.
library("quantmod")
library("coindeskr")
getSymbols("AAPL")
> [1] "AAPL"
row.names(head(Cl(AAPL)))
> NULL
btc <- get_historic_price(start = "2017-01-01")
row.names(head(btc))
> [1] "2017-01-01" "2017-01-02" "2017-01-03" "2017-01-04" "2017-01-05" "2017-01-06"
The values of stock-prices in quantmod are not really either data.frames or matrices although they are built from components that have matrix structures and they are somewhat similar to zoo-objects:
str(AAPL)
#---------
An ‘xts’ object on 2007-01-03/2018-05-01 containing:
Data: num [1:2852, 1:6] 12.3 12 12.3 12.3 12.3 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
$ src : chr "yahoo"
$ updated: POSIXct[1:1], format: "2018-05-02 16:49:23"
So the AAPL object has a numeric matrix called "Data" and a separate index of class Date:
> str( index(AAPL) )
Date[1:2852], format: "2007-01-03" "2007-01-04" "2007-01-05" "2007-01-08" "2007-01-09" "2007-01-10" "2007-01-11" "2007-01-12" ...
> str( coredata(AAPL) )
num [1:2852, 1:6] 12.3 12 12.3 12.3 12.3 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:6] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
You need to learn the structure and the special functions needed to access them cleanly. Look at:
?xts
?xtsAttributes
I suspect you may want to try coercion to a dataframe object:
str( as.data.frame(AAPL) )
#--------
'data.frame': 2852 obs. of 6 variables:
$ AAPL.Open : num 12.3 12 12.3 12.3 12.3 ...
$ AAPL.High : num 12.4 12.3 12.3 12.4 13.3 ...
$ AAPL.Low : num 11.7 12 12.1 12.2 12.2 ...
$ AAPL.Close : num 12 12.2 12.2 12.2 13.2 ...
$ AAPL.Volume : num 309579900 211815100 208685400 199276700 837324600 ...
$ AAPL.Adjusted: num 8.1 8.28 8.23 8.27 8.95 ...
row.names( as.data.frame(AAPL) )[:16]
#---------
[1] "2007-01-03" "2007-01-04" "2007-01-05" "2007-01-08" "2007-01-09" "2007-01-10" "2007-01-11" "2007-01-12"
[9] "2007-01-16" "2007-01-17" "2007-01-18" "2007-01-19" "2007-01-22" "2007-01-23" "2007-01-24" "2007-01-25"
The creators of anomalize have another package which you might find useful. It's called tidyquant and is perfect for what you are looking to do.
library(anomalize)
library(tidyquant)
library(tidyverse)
The Tidy Way to get Financial data
aapl <- tq_get("AAPL")
Referencing your end goal, we can chart AAPL's anomalies since 2008.
aapl %>%
time_decompose(adjusted) %>%
anomalize(remainder) %>%
time_recompose() %>%
plot_anomalies(time_recomposed = TRUE, ncol = 3, alpha_dots = 0.5)
So I have large list of xts objects which is OHLC data for 900 plus names. It orginally came in a dataframe coupled with other stuff I didn't need. I now want to use it in quantmod using getsymbols and load it into my environment however it is taking far too long. Anyone know a more efficient way of doing this or what I might be doing wrong? Can getSymbols handle a list of xts?
load(file = "biglistofdataframes.Rdata")
### Convert the list of dataframes to xts
list_of_xts <- lapply(listofdataframes,function(x) xts(x[,2:6],x$date))
####change column names to match quantmod
list_of_xts <- lapply(list_of_xts, setNames,nm = c("Open","High","Low","Close","Volume"))
####Save to Rdatafile
save(list_of_xts, file="1.RData")
#First I clear the environment then I load the data back into the environment
load("1.RData")
##
getSymbols("list_of_xts", src="RData", auto.assign=TRUE)#this craps out on me
The reason I am trying to get it into this format is so that I can replicate Ross Bennett's momentum code. See below
https://rbresearch.wordpress.com/2012/10/20/momentum-in-r-part-2/
I wouldn't expect this code to work:
getSymbols("list_of_xts", src = "RData", auto.assign = TRUE)
?getSymbols.RData says that Symbols (the first argument) should be "a character vector specifying the names of each symbol to be loaded". You don't have a symbol and file named "list_of_xts.RData".
Also, getSymbols.RData() expects each symbol to be in its own file, so you would have to write each xts object in your list to a separate file.
# Get some data
env_of_xts <- new.env()
getSymbols(symbols, env=env_of_xts)
# Write it to a temporary directory
tdir <- tempdir()
for (nm in names(env_of_xts)) {
save(list = nm, file = file.path(tdir, paste0(nm, ".RData")), envir = env_of_xts)
}
# Now you can use getSymbols() to load from file
getSymbols(symbols[1], src = "RData", dir = tdir, extension = "RData")
# [1] "AAPL"
head(AAPL)
# AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
# 2007-01-03 12.32714 12.36857 11.70000 11.97143 309579900 8.137179
# 2007-01-04 12.00714 12.27857 11.97429 12.23714 211815100 8.317789
# 2007-01-05 12.25286 12.31428 12.05714 12.15000 208685400 8.258555
# 2007-01-08 12.28000 12.36143 12.18286 12.21000 199276700 8.299341
# 2007-01-09 12.35000 13.28286 12.16429 13.22429 837324600 8.988768
# 2007-01-10 13.53571 13.97143 13.35000 13.85714 738220000 9.418928
i am unsure about what symbols RData contains, I do know however that you can generate a list of many xts with lapply and getsymbols. Here is my example using google for current stock data.
symbols<-c("AAPL", "AMZN","GOOGL", "F", "GM", "IBM", "ORCL")
List_of_xts<-lapply(symbols, function(sym){
List_of_Xts<-getSymbols(Symbols = sym, src = "google", auto.assign = FALSE)
})
str(List_of_xts)
resulting in:
> str(List_of_xts)
List of 7
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 12.3 12 12.2 12.3 12.3 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:05"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 38.7 38.6 38.7 38.2 37.6 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "AMZN.Open" "AMZN.High" "AMZN.Low" "AMZN.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:06"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 233 235 241 244 243 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "GOOGL.Open" "GOOGL.High" "GOOGL.Low" "GOOGL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:07"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 7.56 7.56 7.72 7.63 7.75 7.79 7.73 7.77 7.89 7.97 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "F.Open" "F.High" "F.Low" "F.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:08"
$ :An ‘xts’ object on 2010-11-18/2018-02-07 containing:
Data: num [1:1817, 1:5] 35 34.1 34.2 34 33.7 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "GM.Open" "GM.High" "GM.Low" "GM.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:08"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 97.2 97.2 97.6 98.5 99.1 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "IBM.Open" "IBM.High" "IBM.Low" "IBM.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:09"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 17.2 17.6 17.6 17.6 17.9 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "ORCL.Open" "ORCL.High" "ORCL.Low" "ORCL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:10"> str(List_of_xts)
List of 7
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 12.3 12 12.2 12.3 12.3 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:05"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 38.7 38.6 38.7 38.2 37.6 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "AMZN.Open" "AMZN.High" "AMZN.Low" "AMZN.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:06"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2794, 1:5] 233 235 241 244 243 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "GOOGL.Open" "GOOGL.High" "GOOGL.Low" "GOOGL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:07"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 7.56 7.56 7.72 7.63 7.75 7.79 7.73 7.77 7.89 7.97 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "F.Open" "F.High" "F.Low" "F.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:08"
$ :An ‘xts’ object on 2010-11-18/2018-02-07 containing:
Data: num [1:1817, 1:5] 35 34.1 34.2 34 33.7 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "GM.Open" "GM.High" "GM.Low" "GM.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:08"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 97.2 97.2 97.6 98.5 99.1 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "IBM.Open" "IBM.High" "IBM.Low" "IBM.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:09"
$ :An ‘xts’ object on 2007-01-03/2018-02-07 containing:
Data: num [1:2795, 1:5] 17.2 17.6 17.6 17.6 17.9 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr [1:5] "ORCL.Open" "ORCL.High" "ORCL.Low" "ORCL.Close" ...
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
..$ src : chr "google"
..$ updated: POSIXct[1:1], format: "2018-02-07 21:48:10"
I would like to subset dates from an xts object based on logical values in another xts object, but R returns an out-of-range error despite having in-range values.
For example I would like to filter dates and prices where RSI is above 60.
> strength <- RSI(d, 14)>60
> strength["2016-10-17::"]
RSI
2016-10-17 TRUE
2016-10-18 TRUE
2016-10-19 TRUE
2016-10-20 FALSE
2016-10-21 FALSE
> d["2016-10-17::"]
Open
2016-10-17 642.2760
2016-10-18 640.5988
2016-10-19 637.0000
2016-10-20 631.9800
2016-10-21 633.6470
> d["2016-10-17::"][strength == TRUE]
Error in `[.xts`(d["2016-10-17::"], strength == TRUE) :
'i' or 'j' out of range
This is not the output I expect because both my objects have data until 2016-10-21. What could be wrong? I would like something like :
> d["2016-10-17::"][strength == TRUE]
Open
2016-10-17 642.2760
2016-10-18 640.5988
2016-10-19 637.0000
This is the str of my xts objects :
> str(d)
An ‘xts’ object on 2013-09-02/2016-10-21 containing:
Data: num [1:1146, 1] 127 128 121 121 116 ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr "Open"
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
List of 2
$ dateFormat: chr "Date"
$ na.action :Class 'omit' atomic [1:92] 1 2 3 4 5 6 7 8 9 10 ...
.. ..- attr(*, "index")= num [1:92] 1.37e+09 1.37e+09 1.37e+09 1.37e+09 1.37e+09 ...
> str(strength)
An ‘xts’ object on 2013-09-16/2016-10-21 containing:
Data: logi [1:1132, 1] FALSE FALSE FALSE FALSE FALSE FALSE ...
- attr(*, "dimnames")=List of 2
..$ : NULL
..$ : chr "RSI"
Indexed by objects of class: [Date] TZ: UTC
xts Attributes:
NULL
>
Thank you
You didn't make a reproducible example, so here is some toy data. Your issue is that you did not subset strength with the same time window (so your inner strength == TRUE logical series has different row length to your d row length, generating your error. i.e. NROW(strength == TRUE) >> NROW(d["2016-10-17::"]) ):
library(quantmod)
getSymbols("AAPL")
d <- AAPL
strength <- RSI(Cl(d)) > 60
You shouldn't get an error if you do this:
d["2016-10-17::"][strength["2016-10-17::"] == TRUE]