I am merging two xts objects with join="left" i.e. (all rows in the left object, and those that match in the right). I loaded these objectd in myEnv.
library(quantmod)
myEnv <- new.env()
getSymbols("AAPL;FB", env=myEnv)
[1] "AAPL" "FB"
MainXTS <- do.call(merge, c(eapply(myEnv, Cl), join = "left"))
head(MainXTS)
AAPL.Close FB.Close
2007-01-03 2.992857 NA
2007-01-04 3.059286 NA
2007-01-05 3.037500 NA
2007-01-08 3.052500 NA
2007-01-09 3.306072 NA
2007-01-10 3.464286 NA
range(index(myEnv$AAPL))
[1] "2007-01-03" "2020-10-27"
range(index(myEnv$FB))
[1] "2012-05-18" "2020-10-27"
So far it is working as expected since the time index in above merged object is being picked up from APPL. The issue is that when I change the order of the tickers so that FB comes first, the merged object still picks up time indexes from AAPL.
myEnv <- new.env()
getSymbols("FB;AAPL", env=myEnv)
[1] "FB" "AAPL"
MainXTS <- do.call(merge, c(eapply(myEnv, Cl), join = "left"))
head(MainXTS)
AAPL.Close FB.Close
2007-01-03 2.992857 NA
2007-01-04 3.059286 NA
2007-01-05 3.037500 NA
2007-01-08 3.052500 NA
2007-01-09 3.306072 NA
2007-01-10 3.464286 NA
I was expecting the time index to be picked up from FB. Does any one know what I am missing?
I think this has something to do with the fact that the order of objects being loaded is the same and in both cases above it is:
ls(myEnv)
[1] "AAPL" "FB"
We can change the order with match
out <- do.call(merge, c(lapply(mget(ls(myEnv)[match(ls(myEnv),
c("FB", "AAPL"))], myEnv), Cl), join = "left"))
-output
head(out)
# FB.Close AAPL.Close
#2012-05-18 38.23 18.94214
#2012-05-21 34.03 20.04571
#2012-05-22 31.00 19.89179
#2012-05-23 32.00 20.37714
#2012-05-24 33.03 20.19000
#2012-05-25 31.91 20.08178
Related
I have 3 lists of large XTS objects: "SMA"; "L", "Marubozu". Quick look how it looks:
> names(Marubozu)
[1] "TSLA" "AAPL" "NTES" "GOOGL" "ASML" "GOOG" "NFLX" "ADBE" "AMZN" "MSFT" "ADI" "FB"
> names(SMA)
[1] "TSLA" "AAPL" "NTES" "GOOGL" "ASML" "GOOG" "NFLX" "ADBE" "AMZN" "MSFT" "ADI" "FB"
> names(L)
[1] "TSLA" "AAPL" "NTES" "GOOGL" "ASML" "GOOG" "NFLX" "ADBE" "AMZN" "MSFT" "ADI" "FB"
> head(Marubozu$AAPL, n = 2)
WhiteMarubozu BlackMarubozu
2000-01-03 FALSE FALSE
2000-01-04 FALSE FALSE
> head(SMA$AAPL, n = 2)
UpTrend NoTrend DownTrend Trend
2000-01-03 NA NA NA NA
2000-01-04 NA NA NA NA
> head(L$AAPL, n =2)
AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2000-01-03 0.936384 1.004464 0.907924 0.999442 535796800 0.856887
2000-01-04 0.966518 0.987723 0.903460 0.915179 512377600 0.784643
I want to merge corresponding XTS objects in that lists so that it creates one big lig list. For example, the output for New_List$AAPL would be:
AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted WhiteMarubozu BlackMarubozu UpTrend NoTrend DownTrend Trend
2000-01-03 0.936384 1.004464 0.907924 0.999442 535796800 0.856887 0 0 NA NA NA NA
2000-01-04 0.966518 0.987723 0.903460 0.915179 512377600 0.784643 0 0 NA NA NA NA
I tried to create a list of lists and merging it, but it didnt work. Here you can see:
#That works for a single ticker AAPL
full <- merge.xts(L$AAPL, Marubozu$AAPL, SMA$AAPL)
#This doesn't work
out3 <- Map(function(x) {full$x <- merge.xts(lista[[1]]$x, lista[[2]]$x)}, lista)
I guess it is just some simple 2-lines thing but can't really find the solution, thanks for any responses!
We could do this with Map - as the list of xts elements have the same tickers in the same order, just use Map instead of creating a list of lists
library(xts)
out <- Map(merge.xts, L, Marubozu, SMA)
Here's a small function u() that binds the xts-index to an xts object and converts to 'data.frame'.
u <- function(x) cbind.data.frame(index=index(x), unclass(x))
To test it, we create some data using sample_matrix which comes with xts. We split first two and last two columns into two separate xts objects with same index.
library(xts)
data(sample_matrix)
sample.xts <- as.xts(sample_matrix, descr='my new xts object')
S1 <- as.xts(sample_matrix[,1:2]) ##
S2 <- as.xts(sample_matrix[,3:4])
Now we may easily apply merge and create a new xts object out of it.
res <- merge(u(S1), u(S2)) |>
(\(x) xts(x[-1], x$index, descr='my new xts object'))()
class(res)
# [1] "xts" "zoo"
stopifnot(all.equal(res, sample.xts)) ## proof
quantmode newbie here,
My end goal is to have a CSV file including monthly stock prices, I've downloaded the data using getSymbols using this code:
Symbols <- c("DIS", "TSLA","ATVI", "MSFT", "FB", "ABT","AAPL","AMZN",
"BAC","NFLX","ADBE","WMT","SRE","T","MS")
Data <- new.env()
getSymbols(c("^GSPC",Symbols),from="2015-01-01",to="2020-12-01"
,periodicity="monthly",
env=Data)
the line above works fine, now I need to create a data frame that only includes the adjusted prices for all the symbols with a data column ofc,
any help, please? :)
Desired output would be something similar to this
enter image description here
Another straightforward way to get your monthly data:
tickers <- c('AMZN','FB','GOOG','AAPL')
getSymbols(tickers,periodicity="monthly")
head(do.call("merge.xts",c(lapply(mget(tickers),"[",,6),all=FALSE)),3)
AMZN.Adjusted FB.Adjusted GOOG.Adjusted AAPL.Adjusted
2012-06-01 228.35 31.10 288.9519 17.96558
2012-07-01 233.30 21.71 315.3032 18.78880
2012-08-01 248.27 18.06 341.2658 20.46477
Note the logical argument all = FALSE is the equivalent of an innerjoin and you get data when all of your stocks have prices. all = TRUE fills data which is not available with NAs (outerjoin).
To write the file you can use:
write.zoo(monthlyPrices,file = 'filename.csv',sep=',',quote=FALSE)
First get your data from the environment:
require(quantmod)
# your code
dat <- mget(ls(Data), env=Data)
Then draw the data from the Objects:
newdat <- as.data.frame(sapply( names(dat), function(x) coredata(dat[[x]])[,1] ))
Note that this takes the Opening values (see: dat[[x]])[,1]), the Objects have more, e.g.:
names(dat[["AAPL"]])
[1] "AAPL.Open" "AAPL.High" "AAPL.Low" "AAPL.Close"
[5] "AAPL.Volume" "AAPL.Adjusted"
Last, get the dates (assumes symmetric dates for all symbols):
rownames(newdat) <- index(dat[["AAPL"]])
# OR, more universal, by extracting from the complete list:
rownames(newdat) <-
as.data.frame( sapply( names(dat), function(x) as.character(index(dat[[x]])) ) )[,1]
head(newdat, 3)
AAPL ABT ADBE AMZN ATVI BAC DIS FB GSPC MS
2015-01-01 27.8475 45.25 72.70 312.58 20.24 17.99 94.91 78.58 2058.90 39.05
2015-02-01 29.5125 44.93 70.44 350.05 20.90 15.27 91.30 76.11 1996.67 33.96
2015-03-01 32.3125 47.34 79.14 380.85 23.32 15.79 104.35 79.00 2105.23 35.64
MSFT NFLX SRE T TSLA WMT
2015-01-01 46.66 49.15143 111.78 33.59 44.574 86.27
2015-02-01 40.59 62.84286 112.38 33.31 40.794 84.79
2015-03-01 43.67 67.71429 108.20 34.56 40.540 83.93
Writing the csv:
write.csv(newdat, "file.csv")
I am looking for a way to rename the columns of several objects with a for loop or other method in R. Ultimately, I want to be able to bind the rows of each Stock object into one large data frame, but cannot due to differing column names. Example below:
AAPL <-
Date AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted Stock pct_change
2020-05-14 304.51 309.79 301.53 309.54 39732300 309.54 AAPL 0.61
2020-05-15 300.35 307.90 300.21 307.71 41561200 307.71 AAPL -0.59
GOOG <-
Date GOOG.Open GOOG.High GOOG.Low GOOG.Close GOOG.Volume GOOG.Adjusted Stock pct_change
2020-05-14 1335.02 1357.420 1323.910 1356.13 1603100 1356.13 GOOG 0.50
2020-05-15 1350.00 1374.480 1339.000 1373.19 1705700 1373.19 GOOG 1.26
For this example I have 2 objects (AAPL and GOOG), but realistically I would be working with many more. Can I create a for loop to iterate through each object, and rename the 2nd column of each to "Open", 3rd column to "High", 4th column to "Low",.... etc so I can then bind all these objects together?
I already have a column named "Stock", so I do not need the Ticker part of the column name.
Using quantmod we can read a set of stock ticker symbols, clean their names & rbind() into a single data frame.
There are three key features illustrated within this answer, including:
Use of get() to access the objects written by quantmod::getSymbols() once they are loaded into memory.
Use of the symbol names passed into lapply() to add a symbol column to each data frame.
Conversion of the dates stored as row names in the xts objects written by getSymbols() to a data frame column.
First, we'll use getSymbols() to read data from yahoo.com.
library(quantmod)
from.dat <- as.Date("12/02/19",format="%m/%d/%y")
to.dat <- as.Date("12/06/19",format="%m/%d/%y")
theSymbols <- c("AAPL","AXP","BA","CAT","CSCO","CVX","XOM","GS","HD","IBM",
"INTC","JNJ","KO","JPM","MCD","MMM","MRK","MSFT","NKE","PFE","PG",
"TRV","UNH","UTX","VZ","V","WBA","WMT","DIS","DOW")
getSymbols(theSymbols,from=from.dat,to=to.dat,src="yahoo")
# since quantmod::getSymbols() writes named data frames, need to use
# get() with the symbol names to access each data frame
head(get(theSymbols[[1]]))
> head(get(theSymbols[[1]]))
AAPL.Open AAPL.High AAPL.Low AAPL.Close AAPL.Volume AAPL.Adjusted
2019-12-02 267.27 268.25 263.45 264.16 23621800 262.8231
2019-12-03 258.31 259.53 256.29 259.45 28607600 258.1370
2019-12-04 261.07 263.31 260.68 261.74 16795400 260.4153
2019-12-05 263.79 265.89 262.73 265.58 18606100 264.2359
Having illustrated how to access the symbol objects in the global environment, we'll use lapply() to extract the dates from the row names, clean the column headings, and write the symbol name as a column for each symbol's data object.
# convert to list
symbolData <- lapply(theSymbols,function(x){
y <- as.data.frame(get(x))
colnames(y) <- c("open","high","low","close","volume","adjusted")
y$date <- rownames(y)
y$symbol <- x
y
})
Finally, we convert the list of data frames to a single data frame.
#combine to single data frame
combinedData <- do.call(rbind,symbolData)
rownames(combinedData) <- 1:nrow(combinedData)
...and the output:
> nrow(combinedData)
[1] 120
> head(combinedData)
open high low close volume adjusted date symbol
1 267.27 268.25 263.45 264.16 23621800 262.8231 2019-12-02 AAPL
2 258.31 259.53 256.29 259.45 28607600 258.1370 2019-12-03 AAPL
3 261.07 263.31 260.68 261.74 16795400 260.4153 2019-12-04 AAPL
4 263.79 265.89 262.73 265.58 18606100 264.2359 2019-12-05 AAPL
5 120.31 120.36 117.07 117.26 5538200 116.2095 2019-12-02 AXP
6 116.04 116.75 114.65 116.57 3792300 115.5256 2019-12-03 AXP
>
If you can guarantee the order of these columns this should do it:
for(df in list(AAPL, GOOG))
colnames(df) <- c("Date", "Open", "High", "Low", "Close", "Volume", "Adjusted", "Stock", "pct_change")
With lapply, we can loop over the list and remove the prefix in the column names with sub. This can be done without any external packages
lst1 <- lapply(list(AAPL, GOOG), function(x) {
colnames(x) <- sub(".*\\.", "", colnames(x))
x})
I have more than 300 stocks downloaded with getsymbols() and I have the name of this stocks in a vector, for example:
USA_STOCKS = c("AAL","AAPL","ADBE","ADI","ADP","ADSK","ALGN",
"ALXN","AMAT","AMGN","AMZN","ASML","ATVI","AVGO",
"BIDU","BIIB") # This is just an extract from 300
getSymbols(AAL) # this is just one of the 300 "getsymbols"
With that, I have a XTS object called AAL and a vector USA_TOCKS with all the name of the XTS Objets.
I would like to do:
AAL = na.omit(AAL)
But, instead of use the Object AAL, I want to refer the object using the name inside the vector. Something like this:
USA_STOCKS[1] = na.omit(USA_STOCKS[1])
Obviusly if i did this, I will change only the name of "AAL" inside the vector. But what I want is to refer the object AAL.
Hmm, still a bit unclear, but I think you want to do something like this:
library(quantmod)
USA_STOCKS = c("AAL","AAPL","ADBE")
# Put all requested quotes in big list
stocks_usa <- lapply(USA_STOCKS,
getSymbols,
from = "2018-10-01",
to = "2018-11-01",
auto.assign = F)
# set the names of the list
names(stocks_usa) <- USA_STOCKS
#reference AAL
head(stocks_usa$AAL)
AAL.Open AAL.High AAL.Low AAL.Close AAL.Volume AAL.Adjusted
2018-10-01 41.41 41.75 39.60 39.61 7210700 39.50097
2018-10-02 39.60 39.60 38.40 38.50 7625000 38.39403
2018-10-03 38.70 39.26 38.42 38.80 6370300 38.69320
2018-10-04 38.80 39.01 37.48 37.92 5916500 37.81562
2018-10-05 37.93 38.13 36.21 36.44 9127000 36.33969
2018-10-08 36.44 36.85 35.60 35.90 7879300 35.80119
# more referencing
stocks_usa$AAL <- na.omit(stocks_usa$AAL)
I am having difficulty using dailyReturn function on an xts object with multiple return series.
a<-Cl(getSymbols("INTC",auto.assign=FALSE))
b<-Cl(getSymbols("IBM",auto.assign=FALSE))
a<-merge(a,b)
dailyReturn(a[,1]) #This works!
dailyReturn(a) #Only return the result for first series
apply(a,2, dailyReturn)
#Error in array(ans, c(len.a%/%d2, d.ans), if (!all(vapply(dn.ans, is.null, :
length of 'dimnames' [1] not equal to array extent
How do I get dailyReturn to return the daily returns for multiple series in xts object?
I prefer ROC also, but if you must use dailyReturn, you can lapply over the columns and cbind them back together.
> head(do.call(cbind, lapply(a, dailyReturn)))
daily.returns daily.returns.1
2007-01-03 0.0000000000 0.000000000
2007-01-04 0.0402948403 0.010691889
2007-01-05 -0.0033065659 -0.009052996
2007-01-08 -0.0042654028 0.015191952
2007-01-09 0.0009519277 0.011830131
2007-01-10 0.0233000476 -0.011791746
I used do.call so that it will work with any number of columns.
I would just use TTR::ROC instead.
> head(r <- ROC(a, type="discrete"))
INTC.Close IBM.Close
2007-01-03 NA NA
2007-01-04 0.0402948403 0.010691889
2007-01-05 -0.0033065659 -0.009052996
2007-01-08 -0.0042654028 0.015191952
2007-01-09 0.0009519277 0.011830131
2007-01-10 0.0233000476 -0.011791746