I installed the quantmod package and I'm trying to import a csv file with 1 minute intraday data. Here is a sample GAZP.csv file:
"D";"T";"Open";"High";"Low";"Close";"Vol"
20130902;100100;132.2000000;133.0500000;131.9200000;132.5000000;131760
20130902;100200;132.3700000;132.5700000;132.2500000;132.2900000;66090
20130902;100300;132.3600000;132.5000000;132.2600000;132.4700000;37500
I've tried:
> getSymbols('GAZP',src='csv')
Error in `colnames<-`(`*tmp*`, value = c("GAZP.Open", "GAZP.High", "GAZP.Low", :
length of 'dimnames' [2] not equal to array extent
> getSymbols.csv('GAZP',src='csv')
> # or
> getSymbols.csv('GAZP',env,dir="c:\\!!",extension="csv")
Error in missing(verbose) : 'missing' can only be used for arguments
How should I properly use the getSymbols.csv command to read such data?
#Vladimir, if you are not insisting to use the "getSymbols" function from the quantmod package you can import your csv file - assuming it is in your working directory - as zoo object with the line:
GAZP=read.zoo("GAZP.csv",sep=";",header=TRUE,index.column=list(1,2),FUN = function(D,T) as.POSIXct(paste(D, T), format="%Y%m%d %H%M%S"))
and convert it to a xts object if you want.
GAZP.xts <- as.xts(GAZP)
> GAZP
Open High Low Close Vol
2013-09-02 10:01:00 132.20 133.05 131.92 132.50 131760
2013-09-02 10:02:00 132.37 132.57 132.25 132.29 66090
2013-09-02 10:03:00 132.36 132.50 132.26 132.47 37500
Related
I inherited some R code that analyses simulation results. At one point, that code calls the xts package's to.monthly function with indexAt = 'yearmon' to summarize some values in a zoo.
That code normally runs without issue. Recently, however, when analysing simulations over much older data, the call to to.monthly generated some disturbing Warning messages like this:
Warning in zoo(xx, order.by = index(x), ...) :
some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
I culled my data down to the minimum size that still exhibits this Warning. Start with this R code:
library(xts)
z = structure(c(-1062503.35419463, -1080996.55425821, -1099783.92018741,
-1122831.06978888, -1138804.79976585, -1158620.33101501, -1163717.44859603,
-1183250.17288897, -1212428.97863421, -1234981.23171341, -1253605.89670471,
-1269885.84780747, -1272023.98376509, -1284471.17954946, -1313114.61914572,
-1334861.551294, -1349971.87378146, -1360596.77251109, -1363047.71977556,
-1383840.30131117, -1407963.97518998, -1427010.7195352, -1451908.36211767,
-1464563.94519573, -1470017.67402451, -1503642.02732151, -1529231.67395429,
-1560593.79655716, -1582052.24505653, -1595391.99583389), index = structure(c(1111985820,
1112072340, 1112158740, 1112245140, 1112331540, 1112392740, 1112587140,
1112673540, 1112759880, 1112846340, 1112932200, 1112993940, 1113191940,
1113278340, 1113364560, 1113451080, 1113537540, 1113598740, 1113796560,
1113883140, 1113969540, 1114055940, 1114142220, 1114203540, 1114401480,
1114487940, 1114574280, 1114660740, 1114747080, 1114808340), class = c("POSIXct",
"POSIXt")), class = "zoo")
class(z)
head(z)
tail(z)
Then execute this call to to.monthly:
to.monthly(z, indexAt = 'yearmon', name = "Monthly")
On my machine that generates this output:
Warning in zoo(xx, order.by = index(x), ...) :
some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
Warning in zoo(xx, order.by = index(x), ...) :
some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
Monthly.Open Monthly.High Monthly.Low Monthly.Close
Apr 2005 -1062503 -1062503 -1138805 -1138805
Apr 2005 -1158620 -1158620 -1595392 -1595392
Note the Warning messages, followed by the result of to.monthly, which is a zoo that has the duplicate position of "Apr 2005".
I spent some time executing the code in to.monthly line by line, and determined that the bug actually happens inside to.monthly's call to to.period.
In particular, I found that the xx local variable inside to.period is initially calculated correctly, but after the line
indexClass(xx) <- indexAt
is executed that is when the positions of xx become non-unique.
That behavior sure looks like a bug in the xts package's to.period function to me.
I would love to hear from someone who knows how to.monthly/to.period/yearmon really works either confirm that this is a bug, or explain to me why it is not and give me a work around.
I found this possibly related report on the xts github page (which I do not fully understand).
Concerning my machine:
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17134)
...
other attached packages:
...
xts_0.10-0
zoo_1.8-0
When I startup Rgui, I see this Warning message about xts:
Warning: package ‘xts’ was built under R version 3.4.2
This looks like a bug, unrelated to #158. The problem is that the index of z is POSIXct in your local timezone. You aggregate to monthly, which doesn't have a timezone (so xts sets the timezone attribute to "UTC").
But the change in timezone occurs on the POSIXct index, which changes the local time before the index is converted to "yearmon". So, depending on your local timezone's offset from UTC, this may convert the first (last) observation in a month into the last (first) observation of the prior (next) month.
To illustrate:
Sys.setenv(TZ = "America/Chicago")
debugonce(xts:::`indexClass<-.xts`)
to.monthly(z, indexAt="yearmon", name="monthly")
# <snip>
# Browse[2]>
# debug: attr(attr(x, "index"), "tzone") <- "UTC"
# Browse[2]> print(x) # When timezone is "America/Chicago"
# monthly.Open monthly.High monthly.Low monthly.Close
# 2005-03-31 22:59:00 -1062503 -1062503 -1138805 -1138805
# 2005-04-29 15:59:00 -1158620 -1158620 -1595392 -1595392
# Browse[2]>
# debug: attr(attr(x, "index"), "tclass") <- value
# Browse[2]> print(x) # When timezone is "UTC"
# monthly.Open monthly.High monthly.Low monthly.Close
# 2005-04-01 04:59:00 -1062503 -1062503 -1138805 -1138805
# 2005-04-29 20:59:00 -1158620 -1158620 -1595392 -1595392
# Warning message:
# timezone of object (UTC) is different than current timezone ().
You can see that the call to attr(attr(x, "index"), "tzone") <- "UTC" pushed the last observation in March into the first day of April (note that the debugger lists the next call it will evaluate above my calls to print(x)).
Thanks for narrowing it down to the indexClass<- call. That made it a lot easier for me to debug!
I have downloaded the DJI historical data from Yahoo as a csv for further analysis in R. Out of curiosity getSymbols("^DJI") didn't seem to work, but I digress.
The point is that I don't know how to turn this csv file into a time series format.
Here is the output and problem so far:
> DJI = read.csv("^DJI.csv")
> head(DJI)
Date Open High Low Close Adj.Close Volume
1 1/29/1985 1277.72 1295.49 1266.89 1292.62 1292.62 13560000
2 1/30/1985 1297.37 1305.10 1278.93 1287.88 1287.88 16820000
3 1/31/1985 1283.24 1293.40 1272.64 1286.77 1286.77 14070000
4 2/1/1985 1276.94 1286.11 1269.77 1277.72 1277.72 10980000
5 2/4/1985 1272.08 1294.94 1268.99 1290.08 1290.08 11630000
6 2/5/1985 1294.06 1301.13 1278.60 1285.23 1285.23 13800000
> chartSeries(DJI)
Error in try.xts(x, error = "chartSeries requires an xtsible object") :
chartSeries requires an xtsible object
So the {quantmod} function chartSerie is requesting an .xts file, but the Date column in DJI is not immediately recognized as such:
> DJI = as.Date(DJI$Date)
Error in charToDate(x) :
character string is not in a standard unambiguous format
EDIT after the answer below:
> head(DJI)
Open High Low Close Adj.Close Volume
1985-01-29 1277.72 1295.49 1266.89 1292.62 1292.62 13560000
1985-01-30 1297.37 1305.10 1278.93 1287.88 1287.88 16820000
1985-01-31 1283.24 1293.40 1272.64 1286.77 1286.77 14070000
1985-02-01 1276.94 1286.11 1269.77 1277.72 1277.72 10980000
1985-02-04 1272.08 1294.94 1268.99 1290.08 1290.08 11630000
1985-02-05 1294.06 1301.13 1278.60 1285.23 1285.23 13800000
> is.ts(DJI)
[1] FALSE
To convert the dates you need a format statement...
DJI$Date <- as.Date(DJI$Date,format="%m/%d/%Y")
quantmod needs dates in xts objects to be row names rather than a separate column. You should therefore also do
rownames(DJI) <- DJI$Date
DJI$Date <- NULL #to remove the column
chartSeries(DJI)
I have a dataframe data,Which Contains the columns having integers,and columns containing date and time,As shown
>head(data,2)
PRESSURE AMBIENT_TEMP OUTLET_PRESSURE COMP_STATUS DATE TIME predict
1 14 65 21 0 2014-01-09 12:45:00 0.6025863
2 17 65 22 0 2014-01-10 06:00:00 0.6657910
And Now i'm going to write this back to Sql database by the chunck
sqlSave(channel,data,tablename = "ANL_ASSET_CO",append = T)
Where channel is connection name,But this gives error
[RODBC] Failed exec in Update
22018 1722 [Oracle][ODBC][Ora]ORA-01722: invalid number
But When i try excluding the date column ,it writes back without any error.
> sqlSave(channel,data[,c(1:4,7)],tablename = "ANL_ASSET_CO",append = T)
> sqlSave(channel,data[,c(1:4,6:7)],tablename = "ANL_ASSET_CO",append = T)
Because of the date column the data is not writing to ORACLE SQL developer,Could be problem with the hyphen.
How can i write , Any help !!
>class(data$DATE)
[1] "POSIXct" "POSIXt"
So had to change the data type as character
>data$DATE <- as.character(data$DATE)
>sqlSave(channel,data,tablename = "ANL_ASSET_CO",append=T)
This one worked!!
I have data frame(df) that contains daily stock index prices covering over 4000 days. It looks like:
Date Prices
1986-1-1 20
. .
. .
. .
. .
2001-08-31 40
I am trying to convert the data frame into zoo object using read.zoo(df) (read.zoo is a function in zoo package). However it gives me the following error:
Warning message:
In zoo(rval3, ix) :
some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
that affects the subsequent codes I apply to the object.
For a reproducibility purpose, the original data (FTSE100jensen.csv) and code (JensenPaper.R) is available on https://github.com/ahmedfsalhin/1stpaper
The problem is that you called read.zoo() without providing a value for format=, but your dates are formated like "%d/%m/%Y", not "%Y-%m-%d"
I'm not quite sure why this error was occurring, but I first converted Date to the Date class and was able to call read.zoo without error using this:
options(stringsAsFactors=FALSE)
library(zoo)
##
Data <- read.csv(
"F:/gitData.csv",
header=TRUE)
#
Data$Date <- as.Date(
Data$Date,
"%d/%m/%Y")
##
zData <- read.zoo(Data)
##
> head(zData)
Open High Low Close Volume Adj.Close
1986-01-01 1412.6 1412.6 1412.6 1412.6 0 1412.6
1986-01-02 1412.6 1420.8 1412.0 1420.5 0 1420.5
1986-01-03 1420.5 1430.0 1419.6 1429.8 0 1429.8
1986-01-06 1429.8 1436.3 1424.1 1424.1 0 1424.1
1986-01-07 1419.8 1419.8 1411.6 1415.2 0 1415.2
1986-01-08 1415.2 1419.3 1400.3 1404.2 0 1404.2
and everything seems to be in order, e.g. I can call .zoo methods properly, etc...
> plot(zData)
To address the comments above, the error message does seem to indicate that there are duplicated dates, but this is not the case:
> dim(Data)
[1] 4088 7
> length(unique(Data$Date))
[1] 4088
I am trying to read in a CSV file and change it to XTS format. However, I am running into and issue with the CSV format have date and time fields in separate columns.
2012.10.30,20:00,1.29610,1.29639,1.29607,1.29619,295
2012.10.30,20:15,1.29622,1.29639,1.29587,1.29589,569
2012.10.30,20:30,1.29590,1.29605,1.29545,1.29574,451
2012.10.30,20:45,1.29576,1.29657,1.29576,1.29643,522
2012.10.30,21:00,1.29643,1.29645,1.29581,1.29621,526
2012.10.30,21:15,1.29621,1.29644,1.29599,1.29642,330
I am trying to pull it in with
euXTS <- as.xts(read.zoo(file="EURUSD15.csv", sep=",", format="%Y.%m.%d", header=FALSE))
But it gives me this warning message so I think somehow I have to attached the time stamp but I am not sure the best way to do that.
Warning message:
In zoo(rval3, ix) :
Some methods for “zoo” objects do not work if the index entries in ‘order.by’ are not unique
It is better to use read.zoo to read directly your ts in a zoo object, easily coerced to xts one:
library(xts)
ts.z <- read.zoo(text='2012.10.30,20:00,1.29610,1.29639,1.29607,1.29619,295
2012.10.30,20:15,1.29622,1.29639,1.29587,1.29589,569
2012.10.30,20:30,1.29590,1.29605,1.29545,1.29574,451
2012.10.30,20:45,1.29576,1.29657,1.29576,1.29643,522
2012.10.30,21:00,1.29643,1.29645,1.29581,1.29621,526
2012.10.30,21:15,1.29621,1.29644,1.29599,1.29642,330',
sep=',',index=1:2,tz='',format="%Y.%m.%d %H:%M")
as.xts(ts.z)
V3 V4 V5 V6 V7
2012-10-30 20:00:00 1.29610 1.29639 1.29607 1.29619 295
2012-10-30 20:15:00 1.29622 1.29639 1.29587 1.29589 569
2012-10-30 20:30:00 1.29590 1.29605 1.29545 1.29574 451
2012-10-30 20:45:00 1.29576 1.29657 1.29576 1.29643 522
2012-10-30 21:00:00 1.29643 1.29645 1.29581 1.29621 526
2012-10-30 21:15:00 1.29621 1.29644 1.29599 1.29642 330