Below is the code I am trying to use to convert the csv file to xts so that I can perform analysis to it but nothing seems to work. I have even used answers for similar issue that have been posted on this platform but nothing seems to be working.
toDate <- function(x) as.Date(x, origin = "2015-02-15")
z <- read.zoo("Nasdaq.csv", header = TRUE, sep = ",", FUN = toDate)
x <- as.xts(z)
I get below error:
7. stop("character string is not in a standard unambiguous format")
6. charToDate(x)
5. as.Date.character(x, origin = "2015-02-15")
4. as.Date(x, origin = "2015-02-15")
3. FUN(...)
2. processFUN(ix)
1. read.zoo("Nasdaq.csv", header = TRUE, sep = ",", FUN = toDate)
Problem is when you reading file as zoo.
If is a time series, try load file, than turn it into ts.
data <- read.csv("anyfile.csv")
Y <- ts(data$Y, start = c(2015, 2), end = c(2017, 1), frequency = 12)
my_xts <- as.xts(Y)
Related
I am importing 500 csv's that have the following similar format:
"https://www.quandl.com/api/v3/datasets/WIKI/stockname/data.csv?column_index=11&transform=rdiff&api_key=keyname"
Where stockname is the ticker symbol of a single stock. I have the list of stock tickers saved in a dataframe called stocklist.
I'd like to use lapply to iterate through my list of stocks. Here's what I have so far:
lst <- lapply(stocklist, function(i){
url <- paste0("https://www.quandl.com/api/v3/datasets/WIKI/",i,"/data.csv?column_index=11&transform=rdiff&api_key=XXXXXXXXXXXXXXX")
spdata <- read.csv(url, stringsAsFactors = FALSE)
})
I get the following error:
Error in file(file, "rt") : invalid 'description' argument
What could be causing this error? I tried using a for loop as well, but was unsuccessful and I have been told lapply is a better method in R for this type of task.
Edit:
Stucture of stocklist:
> dput(droplevels(head(stocklist)))
structure(list(`Ticker symbol` = c("MMM", "ABT", "ABBV", "ABMD",
"ACN", "ATVI")), .Names = "Ticker symbol", row.names = c(NA,
6L), class = "data.frame")
Second Edit (solution):
stockdata<-lapply(paste0("https://www.quandl.com/api/v3/datasets/WIKI/",stocklist[1][[1]],"/data.csv?column_index=11&transform=rdiff&api_key=XXXXXXX"),read.csv,stringsAsFactors=FALSE)
Add names to stockdata:
names(stockdata)<-stocklist[1][[1]]
I believe your 'i' variable is a vector.
Make sure you are sub-scripting it properly and only passing one stock at a time.
This: i would look something like this: i[x]
I can't tell what i is, but I'm guessing it's a number. Can you try this and see if it works for you? I think it's pretty close. Just make sure the URL matches the pattern of the actual URL you are fetching data from. I tried to find it; I couldn't find it.
seq <- c(1:10)
for (value in seq) {
mydownload <- function (start_date, end_date) {
start_date <- as.Date(start_date) ## convert to Date object
end_date <- as.Date(end_date) ## convert to Date object
dates <- as.Date("1970/01/01") + (start_date : end_date) ## date sequence
## a loop to download data
for (i in 1:length(dates)) {
string_date <- as.character(dates[i])
myfile <- paste0("C:/Users/Excel/Desktop/", string_date, ".csv")
string_date <- gsub("-", "-", string_date) ## replace "-" with "/"
myurl <- paste("https://www.quandl.com/api/v3/datasets/WIKI/", i, "/data.csv?column_index=11&transform=rdiff&api_key=xxxxxxxxxx", sep = "")
download.file(url = myurl, destfile = myfile, quiet = TRUE)
}
}
}
I am facing difficulties after running the code and trying to export the dataset to a spreadsheet or txt.file.
I am newbie to R, so maybe this question is trivial.
After running the following code:
eia_series <- function(api_key, series_id, start = NULL, end = NULL, num = NULL, tidy_data = "no", only_data = FALSE){
# max 100 series
# test if num is not null and either start or end is nut null. Not allowed
# api_key test for character.
# series_id test for character.
# if start/end not null, then check if format matches series id date format
# parse date and numerical data
# parse url
series_url <- httr::parse_url("http://api.eia.gov/series/")
series_url$query$series_id <- paste(series_id, collapse = ";")
series_url$query$api_key <- api_key
series_url$query$start <- start
series_url$query$end <- end
series_url$query$num <- num
# get data
series_data <- httr::GET(url = series_url)
series_data <- httr::content(series_data, as = "text")
series_data <- jsonlite::fromJSON(series_data)
# Move data from data.frame with nested list and NULL excisting
series_data$data <- series_data$series$data
series_data$series$data <- NULL
# parse data
series_data$data <- lapply(X = series_data$data,
FUN = function(x) data.frame(date = x[, 1],
value = as.numeric(x[, 2]),
stringsAsFactors = FALSE))
# add names to the list with data
names(series_data$data) <- series_data$data
# parse dates
series_data$data <- eia_date_parse(series_list = series_data$data, format_character = series_data$series$f)
# tidy up data
if(tidy_data == "tidy_long"){
series_data$data <- lapply(seq_along(series_data$data),
function(x) {cbind(series_data$data[[x]],
series_time_frame = series_data$series$f[x],
series_name = series_data$series$series_id[x],
stringsAsFactors = FALSE)})
series_data$data <- do.call(rbind, series_data$data)
}
# only data
if(only_data){
series_data <- series_data$data
}
return(series_data)
}
After running the function
eia_series(api_key = "XXX",series_id = c("PET.MCRFPOK1.M", "PET.MCRFPOK2.M"))
I tried to "transfer" the data in order to export it but got the following error:
No encoding supplied: defaulting to UTF-8.
I don't understand why. Could you help me out?
That doesn't look like an error, rather a statement. Probably coming from httr::content(series_data, as = "text"). Look in https://cran.r-project.org/web/packages/httr/vignettes/quickstart.html in The body section. It shouldn't be a problem, as long as your data returns what you expect. Otherwise you can try different encoding or there is a bug elsewhere.
Try:
series_data <- httr::content(series_data, as = "text", encoding = "UTF-8")
Trying to fetch the closing prices from a list of tickers listed in a csv. file using the following code:
date <- "2017-03-03"
tickers <- read.csv("us_tickerfeed.csv", header = TRUE)
for(i in 1:nrow(tickers)){
data <- getSymbols(tickers$ticker_th[i], from = date, to = date, src = "yahoo")
tickers$close_price[i] <- Cl(get(data))[[1]]
}
these codes worked before but now I'm getting the following error message:
Error in do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, :
could not find function "getSymbols.6"
Thanks!
I had this problem previously, could you try as.character(tickers$ticker_th)?
I'm trying to download TRMM 3B42 3-hour binary data for a given time span from this NASA FTP server.
There is an excellent code made by Florian Detsch to download the daily product (here is the link: https://github.com/environmentalinformatics-marburg/Rsenal/blob/master/R/downloadTRMM.R) included in the GitHub-only Rsenal package. Unfortunately it is not working for the 3-hour data.
I changed the code:
downloadTRMM <- function(begin, end, dsn = ".", format = "%Y-%m-%d.%H") {
## transform 'begin' and 'end' to 'Date' object if necessary
if (!class(begin) == "Date")
begin <- as.Date(begin, format = format)
if (!class(end) == "Date")
end <- as.Date(end, format = format)
## trmm ftp server
ch_url <-"ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7/"
## loop over daily sequence
ls_fls_out <- lapply(seq(begin, end, 1), function(i) {
# year and julian day (name of the corresponding folder)
tmp_ch_yr <- strftime(i, format = "%Y%m")
#tmp_ch_dy <- strftime(i, format = "%j")
# trmm date format
tmp_dt <- strftime(i+1, format = "%Y%m%d.%H")
# list files available on server
tmp_ch_url <- paste(ch_url, tmp_ch_yr, "", sep = "/")
tmp_ch_fls <- tmp_ch_fls_out <- character(2L)
for (j in 1:2) {
tmp_ch_fls[j] <- paste0("3B42.", tmp_dt, "z.7.precipitation",
ifelse(j == 1, ".bin"))
tmp_ch_fls[j] <- paste(tmp_ch_url, tmp_ch_fls[j], sep = "/")
tmp_ch_fls_out[j] <- paste(dsn, basename(tmp_ch_fls[j]), sep = "/")
download.file(tmp_ch_fls[j], tmp_ch_fls_out[j], mode = "wb")
}
# return data frame with *.bin and *.xml filenames
tmp_id_xml <- grep("xml", tmp_ch_fls_out)
data.frame(bin = tmp_ch_fls_out[-tmp_id_xml],
xml = tmp_ch_fls_out[tmp_id_xml],
stringsAsFactors = FALSE)
})
## join and return names of processed files
ch_fls_out <- do.call("rbind",ls_fls_out)
return(ch_fls_out)
}
getwd()
setwd("C:/Users/joaoreis/Documents/Bases_Geograficas/trmm_3h/")
fls_trmm <- downloadTRMM(begin = "2008-01-01.00", end = "2008-01-05.00")
fls_trmm
But I get the following error:
trying URL
'ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7//200801//3B42.20080102.00z.7.precipitation.bin'
Error in download.file(tmp_ch_fls[j], tmp_ch_fls_out[j], mode = "wb")
: cannot open URL
'ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B42_V7//200801//3B42.20080102.00z.7.precipitation.bin'
In addition: Warning message: In download.file(tmp_ch_fls[j],
tmp_ch_fls_out[j], mode = "wb") : InternetOpenUrl failed: '' Called
from: download.file(tmp_ch_fls[j], tmp_ch_fls_out[j], mode = "wb")
Does anyone know how to fix it using R?
Thanks!
As of commit 909f98a, I have enabled the automated retrieval of 3-hourly data from ftp://disc3.nascom.nasa.gov/data/s4pa/TRMM_L3. Make sure you have the latest version of Rsenal installed using
devtools::install_github("environmentalinformatics-marburg/Rsenal")
and then have a look at the examples in ?downloadTRMM. For now, the function supports both character (requires 'format' argument passed on to strptime) and POSIXlt input. For example, something like
downloadTRMM(begin = "2015-01-01 12:00", end = "2015-01-03 12:00",
type = "3-hourly", format = "%Y-%m-%d %H:%M")
to download 3-hourly data from 1-3 January 2015 (noon to noon) should now work just fine.
Note that in contrast to the FTP server you mentioned, the data comes in .HDF format and a rasterize method has not been implemented so far, meaning that you have to deal with the container files yourself. I'll try to figure out something more convenient soon regarding the automated rasterization of the data.
I want to load time series data from a csv file. I am confused if it is possible using the ts() function?
The data looks like this:
time(ms),value
1390933817000,3775.89624023438
1390933847000,3765.65698242188
1390933877000,3757.01416015625
1390933907000,3768.63623046875
1390933937000,3775.84497070312
1390933967000,3774.53588867188
1390933997000,3771.6240234375
1390934027000,3763.83081054688
As you can observe, the value is fetched every 30 seconds.
Try this (noting comment):
Lines <- "time(ms),value
1390933817000,3775.89624023438
1390933847000,3765.65698242188
1390933877000,3757.01416015625
1390933907000,3768.63623046875
1390933937000,3775.84497070312
1390933967000,3774.53588867188
1390933997000,3771.6240234375
1390934027000,3763.83081054688"
library(zoo)
# z <- read.zoo("myfile.dat", sep = ",", header = TRUE, FUN = identity)
z <- read.zoo(text = Lines, sep = ",", header = TRUE, FUN = identity)
as.ts(z)