I would like to download multiple data files in r from multiple urls, where only the number changes. I succesfully downloaded this code, but i need to download a string of numbers (e.g. 29208, 49510, 54604 62759,62760,7002,38175) which have to replace 29208 in the url. I am a total newbie, and eventhough I have read and seen some examples, I cannot seem to write the right code.
install.packages("jsonlite")
library(jsonlite)
df <- fromJSON('https://api.euroinvestor.dk/instruments/29208/closeprices?fromDate=1970-1-1')
You can not do this , you can just do a loop for , and in everytime you change the url parameters, the loop for loop in all values that you want to use in the url
Related
I am trying to merge two files in R in an attempt to compute the correlation. One file is located at http://richardtwatson.com/data/SolarRadiationAthens.csv and the other at http://richardtwatson.com/data/electricityprices.csv
Currently my code looks as follows:
library(dplyr)
data1<-read.csv("C:/Users/nldru/Downloads/SolarRadiationAthens.csv")
data2 <- read.csv("C:/Users/nldru/Downloads/electricityprices.csv")
n <- merge(data1,data2)
I have the code stored locally on my computer just for ease of access. The files are being read in properly, but for some reason when I merge, variable n receives no data, just the headers of the columns from the csv files. I have experimented with using inner_join to no avail as well as pulling the files directly from the http address linked above and using read_delim() commands but can't seem to get it to work. Any help or tips are much appreciated.
I'm trying to download data on margin requirements from the MCX website using R
However, I am unable to recognise the appropriate url to use in order to download this data.
The link is here
files for different dates have seemingly different urls
for instance:
DailyMargin_20170919223427.csv
DailyMargin_20170919223104.csv
DailyMargin_20170919223039.csv
They seem to be of the form
DailyMargin_2017091922****.csv
(20170919 is the date on which I'm trying to download the data)
My code has the line:
myURL = paste("https://www.mcxindia.com/market-operations/clearing-settlement/daily-margin", "DailyMargin_2017091922","****", ".csv", sep = "")
the ****** part seems to be random.
From what I can tell, the remaining **** appears to be a timestamp when the data are created by the webpage using javascript. You will probably not be able to directly download data as it will not exist until it is created. That said, you might be able to utilize a package like rvest to do the scrapping for you.
https://stat4701.github.io/edav/2015/04/02/rvest_tutorial/
I am programming in R. I need to download a set of files from an http: address. The naming format of the file refers to a date/time period but also contains additional numbering that is not recognizable. For example for the file below the first set of numbers refers to the date 2014/10/24 at 05:10am but the second batch of numbers is not recognizable. All files on the webpage follow this standard format.
http://www.nemweb.com.au/REPORTS/CURRENT/MCCDispatch/PUBLIC_MCCDISPATCH_201410240510_0000000258279329.zip
My question is: How do I download the file with only partial name information?
For example if I wanted to download the file relating to the 6:30 time period I know that the url prefix is as below, but would not know the numbers that follow after: http://www.nemweb.com.au/REPORTS/CURRENT/MCCDispatch/PUBLIC_MCCDISPATCH_201410240630_??????????????.zip
You're actually in luck. Because you have a directory listing. Essentially, you have to download the list of links and then grep them. Here's how you would go about doing that.
library(XML)
url <- "http://www.nemweb.com.au/REPORTS/CURRENT/MCCDispatch/"
parsed <- htmlParse(url)
links <- xpathSApply(parsed, "//#href")
Now you have a list of URLs that you can search through and choose the one that's appropriate.
Hint: grep("pattern",links)
I am EXTREMELY new to R, and programming in general, so thank you for your patience.
I am trying to write a script which reads values from a .txt file and after some manipulation plots the results. I have two questions which are somewhat coupled.
First, is there a function which asks the user to identify the location of a file? i.e. User runs script. Script opens up file navigation prompt and requests user to navigate to and select relevant file.
Currently, I have to manually identify the file and location in R. e.g.
spectra.raw <- read.table("C:\Users\...\file1.txt", row.names=NULL, header = TRUE)
I'd rather have the user identify the file location each time the script is run. This will be used by non-tech people, and I don't trust them to copy/paste file locations into R.
The second question I've been struggling with is, is it possible to create a variable name based off the file selected? For example, if the user selects "file1.txt" I'd like R to assign the output of read.table() to a variable named "file1.raw" much like the above "spectra.raw"
If it helps, all the file names will have the exact same number of characters, so if it's possible to select the last say 5 characters from the file location, that would work.
Thank you very much, and please excuse my ignorance.
See file.choose. Though I believe it behaves slightly differently on different platforms, so beware of that.
See assign, i.e. assign("fileName",value). You'll want to parse the file path that file.choose spits back using string manipulation functions like substr or strsplit.
Try
file.choose
I think it can do what you want.
For example,
myfile <- file.choose()
Enter file name: adataset.Rdata
load(myfile)
myfile contains the name of the file so you don't have to do anything special.
I am trying to get specific disease-related information from the GWAS catalog. This can be done directly from the website via a spreadsheet download. But I was wondering if I could possibly do it programmatically in R. Any suggestions will be greatly appreciated.
Thanks.
Avoks
Checkout the function download.file() and the package rcurl (http://cran.r-project.org/web/packages/RCurl/index.html) - this should do what you are looking for
You will have to download .tsv file(s) first and manually edit them.
This is because GWAS Catalog files contain HTML symbols, like § in "Behçet's disease" (defining that special fourth letter). The # in these symbols will be interpreted by R as an end of line, thus you will get an error message, e.g.:
line 2028 did not have 34 elements
So you downlad it first, open in plain text editor, automatically replace every # with empty character, and only then load it into R with:
read.table("gwas_catalog_v1.0-associations_e91_r2018-02-21.tsv",sep="\t",h=T,stringsAsFactors = F,quote="")