I'm currently doing a project that uses R to process some large csv files that are saved in my local directory linked to my repo.
So far, I managed to create the R project and commit and push R scripts into the repo with no problem.
However, the scripts read in the data from the csv files saved in my local directory, so the code goes in a form
df <- read.csv("mylocaldirectorylink")
However, this is not helpful if my partner and I working on the same project have to change that url to our own local directory every time we pull it off the repo. So I was thinking that maybe we can upload the csv files onto GitHub Repo and let the R script refer directly to the csv files online.
So my questions are:
Why can't I upload csv files onto GitHub? They keep saying that my file is too large.
If I can upload the csv files, how to I read the data from these csv files?
Firstly, it's generally a bad idea to store data on Github, especially if it's large. If you want to save it somewhere on the Internet, you can use, say, Dataverse, and then can access your data with URL (through the API), or Google Drive, as Jake Kaupp suggested.
Now back to your question. If your data doesn't change, I would just use not the absolute paths to CSV but relative ones. In other words, instead of
df<-read.csv("C:/folder/subfolder/data.csv")
I would use
df <- read.csv("../data.csv")
If you are working with R project, then the initial working directory is inside the folder of the project. You can check it with getwd(). This working directory changes as you move the R project. Just agree with your colleague that your data file should be in the same folder where the folder with R project is situated.
This is for a Python script.
You can track csv files by editing your .gitignore file.
**OR**
You can add csv files in your github repo, which can be used by others.
I did so by following steps:
Checkout the branch on github.com
Go to the folder where you want to keep csv files.
Here, you will see an option "Add file" in top right area as shown below:
Here you can upload csv files and commit the changes in same branch or by creating a new branch.
Related
I'm providing a .zip with a .R file and a .xlsx file to some people
I need to make a code that can read this .xlsx file in any directory of any pc.
But as the directories vary from computer to computer, I couldn't find a solution.
IMPORTANT: I'm not using Rstudio for read this .R, so i just can use base functions
Using R - How do I search for a file/folder on all drives (hard drives as well as USB drives) This question don't solve my problem..
Take a look at the here package. When you load the library (library("here")) it sets "base" working directory and then you can use the package to construct relative file paths given that location. For example, if inside your .zip file you have an R script (e.g., My Data Analysis.R) that analyzes data that is kept within a folder called data you could read it in using, for example, read.csv(here("data", "my_csv_file.csv")) and it will construct the full appropriate file path no matter what computer it is on. Of course the file structure of the program needs to stay the same across programs.
I've been building an R package via an R project file (.Rproj) in R studio with the project file linked to my github. When I was working on it this morning, all was as it should be, but when I returned to it this afternoon, the .Rproj file had mysteriously vanished. All other files and folders associated with the package are where they should be (e.g., the "R" folder with the scripts for the package's functions, the DESCRIPTION file, the man folder, etc.) but the .Rproj file is gone. I have not deleted it and it is not in my trash can, nor does git have any record of it for some reason.
If I do a search for the file name in the windows search bar (I'm suing windows 7, btw) the file shows up and says that it is in the correct location, but if I click on it, I get an error message saying that the file has been moved or deleted. Similarly, if I try to navigate to where the file should be via "open project" in R, there is a greyed out ".Rproj.user" folder that I don't recall having seen before, but no project file.
I'm at my wits end for what is going on. The package on github appears to all still be correct (as do the files on my computer), but without being able to find or access the .Rproj file I can't easily edit the package and push the changes to github.
Does anyone have any idea what is going on or how to retrieve my package file? Since I have all the source files for it, I could always build a new project using the source files, but then I need that to link up to my github which is linked to a now non-existent project file, and everything becomes really messy and tedious quickly.
PS I have restarted both my computer and R. It didn't help.
You can probably just create a "new" project in the existing folder to generate a new .Rproj file.
The .Rproj file doesn't actually do that much. It mostly lets RStudio know that "this folder is the root directory of a project named X". Git keeps its own hidden files in the directory to track things (that RStudio will look for if it's a project).
If you haven't done this, I'd also recommend adding your .Rproj files to your .gitignore file - Rproj files are user-specific so shouldn't be tracked in version control. (If you used RStudio to create the git repo, this will be done automatically.)
I'm creating an app where users need to work with large databases. Rather than having the users download the data and populate an SQLite database on the client (which would take a long time), I'm hoping I can use downloadable, pre-populated databases.
I found cordova-sqlite-ext, which allows working with pre-populated databases, but SQLite files must be located in the www folder for this to work. Is it actually possible to download files to this folder in Ionic/Cordova (on non-rooted devices)?
It's always good practise to store your files in app's directory. Check this comment of mine and see if it helps you:
https://github.com/driftyco/ionic-native/issues/881#issuecomment-270832521
I had a requirement of downloading a zip file(with sqlite file in it), unzip the file, store the file in app's directory and query the DB. I was able to achieve it using plugins and works quite well.
If I put a very small csv file in my GitHub directory so that it gets copied to /ocpu/github/username/projectname/www/ , will I be able to access the contents of the csv for use in a R function? I tried to ajax the file, but I get a 404 error even though I can see the csv file sitting in the www directory of my local server. I need to have the csv on the server as a static file rather than being uploaded by a function. Thanks
You should be able to access them like any other file. Can you post an example that shows what you are doing and what error you are getting?
That said, if you just want to use this data in your R functions, it is better to include it in the R package as an actual data file. Also see section 1.1.6 of Writing R Extensions. An example is the mapapp package, which includes a dataset called countryExData. Also see the live app.
I was wondering if its possible to somehow given a dropbox folder link, download all files in that folder in to R?
If not is is possible, how would one be able to upload files onto Shiny Server so maybe I could source them locally?
Thanks,