I have made some changes in following folders structure for a Python project-
Inside the categories folder my csv file is there and I am reading it from a file reside inside merchants folder.
I am facing FileNotFoundError: File b'/categories/abc.csv' does not exist in folllowing line-
rfile = pd.read_csv('categories/abc.csv')
And I am generating this csv file from a python file reside inside the utils folder.
Now if I re run my csv file generation file from utils folder I am getting-
FileNotFoundError: [Errno 2] No such file or directory: 'categories/abc.csv'
Can anyone please point out what I am doing wrong here?
rfile = pd.read_csv('../categories/abc.csv')
First of all come out of your merchants directory and navigate to categories directory, as mentioned in my code.
The path you are passing is relative to python file. So make sure either pass absolute path to CSV file, or relative path is correct.
Related
I am trying to use the load() function to load an .RData file into my current R session. I open my code base by double clicking on the R project file in my directory and then attempt to load the .RData file by running load('./Data/mydata.rdata'). However, this returns a 'No such file or directory' error. I verified that the working directory is correct by using getwd().
I figure I must be using incorrect syntax because I have no issues loading the file when I type in the full file path.
Working directory for R Project file: "/Users/Me/Library/OneDrive/RStuff"
Directory containing .RData file: "/Users/Me/Library/OneDrive/RStuff/Data"
Code that works: load("/Users/Me/Library/OneDrive/RStuff/Data/mydata.rdata")
Code that fails: load('./Data/mydata.rdata')
Do relative paths not work with load() or is my syntax wrong?
Do relative paths not work with load() or is my syntax wrong?
It looks like there is an error with your paths. You said the working directory is "/Users/Me/Library/OneDrive/RStuff"
But according to the code that works:
load("/Users/Me/Library/OneDrive/RStuff/Data/mydata.rdata")
...the data file is in "/Users/Me/Library/OneDrive/RStuff/Data"
Therefore to use relative paths, you would use:
load("./Data/mydata.rdata")
mission on online course:
Download this RData file to your working directory. Then load the data into R with the following command:
load("skew.RData")
so I have downloaded it to my computer but where is my working directory? or how do I load the downloaded file to Rstudio
You can see your working directory by executing getwd().
In order to load the data you need to change it to the folder path where the data is stored.
setwd("C:/your/path")
load("skew.RData")
You can set you working directory in RStudio on the right side manually and then save the files there and open them like :
load("skew.RData")
If your file is saved somewhere else, you should define the path to it:
load("path/to/your/file/skew.RData")
I have two projects in R. I've moved an .Rmd document from project 1 to project 2.
When I used the .Rmd file which I had moved to project 2 to try and read in some data I get the following error message:
cannot open file '"mydata.csv"': No such file or directoryError in file(file, "rt") : cannot open the connection.
This kind of error usually suggests to me it's a working directory issue, however when I run getwd() in the command line it's the correct working directory that is listed and points to where the csv is stored. I've also run getwd() within the rmd doc and again the wd is correct.
Does anyone else have this experience of moving one .Rmd file to another project and then it not working in the new project?
The code in the .Rmd file that I am trying to run is:
Data <- read.csv("mydata.csv", stringsAsFactors = T) and the data is definitely within the project and has the correct title, is a csv etc.
Has anyone else seen this issue when moving an RMarkdown document into another project before?
Thanks
This may not be the answer, but rmarkdown and knitr intentionally don't respect setwd(): the code in each block is run from the directory holding the .rmd file. So, if you've moved your .rmd file but are then using setwd() to change to the directory holding the data, that does not persist across code chunks.
If this is the issue, then one solution is to use the knitr options to set the root.dir to the data location:
opts_knit$set(root.dir = '/path/to/the/data')
See here.
Maybe not relevant but it seems to be the most likely explanation for what's happening here:
The project shouldn't really interfere with your code here. When opening the project it will set your working directory to the root location of the project. However, this shouldn't matter in this case since RMarkdown files automatically set the working directory to the location where the RMarkdown file is saved. You can see this when running getwd() once in the Console and once from the RMarkdown file via run current chunk.
The behavior is the same when the file is knitted. So except in the case when "mydata.csv" is in the same directory as the RMarkdown file, the code above won't work.
There are two workarounds: you can use relative or absolute paths to navigate to your data file. In a project I prefer relative paths. Let's say the rmd file is in a folder called "scripts" and your data file is in a folder called "data" and both are in the same project folder. Then this should work:
Data <- read.csv("../data/mydata.csv", stringsAsFactors = TRUE)
The other option, which I do not recommend, is to set the working diretory in the rmd file via:
opts_knit$set(root.dir = '../data/')
The reason why I wouldn't do that is because the working direcotry is only changed when knitting the document but using the rmd file interactivly, you have a different working directory (the location of the rmd file)
This is a great application of the here package for dealing with these types of issues.
here https://github.com/jennybc/here_here looks around the Rmd file (and all R files) for the .Rproj file and then uses that as an "anchor" to search for other files using relative references. If for instance you had the following project structure:
-data
|--mydata.csv
-src
|-00-import.R
|-01-make-graphs.R
|-02-do-analysis.R
-report
|--report.Rmd
-yourproject.Rproj
And you wanted to use mydata.csv in your report.Rmd you could do the following:
library(here)
dat <- read.csv(here("data", "mydata.csv"))
here will then convert this path to "~/Users/../data/mydata.csv" for you. Note that you have to be in the Rproject for this use case.
Here is such a great package and solves a lot of issues.
On a mac using iCloud file optimization, large files that are seldom used are uploaded to iCloud and only a small pointer file is left. When I look for the file in Finder, I see the file name and to the left is an icon that indicates that the file is in the cloud. To access the file, I click on the icon and the file is downloaded. With the file.exist command, R returns FALSE for the existence of the file. But after some research I found that the file link is stored in a directory below ~/Library/Mobile\ Documents/com~apple~CloudDocsand the file name is changed to xxx.icloud where xxx is the original file name.
Here's an example of the path to a a directory that holds a .icloud file from a shell in my mac
/Users/gcn/Library/Mobile Documents/com~apple~CloudDocs/Documents/workspace/nutmod/data-raw/NutrientData
I can query for the existence of the file with exists(xxx.icloud). But how do I tell my mac to download the iCloud file and then read it in? Using something like read.table or read.csv doesn't work because the pointer file is not csv.
You can read a csv file directly from a iCloud folder on the Mac by using the path to that folder. Get the path by finding it in the finder. Then right click on the filename at the bottom of the finder window where it shows all the folders leading to the file and choose: Copy "YourFile" as Pathname.
That path will look something like this:
"/Users/NAME/Library/Mobile
Documents/com~apple~CloudDoc/Docs/YourFile.csv"
Use that in your read code:
iCloudDat <- read_csv("/Users/NAME/Library/Mobile Documents/com~apple~CloudDocs/Documents/YourFileName.csv")
That should work.
If the extension isn't .txt or .csv read.table and read.csv won't work.
you have to download the file and extract the tables to a readable format.
you can download the file using download.file() which is is the utils basic library.
I am running some R code on a Windows computer using RStudio and my code generates Excel files and netCDF files periodically (dozens of them eventually). I don't want them to clutter my working directory. Is there a way to save the files to a directory called "Output" (ex: C:/.../original file path/Output) in the parent directory? I would like a way to change my current working directory to a different directory. I understand there is getwd() and setwd() but how do I set the path to the output directory without typing out the entire windows path (for example: setwd(current source file path for windows or Mac/output). My collaborator uses a Mac and he would have his output stored there as well.
You have a file argument in your write* function. If your Output directory is in your working directory, it works like this:
write.xlsx(df, file = "Output/table.xlsx")
write.csv(df, file = "Output/table.csv")
You can specify an argument in your write.csv function and other similar write functions which specifies your path.
#Output path
OutPath<- "C:/blah/blahblah/op/"
#Table to dump as output
OutTbl <- iris
write.csv(OutTbl, file = OutPath)
Source: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/write.table.html