I have some .txt files in a folder that is not the folder where my script is. But when I try to open those files I get LoadError: SystemError: opening file "/some/folder/filename.txt": No such file or directory
path = "/some/folder/"
files = filter(file -> endswith(file, ".txt"), readdir(path))
for file in files
open(file, "r")
end
If I just do a println(file) in the for loop I can see that the files are there. But if I try to do anything to the files I get this error. I have used pwd() to get the correct directory. Really confused to as I'm getting this error.
From the docstring:
help?> readdir
search: readdir
readdir(dir::AbstractString=pwd();
join::Bool = false,
sort::Bool = true,
) -> Vector{String}
Return the names in the directory dir or the current working directory if not given. When join is false, readdir returns just the names in the directory as is; when join is true, it returns joinpath(dir,
name) for each name so that the returned strings are full paths. If you want to get absolute paths back, call readdir with an absolute directory path and join set to true.
i.e. you want readdir(path; join = true) to get the full paths to your files.
In my case the Terminal as part of the editor (at the bottom of the attached screenshot) had a different directory than my currently opened .jl file.
I use VSCode on Linux.
I manually changed the directory in the embedded terminal as part of the VSCode editor in order for julia to detect my file.
Related
I want to know how can I import a .txt file in R, but avoiding pathing to my file. I usually import like this: "Import Dataset" and the a select "From text(base)", but when I write in the program file.exists("myfilename.txt") it tells me FALSE. How can I do it correctly?
When you run file.exists("myfilename.txt"), R will search your current working directory for a file called myfilename.txt. If you have a file called myfilename.txt that you imported from some other directory, then file.exists("myfilename.txt") will return FALSE.
Solution 1:
Put your R script and the myfilename.txt file in the same folder
Change your working directory to that folder, either using the session menu or using setwd("path/to/folder")
file.exists("myfilename.txt") should now return TRUE
You can read your table with read.delim("myfilename.txt")
Solution 2:
Create an Rstudio project
Place your R script and the myfilename.txt file in the project folder.
Every time you open the project, your working directory will point to the project folder.
file.exists("myfilename.txt") is TRUE
You can read your table using read.delim("myfilename.txt").
Solution 3:
Leave myfilename.txt where it is and read it by providing the absolute path, for example: read.delim("C:/Users/Jiakai/Documents/myfilename.txt")
In this case file.exists("myfilename.txt") is FALSE and file.exists("C:/Users/Jiakai/Documents/myfilename.txt") is TRUE.
If you want file.exists("myfilename.txt") to return TRUE change your working directory to "C:/Users/Jiakai/Documents/myfilename.txt".
To import a txt file, you have several options. The two best options are
readr::read_delim("path/tomyfile/myfilename.txt", delim = "\t")
or
data.table::fread("path/tomyfile/myfilename.txt", sep = "\t")
They are preferrable to the base R read.delim function that is slower.
You can provide absolute paths or relative path, if you know your working directory
Edit
If you don't know your working directory, you can run
getwd()
If you don't find your file with file.exists it means you need to change your working directory or change the path in your import and file.exists command
I'm trying to open a file that does not exist in my current directory. This file is named testFile.r existing in the data folder in my current directory. I tried file.path("./data") and then wanted to show the file with this command file.show("testFile.r") but it gives this error:
Error: File testFile.r does not exist.
And the command getwd() gives the previous current directory. So any thoughts on this?
You change your current directory using the command setwd(new_directory)
(replacing new_directory with the path to the directory you want).
If you'd rather stay in your current directory, just do
file.show("./data/testFile.r")
To keep track of multiple paths, you can save each as a variable and use the paste function to reference the file:
path1 <- "./data/"
path2 <- "./second_folder_name/"
file.show(paste0(path1, "testFile.R"))
file.show(paste0(path2, "testFile.R"))
I am downloading a 78MB zip file from the UN FAO, which contains a 2.66GB csv. I am able to unzip the the downloaded file from a folder using winzip, but have been unable to unzip the file using unzip() in R:
Warning - 78MB download!
url <- "http://fenixservices.fao.org/faostat/static/bulkdownloads/FoodBalanceSheets_E_All_Data_(Normalized).zip"
path <- file.path(getwd(),"/zipped_data.zip")
download.file(url, path, mode = "wb")
unzipped_data <- unzip(path)
This results in a warning and a failure to unzip the file:
Warning message
In unzip(path) : zip file is corrupt
In the ?unzip documentation I see
"It does have some support for bzip2 compression and > 2GB zip files (but not >= 4GB files pre-compression contained in a zip file: like many builds of unzip it may truncate these, in R's case with a warning if possible)"
This makes me believe that unzip() should handle my file, but this same process has successfully downloaded, unzipped, and read multiple other smaller tables from the FAOstat. Is there a chance that the size of my csv is the source of this error? If so, what is the workaround?
I can't test my solution and it also depends on your installation but hopefully that'll work or at least point you to a suitable solution:
You can run winzip through command line, this page shows the structure of the call
And you can also run command lines from R, with system or shell (which is just a wrapper for system
The command line general structure to extract would be:
winzip32 -e [options] filename[.zip] folder
So we create a string with this structure and your input paths, and we create a function around it that mimics unzip with parameters zipfile and exdir
unzip_wz <- function(zipfile,exdir){
dir.create(exdir,recursive = FALSE,showWarnings=FALSE) # I don't know how/if unzip creates folders, you might want to tweak or remove this line altogether
str1 <- sprintf("winzip32 -e '%s' '%s'",zipfile,exdir)
shell(str1,wait = TRUE) # set to FALSE if you want the program to keep running while unzipping, proceed with caution but in some cases that could be an improvement of your current solution
}
You can try to use this function in place of unzip. It assumes that winzip32 was added to your system path variables, if it isn't, either add it, or replace it by the exec full name so you have something like:
str1 <- sprintf("'C://probably/somewhere/in/program/files/winzip32.exe' -e '%s' '%s'",zipfile,exdir)
PS: use full paths! the command line doesn't know your working directories (we could implement the feature in our function if needed).
I had the same problem running unzip() on Ubuntu Server 20.04. Setting argument unzip(..., unzip = "/usr/bin/unzip"), instead of unzip = "internal", did the trick.
Environment:
Windows 7 OS
RStudio Version 0.99.491
I have been programming in R for about 4 months via the Coursera Data Science curriculum, but I have NEVER been successful in using the unzip function.
I've looked at the forums for hours for potential solutions, syntax problems, undefined arguments, etc., but to no avail. I eventually unzip the contents manually and proceed with the assignment, but I am tired of not knowing why it is not working.
Here are a few examples of the error:
fileName <- "StormData.zip"
unzip(fileName, exdir = mainDir,subDir)
Warning message: In unzip(fileName, exdir = mainDir, subDir) : error
1 in extracting from zip file
unzip(fileName)
Warning message: In unzip(fileName) : error 1 in extracting from zip
file
unzip(fileName, "stormdata.csv")
Warning message: In unzip(fileName, "stormdata.csv") : error 1 in
extracting from zip file
unzip(fileName, "stormdata.csv", list = TRUE)
Error in unzip(fileName, "stormdata.csv", list = TRUE) : zip file
'StormData.zip' cannot be opened
Any suggestions would be greatly appreciated.
I was getting the same error.
I changed the path --
from :
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata"
to
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata.zip"
and it works fine!
setwd("C:\\Users\\Sharvari\\Downloads")
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata.zip"
unzip(uzp, exdir = "C:\\Users\\Sharvari\\Desktop\\specdata")
I too was getting that error 1 message when trying to unzipping a zip file. Glitch in my case was the conflict between working directory and zip file path.
My case was:
My working directory was like "C:/Users/SCOTT/Desktop/Training"
While my zip file was located in "C:/Users/SCOTT/Desktop/Training/house_consumption_data"
When I was trying to execute this:
unzip("house_data.zip")
Possibly your file is in a different folder.
I have had the same problem trying to download and unzip the same file, for the same course. And I have had problems with unzip in the past and was determined to solve it this time too.
Eventually the extension of the file turned out to be csv.bz2. And than this Extract bz2 file in R post solved my problem.
After downloading the file I was able to read it directly with
stormdata <- read.csv("stormdata.zip")
without using unzip.
This error seems to appear whenever openXLS is unable to open the specified file.
It could be a wrong name, wrong directory or the file might be encrypted or password protected
change your zip file format this error will appear while the zip format problems occur, look at your zip file it should be "rar" change it to "zip". the function works only for "zip" format files.
I faced the same issue. Make sure that, you specify the correct name of the file(get it from the properties of .zip file) in the following code.
file = read.table(unzip("file_name.csv.zip"), sep = ",", header = TRUE)
In my case, Was just mentioning file_name.zip and R was throwing the error.
Also, there are two functions for unzipping files in R
1) unz - to extract single element from zip file/s
2) unzip - to extract all the present elements from the .zip file
I usually prefer unzip.
If you will use unz in the above code, R will throw error again.
I encountered the same error using install_course_zip' with a zip file. I followed all the instructions for the command faithfully but kept getting errors relating to the 'exdir'. I moved the zip file to various directories without success.
I finally used getwd() to get the working directory and then placed the zip file in that directory. I then was able to use the zip file name without having to use any folder structure and this worked. I still have no idea why R would not accept a different directory.
I had list of files to be unzipped and processed; I was facing same error
"error 1 in extracting from zip file"
used full directory and set working directory code worked
files <- list.files(path="C:\\Users\\Tejas naik\\Documents\\land", pattern=".zip$")
out_dir<- "C:\\Users\\Tejas naik\\Documents\\input"
setwd("C:\\Users\\Tejas naik\\Documents\\land")
for (i in files) {
#nzip(paste(out_dir,i), exdir=out_dir)
unzip(i,exdir=out_dir)
}
This error was happening bit differently in my case . As there was no zip file ,the issue was file was open in excel so this error was poping up .
It's crucial to give the full name (including the path) of the zip-file to the unzip function.
So instead of file.zip, it should be C:\user\name\file.zip.
In case you're using the list.files function, one should set the full.names option to TRUE.
For me the error is fixed after I add \ backslash character to the filepath.
Example:
from
unzip("abc\aaa.zip")
to
unzip("abc\\aaa.zip")
I am trying to use python to create the files needed to run some other software in batch.
For part of this I need to produce a text file that loads the needed data files into the software.
My problem is that the files I need to enter into this text file are stored in a set of structured folders.
I need to loop over a set of folders (up to 20), which each could contain up to 3 more folders which contain the files I need. The bottom level of the folders contain a set of files needed for each run of the software. The text file should have the path+name of these files printed line by line, add an instruction line and then move to the next set of files from a folder and so on until all of sub level folders have been checked.
Charles' answer is good, but can be improved upon to increase speed and efficiency. Each item produced by os.walk() (See docs) is a tuple of three items. Those items are:
The working directory
A list of strings naming any sub-directories present in the working directory
A list of files present in the working directory
Knowing this, much of Charles' code can be condensed with the modification of a forloop:
import os
def list_files(dir):
r = []
for root, dirs, files in os.walk(dir):
for name in files:
r.append(os.path.join(root, name))
return r
Use os.walk(). The following will output a list of all files within the subdirectories of "dir". The results can be manipulated to suit you needs:
import os
def list_files(dir):
r = []
subdirs = [x[0] for x in os.walk(dir)]
for subdir in subdirs:
files = os.walk(subdir).next()[2]
if (len(files) > 0):
for file in files:
r.append(os.path.join(subdir, file))
return r
For python 3, change next() to __next__().
This will help to list specific file extension. In my sub-folders i have many files but i am only interested parquet files.
import os
dir = r'/home/output/'
def list_files(dir):
r = []
for root, dirs, files in os.walk(dir):
for name in files:
filepath = root + os.sep + name
if filepath.endswith(".snappy.parquet"):
r.append(os.path.join(root, name))
return r