I created a folder in the local R directory
str1 = 'Test_run_'
filename = paste0(sub('\\..*', ' ', str1), format(Sys.time(),'%d_%m%__%H_%M'))
dir.create(filename)
how can I save files to this folder (filename)?
I am looking a way where I can upload files with in a nested folder using R Programming
I tried below functions which uploads either file or folder from aws.s3 package
library("aws.s3")
put_object("pathoftheobject", object = "filename", bucket = "bucketname")
put_folder("foldername", bucket = "bucketname")
Folder Structure and Files :
ParentFolder
SubFolder1
File1
File2
SubFoler2
File3
File4
............
SubFoldern
Filen
Any guidance here will be really useful.
I'm not an R developer but in C# instead of creating folders under buckets, I use filename with / so S3 recognizes that as sub-folder.
my file name when uploading:
ParentFolder/SubFolder1/Fil1
ParentFolder/SubFolder1/Fil2
ParentFolder/SubFolder2/Fil3
aws.s3::s3sync function does the needful.
library(aws.s3)
Sys.setenv("AWS_ACCESS_KEY_ID" = access_key_id,
"AWS_SECRET_ACCESS_KEY" = secret_access_key,
"AWS_DEFAULT_REGION" = "eu-central-1",
"AWS_SESSION_TOKEN" = session_token)
s3sync(files =dir(paste0(getwd(), "/Folder1Name/","Folder2Name"),recursive = T),
bucket = "BucketName", direction = "upload",verbose = TRUE)
I was following along this SO post on how to download and unzip a file.
url <- 'https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz'
file <- basename(url)
download.file(url, file)
tmpdir <- tempdir()
untar(file, compressed = 'gzip', exdir = tmpdir)
Everything runs fine in the above block except the last line which returns
> untar(file, compressed = 'gzip', exdir = tmpdir)
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
Warning message:
In untar(file, compressed = "gzip", exdir = tmpdir) :
‘/usr/bin/tar -xf 'GoogleNews-vectors-negative300.bin.gz' -C '/var/folders/ll/g08vjcnd33vdhf250bbg9230fdw13f/T//RtmpAICV5a'’ returned error code 1
How can I unzip GoogleNews-vectors-negative300.bin.gz after downloading it?
I am trying to extract(unzip) folder (namely "pakistan.zip" which contains 5 files Pak_admin0.shp, Pak_admin0.shx, Pak_admin0.dbf, Pak_admin0.prj, Pak_admin0.qpj) and copying the files of .shp, .shx, .dbf files from that folder to destination folder using Rstudio 0.99.451 version with the following codes:
for(j in list(".shp", ".shx", ".dbf"))
{
fname <- unzip(file=paste("pakistan", j, sep=""), zipfile= "pakistan.zip")
file.copy(fname, paste("./pakistan", j, sep="/"), overwrite=TRUE)
}
unlink("pakistan.zip")
but it gives me following error
Warning messages:
1: In unzip(file = paste("zupanije", j, sep = ""), zipfile = "pakistan.zip") : requested file not found in the zip file
2: In unzip(file = paste("zupanije", j, sep = ""), zipfile = "pakistan.zip") : requested file not found in the zip file
3: In unzip(file = paste("zupanije", j, sep = ""), zipfile = "pakistan.zip") : requested file not found in the zip file
Please provide any possible solution to deal with this error.
These are actual codes which I have found but zip.file.extract function is no longer part of R:
for(j in list(".shp", ".shx", ".dbf")){
fname <- zip.file.extract(file=paste("zupanije", j, sep=""),
zipname="zupanije.zip")
file.copy(fname, paste("./zupanije", j, sep=""), overwrite=TRUE)
}
unlink("zupanije.zip")
I want to automate the structure of downloading the shape file from website and unzip it and place into another folder then will display it using maptools library using readShapePoly() function.
Your code works for me for a zip file that contains those files. The error suggests those files are not contained in the zip file. Since you say you are trying to extract a "directory" perhaps they are in a subdirectory in the zipfile? For example, if I put the files in a "temp" directory and then create a zip file of that directory, I must add the directory to the file path, like this:
f <- "test.zip"
for(j in list(".shp", ".shx", ".dbf"))
{
# note "pakistan" directory added to path
# unzip pakistan/zupanije.shp (or .shx or .dbf) out of test.zip
fname <- unzip(file=paste("pakistan/zupanije", j, sep=""), zipfile= f)
#copy extracted file to destination directory
file.copy(fname, paste("./destination", j, sep="/"), overwrite=TRUE)
}
If you are in a Linux like environment, you could try the following command to inspect the zip file and ensure it contains what you think it contains and at the path you expect:
unzip -vl pakistan.zip
By the way, your code will output the file "./pakistan/.dbf", "./pakistan/.shx" and "./pakistan/.shp". Is that what you want? Or do you perhaps want "pakistan.shx", etc. in which case this change is needed:
-file.copy(fname, paste("./pakistan", j, sep="/"), overwrite=TRUE)
+file.copy(fname, paste("./pakistan", j, sep=""), overwrite=TRUE)
I'm trying to make my research reproducible storing the data at figshare.
Something strange happens when I download and unzip the data in R.
here is the zip
If I download it manually, it opens ok; but when I try to get it with an R script, the downloaded archive is corrupt. Any ideas where is the problem?
the code to reproduce my error
url <- 'https://ndownloader.figshare.com/files/4797355'
path <- 'test/missing_data_raw.zip'
ifelse(file.exists(path1), yes = 'file alredy exists', no = download.file(url1, path1))
unzip(zipfile = path1,exdir = 'test')
Try setting the download mode to binary explicitly:
url <- 'https://ndownloader.figshare.com/files/4797217'
path1 <- tempfile(fileext = ".zip")
if (file.exists(path1)) 'file alredy exists' else download.file(url, path1, mode="wb")
unzip(zipfile = path1,exdir = tempdir())