I have an array with several files in it. And i want to loop through these files. For each file i want to run a command.
result = [rtlvis_20190518_13.35.48_00001.bin, rtlvis_20190518_13.35.48_00002.bin, rtlvis_20190518_13.35.48_00003.bin, rtlvis_20190518_13.35.48_00004.bin, rtlvis_20190518_13.35.48_00005.bin]
something like: for each file in result run the following command read_rtlvis_v12,a,c,t,g,FILE="file",/CFILEONLY
Where file is each one of the files in result
Ive tried the following
FOREACH file, result do begin read_rtlvis_v12,a,c,t,g,FILE="file",/CFILEONLY
The error i get is in the read_rtlvis_v12 code. But my question is, is this the right way to go about doing a for loop with this kind of command.
Am i setting FILE="file" correctly, where file is each one of the files in result.
Do not use quotes around "file" — that is trying to read a file literally named "file". Use:
foreach file, result do read_rtlvis_v12, a, c, t, g, FILE=file, /CFILEONLY
Related
I need to download many files from a server (specifically tectia) ideally using the ssh package. These files all follow the a predictable pattern across multiple sub folders. The filepath is formatted like this
/directory/subfolder/A001/abcde001.csv
Where A001 counts up alongside the last 3 digits of the filename (/A002/abcde002.csv and so on)
In the vignette for scp_download it states that the files parameter may contain wildcards so I have tried to do something like
scp_download(session, "/directory/subfolder/A.*/abcde.*[.]csv", to=tempdir())
and
scp_download(session, "directory/subfolder/A\\d{3}/abcde\\d{3}[.]csv", to=tempdir())
but no matter which combination of patterns or wildcards I can think of (which isn't many) I only get something like
Warning: SSH warning: scp: /directory/subfolder/A\d{3}/abcde\d{3}[.]csv: No such file or directory
What I'm hoping to do is either find a way to do pattern matching here, or to find a way to store tectia directories as a string to be read by scp_download. I've made sure that my session is connected properly and it works without attempting to pattern match, which it does.
I had the same problem. The problem is that when you use * in your pattern it gets escaped when you send it to the server. However, when you request a special file name like this /directory/subfolder/A001/abcde001.csv, it works fine.
Finally I changed my code based on the below steps:
I got the list of files/folders using ls command with ssh_exec_wait function and then store them on a variable.
Download files in the variable separately
session <- ssh_connect("username#ip",passwd="password")
files<-capture.output(ssh_exec_wait(session, command = 'ls /directory/subfolder/A001/*'))
dnc1<- scp_download(session, files[1], to = paste0(getwd(),"/data/"))
dnc2<- scp_download(session, files[2], to = paste0(getwd(),"/data/"))
dnc3<- scp_download(session, files[3], to = paste0(getwd(),"/data/"))
The bottom 3 commands can be done in a loop as this could be hundreds or thousands of records.
I am using system() to run several files iteratively through a program via CMD. It deposits each outputs into a sub-directory designated for specifically and only that input file. So # of inputs is exactly equal to the number of output directories/outputs.
My code works for the first iteration, but I can see in the console that it won't move on to the second file after completing the first. The stop sign remains active so I know R is still 'running', but since the for loop environment is unique I can't really tell what it's stuck on. It just stays like this for hours. Therefore I'm not sure how to begin to diagnose the issue I'm having. Is there a way of tracing what happened after cancelling the code, for example?
If your curious, the code looks like this btw. I don't know how to make it reproducible, so I just commented each line:
for (i in 1:length(flist)) {
##flist is a vector of character strings. Each
row of characters is both the name of the input file and the name of the
output directory
setwd(paste0(solutions_dir, "\\", flist[i]))
#sets the appropriate dir
system(paste0(program_dir,"\\program.exe I=",
file_dir, "\\", flist[i], " O=",solutions_dir, "\\", flist[i],
"\\solv"))
##line that inputs program's exe file and the appropriate input/output
locations
}
Using QMake, I read some boiler plate code, make modifications and write the modified code to a file.
However, I get very strange results. I have simplified the problem down to the following:
# Open boiler plate file
interfaceBoilerPlateCode = $$cat($$boilerPlateFile, blob)
# Make sure we read the right content
message("Content read: $$interfaceBoilerPlateCode")
# Write the read text into a file
output = $$system(echo $$interfaceBoilerPlateCode >> $$targetFile) # Doesnt work
output = $$system(echo "Howde" >> $$targetFile) # This works
The file being read is a plain text file containing only the string "Howde".
The contents of the file get read correctly.
However, when I try and write the contents of the file to another target file, I get no output (literally: no errors/warnings but no new file generated). However, if I use echo with just a string defined in the code itself (as in the last line of snippet above), a new file gets generated with the string "Howde" inside it.
What is going on? What am I doing wrong that the penultimate line does not generate a new file?
Use write_file. Instead of:
$$system(echo $$content >> $$file_path)
use
write_file($$file_path, $$content)
I am trying to run an R-script file using windows task scheduler that runs it every two hours. What I am trying to do is gather some tweets through Twitter API and run a sentiment analysis that produces two graphs and saves it in a directory. The problem is, when the script is run again it replaces the already existing files with that name in the directory.
As an example, when I used the pdf("file") function, it ran fine for the first time as no file with that name already existED in the directory. Problem is I want the R-script to be running every other hour. So, I need some solution that creates a new file in the directory instead of replacing that file. Just like what happens when a file is downloaded multiple times from Google Chrome.
I'd just time-stamp the file name.
> filename = paste("output-",now(),sep="")
> filename
[1] "output-2014-08-21 16:02:45"
Use any of the standard date formatting functions to customise to taste - maybe you don't want spaces and colons in your file names:
> filename = paste("output-",format(Sys.time(), "%a-%b-%d-%H-%M-%S-%Y"),sep="")
> filename
[1] "output-Thu-Aug-21-16-03-30-2014"
If you want the behaviour of adding a number to the file name, then something like this:
serialNext = function(prefix){
if(!file.exists(prefix)){return(prefix)}
i=1
repeat {
f = paste(prefix,i,sep=".")
if(!file.exists(f)){return(f)}
i=i+1
}
}
Usage. First, "foo" doesn't exist, so it returns "foo":
> serialNext("foo")
[1] "foo"
Write a file called "foo":
> cat("fnord",file="foo")
Now it returns "foo.1":
> serialNext("foo")
[1] "foo.1"
Create that, then it returns "foo.2" and so on...
> cat("fnord",file="foo.1")
> serialNext("foo")
[1] "foo.2"
This kind of thing can break if more than one process might be writing a new file though - if both processes check at the same time there's a window of opportunity where both processes don't see "foo.2" and think they can both create it. The same thing will happen with timestamps if you have two processes trying to write new files at the same time.
Both these issues can be resolved by generating a random UUID and pasting that on the filename, otherwise you need something that's atomic at the operating system level.
But for a twice-hourly job I reckon a timestamp down to minutes is probably enough.
See ?files for file manipulation functions. You can check if file exists with file.exists, and then either rename the existing file, or create a different name for the new one.
I am writing a CGI script in Perl with a section of embedded R script which produces a graph. The original data filename is unknown as it has been uploaded by the CGI script and is stored in a Perl variable called $filename.
My question is that I now would like to open that file in R using read.table(). I am using Statistics::R and so I have tried:
my $R = Statistics::R->new();
$R->set('filename',$filename);
my $out1 = $R->run(
q`rm(list=ls())`,
# Fetch data
q`setwd("/var/www/uploads")`,
q`peakdata<-read.table(filename, sep="",col.names=c("mz","intensity","ionsscore","matched","query","index","hit"))`,
q`attach(peakdata)` ...etc
I can get this to work ONLY if I change $filename into something static and known like 'data.txt' before trying to open the file in read.table - is there a way for me to open a file with a variable for a name?
Thank you in advance.
One possible way to do this is by doing a little more work in Perl.
This is untested code, to give you some ideas:
my $filename = 'fileNameIGotFromSomewhere.txt'
my $source_dir = '/var/www/uploads';
my $file = "$source_dir/$fielname";
# make sure we can read it
unless ( -r $file ) {
die 'can read that data file: $!";
}
Then instead of $R->set, you could interpolate the file name into the R program. Where you've used the single-quote operator, use the double-quote operator instead:
So instead of:
q`peakdata<-read.table(filename, sep="",col.names= .... )`
Use:
qq`peakdata<-read.table($filename, sep="",col.names= .... )`
Now this looks like it would be inviting problems similar to SQL/Code Injections, so that's why I put int the logic to insure that the file exists and is readable. You might be able to think other checks to add to safeguard your use of user-supplied info.