I am writing a CGI script in Perl with a section of embedded R script which produces a graph. The original data filename is unknown as it has been uploaded by the CGI script and is stored in a Perl variable called $filename.
My question is that I now would like to open that file in R using read.table(). I am using Statistics::R and so I have tried:
my $R = Statistics::R->new();
$R->set('filename',$filename);
my $out1 = $R->run(
q`rm(list=ls())`,
# Fetch data
q`setwd("/var/www/uploads")`,
q`peakdata<-read.table(filename, sep="",col.names=c("mz","intensity","ionsscore","matched","query","index","hit"))`,
q`attach(peakdata)` ...etc
I can get this to work ONLY if I change $filename into something static and known like 'data.txt' before trying to open the file in read.table - is there a way for me to open a file with a variable for a name?
Thank you in advance.
One possible way to do this is by doing a little more work in Perl.
This is untested code, to give you some ideas:
my $filename = 'fileNameIGotFromSomewhere.txt'
my $source_dir = '/var/www/uploads';
my $file = "$source_dir/$fielname";
# make sure we can read it
unless ( -r $file ) {
die 'can read that data file: $!";
}
Then instead of $R->set, you could interpolate the file name into the R program. Where you've used the single-quote operator, use the double-quote operator instead:
So instead of:
q`peakdata<-read.table(filename, sep="",col.names= .... )`
Use:
qq`peakdata<-read.table($filename, sep="",col.names= .... )`
Now this looks like it would be inviting problems similar to SQL/Code Injections, so that's why I put int the logic to insure that the file exists and is readable. You might be able to think other checks to add to safeguard your use of user-supplied info.
Related
For my job, I have to use an .my.cnf file to pass a password key into my R code, since I cannot (for security reasons) keep my password key in the R code itself. So I have R code that looks like this:
library(configr)
my_configs = read.config(file = "~/.my.cnf")
my_googleservicekey = my_configs$api_authentication$googleServiceKeyJsonString
Separately, I have a .my.cnf file saved on my computer, and that file looks like this:
[api authentication]
googleServiceKeyJsonString = {\r\n \"type"\: \"service_account\",\r\n \"project_id\": ... ... \r\n}
where the ... ... signifies that the actual googleServiceKey is much longer. This key was provided by a coworker of mine, and needs to be set exactly as such in the R variable my_googleservicekey. However, when I save the key like this in the .my.cnf file, and read it into R in the manner that I did, I get the following:
> my_googleservicekey
[1] "{\\r\\n \\\"type\\\": \\\"service_account\\\",\\r\\n \\\"project_id\\\": ... ... \\r\\n}"
So what happens is, sometime when read.config(file = "~/.my.cnf") is run, the
object/string I passed/saved into the .my.cnf file looks like it had all of the quotes and backslashes escaped, and as a result I don't have the correct variable in R.
My question then is this:
Is it possible to save the key in the .my.cnf and have read.config in R read the key EXACTLY as it is saved in the .my.cnf file, so that it doesn't add the additional escapes?
Thanks in advance for any help with this!!
EDIT: In particular, are there any parameters to R's read.config function, or any other functions in R that could help out with this?
I want to use a relative file path as a command line argument but as the example and assessment below will demonstrate, the variable passes \..\ as a string, it doesn't evaluate it.
Can I can force the command line to parse and expand the variable as a string?
: For example: I have a R script file I want to launch from the command line:
Set RPath=C:\Program Files\R\R-3.1.0\bin\Rscript.exe
SET RScript=%CD%\..\..\HCF_v9.R
SET SourceFile=%CD%\..\Source\
ECHO String used for Source Location - %SourceFile%
"%RPath%" "%RScript%" %SourceFile%
The inclusion of \..\ works in the call to R as an external program because the batch file can resolve it's own commands.
The variable of SourceFile however doesn't work because the SourceFile variable hasn't expanded \..\, it has just included it as part of the string and R can't process \..\
You can use the for replaceable parameters to resolve to the real path
for %%a in ("..\..\HCF_v9.R") do set "RScript=%%~fa"
#MC ND has provided the batch file approach; an R-centric approach would be to pass the current directory to R, and modify it there.
; batch file
Set RPath=C:\Program Files\R\R-3.1.0\bin\Rscript.exe
SET RScript=%CD%\..\..\HCF_v9.R
"%RPath%" "%RScript%" %CD%
# in R
srcpath <- commandArgs(TRUE)[1]
srcpath <- normalizePath(file.path(srcpath, "../Source"))
Using QMake, I read some boiler plate code, make modifications and write the modified code to a file.
However, I get very strange results. I have simplified the problem down to the following:
# Open boiler plate file
interfaceBoilerPlateCode = $$cat($$boilerPlateFile, blob)
# Make sure we read the right content
message("Content read: $$interfaceBoilerPlateCode")
# Write the read text into a file
output = $$system(echo $$interfaceBoilerPlateCode >> $$targetFile) # Doesnt work
output = $$system(echo "Howde" >> $$targetFile) # This works
The file being read is a plain text file containing only the string "Howde".
The contents of the file get read correctly.
However, when I try and write the contents of the file to another target file, I get no output (literally: no errors/warnings but no new file generated). However, if I use echo with just a string defined in the code itself (as in the last line of snippet above), a new file gets generated with the string "Howde" inside it.
What is going on? What am I doing wrong that the penultimate line does not generate a new file?
Use write_file. Instead of:
$$system(echo $$content >> $$file_path)
use
write_file($$file_path, $$content)
I have a 5GB 1 liner file with JSON data and each line starts from this pattern "{"created". I need to be able to use Unix commands on my Mac to convert this monster of a 1 liner into as many lines as it deserves. Any commands?
ASCII English text, with very long lines, with no line terminators
If you have enough memory you can open the file once with the TextWrangler application (the free BBEdit cousin) and use regular search/replace on the whole file. Use \r in replace to add a return. Will be very slow at opening the file, may even hang if low on memory, but in the end it may probably work. No scripting, no commands,.. etc.. I did this with big SQL files and sometimes it did the job.
You have to replace your line-start string with the same string with \n or \r or \r\n in front of it.
Unclear how it can be a “one liner” file but then each line starts with "{"created", but perhaps python -mjson.tool can help you get started:
cat your_source_file.json | python -mjson.tool > nicely_formatted_file.json
Piping raw JSON through ``python -mjson.tool` will cleanly format the JSON to be more human readable. More info here.
OS X ships with both flex and bison, you can use those to write a parser for your data.
You can use PHP as a shell command (if PHP is installed), just save a text file with name "myscript" and appropriate code (I cannot test code now, but the idea is as follows)
UNTESTED CODE
#!/usr/bin/php
<?php
$REPLACE_STRING='{"created'; // anything you like
// open input file with fopen() in read mode
$inFp=fopen('big_in_file.txt', "r");
// open output file with fopen() in write mode
$outFp=fopen('big_out_file.txt', "w+");
// while not end of file
while (!feof($inFp)) {
// read file chunks here with fread() in variable $chunk
$chunk = fread($inFp, 8192);
// do a $chunk=str_replace($REPLACE_STRING,"\r".$REPLACE_STRING; // to add returns
// (or use \r\n for windows end of lines)
$chunk=str_replace($REPLACE_STRING,"\r".$REPLACE_STRING,$chunk);
// problem: if chunk contains half the string at the end
// easily solved if $REPLACE_STRING is a one char like '{'
// otherwise test for fist char { in the end of $chunk
// remove final part and save it in a var for nest iteration
// write $chunk to output file
fwrite($outFp, $chunk);
// End while
}
?>
After you save it you must make it executable whith sudo chmod a+x ./myscript
and then launch it as ./myscript in terminal
After this, the myscript file is a full unix command
How do I find/use op:except with the multiple xml files?
I've gotten the nodes from file 1 and file 2, and in the xquery expression I'm tring to find the op:except of those two. When I use op:except, I end up getting an empty set.
XML File 1:
<a>txt</a>
<a>txt2</a>
<a>txt3</a>
XML File 2:
<a>txt2</a>
<a>txt4</a>
<a>txt3</a>
I want output from op:($nodesfromfile1, $nodesfromfile2) to be:
<a>txt</a>
It effectively comes down to the single line behind the return in the following code. You could put that in a function if you like, but it is already very dense, so maybe not worth it..
let $file1 := (
<a>txt</a>,
<a>txt2</a>,
<a>txt3</a>
)
let $file2 := (
<a>txt2</a>,
<a>txt4</a>,
<a>txt3</a>
)
return
$file1[not(. = $file2)]
Note, you also have the 'except' keyword ($file1 except $file2), but that works on node identity which won't work if the nodes comes from different files.
By the way, above code uses string-equality for comparison. If you would prefer to do a comparison on full node-structure, you could also use the deep-equal() function.
HTH!