Error in running a Python code from R with the package rPithon - r

I would like to run this Python code from R:
>>> import nlmpy
>>> nlm = nlmpy.mpd(nRow=50, nCol=50, h=0.75)
>>> nlmpy.exportASCIIGrid("raster.asc", nlm)
Nlmpy is a Python package to build neutral landscape models. The example comes from the website
To run this Python code from R, I 'm trying to use the package rPithon. However, I obtain this error message:
if (pithon.available())
{
nRow <- 50
nCol <- 50
h <- 0.75
# this file contains the definition of function concat
pithon.load("C:/Users/Anaconda2/Lib/site-packages/nlmpy/nlmpy.py")
pithon.call( "mpd", nRow, nCol, h)
} else {
print("Unable to execute python")
}
Error in pithon.get("_r_call_return", instance.name = instname) :
Couldn't retrieve variable: Traceback (most recent call last):
File "C:/Users/Documents/R/win-library/3.3/rPithon/pythonwrapperscript.py", line 110, in <module>
reallyReallyLongAndUnnecessaryPrefix.data = json.dumps([eval(reallyReallyLongAndUnnecessaryPrefix.argData)])
File "C:\Users\ANACON~1\lib\json\__init__.py", line 244, in dumps
return _default_encoder.encode(obj)
File "C:\Users\ANACON~1\lib\json\encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Users\ANACON~1\lib\json\encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "C:\Users\ANACON~1\lib\json\encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: array([[ 0.36534654, 0.31962481, 0.44229946, ..., 0.11513079,
0.07156331, 0.00286971], [ 0.41534291, 0.41333479, 0.48118995, ..., 0.19203674,
0.04192771, 0.03679473], [ 0.5188
Is this error caused by a syntax issue in my code ? I work with the Anaconda 4.2.0 platform for Windows which uses the Python 2.7 version.

I haven't used the nlmpy package hence, I am not sure what would be your expected output. However, this code successfully communicates between R and Python.
There are two files,
nlmpyInR.R
command ="python"
path2script="path_to_your_pythoncode/nlmpyInPython.py"
nRow <-50
nCol <-50
h <- 0.75
# Build up args in a vector
args = c(nRow, nCol, h)
# Add path to script as first arg
allArgs = c(path2script, args)
Routput = system2(command, args=allArgs, stdout=TRUE)
#The command would be python nlmpyInPython.py 50 50 0.75
print(paste("The Output is:\n", Routput))
nlmpyInPython.py
import sys
import nlmpy
#Getting the arguments from the command line call
nRow = sys.argv[1]
nCol = sys.argv[2]
h = sys.argv[3]
nlm = nlmpy.mpd(nRow, nCol, h)
pyhtonOutput = nlmpy.exportASCIIGrid("raster.asc", nlm)
#Whatever you print will get stored in the R's output variable.
print pyhtonOutput

The cause of the error that you're getting is hinted at by the
"is not JSON serializable" line. Your R code calls the mpd
function with certain arguments, and that function itself will
execute correctly. The rPithon library will then try to send the
return value of the function back to R, and to do this it will try
to create a JSON object
that describes the return value.
This works well for integers, floating point values, arrays, etc,
but not every kind of Python object can be converted to such a
JSON representation. And because rPithon can't convert the return value
of mpd this way, an error is generated.
You can still use rPithon to call the mpd function though. The following
code creates a new Python function that performs two steps: first
it calls the mpd function with the specified parameters, and then it
exports the result to a file, of which the filename is also an argument.
Using rPithon, the new function is then called from R. Because myFunction doesn't return anything, representing the return value in JSON format will not be a problem.
library("rPithon")
pythonCode = paste("import nlmpy.nlmpy as nlmpy",
"",
"def myFunction(nRow, nCol, h, fileName):",
" nlm = nlmpy.mpd(nRow, nCol, h)",
" nlmpy.exportASCIIGrid(fileName, nlm)",
sep = "\n")
pithon.exec(pythonCode)
nRow <- 50
nCol <- 50
h <- 0.75
pithon.call("myFunction", nRow, nCol, h, "outputraster.asc")
Here, the Python code defined as an R string, and executed using
pithon.exec. You could also put that Python code in a separate file
and use pithon.load to process the code so that the myFunction
function is known.

Related

NetCDF - What is is 'phony_dim_0', 'phony_dim_1', 'phony_dim_2'?

I am very new to using NetCDF files and I am at the exploratory stage trying to understand what this file could do. I am using 'netCDF4' python library. I am trying to find what does 'phony_dim_0', 'phony_dim_1', 'phony_dim_2' mean and contains? I am thinking it could be 'lat','lon', and/or 'time'?
Loading nc file
ds = nc.Dataset('my_file.nc')
type(ds)
>>> netCDF4._netCDF4.Dataset
Printing Keys
print(ds.dimensions.keys())
>>> dict_keys(['phony_dim_0', 'phony_dim_1', 'phony_dim_2'])
Extract what is in this key?
ds.dimensions['phony_dim_0']
>>> <class 'netCDF4._netCDF4.Dimension'>: name = 'phony_dim_0', size = 1179
Error:
for c in ds.dimensions['phony_dim_0']:
print(c) # Want to see what is in this? # Errors: TypeError: 'netCDF4._netCDF4.Dimension' object is not iterable

Evaluating strings in Julia_Eval for diffeqr solver

I am trying to evaluate strings within a for loop within an R script using JuliaCall::julia_eval. While I was able to accomplish this in R using the deSolve package, I am running into issues when converting the code to one that is compatible with Julia. The base code for the correctly functioning R deSolve code is shown below.
library(deSolve)
library(dplyr)
Combine <- c(" - 1*0.4545*(H2O2^1) - 1*27000000*(`$OH`^1)*(H2O2^1)", " - 1*3100000000*(`1,4-dioxane`^1)*(`$OH`^1)",
" - 1*33000*(TOC^1)*(`$OH`^1)", "2*0.4545*(H2O2^1) - 1*3100000000*(`1,4-dioxane`^1)*(`$OH`^1) - 1*33000*(TOC^1)*(`$OH`^1) - 1*27000000*(`$OH`^1)*(H2O2^1) - 1*8500000*(`$OH`^1)*(`HCO3-`^1) - 1*390000000*(`$OH`^1)*(`CO3 2-`^1)",
" - 1*8500000*(`$OH`^1)*(`HCO3-`^1)", " - 1*390000000*(`$OH`^1)*(`CO3 2-`^1)"
)
time <- seq(from=0, to=0.01, by = 1E-4)
State <- c(H2O2 = 0.000294117647058824, `1,4-dioxane` = 0.00000113494,
TOC = 0, `$OH` = 0, `HCO3-` = 0.003766104, `CO3 2-` = 0.0000167638711956647)
ODEcreater2 <- function(t, state, parameters){
with(as.list(c(state)),{
for (i in 1:6) { #
dY[i] <- eval(parse(text=Combine[i]))}
return(list(dY))
} )}
out1<- ode(y = state, times = time, func = ODEcreater2, parms = NULL)
I am trying to use replicate the code and run it in Julia to improve the speed of the ODE solver by using diffeqr vs. deSolve. Unfortunately, I am running into evaluating the string/expression within a for loop in julia_call.
library(diffeqr)
diffeqr::diffeq_setup()
library(JuliaCall)
julia <- julia_setup()
ODEcreater <- JuliaCall::julia_eval("
function (dY,t,state)
for i in 1:6
dY[i] = eval(Meta.parse(:Combine[i]))
end
end")
tspan <- list(1E-6, 1E-3)
sol = diffeqr::ode.solve(ODEcreater,state,tspan, abstol=1e-8, reltol=1e-8)
Does anyone have any insight into the best way to evaluate the strings within the for loop? I have been investigating metaexpressions on the JuliaLang website but am still lost.
As mentioned in the duplicate question https://stackoverflow.com/a/58766919/1544203 , building the string and then doing
sprintf("function f(du,u,p,t)\n%s\nend", paste(Combine, collapse="\n"))
from the R side builds a single string which matches the format that works. This is also optimal since it excludes any extra function calls from the generated function.
from julia docs:
parse(str; raise=true, depwarn=true)
Parse the expression string greedily, returning a single expression. An error is thrown if there are additional
characters after the first expression. If raise is true (default), syntax errors will raise an error; otherwise, parse
will return an expression that will raise an error upon evaluation. If depwarn is false, deprecation warnings will be
suppressed.
julia> Meta.parse("x = 3")
:(x = 3)
so, Meta.parse accepts a string and returns an expression. this should evaluate correctly:
eval(Meta.parse(Combine[i]))
one problem i see is the use of non-valid julia variable names, like $OH

Unexpected symbol error in R that doesn't match my code

I am coding in R-studio and have a function called saveResults(). It takes:
sce - a Single Cell Experiment object.
opt - a list with five things
clusterLabels - simple dataframe with two columns
The important thing is that I receive an error stating:
Error: unexpected symbol in:
"saveResults(sce = sce, opt = opt, clusteInputs()
zhengMix"
which doesn't agree at all with the parameters I pass into the function. You can see this on the last line of the code block below: I pass in proper parameters, but I receive an error that says I have passed in clusteInputs(), and zhengMix instead of clusterLabels. I don't have a function called clusteInputs(), and zhengMix was several lines above.
# Save the clustering data
InstallAndLoadPackagesForSC3Clustering()
opt <- GetOptionInputs()
zhengMix <- FetchzhengMix(opt)
sce <- CreateSingleCellExperiment(zhengMix)
clusterLabels <- getClusterLabels(sce)
opt <- createNewDirectoriesToSaveData(opt)
saveResults <- function(sce, opt, clusterLabels){
print("Beginning process of saving results...")
maxClusters = ncol(clusterLabels)/2+1
for (n in 2:maxClusters){
savePCAasPDF(sce, opt, numOfClusters = n, clusterLabels)
saveClusterLabelsAsRDS(clusterLabels, numOfClusters = n, opt)
}
saveSilhouetteScores(sce, opt)
print("Done.")
}
saveResults(sce = sce, opt = opt, clusterLabels = clusterLabels)
Does anyone have an idea what is going on? I'm pretty stuck on this.
This isn't the best solution, but I fixed my own problem by removing the code out of the function and running it there caused no issues.

calling a R function from python code with passing arguments

rtest = function(input ,output) {
a <- input
b <- output
outpath <- a+b
print(a+b)
return(outpath)
}
I have just return this R code for as a function for getting sum of two numbers. I tried to run this function from my python code using subprocess by passing 2 numbers as arguments. But it does not return sum value as return output. Do you know any method for implement this in python3 by passing function arguments.
my python code using subprocess is:
args=['3','10'] # (i tried to pass aruments like this)
command="Rscript"
path2script = '/...path/rtest.R'
cmd = [command, path2script] +args
x = subprocess.check_output(cmd, universal_newlines=True)
print(x)
but x return ' ' null value
This could be easily done by rpy2 library in python.
import rpy2.robjects as ro
path="specify/path to/ R file"
def function1(input,output):
r=ro.r
r.source(path+"rtest.R")
p=r.rtest(input,output)
return p
a=function1(12,12) # calling the function with passing arguments
Thanks.

Porting to Python3: PyPDF2 mergePage() gives TypeError

I'm using Python 3.4.2 and PyPDF2 1.24 (also using reportlab 3.1.44 in case that helps) on windows 7.
I recently upgraded from Python 2.7 to 3.4, and am in the process of porting my code. This code is used to create a blank pdf page with links embedded in it (using reportlab) and merge it (using PyPDF2) with an existing pdf page. I had an issue with reportlab in that saving the canvas used StringIO which needed to be changed to BytesIO, but after doing that I ran into this error:
Traceback (most recent call last):
File "C:\cms_software\pdf_replica\builder.py", line 401, in merge_pdf_files
input_page.mergePage(link_page)
File "C:\Python34\lib\site-packages\PyPDF2\pdf.py", line 2013, in mergePage
self.mergePage(page2)
File "C:\Python34\lib\site-packages\PyPDF2\pdf.py", line 2059, in mergePage
page2Content = PageObject._pushPopGS(page2Content, self.pdf)
File "C:\Python34\lib\site-packages\PyPDF2\pdf.py", line 1973, in _pushPopGS
stream = ContentStream(contents, pdf)
File "C:\Python34\lib\site-packages\PyPDF2\pdf.py", line 2446, in __init
stream = BytesIO(b_(stream.getData()))
File "C:\Python34\lib\site-packages\PyPDF2\generic.py", line 826, in getData
decoded._data = filters.decodeStreamData(self)
File "C:\Python34\lib\site-packages\PyPDF2\filters.py", line 326, in decodeStreamData
data = ASCII85Decode.decode(data)
File "C:\Python34\lib\site-packages\PyPDF2\filters.py", line 264, in decode
data = [y for y in data if not (y in ' \n\r\t')]
File "C:\Python34\lib\site-packages\PyPDF2\filters.py", line 264, in
data = [y for y in data if not (y in ' \n\r\t')]
TypeError: 'in <string>' requires string as left operand, not int
Here is the line and the line above where the traceback mentions:
link_page = self.make_pdf_link_page(pdf, size, margin, scale_factor, debug_article_links)
if link_page != None:
input_page.mergePage(link_page)
Here are the relevant parts of that make_pdf_link_page function:
packet = io.BytesIO()
can = canvas.Canvas(packet, pagesize=(size['width'], size['height']))
....# left out code here is just reportlab specifics for size and url stuff
can.linkURL(url, r1, thickness=1, color=colors.green)
can.rect(x1, y1, width, height, stroke=1, fill=0)
# create a new PDF with Reportlab that has the url link embedded
can.save()
packet.seek(0)
try:
new_pdf = PdfFileReader(packet)
except Exception as e:
logger.exception('e')
return None
return new_pdf.getPage(0)
I'm assuming it's a problem with using BytesIO, but I can't create the page with reportlab with StringIO. This is a critical feature that used to work perfectly with Python 2.7, so I'd appreciate any kind of feedback on this. Thanks!
UPDATE:
I've also tried changing from using BytesIO to just writing to a temp file, then merging. Unfortunately I got the same error.
Here is tempfile version:
import tempfile
temp_dir = tempfile.gettempdir()
temp_path = os.path.join(temp_dir, "tmp.pdf")
can = canvas.Canvas(temp_path, pagesize=(size['width'], size['height']))
....
can.showPage()
can.save()
try:
new_pdf = PdfFileReader(temp_path)
except Exception as e:
logger.exception('e')
return None
return new_pdf.getPage(0)
UPDATE:
I found an interesting bit of information on this. It seems if I comment out the can.rect and can.linkURL calls it will merge. So drawing anything on a page, then trying to merge it with my existing pdf is causing the error.
After digging in to PyPDF2 library code, I was able to find my own answer. For python 3 users, old libraries can be tricky. Even if they say they support python 3, they don't necessarily test everything. In this case, the problem was with the class ASCII85Decode in filters.py in PyPDF2. For python 3, this class needs to return bytes. I borrowed the code for this same type of function from pdfminer3k, which is a port for python 3 of pdfminer. If you exchange the ASCII85Decode() class for this code, it will work:
import struct
class ASCII85Decode(object):
def decode(data, decodeParms=None):
if isinstance(data, str):
data = data.encode('ascii')
n = b = 0
out = bytearray()
for c in data:
if ord('!') <= c and c <= ord('u'):
n += 1
b = b*85+(c-33)
if n == 5:
out += struct.pack(b'>L',b)
n = b = 0
elif c == ord('z'):
assert n == 0
out += b'\0\0\0\0'
elif c == ord('~'):
if n:
for _ in range(5-n):
b = b*85+84
out += struct.pack(b'>L',b)[:n-1]
break
return bytes(out)

Resources