I have a pickle file that is written by using python (By using cPickle). I need to use it inside julia. I guess I can use PyCall for that purpose.Here is what I've done so far:
julia> using PyCall
julia> #pyimport cPickle as pickle
julia> f=open("mypicklefile.picle","r")
julia> PyTextIO(f)[:seek](0) #if I don't do this, I got error.
julia> trn,tst= pickle.load(PyTextIO(f))
Here,I have two questions. As far as I know, unlike Julia, python(numpy) is rowmajor. So, what should I do to read the content of file in a correct way? I guess somehow I have to tell the PyCall it is written in rowmajor order.
My second question is about performance. The pickle file is about 4.5GB. When I read it inside python, it is completed in a minute whereas in julia it takes at least 15 minutes. Am I doing something wrong ? Since the file is not created by me, I cannot use another format such as hdf5 or jld.
Related
I have a problem with using packages in Julia. It has worked before, and I'm not really sure why this has changed or how to troubleshoot.
I have a folder
/my_path/julia/packages
with Julia packages. For example, there is a folder
/my_path/julia/packages/FFTW/
with the FFTW package.
Further, I have changed the depot path to point at this directory by assigning JULIA_DEPOT_PATH before starting julia, so that
Base.DEPOT_PATH = ["/my_path/julia/"]
However, if I run
julia> using FFTW
I get the following error message:
ERROR: ArgumentError: Package FFTW not found in current path:
- Run `import Pkg; Pkg.add("FFTW")` to install the FFTW package.
Any idea how I can troubleshoot or fix this?
Manipulating Base.DEPOT_PATH does not seem like a good idea.
The code proposed by #cmc will does not work (at least on Julia 1.3.1):
julia> Base.DEPOT_PATH = ["/some/path"]
ERROR: cannot assign variables in other modules
There is a workaround:
Base.DEPOT_PATH[1] = "/some/path"
However, the correct way is to assign the JULIA_DEPOT_PATH system variable before starting Julia, Windows:
set JULIA_DEPOT_PATH=c:\some\path
or
set JULIA_DEPOT_PATH=c:\some\path1;c:\some\path2
Linux/OSX:
export JULIA_DEPOT_PATH=/some/path
or
export JULIA_DEPOT_PATH=/some/path1:/some/path2
Unless you have a specific reason to do so (and if this is the case I'd be interested to hear it!), you don't need to fiddle with the DEPOT_PATH or LOAD_PATH variables: using Julia's package manager should be enough to cover your needs most of the time.
In this specific instance, have you tried to do what the error message suggests?
julia> import Pkg
julia> Pkg.add("FFTW")
LOAD_PATH, not DEPOT_PATH, will modify code loading.
You want to do something like push!(LOAD_PATH, /my_path/julia/packages).
I will echo #ffevotte and strongly suggest to not modify LOAD_PATH unless necessary. The benefits of organizing dependencies into Pkg environments far outweigh the small overhead of declaring them explicitly through Pkg.add.
I am new in Julia and saying good bye to MATLAB, The issue here is that I am trying to set a script with all my functions where I can go later and perform some operations.
I have tried to do so in a MATLAB style, I also tried to read the documentation, but neither help that much.
So the question here is: how do I set a .jl file in another folder so that I can call this file later on in Julia?.
Hint: I am using Atom as an editor
KR
Rubén
You can include any .jl file in a Julia session:
include("/path/to/my/jl/file/functions.jl")
Afterwards, all the function definitions etc. in functions.jl are available.
I am trying to load a matlab file with the R.matlab package. The problem is that it keeps loading indefinitely (e.g. table <- readMat("~/desktop/hg18_with_miR_20080407.mat"). I have a genome file from the Broad Institute (hg18_with_miR_20080407.mat).
You can find it at:
http://genepattern.broadinstitute.org/ftp/distribution/genepattern/dev_archive/GISTIC/broad.mit.edu:cancer.software.genepattern.module.analysis/00125/1.1/
I was wondering: has anyone tried the package and have similar issues?
(hopefully helpful, but not really an answer, though there was too much formatting for a comment)
I think you may need to get a friend with matlab access to save the file into a more reasonable format or use python for data processing. It "hangs" for me as well (OS X 10.9, R 3.1.1). The following works in python:
import scipy.io
mat = scipy.io.loadmat("hg18_with_miR_20080407.mat")
(you can see and work with the rg and cyto' crufty numpy arrays, but they can't be converted to JSON withjson.dumpsand evenjsonpickle.encodecoughs up a lung-full of errors (i.e. you won't be able to userPython` to get access to the object which was the original workaround I was looking for), so no serialization to a file either (and, I have to believe the resultant JSON would have been ugly to deal with).
Your options are to:
get a friend to convert it (as suggested previous)
make CSV files out of the numpy arrays in python
use matlab
I would like to display to the user a human-readable file size in Julia.
Python, for instance, has humanize.naturalsize():
>>> humanize.naturalsize(1000000)
'1.0 MB'
>>> humanize.naturalsize(1000000, binary=True)
'976.6 KiB'
How can I achieve this in Julia?
I don't believe there is a package to do this, but the Python code is very short and should translate nicely to Julia.
EDIT: In fact, I decided to make a package out of it: Humanize.jl, which you can install with Pkg.add("Humanize")
I am getting the weirdest error in using the write()function:
julia> write(0.1)
�8
julia>
������?
These are just the commands I'm writing in the console, as you can see. What could be going wrong?
Note: I'm using Julia Studio 0.4.4
From the documentation:
write(stream, x): Write the canonical binary representation of a value to the given stream.
I suspect that you want to use print rather than write.