I'd like to capture the result of a shell command in julia
in ipython this works:
[1]: x = ! date
In [2]: x
Out[2]: ['Thu Dec 14 15:34:06 PST 2017']
I've already tried
julia> x = ;date
ERROR: syntax: unexpected ;
julia> x = readstring(`date`)
"Thu Dec 14 21:33:48 PST 2017\n"
julia> x
"Thu Dec 14 21:33:48 PST 2017\n"
is readstring the best way to accomplish this? or is there a ; way, too?
Pressing ; works only in Julia REPL moving it to >shell mode and only if you are at the beginning of the line.
In programs ; is expression terminator. Therefore x = ;date is an error because x = is a malformed expression.
Starting from Julia 0.7 you will not be able to use readstring but rather
read(`date`, String)
If you do not want a string but an array of raw bytes simply write
read(`date`)
Related
Processing ASCII characters beyond the range 1-127 can easily crash Julia.
mystring = "A-Za-zÀ-ÿŽž"
for i in 1:length(mystring)
print(i,":::")
print(Int(mystring[i]),"::" )
println( mystring[i] )
end
gives me
1:::65::A
2:::45::-
3:::90::Z
4:::97::a
5:::45::-
6:::122::z
7:::192::À
8:::ERROR: LoadError: StringIndexError("A-Za-zÀ-ÿŽž", 8)
Stacktrace:
[1] string_index_err(::String, ::Int64) at .\strings\string.jl:12
[2] getindex_continued(::String, ::Int64, ::UInt32) at .\strings\string.jl:220
[3] getindex(::String, ::Int64) at .\strings\string.jl:213
[4] top-level scope at R:\_LV\STZ\Web_admin\Languages\Action\Returning\chars.jl:5
[5] include(::String) at .\client.jl:457
[6] top-level scope at REPL[18]:1
It crashes after outputting the first character outside the normal range, rather than during that output, which is mentioned in the answer to String Index Error (Julia)
If declaring the values in Julia one should declare them as Unicode, but I have these characters in my input.
The manual says that Julia looks at the locale, but is there an "everywhere" locale?
Is there some way to handle input and output of these characters in Julia?
I am working on Windows10, but I can switch to Linux if that works better for this.
Use eachindex to get a list of valid indices in your string:
julia> mystring = "A-Za-zÀ-ÿŽž"
"A-Za-zÀ-ÿŽž"
julia> for i in eachindex(mystring)
print(i, ":::")
print(Int(mystring[i]), "::")
println(mystring[i])
end
1:::65::A
2:::45::-
3:::90::Z
4:::97::a
5:::45::-
6:::122::z
7:::192::À
9:::45::-
10:::255::ÿ
12:::381::Ž
14:::382::ž
Your issue is related to the fact that Julia uses byte-indexing of strings, as is explained in the Julia Manual.
For example character À takes two bytes, therefore, since its location is 7 the next index is 9 not 8.
In UTF-8 encoding which is used by default by Julia only ASCII characters take one byte, all other characters take 2, 3 or 4 bytes, see https://en.wikipedia.org/wiki/UTF-8#Encoding.
For example for À you get two bytes:
julia> codeunits("À")
2-element Base.CodeUnits{UInt8, String}:
0xc3
0x80
I have also written a post at https://bkamins.github.io/julialang/2020/08/13/strings.html that tries to explain how byte-indexing vs character-indexing works in Julia.
If you have additional questions please comment.
String indices in Julia refer to code units (= bytes for UTF-8), the fixed-width building blocks that are used to encode arbitrary characters (code points). This means that not every index into a String is necessarily a valid index for a character. If you index into a string at such an invalid byte index, an error is thrown.
You can use enumerate to get the value and the number of iteration.
mystring = "A-Za-zÀ-ÿŽž"
for (i, x) in enumerate(mystring)
print(i,":::")
print(Int(x),"::")
println(x)
end
#1:::65::A
#2:::45::-
#3:::90::Z
#4:::97::a
#5:::45::-
#6:::122::z
#7:::192::À
#8:::45::-
#9:::255::ÿ
#10:::381::Ž
#11:::382::ž
In case you need the value and index of the string in bytes you can use pairs.
for (i, x) in pairs(mystring)
print(i,":::")
print(Int(x),"::")
println(x)
end
#1:::65::A
#2:::45::-
#3:::90::Z
#4:::97::a
#5:::45::-
#6:::122::z
#7:::192::À
#9:::45::-
#10:::255::ÿ
#12:::381::Ž
#14:::382::ž
In preparation for de-minimising my MCVE for what I want to do, which involves advancing the string position not just in a for-all loop, I used the information in the post written by Bogumił Kamiński, to come up with this:
mystring = "A-Za-zÀ-ÿŽž"
for i in 1:length(mystring)
print(i,":::")
mychar = mystring[nextind(mystring, 0, i)]
print(Int(mychar), "::")
println( mychar )
end
syntax: invalid type signature around C:\Users\admin.julia\pluto_notebooks\Revolutionary blueprint.jl#==#396279cf-6e6b-468e-8aa6-772e2279058d:1
enter image description here
enter image description here
You cannot use a dash as part of a variable name, as it is the subtraction operator:
julia> x = 5
5
julia> y = 2
2
julia> x-y
3
So there's no way to distinguish x minus y from a variable called "x-y". Use underscores instead:
julia> struct My_Struct end
julia> struct My-Struct end
ERROR: syntax: invalid type signature around REPL[47]:1
Stacktrace:
[1] top-level scope
# REPL[47]:1
If I understand correctly, in Julia m = [[1,2],[3,4]] is a two-dimensional vector with each vector of two elements. And this fact won't change after the operation of sympify
ms = sympy.sympify(m)
size(ms)
The output will still be (2,), as expected.
However, if I create a file test.txt with just one line [[1,2],[3,4]] and run
using SymPy
open("test.txt") do io
while ! eof(io)
m = readline(io)
println(m)
ms = sympy.sympify(m)
println(ms)
end
end
The output will be
[[1,2],[3,4]]
Sym[1 2; 3 4]
Namely, now ms suddenly changes to a two-by-two matrix! I really cannot understand this (since the dimension of m after readline stays to be (2,) as before). Could someone explain this for me?
As a workaround, you can define your own parsing function to read lines from a file into the vector of vectors:
function myparse(T::Type, s::String)
[[parse(T, c) for c in element if isnumeric(c)] for element in split(s, "],[")]
end
Then, you can simply do:
using SymPy
m = open("test.txt") do io
myparse(Int, readline(io))
end
println(typeof(m)) # Vector{Vector{Int64}} (alias for Array{Array{Int64, 1}, 1})
Then,
julia> ms = sympy.sympify(m)
2-element Vector{Sym}:
[1 2]
[3 4]
I already have output some content in the REPL. Is there any function to print all this content to a file?
If those outputs have already been printed in the REPL, I guess the only way to save them to a file is copy-pasting manually. But if you would like to save the REPL output history for future use, one way is to overload display:
shell> touch repl_history.txt
julia> using REPL
julia> function REPL.display(d::REPL.REPLDisplay, mime::MIME"text/plain", x)
io = REPL.outstream(d.repl)
get(io, :color, false) && write(io, REPL.answer_color(d.repl))
if isdefined(d.repl, :options) && isdefined(d.repl.options, :iocontext)
# this can override the :limit property set initially
io = foldl(IOContext, d.repl.options.iocontext,
init=IOContext(io, :limit => true, :module => Main))
end
show(io, mime, x)
println(io)
open("repl_history.txt", "a") do f
show(f, mime, x)
println(f)
end
nothing
end
then, let's print something random in the REPL:
julia> rand(10)
10-element Array{Float64,1}:
0.37537591915616497
0.9478991508737484
0.32628512501942475
0.8888960925262224
0.9967927432272801
0.4910769590205608
0.7624517049991089
0.26310423494973545
0.5117608425961135
0.0762255311602309
help?> gcd
search: gcd gcdx significand
gcd(x,y)
Greatest common (positive) divisor (or zero if x and y are both zero).
Examples
≡≡≡≡≡≡≡≡≡≡
julia> gcd(6,9)
3
julia> gcd(6,-9)
3
And here is what the file content looks like:
shell> cat repl_history.txt
10-element Array{Float64,1}:
0.37537591915616497
0.9478991508737484
0.32628512501942475
0.8888960925262224
0.9967927432272801
0.4910769590205608
0.7624517049991089
0.26310423494973545
0.5117608425961135
0.0762255311602309
gcd(x,y)
Greatest common (positive) divisor (or zero if x and y are both zero).
Examples
≡≡≡≡≡≡≡≡≡≡
julia> gcd(6,9)
3
julia> gcd(6,-9)
3
If there is no need to work with REPL interactively, simply use julia script.jl > output.txt might also do the trick.
What you've printed isn't saved anywhere, so no, there's no way to do that. It may be that there's something easy to do but without more details it's impossible to way really.
If you want to save variables to file, you can you JLD2 package. Then you can save each variable as below:
using JLD2, FileIO
hello = "world"
foo = :bar
#save "example.jld2" hello foo
The documentation for redirect_stdouton version 1.1.0, that I am currently using, does not seem to give an example of how to use that function. Maybe I missed it?
I want to capture the output of println and get it back as a string.
Here is an example:
julia> VERSION
v"1.1.0"
julia> (rd, wr) = redirect_stdout();
julia> println("This is a test.")
julia> # Get back the string "This is a test."
julia> # s = do_something_with_rd(rd)
julia> # s == "This is a test."
julia> # true
Any suggestions?
Edit
Based on the accepted answer below, here is a complete solution to my question:
julia> original_stdout = stdout;
julia> (rd, wr) = redirect_stdout();
julia> println("This is a test.")
julia> s = readline(rd)
"This is a test."
julia> s == "This is a test."
true
julia> redirect_stdout(original_stdout);
julia> println("Test of orig. stdout.")
Test of orig. stdout.
Edit 2: A More Complete Example
Here is an example of testing a variety of print and println function outputs using redirection of stdout. Thanks to #Bogumił Kamiński for his answer and edit that made this more clear to me:
using Test
# Test redirect_stdout.
#testset "Example tests using redirect_stdout" begin
original_stdout = stdout;
(read_pipe, write_pipe) = redirect_stdout();
print("Using print function.")
println("Using println function.")
println("Second use of println function.")
println("Line 1.\nLine 2.\nLine 3.\nEND")
println("""
This is new line 1.
This is new line 2. Next a Char = """)
print('A')
redirect_stdout(original_stdout);
close(write_pipe)
#test readline(read_pipe) == "Using print function.Using println function."
#test readline(read_pipe) == "Second use of println function."
#test read(read_pipe, String) == "Line 1.\nLine 2.\nLine 3.\nEND\n" *
"This is new line 1.\nThis is new line 2. Next a Char = \nA"
end
# Suppress unnecessary output when this file.
return nothing
Here is the output:
julia> include("test_redirect_stdout.jl")
Test Summary: | Pass Total
Example tests using redirect_stdout | 3 3
Just run readline on rd (or any other reading operation).
You have just to be careful that read operations on rd are blocking, i.e. the terminal will seem to hang when the operation cannot be completed. One solution is to use #async for this. For instance:
julia> (rd, wr) = redirect_stdout();
julia> #async global x = readline(rd) # if we did not put #async here the terminal would be blocked
Task (runnable) #0x0000000004e46e10
julia> x # x is yet undefined as readline is waiting for an input
ERROR: UndefVarError: x not defined
julia> println("something") # we feed data to stdout
julia> x # and readline has finished its work and bound the value to variable x
"something"
Of course if you know exactly that the data you want to read in is there just run readline or some other function and all will work without #async.
EDIT
Given the comments from SalchiPapa I think it is also add this pattern of possible usage as it is simplest to think of IMO:
original_stdout = stdout
(rd, wr) = redirect_stdout();
println("This is a test 1.")
println("This is a test 2.")
println("This is a test 3.")
redirect_stdout(original_stdout)
# you can still write to wr
println(wr, "This is a test 4.")
# you have to close it to make the read non-blocking
close(wr)
# the pipe is redirected to original stdout and wr is closed so this is non-blocking
s = read(rd, String)
You could use sprint AKA "string print":
https://docs.julialang.org/en/v1/base/io-network/index.html#Base.sprint
julia> sprint(println, "This is a test")
"This is a test\n"