What is the right way to call an external command and collect its output in OCaml?
In Python, I can do something like this:
os.popen('cmd').read()
How I can get all of an external program's output in OCaml? Or, better, OCaml with Lwt?
Thanks.
You want Unix.open_process_in, which is described on page 388 of the OCaml system manual, version 3.10.
For Lwt,
val pread : ?env:string array -> command -> string Lwt.t
seems to be a good contender. Documentation here: http://ocsigen.org/docu/1.3.0/Lwt_process.html
let process_output_to_list2 = fun command ->
let chan = Unix.open_process_in command in
let res = ref ([] : string list) in
let rec process_otl_aux () =
let e = input_line chan in
res := e::!res;
process_otl_aux() in
try process_otl_aux ()
with End_of_file ->
let stat = Unix.close_process_in chan in (List.rev !res,stat)
let cmd_to_list command =
let (l,_) = process_output_to_list2 command in l
There are lots of examples on PLEAC.
You can use the third party library Rashell which uses Lwt to define some high-level primitives to read output from processes. These primitives, defined in the module Rashell_Command, are:
exec_utility to read the output of a process as a string;
exec_test to only read the exit status of a process;
exec_query to read the output of a process line by line as a string Lwt_stream.t
exec_filter to use an external program as a string Lwt_stream.t -> string Lwt_stream.t transformation.
The command function is used to create command contexts on which the previous primitives can be applied, it has the signature:
val command : ?workdir:string -> ?env:string array -> string * (string array) -> t
(** [command (program, argv)] prepare a command description with the
given [program] and argument vector [argv]. *)
So for instance
Rashell_Command.(exec_utility ~chomp:true (command("", [| "uname" |])))
is a string Lwt.t which returns the “chomped” string (new line removed) of the “uname” command. As a second example
Rashell_Command.(exec_query (command("", [| "find"; "/home/user"; "-type"; "f"; "-name"; "*.orig" |])))
is a string Lwt_stream.t whose elements are the paths of the file found by the command
find /home/user -type f -name '*.orig'
The Rashell library defines also interfaces to some commonly used commands, and a nice interface to the find command is defined in Rashell_Posix – which by the way guarantees POSIX portability.
Related
I'm trying to learn F# at the moment and have come up on a problem I can't solve and can't find any answers for on google.
Initially I wanted a log function that would work like the printf family of functions whereby I could provide a format string and a number of arguments (statically checked) but which would add a little metadata before printing it out. With googling, I found this was possible using a function like the following:
let LogToConsole level (format:Printf.TextWriterFormat<'T>) =
let extendedFormat = (Printf.TextWriterFormat<string->string->'T> ("%s %s: " + format.Value))
let date = DateTime.UtcNow.ToString "yyyy-MM-dd HH:mm:ss.fff"
let lvl = string level
printfn extendedFormat date lvl
having the printfn function as the last line of this function allows the varargs-like magic of the printf syntax whereby the partially-applied printfn method is returned to allow the caller to finish applying arguments.
However, if I have multiple such functions with the same signature, say LogToConsole, LogToFile and others, how could I write a function that would call them all keeping this partial-application magic?
Essential I'm looking for how I could implement a function MultiLog
that would allow me to call multiple printf-like functions from a single function call Such as in the ResultIWant function below:
type LogFunction<'T> = LogLevel -> Printf.TextWriterFormat<'T> -> 'T
let MultiLog<'T> (loggers:LogFunction<'T>[]) level (format:Printf.TextWriterFormat<'T>) :'T =
loggers
|> Seq.map (fun f -> f level format)
|> ?????????
let TheResultIWant =
let MyLog = MultiLog [LogToConsole; LogToFile]
MyLog INFO "Text written to %i outputs" 2
Perhaps the essence of this question can be caught more succintly: given a list of functions of the same signature how can I partially apply them all with the same arguments?
type ThreeArg = string -> int -> bool -> unit
let funcs: ThreeArg seq = [func1; func2; func3]
let MagicFunction = ?????
// I'd like this to be valid
let partiallyApplied = MagicFunction funcs "string"
// I'd also like this to be valid
let partiallyApplied = MagicFunction funcs "string" 255
// and this (fullyApplied will be `unit`)
let fullyApplied = MagicFunction funcs "string" 255 true
To answer the specific part of the question regarding string formatting, there is a useful function Printf.kprintf which lets you do what you need in a very simple way - the first parameter of the function is a continuation that gets called with the formatted string as an argument. In this continuation, you can just take the formatted string and write it to all the loggers you want. Here is a basic example:
let Loggers = [printfn "%s"]
let LogEverywhere level format =
Printf.kprintf (fun s ->
let date = DateTime.UtcNow.ToString "yyyy-MM-dd HH:mm:ss.fff"
let lvl = string level
for logger in Loggers do logger (sprintf "%s %s %s" date lvl s)) format
LogEverywhere "BAD" "hi %d" 42
I don't think there is a nice and simple way to do what you wanted to do in the more general case - I suspect you might be able to use some reflection or static member constraints magic, but fortunately, you don't need to in this case!
There is almost nothing to add to a perfect #TomasPetricek answer as he is basically a "semi-god" in F#. Another alternative, which comes to mind, is to use a computation expression (see, for example: https://fsharpforfunandprofit.com/series/computation-expressions.html). When used properly it does look like magic :) However, I have a feeling that it is a little bit too heavy for the problem, which you described.
Since I am an absolute Haskell beginner, but determined to conquer it, I am asking for help again.
using:
fetchData2 = do
conn <- connectSqlite3 "dBase.db"
statement <- prepare conn "SELECT * FROM test WHERE id > 0"
execute statement []
results <- fetchAllRows statement
print results
returns:
[[SqlInt64 3,SqlByteString "Newco"],[SqlInt64 4,SqlByteString "Oldco"],[SqlInt64 5,SqlByteString "Mycom"],[SqlInt64 4,SqlByteString "Oldco"],[SqlInt64 5,SqlByteString "Mycom"]]
Is there a clever way to clean this data into Int and [Char], in other words omitting types SqlInt64 and SqlByteString.
You could define a helper:
fetchRowFromSql :: Convertible SqlValue a => Statement -> IO (Maybe [a])
fetchRowFromSql = fmap (fmap (fmap fromSql)) . fetchRow
The implementation looks a bit daunting, but this is just because we need to drill down under the layered functors as you already noted (first IO, then Maybe and lastly []). This returns something that is convertible from a SqlValue. There are a bunch of these defined already. See e.g. the docs. An example (using -XTypeApplications):
fetchRowFromSql #String :: Statement -> IO (Maybe [String])
I should perhaps add that the documentation mentions that fromSql is unsafe. Meaning that if you try to convert a sql value to an incompatible Haskell value the program will halt.
I need to pass two list as command line arguments in ocaml.
I used the following code to access it in the program.
let list1=Sys.argv.(1);;
let list2=Sys.argv.(2);;
I need to have the list1 and list2 as list of integers.
I am getting the error
This expression has type string but an expression was expected of type
int list
while processing.
How can I convert that arguments to a list of integers.
The arguments are passed in this format [1;2;3;4] [1;5;6;7]
Sys.argv.(n) will always be a string. You need to parse the string into a list of integers. You could try something like this:
$ ocaml
OCaml version 4.01.0
# #load "str.cma";;
# List.map int_of_string (Str.split (Str.regexp "[^0-9]+") "[1;5;6;7]");;
- : int list = [1; 5; 6; 7]
Of course this doesn't check the input for correct form. It just pulls out sequences of digits by brute force. To do better you need to do some real lexical analysis and simple parsing.
(Maybe this is obvious, but you could also test your function in the toplevel (the OCaml read-eval-print loop). The toplevel will handle the work of making a list from what you type in.)
As Sys.argv is a string array, you need to write your own transcription function.
I guess the simplest way to do this is to use the Genlex module provided by the standard library.
let lexer = Genlex.make_lexer ["["; ";"; "]"; ]
let list_of_string s =
let open Genlex in
let open Stream in
let stream = lexer (of_string s) in
let fail () = failwith "Malformed string" in
let rec aux acc =
match next stream with
| Int i ->
( match next stream with
| Kwd ";" -> aux (i::acc)
| Kwd "]" -> i::acc
| _ -> fail () )
| Kwd "]" -> acc
| _ -> fail ()
in
try
match next stream with
| Kwd "[" -> List.rev (aux [])
| _ -> fail ()
with Stream.Failure -> fail ()
let list1 = list_of_string Sys.argv.(1)
let list2 = list_of_string Sys.argv.(2)
Depending on the OCaml flavor you want to use, some other library may look more interesting. If you like yacc, Menhir may solve your problem in a few lines of code.
I have the following code which I intend to create a Map with self defined types variable and location. I understand that the key type should be ordered (some comparator function). How shall I add these rules to make this work? Also, I find the code ugly here. Do I really need the ;; at the end of a type and module?
type variable = string;;
type location = int;;
module LocationMap = Map.Make(variable);;
module EnvironmentMap = Map.Make(location);;
EDIT
This is the rest of my code:
type variable = Variable of string
type location = Location of int
module LocationMap = Map.Make(struct type t = variable let compare = compare end)
module EnvironmentMap = Map.Make(struct type t = variable let compare = compare end)
(*file read function*)
let read_file filename =
let lines = ref [] in
let chan = open_in filename in
try
while true do
lines := input_line chan :: !lines
done;
!lines
with End_of_file ->
close_in chan;
List.rev !lines
in
(*get the inputs*)
let inputs = read_file Sys.argv.(1) in
for i = 0 to List.length inputs - 1 do
Printf.printf "%s\n" (List.nth inputs i)
done;
This has a syntax error. I am not sure why.
EDIT2
I make this work with the following edit:
type variable = Variable of string
type location = Location of int
module LocationMap = Map.Make(struct type t = variable let compare = compare end)
module EnvironmentMap = Map.Make(struct type t = variable let compare = compare end)
(*file read function*)
let read_file filename =
let lines = ref [] in
let chan = open_in filename in
try
while true do
lines := input_line chan :: !lines
done;
!lines
with End_of_file ->
close_in chan;
List.rev !lines
(*get the inputs*)
let () =
let inputs = read_file Sys.argv.(1) in
for i = 0 to List.length inputs - 1 do
Printf.printf "%s\n" (List.nth inputs i)
done;
Sorry for the long list of questions, what does let () = do here? Is it true that when I define a function with let, I do not need in?
When applying the Map.Make functor, you need to supply a struct containing your type and a compare function:
module LocationMap =
Map.Make(struct type t = variable let compare = compare end)
module EnvironmentMap =
Map.Make(struct type t = location let compare = compare end)
You never need to use ;; in compiled code. It's only required when using the toplevel, to tell it when it should evaluate what you've typed in so far.
Some people do use ;; in compiled code, but you never need to do this and I personally never do. There is always a way to get the same effect without using ;;.
Update
The let compare = compare binds the pre-existing OCaml function compare (the infamous polymorphic comparison function) to the name compare inside the struct. So, it creates a Map that uses polymorphic compare to do its comparisons. This is often what you want.
I created a file containing your definitions (without ;;) and the above code, then compiled it with ocamlc -c. There were no syntax errors. I'm positive you don't need to use ;;, as I've written many many thousands of lines of code without it.
Note that I'm not saying that if you remove ;; from syntactically correct OCaml code, the result is always syntactically correct. There are a few idioms that only work when you use ;;. I personally just avoid those idioms.
Update 2
A let at top level of a module is special, and doesn't have an in. It defines a global value of the module. OCaml treats every source file as a module (for free, as I like to say), with a name that's the same as the source file name (capitalized).
You can actually have any pattern in let pattern = expression. So let () = ... is completely normal. It just says that the expression has unit type (hence the pattern matches).
How to get 'System.Type' of the module?
For example module:
module Foo =
let bar = 1
And this does not work:
printfn "%s" typeof<Foo>.Name
Error is:
The type 'Foo' is not defined
You could add a marker type to the module and then discover the module's type from that:
module Foo =
type internal Marker = interface end
let t = typeof<Marker>.DeclaringType
It would certainly be nice to have a moduleof operator... Since there's not one, the easiest way to do what you want is probably to use the Metadata library in the F# PowerPack:
#r "FSharp.PowerPack.Metadata.dll"
open Microsoft.FSharp.Metadata
// get .NET assembly by filename or other means
let asm = ...
let fasm = FSharpAssembly.FromAssembly asm
let t = fasm.GetEntity("Foo").ReflectionType
Unfortunately, this won't work with dynamic assemblies (such as those generated via F# Interactive). You can do something similar using vanilla System.Reflection calls, but that's more dependent on having a good understanding of the compiled form that your module takes.
It can also be done using Quotations. First, define this helper function somewhere:
open Microsoft.FSharp.Quotations.Patterns
let getModuleType = function
| PropertyGet (_, propertyInfo, _) -> propertyInfo.DeclaringType
| _ -> failwith "Expression is no property."
Then, you can define a module and get its type like this:
module SomeName =
let rec private moduleType = getModuleType <# moduleType #>
Hope this helps.
module name is not a type.
List in List.map and let (a:List<int>) = [1;2;3] are different.
The first List is a module name, the second is a type.