How do I retrieve outputs from objects in an array as described in the background?
I have a function in R that returns multiple variables. For eg. if my function is called function_ABC,then:
a<-function_ABC (input_var)
gives a such that a$var1, a$var2, and a$var3 exist.
I have multiple cases to run such that I have put then in an array:
input_var <- c(1, 2, ...15)
for storing the outputs, I declared var such that:
var <- c(v1, v2, v3, .... v15)
Then I run:
assign(v1[i],function(input_var(i)))
However, after that I am unable to access these variables as v1[1]$var1. I can access them as: v1$var1, or v3$var1, etc. But this means I need to write 15*3 commands to retrieve my output.
Is there an easier way to do this?
Push your whole input set into an array Arr[ ].
Open a multi threaded executor E of certain size N.
Using a for loop on the input array Arr[], submit your function calls as a Callable job to the executor E. While submitting each job, hold the reference to the FutureTask in another Array FTArr[ ].
When all the FutureTask jobs are executed, you may retrieve the output for each of them by running another for loop on FTArr[ ].
Note :
• make sure to add synchronized block in your func_ABC, where you are accessing shared resources to avoid deadlocks.
• Please refer to the below link, if you want to know more about the usage of a count-down-latch. A count-down-latch helps you to find out, when exactly, all the child threads have finished execution.
https://www.geeksforgeeks.org/countdownlatch-in-java/
Related
I am trying to figure out if it is possible, with a sane amount of programming, to create a certain debugging function by using R's metaprogramming features.
Suppose I have a block of code, such that each line uses as all or part of its input the output from thee line before -- the sort of code you might build with pipes (though no pipe is used here).
{
f1(args1) -> out1
f2(out1, args2) -> out2
f3(out2, args3) -> out3
...
fn(out<n-1>, args<n>) -> out<n>
}
Where for example it might be that:
f1 <- function(first_arg, second_arg, ...){my_body_code},
and you call f1 in the block as:
f1(second_arg = 1:5, list(a1 ="A", a2 =1), abc = letters[1:3], fav = foo_foo)
where foo_foo is an object defined in the calling environment of f1.
I would like a function I could wrap around my block that would, for each line of code, create an entry in a list. Each entry would be named (line1, line2) and each line entry would have a sub-entry for each argument and for the function output. the argument entries would consist, first, of the name of the formal, to which the actual argument is matched, second, the expression or name supplied to that argument if there is one (and a placeholder if the argument is just a constant), and third, the value of that expression as if it were immediately forced on entry into the function. (I'd rather have the value as of the moment the promise is first kept, but that seems to me like a much harder problem, and the two values will most often be the same).
All the arguments assigned to the ... (if any) would go in a dots = list() sublist, with entries named if they have names and appropriately labeled (..1, ..2, etc.) if they are assigned positionally. The last element of each line sublist would be the name of the output and its value.
The point of this is to create a fairly complete record of the operation of the block of code. I think of this as analogous to an elaborated version of purrr::safely that is not confined to iteration and keeps a more detailed record of each step, and indeed if a function exits with an error you would want the error message in the list entry as well as as much of the matched arguments as could be had before the error was produced.
It seems to me like this would be very useful in debugging linear code like this. This lets you do things that are difficult using just the RStudio debugger. For instance, it lets you trace code backwards. I may not know that the value in out2 is incorrect until after I have seen some later output. Single-stepping does not keep intermediate values unless you insert a bunch of extra code to do so. In addition, this keeps the information you need to track down matching errors that occur before promises are even created. By the time you see output that results from such errors via single-stepping, the matching information has likely evaporated.
I have actually written code that takes a piped function and eliminates the pipes to put it in this format, just using text manipulation. (Indeed, it was John Mount's "Bizarro pipe" that got me thinking of this). And if I, or we, or you, can figure out how to do this, I would hope to make a serious run on a second version where each function calls the next, supplying it with arguments internally rather than externally -- like a traceback where you get the passed argument values as well as the function name and and formals. Other languages have debugging environments like that (e.g. GDB), and I've been wishing for one for R for at least five years, maybe 10, and this seems like a step toward it.
Just issue the trace shown for each function that you want to trace.
f <- function(x, y) {
z <- x + y
z
}
trace(f, exit = quote(print(returnValue())))
f(1,2)
giving the following which shows the function name, the input and output. (The last 3 is from the function itself.)
Tracing f(1, 2) on exit
[1] 3
[1] 3
I'm running a google big query script off of RStudio.
I have one important parameterised variable. Which needs to be replaced with values in a dataframe
health_tags<-read.csv('marker_tags.csv')
health_tags<-tail(tags, 7)
I have built a function which executes my query whilst adding the parameters to my variables.
query_details (MD2_date_start="2018-06-06",
MD2_date_end="2018-07-07",
Sterile_tag="7894")
So "query_details" is a function API call which fills in details for BQ to run. How do I write a looper which replaces the values in "sterile_tag" with the codes found in the health_tags CSV and then run the "query_details" function each time until all iterations have completed.
You can use sapply where column should be the real name of your column:
sapply(health_tags$column, function(x) query_details (MD2_date_start="2018-06-06",
MD2_date_end="2018-07-07",
Sterile_tag=as.character(x)))
I am still trying to understand the Prolog logic and have stumbled upon a problem.
I am trying to save values found within recursive calls, to pass on or gather.
As such:
main([]) :- !.
main([H|Tail]) :- findall(X,something(_,_,X),R),
getValueReturn(R,H,Lin, Lout),
main(Tail).
% X is the Head from main
getValueReturn([H|Tail],X,Lin, Lout) :- subset(X, H) ->
findall(A,something(A,_,H),L1),
append(Lin,L1,Lout),
getValueReturn(Tail,X,Lout,L)
;
getValueReturn(Tail,X,Lin,Lout).
I would like to gather the results from findall in getValueReturn, combine them, and send them back to main, which can then use them.
How do I create and add to a list within getValueReturn?
Similarly, how can I save the list in my main for all recursive calls?
EDIT:
I edited the code above as per a comment reply, however when I run this through trace, the list deletes all elements when the empty list is found.
What am I doing wrong? This is the first time I try to use the concept of building a list through recursion.
You should post complete code that can be run, with example data. I have not tested this.
You need to pass L around on the top-level also. Using the same variable names for different parameters in adjacent procedures does not improve readability.
main([E|Es],L0,L) :-
findall(X,something(_,_,X),Rs),
getValueReturn(Rs,E,L0,L1),
main(Es,L1,L).
main([],L,L).
getValueReturn([R|Rs],E,L0,L) :-
( subset(E,R) ->
findall(A,something(A,_,R),New),
append(L0,New,L1),
getValueReturn(Rs,E,L1,L)
; getValueReturn(Rs,E,L0,L) ).
getValueReturn([],_,L,L).
A variable can only have one value in Prolog. In your code, for example, Lout is the output from append/3, an input to a recursive call of getValueReturn/4, and then also the output on the top-level. This is probably not going to do what you want.
I have found the best way to do what I was trying to was to use asserta/z when a result was found, and then gather these results later on.
Otherwise the code became overly complicated and did not function as intended.
I'd like to fancy up my embedding of Julia in a MATLAB mex function by hooking up Julia's STDIN, STDOUT, and STDERR to the MATLAB terminal. The documentation for redirect_std[in|out|err] says that the stream that I pass in as the argument needs to be a TTY or a Pipe (or a TcpSocket, which wouldn't seem to apply).
I know how I will define the right callbacks for each stream (basically, wrappers around calls to MATLAB's input and fprintf), but I'm not sure how to construct the required stream.
Pipe was renamed PipeEndpoint in https://github.com/JuliaLang/julia/pull/12739, but the corresponding documentation was not updated and PipeEndpoint is now considered internal. Even so, creating the pipe up front is still doable:
pipe = Pipe()
Base.link_pipe(pipe)
redirect_stdout(pipe.in)
#async while !eof(pipe)
data = readavailable(pipe)
# Pass data to whatever function handles display here
end
Furthermore, the no-argument version of these functions already create a pipe object, so the recommended way to do this would be:
(rd,wr) = redirect_stdout()
#async while !eof(rd)
data = readavailable(rd)
# Pass data to whatever function handles display here
end
Nevertheless, all of this is less clear than it could be, so I have created a pull request to clean up this API: https://github.com/JuliaLang/julia/pull/18253. Once that pull request is merged, the link_pipe call will become unnecessary and pipe can be passed directly into redirect_stdout. Further, the return value from the no-argument version will become a regular Pipe.
Suppose I have a function that has multiple returned values (shown below). However, this output is not informative as users do not know what each value stands for unless they look up the function definition. So I would like to use println() to print the results with appropriate names to the screen, while suppressing the the actual returned values from being printed on the screen. In R, the function invisible() does that, but how do you do the same thing in Julia?
function trimci(x::Array; tr=0.2, alpha=0.05, nullvalue=0)
se=sqrt(winvar(x,tr=tr))./((1-2.*tr)*sqrt(length(x)))
ci=cell(2)
df=length(x)-2.*floor(tr.*length(x))-1
ci=[tmean(x, tr=tr)-qt(1-alpha./2, df).*se, tmean(x, tr=tr)+qt(1-alpha./2, df).*se]
test=(tmean(x,tr=tr)-nullvalue)./se
sig=2.*(1-pt(abs(test),df))
return ci, tmean(x, tr=tr), test, se, sig
end
In addition to what Harlan and Stefan said, let me share an example from the ODBC.jl package (source here).
One of my favorite features of Julia over other languages is how dead simple it is to create custom types (and without performance issues either!). Here's a custom type, Metadata, that simply holds several fields of data that describe an executed query. This doesn't necessarily need its own type, but it makes it more convenient passing all this data between functions as well as allowing custom formatting of its output by overloading the Base.show() function.
type Metadata
querystring::String
cols::Int
rows::Int
colnames::Array{ASCIIString}
coltypes::Array{(String,Int16)}
colsizes::Array{Int}
coldigits::Array{Int16}
colnulls::Array{Int16}
end
function show(io::IO,meta::Metadata)
if meta == null_meta
print(io,"No metadata")
else
println(io,"Resultset metadata for executed query")
println(io,"------------------------------------")
println(io,"Columns: $(meta.cols)")
println(io,"Rows: $(meta.rows)")
println(io,"Column Names: $(meta.colnames)")
println(io,"Column Types: $(meta.coltypes)")
println(io,"Column Sizes: $(meta.colsizes)")
println(io,"Column Digits: $(meta.coldigits)")
println(io,"Column Nullable: $(meta.colnulls)")
print(io,"Query: $(meta.querystring)")
end
end
Again, nothing fancy, but illustrates how easy it really is to define a custom type and produce custom output along with it.
Cheers.
One thing you could do would be to define a new type for the return value for this function, call it TrimCIResult or something. Then you could define appropriate methods to show that object in the REPL. Or you may be able to generalize that solution with a type hierarchy that could be used for storing the results from and displaying any statistical test.
The value nothing is how you return a value that won't print: the repl specifically checks for the value nothing and prints nothing if that's the value returned by an expression. What you're looking to do is to return a bunch of values and not print them, which strikes me as rather odd. If a function returns some stuff, I want to know about it – having the repl lie to users seems like a bad idea. Harlan's suggesting would work though: define a type for this value with the values you don't want to expose to the user as fields and customize its printing so that the fields you don't want to show people aren't printed.