I have these lines of code in one program:
source("R:/ML NC8 MENSAL.R")
source("R:/ ML NPC NC8 MENSAL.R")
The mentioned programs both have these lines of code:
# Defining Variable
MONTH <- "01_2021"
I want to make this definition in the first program for the two programs.
Which code should I write?
Thank you for your help.
If both scripts have these lines and you only want to return it from the first script, then I would write a function in both scripts. Only the first one would return the month-value.
month_return_scr1 <- function(){
MONTH <- "01_2021"
#more code
return(list(MONTH, more variables, or data.frames)}
month_return_scr2 <- function(){
MONTH <- "01_2021"
#more code
return(list(more variables, or data.frames)}
The Month would then not be returned by the second source.
I used successfully the following solution:
Create a program - R:/constants.R - with the month variable (and any
others used in all programs)
Create a program - R:/superprogram.R - that executes all the 23 programs
In each 23 programs replace the variables definition for this code
source("R:/constants.R"). This will bring the constants defined in
the source file into the global environment.
Change the variables in program R:/constants.R and save it
Run R:/superprogram.R
Related
I am working on a julia code where I have several files and call functions from these files to a main function called run.jl. Every time I make changes to any one of these files I need to restart the julia REPL which is a bit annoying. Anyway to work around this.
For example
# --------------------------------- run.jl
include("agent_types.jl")
include("resc_funcs.jl")
using .AgentTypes: Cas, Resc
using .RescFuncs: update_rescuers_at_pma!, travel_to_loc
# ... some code
for i = 1:500
update_rescuers_at_pma!(model)
travel_to_loc(model)
end
# ------------------------------ resc_funcs.jl
function travel_to_loc(model)
println(get_resc_by_prop(model, :on_way_to_iz))
for resc_id in get_resc_by_prop(model,:on_way_to_iz)
push!(model[resc_id].dist_traject, model[resc_id].dist_to_agent) #This is just for checking purposes
model[resc_id].dist_to_agent -= model.dist_per_step
push!(model[resc_id].loc_traject, "on_way_to_iz")
end
#If we add new print statement here and execute run.jl it wont print
println(model[resc_id].loc_traject) #new amendment to code
end
But now when I go and update travel_to_loc function for example. I need to restart the julia repl before those changes are reflected. I am wondering if there is a way after you save the file (resc_funcs.jl in this case) those amendments are reflected when you execute run.jl.
The simplest workflow could be the following:
Select some folder to be your current working folder in Julia (eg. by using cd() command from Julia)
Create sample file MyMod.jl
module MyMod
export f1
function f1(x)
x+1
end
end
Add the current folder to LOAD_PATH and load Revise.jl
push!(LOAD_PATH, ".")
using Revise
Load the module and test it:
julia> using MyMod
[ Info: Precompiling MyMod [top-level]
julia> f1(3)
4
Try to edit the file eg. change x+1 to x+3
Run again
julia> f1(3)
6
Notes:
you will still need to restart REPL when your data structures change (you modify a definition of a struct object)
you could generate a full module package using Pkg.generate but I wanted to make things simplified.
I ran
mpiexec -n $nprocs julia --project myfile.jl
on a cluster, where myfile.jl has the following form
using Distributed; using Dates; using JLD2; using LaTeXStrings
#everywhere begin
using SharedArrays; using QuantumOptics; using LinearAlgebra; using Plots; using Statistics; using DifferentialEquations; using StaticArrays
#Defining some other functions and SharedArrays to be used later e.g.
MySharedArray=SharedArray{SVector{Nt,Float64}}(Np,Np)
end
#sync #distributed for pp in 1:Np^2
for jj in 1:Nj
#do some stuff with local variables
for tt in 1:Nt
#do some stuff with local variables
end
end
MySharedArray[pp]=... #using linear indexing
println("$pp finished")
end
timestr=Dates.format(Dates.now(), "yyyy-mm-dd-HH:MM:SS")
filename="MyName"*timestr
#save filename*".jld2"
#later on, some other small stuff like making and saving a figure. (This does give an error "no method matching heatmap_edges(::Surface{Array{Float64,2}}, ::Symbol)" but I think that this is a technical thing about Plots so not very related to the bigger issue here)
However, when looking at the output, there are a few issues that make me conclude that something is wrong
The "$pp finished" output is repeated many times for each value of pp. It seems that this amount is actually equal to 32=$nprocs
Despite the code not being finished, "MyName" files are generated. It should be one, but I get a dozen of them with different timestr component
EDIT: two more things that I can add
the output of the different "MyName" files is not identical, but this is expected since random numbers are used in the inner loops. There are 28 of them, a number that I don't easily recognize except that its again close to the 32 $nprocs
earlier, I wrote that the walltime was exceeded, but this turns out not to be true. The .o file ends with "BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES ... EXIT CODE :9", pretty shortly after the last output file.
$nprocs is obtained in the pbs script through
#PBS -l select=1:ncpus=32:mpiprocs=32
nprocs= `cat $PBS_NODEFILE|wc -l`
As pointed out by adamslc on the Julia discourse, the proper way to use Julia on a cluster is to either
Start a session with one core from the job script, add more with addprocs() in the Julia script itself
Use more specialized Julia packages
https://discourse.julialang.org/t/julia-distributed-redundant-iterations-appearing/57682/3
In tensorflow the training from the scratch produced following 6 files:
events.out.tfevents.1503494436.06L7-BRM738
model.ckpt-22480.meta
checkpoint
model.ckpt-22480.data-00000-of-00001
model.ckpt-22480.index
graph.pbtxt
I would like to convert them (or only the needed ones) into one file graph.pb to be able to transfer it to my Android application.
I tried the script freeze_graph.py but it requires as an input already the input.pb file which I do not have. (I have only these 6 files mentioned before). How to proceed to get this one freezed_graph.pb file? I saw several threads but none was working for me.
You can use this simple script to do that. But you must specify the names of the output nodes.
import tensorflow as tf
meta_path = 'model.ckpt-22480.meta' # Your .meta file
output_node_names = ['output:0'] # Output nodes
with tf.Session() as sess:
# Restore the graph
saver = tf.train.import_meta_graph(meta_path)
# Load weights
saver.restore(sess,tf.train.latest_checkpoint('path/of/your/.meta/file'))
# Freeze the graph
frozen_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
sess.graph_def,
output_node_names)
# Save the frozen graph
with open('output_graph.pb', 'wb') as f:
f.write(frozen_graph_def.SerializeToString())
If you don't know the name of the output node or nodes, there are two ways
You can explore the graph and find the name with Netron or with console summarize_graph utility.
You can use all the nodes as output ones as shown below.
output_node_names = [n.name for n in tf.get_default_graph().as_graph_def().node]
(Note that you have to put this line just before convert_variables_to_constants call.)
But I think it's unusual situation, because if you don't know the output node, you cannot use the graph actually.
As it may be helpful for others, I also answer here after the answer on github ;-).
I think you can try something like this (with the freeze_graph script in tensorflow/python/tools) :
python freeze_graph.py --input_graph=/path/to/graph.pbtxt --input_checkpoint=/path/to/model.ckpt-22480 --input_binary=false --output_graph=/path/to/frozen_graph.pb --output_node_names="the nodes that you want to output e.g. InceptionV3/Predictions/Reshape_1 for Inception V3 "
The important flag here is --input_binary=false as the file graph.pbtxt is in text format. I think it corresponds to the required graph.pb which is the equivalent in binary format.
Concerning the output_node_names, that's really confusing for me as I still have some problems on this part but you can use the summarize_graph script in tensorflow which can take the pb or the pbtxt as an input.
Regards,
Steph
I tried the freezed_graph.py script, but the output_node_name parameter is totally confusing. Job failed.
So I tried the other one: export_inference_graph.py.
And it worked as expected!
python -u /tfPath/models/object_detection/export_inference_graph.py \
--input_type=image_tensor \
--pipeline_config_path=/your/config/path/ssd_mobilenet_v1_pets.config \
--trained_checkpoint_prefix=/your/checkpoint/path/model.ckpt-50000 \
--output_directory=/output/path
The tensorflow installation package I used is from here:
https://github.com/tensorflow/models
First, use the following code to generate the graph.pb file.
with tf.Session() as sess:
# Restore the graph
_ = tf.train.import_meta_graph(args.input)
# save graph file
g = sess.graph
gdef = g.as_graph_def()
tf.train.write_graph(gdef, ".", args.output, True)
then, use summarize graph get the output node name.
Finally, use
python freeze_graph.py --input_graph=/path/to/graph.pbtxt --input_checkpoint=/path/to/model.ckpt-22480 --input_binary=false --output_graph=/path/to/frozen_graph.pb --output_node_names="the nodes that you want to output e.g. InceptionV3/Predictions/Reshape_1 for Inception V3 "
to generate the freeze graph.
It should be simple enough, but I'm just not finding what I need in the documentation. I'm sure I just don't know what to look for.
I'm new to R and have started writing a script that will generate a sequence from 3 user input variables.
basically, I want this
sequence<-seq(1,10,by=2)
to be done with
seqStart<-readline(prompt="Sequence Start")
seqStop<-readline(prompt="Sequence Stop")
seqInt<-readline(prompt="Interval")
sequence<-seq(seqStart, seqStop, by=seqInt)
You need to convert the result of readline to numeric:
seqStart <- as.numeric(readline(prompt="Sequence Start"))
seqStop <- as.numeric(readline(prompt="Sequence Stop"))
seqInt <- as.numeric(readline(prompt="Interval"))
mySequence <- seq(seqStart, seqStop, by=seqInt)
I want to run a program in batch and I want it to wait for the user input in the program.
This is what I have:
library('XLConnect')
FolderPath<-"C:/Documents/Testing/"
load(paste('C:/Documents/Testing/Program.ml',sep=''));Run()
This code is named Test.R, and it calls a compiled file that runs a function that does what I need. I need to have some inputs in this exercise, so I wanted to ask for the user to input some dates.
The program should do something like this:
What are the years for this simulation?
Then the user enter:
>2001, 2002, 2003, 2004
The program will save this vector in variable, lets call y.
Then, when I load the compiled function, I will use y.
The problem is that I run this R code in cmd batch.
Yukio, try the function readline (documentation here) as in the following example, in which a user inputs two years separated by a comma.
> years = readline('What are the years for this simulation? ')
What are the years for this simulation? 2001, 2002
> years = as.numeric(unlist(strsplit(years, ",")))
> years
[1] 2001 2002
By the way, your English is great!
As far as I understand (someone please correct me if this is wrong), but you can't send messages to the user at the command line when executing a script with R CMD BATCH (it writes all input and output to an .Rout file). But executing a script with Rscript.exe does in fact let you send stuff to stdout. However, in order to get input from a user when executing with Rscript.exe, you need to use file('stdin') as the connection to your input function, which, funny enough, won't work if you then try to run the script from the R interpreter with the source function. Here's an example that showing you how to prompt the user for a series of comma separated years, regardless of whether the script runs with Rscript.exe or the source function in interactive mode.
con <- if (interactive()) stdin() else file('stdin')
message('what are the years')
years <- scan(file=con, sep=',', nlines=1, quiet=TRUE)
print(years)
You can deal easily with user inputs in a batch file not in the R.
You create a batch file , say launcher.bat , like this for example
ECHO I am Rscript launcher
set /p years= What are the years for this simulation?
cd R_SCRIPT_PATH
Rscript youscript.R %years%
The user can write as many years he want, it will go in the variable years. Then in your script you parse this variable to set it as a valid list of years.