I'm running Julia code which generates a plot and a text file. There exists an "Output" folder in the same folder where the code in question is located. For the first run, I create a "Run_1" folder, with "Plots" and "Data" subfolders:
fig_path = #__DIR__
mkdir(fig_path*"/Output/Run_1/")
mkdir(fig_path*"/Output/Run_1/Plots/")
mkdir(fig_path*"/Output/Run_1/Data/")
After plotting, I save the figure to "Plots":
fig_name = "test_figure"
savefig(fig_path*"/Output/Run_1/Plots/"*fig_name*".pdf")
and the output file (contained within "output_matrix") is saved to "Data":
outfile_1 = fig_path*"/Output/Run_1/Data/test_data.txt"
open(outfile_1, "w") do f1
writedlm(f1,output_matrix)
end
However, I want to run this code multiple times. Each time it runs, it should create a new "Run" folder in the "Output" folder, i.e. on the first run its Run_1, the second run it's Run_2, and so on. All folders from previous runs are NOT deleted. In each Run folder, there's a "Plots" and a "Data" folder, and I save the plot and data to their respective folders in each run. How can I have Julia update the file name in such a manner?
The ispath function checks whether a file or directory exists in the filesystem.
If you want to keep a naming convention like Run_1...Run_N, something like this could help:
function mk_output_dir()
i = 1
while true
dir_name = joinpath(#__DIR__, "Output", "run_$i")
if !ispath(dir_name)
mkpath(dir_name)
return dir_name
end
i += 1
end
end
This produces:
# First run
julia> top_dir = mk_output_dir()
"/tmp/Output/run_1"
julia> mkdir(joinpath(top_dir, "Plots"))
"/tmp/Output/run_1/Plots"
julia> mkdir(joinpath(top_dir, "Data"))
"/tmp/Output/run_1/Data"
# Second run
julia> top_dir = mk_output_dir()
"/tmp/Output/run_2"
julia> mkdir(joinpath(top_dir, "Plots"))
"/tmp/Output/run_2/Plots"
julia> mkdir(joinpath(top_dir, "Data"))
"/tmp/Output/run_2/Data"
Be aware that race conditions could occur if you start two instances of your program at the same time.
Alternatively, I personally tend to use naming conventions involving timestamps when creating directory structures like this. Here would be a minimal example:
using Dates
function mk_output_dir()
timestamp = Dates.format(now(), "YYYYmmdd-HHMMSS")
dir_name = joinpath(#__DIR__, "Output", "run_$timestamp")
#assert !ispath(dir_name) "Somebody else already created the directory"
mkpath(dir_name)
return dir_name
end
which produces something like this:
julia> top_dir = mk_output_dir()
"/tmp/Output/run_20201229-210835"
julia> mkdir(joinpath(top_dir, "Plots"))
"/tmp/Output/run_20201229-210835/Plots"
julia> mkdir(joinpath(top_dir, "Data"))
"/tmp/Output/run_20201229-210835/Data"
Maybe something like this:
function mkresultdir(fig_path)
for i=1:1000
rundir = joinpath(fig_path, "run_$i")
if !isdir(rundir)
mkdir(rundir)
return rundir
end
end
error("too many results on disk, time for a cleanup!")
end
res_dir_1 = mkresultdir("/home/my_user/results")
res_dir_2 = mkresultdir("/home/my_user/results")
Related
I'm new to Scilab. I have to run the same program with a dozen different input files. Currently I simply uncomment the line and then rerun the program, and change the output file to a new name
// Input data file
data_file = 'data1.txt';
//data_file = 'data2.txt';
//data_file = 'data3.txt';
//data_file = 'data4.txt';
//data_file = 'data5.txt';
//data_file = 'data6.txt';
etc. another 6 lines
// Output data file name
output_data = '/output_files/data1.csv';
Is there a way to read in each file (data1.txt, data2.txt...) execute the body of the program and then output a new output file (data1.csv, data2.csv ...) instead of what I'm doing now, which is running the program and then editing it to use the next file and run again?
Just do something like:
for i=1:6
// Input data file
data_file=msprintf("data%d.txt",i);
// Output data file name
output_data=msprintf("/output_files/data%d.csv",i);
// exec the body of your script
end
I am using Octave 5.1.0 on Windows 10 (x64). I am parsing a series of directories looking for an Excel spreadsheet in each directory with "logbook" in its filename. The problem is these files are created by hand and the filenaming isn't consistent: sometimes it's "LogBook", other times it's "logbook", etc...
It looks like the string passed as input to the dir function is case-sensitive so if I don't have the correct case, dir returns an empty struct. Currently, I am using the following workaround, but I wondered if there was a better way of doing this (for a start I haven't captured all possible upper/lower case combinations):
logbook = dir('*LogBook.xls*');
if isempty(logbook)
logbook = dir('*logbook.xls*');
if isempty(logbook)
logbook = dir('*Logbook.xls*');
if isempty(logbook)
logbook = dir('*logBook.xls*');
if isempty(logbook)
error(['Could not find logbook spreadsheet in ' dir_name '.'])
end
end
end
end
You need to get the list of filenames (either via readdir, dir, ls), and then search for the string in that list. If you use readdir, it can be done like this:
[files, err, msg] = readdir ('.'); # read current directory
if (err != 0)
error ("failed to readdir (error code %d): %s", msg);
endif
logbook_indices = find (cellfun (#any, regexpi (files, 'logbook'));
logbook_filenames = files(logbook_indices);
A much less standard approach could be:
glob ('*[lL][oO][gG][bB][oO][kK]*')
I am unable to define empty IndexedTables, e.g.
using IndexedTables, IndexedTables.Table
t = Table(Columns(a=Int64[],b=String[]),Int64[])
t[1,"a"] = 1
t[1,"b"] = 2
t[1,"c"] = t[1,"a"] + t[1,"b"]
BoundsError: attempt to access 0-element Array{Int64,1} at index [0]
I am aware that creating the IndexedTable with already the data is more efficient that creating an empty one and then insert values, but sometimes you are obliged to go on this way.
Is this a bug ? If so, is there any workaround possible ?
(I already posted this thread on the Julia forum, but so far I had no replies there)
This is probably a bug in IndexedTables.
Inserting into an IndexedTable requires reindexing to access the data. Reindexing is done with flush!.
But flush!(t) fails in the example in the question with the empty t.
Fixing flush! which calls _merge! can be done by:
julia> function IndexedTables._merge!(dst::IndexedTable, src::IndexedTable, f)
if length(dst.index)==0 || isless(dst.index[end], src.index[1])
append!(dst.index, src.index)
append!(dst.data, src.data)
else
# merge to a new copy
new = _merge(dst, src, f)
ln = length(new)
# resize and copy data into dst
resize!(dst.index, ln)
copy!(dst.index, new.index)
resize!(dst.data, ln)
copy!(dst.data, new.data)
end
return dst
end
julia> t[1,"c"] = t[1,"a"] + t[1,"b"]
3
The change is the addition of the length(...) check in the first if.
Of course, a pull request / issue should be opened with IndexedTables.jl. Antonello, will you do this? (or shall I)
I want to create an object in R, which will contain one string, with a few variables (in my case it is file path). When I try to use paste to concatenate the file paths I can see only one last variable instead of all variables in one string. I use next code:
for(i in seq_len(nrow(samples))) {
lib = samples$conditions[i]
txtFile = file.path(lib, "hits.txt")
testfiles = paste(txtFile, sep = ',')
}
print(testfiles)
and get something like
cond/hits.txt,
instead of
cond/hits.txt,cond1/hits.txt,cond2/hits.txt and so on
Thank you very much for help
I am writing a Matlab program that loads a data file created in another C++ program.
planet = input('What is the name of your planet? ', 's')
data_file = strcat(planet, '.dat')
load(data_file);
data_file;
x = data_file(:,1);
y = data_file(:,2);
plot (x,y,'r*')
The program takes the name of the planet as the user input, then concatenates ".dat" to the end of the planet name. This gives, for example, "earth.dat," which is the name of the file created by the other C++ program.
I have made sure that the data file being loaded is in the correct folder; however, MATLAB still gives an error when I run the program.
What is the correct command for loading this file?
Thank you!
try using this instead:
planet = input('What is the name of your planet? ', 's')
filename=[num2str(planet) '.dat'];
data_file=load(filename);
x = data_file(:,1);
y = data_file(:,2);
plot (x,y,'r*')