Saving an OrderedDict to Julia Data Format - julia

Question

I wish to use the JLD package to write an OrderedDict to file in such a way that I can subsequently read it back unchanged.
Here was my first effort:
using JLD, HDF5, DataStructures
function testjld()
res = OrderedDict("A" => 1, "B" => 2)
filename = "c:/temp/test.jld"
save(File(format"JLD", filename), "res", res)
res2 = load(filename)["res"]
#Check if round-tripping works
res == res2
end
But the "round-tripping" doesn't work - the function returns false. It also raises a warning:
julia> testjld()
┌ Warning: type JLD.AssociativeWrapper{Core.String,Core.Int64,OrderedCollections.OrderedDict{Core.String,Core.Int64}} not present in workspace; reconstructing
└ # JLD C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:703
false
After reading the docs, I thought that JLD does not support OrderedDict "out of the box", but does support Dict and I can use that fact to write my own custom serialisation for OrderedDict. Something like this:
struct OrderedDictSerializer
d::Dict
end
JLD.writeas(data::OrderedDict) = OrderedDictSerializer(Dict("contents" => convert(Dict, data),
"keyorder" => [k for (k, v) in data]))
function JLD.readas(serdata::OrderedDictSerializer)
unordered = serdata.d["contents"]
keyorder = serdata.d["keyorder"]
OrderedDict((k, unordered[k]) for k in keyorder)
end
Hardly an exhaustive test, but this does seem to work:
julia> testjld()
true
Am I correct in thinking I need to write my own serializer for OrderedDict, and can my serializer be improved?
EDIT
The answer to to my question "Can my serializer be improved?" seems to be "It will have to be, though I don't yet understand how."
Consider the two following test functions:
function testjld2()
res = OrderedDict("A" => [1.0,2.0],"B" => [3.0,4.0])
#check if round-tripping of readas and writeas methods works:
JLD.readas(JLD.writeas(res)) == res
end
function testjld3()
res = OrderedDict("A" => [1.0,2.0],"B" => [3.0,4.0])
filename = "c:/temp/test.jld"
save(File(format"JLD", filename), "res", res)
res2 = load(filename)["res"]
#Check if round-tripping to jld file and back works
res == res2
end
testjld2 shows that my writeas and readas methods correctly round-trip for an OrderedDict{String,Array{Float64,1}} with 2 entries
julia> testjld2()
true
and yet testjld3 doesn't work at all, but yields an error:
julia> testjld3()
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
#000: E:/mingwbuild/mingw-w64-hdf5/src/hdf5-1.10.5/src/H5Tfields.c line 60 in H5Tget_nmembers(): not a datatype
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.5) thread 0:
#000: E:/mingwbuild/mingw-w64-hdf5/src/hdf5-1.10.5/src/H5Tfields.c line 60 in H5Tget_nmembers(): not a datatype
major: Invalid arguments to routine
minor: Inappropriate type
ERROR: Error getting the number of members
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] h5t_get_nmembers at C:\Users\Philip\.julia\packages\HDF5\rF1Fe\src\HDF5.jl:2279 [inlined]
[3] _gen_h5convert!(::Any) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:638
[4] #s27#9(::Any, ::Any, ::Any, ::Any, ::Any, ::Any) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:664
[5] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at .\boot.jl:524
[6] #write_compound#24(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(JLD.write_compound), ::JLD.JldGroup, ::String, ::JLD.AssociativeWrapper{String,Any,Dict{String,Any}}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:700
[7] write_compound at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:694 [inlined]
[8] #_write#23 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
[9] _write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
[10] write_ref(::JLD.JldFile, ::Dict{String,Any}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:658
[11] macro expansion at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:648 [inlined]
[12] h5convert!(::Ptr{UInt8}, ::JLD.JldFile, ::OrderedDictSerializer, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\jld_types.jl:664
[13] #write_compound#24(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(JLD.write_compound), ::JLD.JldFile, ::String, ::OrderedDictSerializer, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:700
[14] write_compound at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:694 [inlined]
[15] #_write#23 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
[16] _write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:690 [inlined]
[17] #write#17(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(write), ::JLD.JldFile, ::String, ::OrderedDict{String,Array{Float64,1}}, ::JLD.JldWriteSession) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:514
[18] write at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:514 [inlined]
[19] #35 at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:1223 [inlined]
[20] #jldopen#14(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol},NamedTuple{(:compatible, :compress),Tuple{Bool,Bool}}}, ::typeof(jldopen), ::getfield(JLD, Symbol("##35#36")){String,OrderedDict{String,Array{Float64,1}},Tuple{}},
::String, ::Vararg{String,N} where N) at C:\Users\Philip\.julia\packages\JLD\1BoSz\src\JLD.jl:246
[21] testjld3() at .\none:0
[22] top-level scope at REPL[48]:1

Answer 1

Use JLD2 instead:
using JLD2, DataStructures, FileIO
function testjld2()
res = OrderedDict("A" => 1, "B" => 2)
myfilename = "c:/temp/test.jld2"
save(myfilename, "res", res)
res2 = load(myfilename)["res"]
#Check if round-tripping works
res == res2
end
Testing:
julia> testjld2()
true
Personally, whenever I can I use BJSON:
using DataStructures, BSON, OrderedCollections
function testbson()
res = OrderedDict("A" => 1, "B" => 2)
myfilename = "c:/temp/test.bjson"
BSON.bson(myfilename, Dict("res" => res))
res2 = BSON.load(myfilename)["res"]
#Check if round-tripping works
res == res2
end
julia> testbson()
true