This question continues story of this one
For some reason I'd like to manually manage memory in my program. So I want to allocate memory, work with it, and free it. The first and the last are quite easy. However there are some troubles with pointer dereferencing.
Suupose I have such a struct
mutable struct Point
x::Float64
y::Float64
end
I'd like to allocate some memory and store my Point there. In fact I could just say point = Point(0.,0.) but this prevents me from manually freeing up allocated memory and lets me rely on gc. So I write
ptr = convert(Ptr{Point}, Base.Libc.malloc(sizeof(Point))) # N.B. sizeof(Point) == 16
And at this point I want to write into memory pointer by ptr. I found some hacky ways to do it.
Use unsafe_store!. This method is not good for me, because I have to allocate memory for object to be written (Point(0.,0.)) and only after that copy it into ptr.
unsafe_store!(ptr, Point(0.,0.))
Use unsafe_wrap. This could be a good idea however I cannot just wrap a piece of memory into array of mutable types. That's because... array of mutable structs contains only sequential pointers to this structs, not the structs themselves. So because Point has two fields of type Int, I can wrap with type Tuple{Float64,Float64} and then assign values inside of wrapped array. But I also have to store (0.,0.) somewhere, and it's also just copying. And of course I cannot assign to fields of immutable type, so on the one hand I cannot wrap around mutable type, on the other, I cannot modify immutable (and on the third hand I cannot reinterpret them into each other)
ptr2 = convert(Ptr{Tuple{Float64,Float64}}, ptr)
arr = unsafe_wrap(Array, ptr2, 1)
arr[1] = (0.,0.)
Finally I can brutally convert everything to primitive types and this really allows avoid additional allocating and copying. And at this point it's time to ask myself a question, what for do I need the declaration of type Point if I'm working with it internals, guts and bones, directly. (Means I wanted something like p.x = p.y = 0.)
ptr3 = convert(Ptr{Int}, ptr)
arr = unsafe_wrap(Array, ptr3, 2)
arr[1] = arr[2] = 0.
The least hackish and the least working (not working at all) method. I found such a good function, called unsafe_pointer_to_objref and at first sight it should do all the bussines. But. It doesn't. It works quite strange. As it is said in docs (source code)
unsafe_pointer_to_objref(p::Ptr)
Convert a Ptr to an
object reference. Assumes the pointer refers to a valid heap-allocated
Julia object. If this is not the case, undefined behavior results,
hence this function is considered "unsafe" and should be used with
care.
So I have a heap-allocated Point, with ptr pointing on it, I can even unsafe_load value of it (unsafe_load makes a copy, so it is not the solution), and I should be able to make something like
point = unsafe_pointer_to_objref(ptr)
But this leads to
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x278d238 -- typekeyvalue_hash at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1160 [inlined]
lookup_typevalue at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:722
in expression starting at none:0
typekeyvalue_hash at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1160 [inlined]
lookup_typevalue at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:722
jl_inst_arg_tuple_type at /cygdrive/c/buildbot/worker/package_win64/build/src\jltypes.c:1589
jl_f_tuple at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:786 [inlined]
jl_f_tuple at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:781
indexed_iterate at .\pair.jl:37 [inlined]
indexed_iterate at .\pair.jl:37
jfptr_indexed_iterate_29996.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
print_response at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:281
#45 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:275
jfptr_YY.45_50669.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
with_repl_linfo at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:508
print_response at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:273
do_respond at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:844
jfptr_do_respond_48815.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:757
#invokelatest#2 at .\essentials.jl:716 [inlined]
invokelatest at .\essentials.jl:714 [inlined] run_interface at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\LineEdit.jl:2493
jfptr_run_interface_49569.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
run_frontend at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\REPL\src\REPL.jl:1230
#49 at .\task.jl:423
jfptr_YY.49_48790.clone_1 at C:\Users\user\AppData\Local\Programs\Julia-1.7.0\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:877
Allocations: 2721 (Pool: 2712; Big: 9); GC: 0
However it works good in pair with pointer_from_objref (which returns Ptr{Nothing}, so I think unsafe_pointer_to_objref works good only for Julia-owned objects
a === unsafe_pointer_to_objref(pointer_from_objref(a))
Finally it seems that PointerArithmetic.jl uses unsafe_load to dereference pointer, that is not the solution due to mentioned above reason.
So there are 2 questions:
Main: How to dereference pointer on a mutable struct and obtain object located in the place that pointer points on?
Not main: why does not unsafe_pointer_to_objref give a result and what for does it exist?
if you are not requiring a C interface for anything but memory allocation, but do not want the garbage collector to manage your structs, why not just preallocate your own heap?
Consider
struct MyStruct
i::Int
j::Float64
MyStruct(i = 0, j = 0.0) = new(0, 0.0)
end
const heapofMyStruct = [MyStruct() for _ in 1:1000]
m = heapofMyStruct[1]
You will need to keep track of where you are "allocating" the next item in your heap, but heapofMyStruct should not be garbage collected in use.
Related
I have a data structure in Cython that uses a char * member.
What is happening is that the member value seems to lose its scope outside of a function that assigns a value to the member. See this example (using IPython):
[nav] In [26]: %%cython -f
...: ctypedef struct A:
...: char *s
...:
...: cdef char *_s
...:
...: cdef void fn(A *a, msg):
...: s = msg.encode()
...: a[0].s = s
...:
...: cdef A _a
...: _a.s = _s
...: fn(&_a, 'hello')
...: print(_a.s)
...: print(b'hola')
...: print(_a.s)
b'hello'
b'hola'
b"b'hola'"
It looks like _a.s is deallocated outside of fn and is being assigned any junk that is in memory that fits the slot.
This happens only under certain circumstances. For example, if I assign b'hello' to s instead of the encoded string inside fn(), the correct string is printed outside of the function.
As you can see, I also added an extra declaration for the char variable and assigned it to the struct before executing fn, to make sure that the _a.s pointer does not get out of scope. However, my suspect is that the problem is assigning the member to a variable that is in the function scope.
What is really happening here, and how do I resolve this issue?
Thanks.
Your problem is, that the pointer a.s becomes dangling in the fn-function as soon as it is created.
When calling msg.encode() the temporary byte-object s is created and the address of its buffer is saved to a.s. However, directly afterwards (i.e. at the exit from the function) the temporary bytes-object gets destroyed and the pointer becomes dangling.
Because the bytes object was small, Python's memory manager manages its memory in the arena - which guaranties that there is no segfault when you access the address (lucky you).
While the temporary object is destroyed, the memory isn't overwritten/sanatized and thus it looks as if the temporary object where still alive from A.s's point of view.
Whenever you create a new bytes-object similar in size to the temporary object, the old memory from the arena might get reused, so that your pointer a.s could point to the buffer of the newly allocated bytes-object.
Btw, would you use a[0].s = msg.encode() directly (and I guess you did), the Cython would not build and tell you, that you try to say reference to a temporary Python object. Adding an explicit reference fooled the Cython, but didn't help your case.
So what to do about it? Which solution is appropriate depends on the bigger picture, but the available strategies are:
Manage the memory of A.s. I.e. manually reserve memory, copy from the temporary object, free memory as soon as done.
Manage reference counting: Add a PyObject * to the A-struct. Assign the temporary object ot it (don't forget to increase the reference counter manually), decrease reference counter as soon as done.
Collect references of temporary objects into a pool (e.g. a list), which would keep them alive. Clear the pool as soon as objects aren't needed.
Not always the best, but easiest is the option 3 - you neither have to manage the memory not the reference counting:
%%cython
...
pool=[]
cdef void fn(A *a, msg):
s = msg.encode()
pool.append(s)
a[0].s = s
While this doesn't solve the principal problem, using PyUnicode_AsUTF8 (inspired by this answer) might be a satisfactory solution in this case:
%%cython
# it is not wrapped by `cpython.pxd`:
cdef extern from "Python.h":
const char* PyUnicode_AsUTF8(object unicode)
...
cdef void fn(A *a, msg):
a[0].s = PyUnicode_AsUTF8(msg) # msg.encode() uses utf-8 as default.
This has at least two advantages:
the pointer a[0].s is valid as long as msg is alive
calling PyUnicode_AsUTF8(msg) is faster than msg.encode(), because it reuses cached buffer, so it basically O(1) after first call, while msg.encode() needs at least copy the memory and is O(n) with n-number of characters.
Is there a way to enforce a dictionary being constant?
I have a function which reads out a file for parameters (and ignores comments) and stores it in a dict:
function getparameters(filename::AbstractString)
f = open(filename,"r")
dict = Dict{AbstractString, AbstractString}()
for ln in eachline(f)
m = match(r"^\s*(?P<key>\w+)\s+(?P<value>[\w+-.]+)", ln)
if m != nothing
dict[m[:key]] = m[:value]
end
end
close(f)
return dict
end
This works just fine. Since i have a lot of parameters, which i will end up using on different places, my idea was to let this dict be global. And as we all know, global variables are not that great, so i wanted to ensure that the dict and its members are immutable.
Is this a good approach? How do i do it? Do i have to do it?
Bonus answerable stuff :)
Is my code even ok? (it is the first thing i did with julia, and coming from c/c++ and python i have the tendencies to do things differently.) Do i need to check whether the file is actually open? Is my reading of the file "julia"-like? I could also readall and then use eachmatch. I don't see the "right way to do it" (like in python).
Why not use an ImmutableDict? It's defined in base but not exported. You use one as follows:
julia> id = Base.ImmutableDict("key1"=>1)
Base.ImmutableDict{String,Int64} with 1 entry:
"key1" => 1
julia> id["key1"]
1
julia> id["key1"] = 2
ERROR: MethodError: no method matching setindex!(::Base.ImmutableDict{String,Int64}, ::Int64, ::String)
in eval(::Module, ::Any) at .\boot.jl:234
in macro expansion at .\REPL.jl:92 [inlined]
in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at .\event.jl:46
julia> id2 = Base.ImmutableDict(id,"key2"=>2)
Base.ImmutableDict{String,Int64} with 2 entries:
"key2" => 2
"key1" => 1
julia> id.value
1
You may want to define a constructor which takes in an array of pairs (or keys and values) and uses that algorithm to define the whole dict (that's the only way to do so, see the note at the bottom).
Just an added note, the actual internal representation is that each dictionary only contains one key-value pair, and a dictionary. The get method just walks through the dictionaries checking if it has the right value. The reason for this is because arrays are mutable: if you did a naive construction of an immutable type with a mutable field, the field is still mutable and thus while id["key1"]=2 wouldn't work, id.keys[1]=2 would. They go around this by not using a mutable type for holding the values (thus holding only single values) and then also holding an immutable dict. If you wanted to make this work directly on arrays, you could use something like ImmutableArrays.jl but I don't think that you'd get a performance advantage because you'd still have to loop through the array when checking for a key...
First off, I am new to Julia (I have been using/learning it since only two weeks). So do not put any confidence in what I am going to say unless it is validated by others.
The dictionary data structure Dict is defined here
julia/base/dict.jl
There is also a data structure called ImmutableDict in that file. However as const variables aren't actually const why would immutable dictionaries be immutable?
The comment states:
ImmutableDict is a Dictionary implemented as an immutable linked list,
which is optimal for small dictionaries that are constructed over many individual insertions
Note that it is not possible to remove a value, although it can be partially overridden and hidden
by inserting a new value with the same key
So let us call what you want to define as a dictionary UnmodifiableDict to avoid confusion. Such object would probably have
a similar data structure as Dict.
a constructor that takes a Dict as input to fill its data structure.
specialization (a new dispatch?) of the the method setindex! that is called by the operator [] =
in order to forbid modification of the data structure. This should be the case of all other functions that end with ! and hence modify the data.
As far as I understood, It is only possible to have subtypes of abstract types. Therefore you can't make UnmodifiableDict as a subtype of Dict and only redefine functions such as setindex!
Unfortunately this is a needed restriction for having run-time types and not compile-time types. You can't have such a good performance without a few restrictions.
Bottom line:
The only solution I see is to copy paste the code of the type Dict and its functions, replace Dict by UnmodifiableDict everywhere and modify the functions that end with ! to raise an exception if called.
you may also want to have a look at those threads.
https://groups.google.com/forum/#!topic/julia-users/n-lqjybIO_w
https://github.com/JuliaLang/julia/issues/1974
REVISION
Thanks to Chris Rackauckas for pointing out the error in my earlier response. I'll leave it below as an illustration of what doesn't work. But, Chris is right, the const declaration doesn't actually seem to improve performance when you feed the dictionary into the function. Thus, see Chris' answer for the best resolution to this issue:
D1 = [i => sind(i) for i = 0.0:5:3600];
const D2 = [i => sind(i) for i = 0.0:5:3600];
function test(D)
for jdx = 1:1000
# D[2] = 2
for idx = 0.0:5:3600
a = D[idx]
end
end
end
## Times given after an initial run to allow for compiling
#time test(D1); # 0.017789 seconds (4 allocations: 160 bytes)
#time test(D2); # 0.015075 seconds (4 allocations: 160 bytes)
Old Response
If you want your dictionary to be a constant, you can use:
const MyDict = getparameters( .. )
Update Keep in mind though that in base Julia, unlike some other languages, it's not that you cannot redefine constants, instead, it's just that you get a warning when doing so.
julia> const a = 2
2
julia> a = 3
WARNING: redefining constant a
3
julia> a
3
It is odd that you don't get the constant redefinition warning when adding a new key-val pair to the dictionary. But, you still see the performance boost from declaring it as a constant:
D1 = [i => sind(i) for i = 0.0:5:3600];
const D2 = [i => sind(i) for i = 0.0:5:3600];
function test1()
for jdx = 1:1000
for idx = 0.0:5:3600
a = D1[idx]
end
end
end
function test2()
for jdx = 1:1000
for idx = 0.0:5:3600
a = D2[idx]
end
end
end
## Times given after an initial run to allow for compiling
#time test1(); # 0.049204 seconds (1.44 M allocations: 22.003 MB, 5.64% gc time)
#time test2(); # 0.013657 seconds (4 allocations: 160 bytes)
To add to the existing answers, if you like immutability and would like to get performant (but still persistent) operations which change and extend the dictionary, check out FunctionalCollections.jl's PersistentHashMap type.
If you want to maximize performance and take maximal advantage of immutability, and you don't plan on doing any operations on the dictionary whatsoever, consider implementing a perfect hash function-based dictionary. In fact, if your dictionary is a compile-time constant, these can even be computed ahead of time (using metaprogramming) and precompiled.
My goal is to call Windows' GetModuleInformation function to get a MODULEINFO struct back. This is all working fine. The problem comes as a result of me wanting to do pointer arithmetic and dereferences on the LPVOID lpBaseOfDll which is part of the MODULEINFO.
Here is my code to call the function in Lua:
require "luarocks.require"
require "alien"
sizeofMODULEINFO = 12 --Gotten from sizeof(MODULEINFO) from Visual Studio
MODULEINFO = alien.defstruct{
{"lpBaseOfDll", "pointer"}; --Does this need to be a buffer? If so, how?
{"SizeOfImage", "ulong"};
{"EntryPoint", "pointer"};
}
local GetModuleInformation = alien.Kernel32.K32GetModuleInformation
GetModuleInformation:types{ret = "int", abi = "stdcall", "long", "pointer", "pointer", "ulong"}
local GetModuleHandle = alien.Kernel32.GetModuleHandleA
GetModuleHandle:types{ret = "pointer", abi = "stdcall", "pointer"}
local GetCurrentProcess = alien.Kernel32.GetCurrentProcess
GetCurrentProcess:types{ret = "long", abi = "stdcall"}
local mod = MODULEINFO:new() --Create struct (needs buffer?)
local currentProcess = GetCurrentProcess()
local moduleHandle = GetModuleHandle("myModule.dll")
local success = GetModuleInformation(currentProcess, moduleHandle, mod(), sizeofMODULEINFO)
if success == 0 then --If there is an error, exit
return 0
end
local dataPtr = mod.lpBaseOfDll
--Now how do I do pointer arithmetic and/or dereference "dataPtr"?
At this point, mod.SizeOfImage seems to be giving me the correct values that I am expecting, so I know the functions are being called and the struct is being populated. However, I cannot seem to do pointer arithmetic on mod.lpBaseOfDll because it is a UserData.
The only information in the Alien Documentation that may address what I'm trying to do are these:
Pointer Unpacking
Alien also provides three convenience functions that let you
dereference a pointer and convert the value to a Lua type:
alien.tostring takes a userdata (usually returned from a function that has a pointer return value), casts it to char*, and returns a Lua
string. You can supply an optional size argument (if you don’t Alien
calls strlen on the buffer first).
alien.toint takes a userdata, casts it to int*, dereferences it and returns it as a number. If you pass it a number it assumes the
userdata is an array with this number of elements.
alien.toshort, alien.tolong, alien.tofloat, and alien.todouble are like alien.toint, but works with with the respective typecasts.
Unsigned versions are also available.
My issue with those, is I would need to go byte-by-byte, and there is no alien.tochar function. Also, and more importantly, this still doesn't solve the problem of me being able to get elements outside of the base address.
Buffers
After making a buffer you can pass it in place of any argument of
string or pointer type.
...
You can also pass a buffer or other userdata to the new method of your
struct type, and in this case this will be the backing store of the
struct instance you are creating. This is useful for unpacking a
foreign struct that a C function returned.
These seem to suggest I can use an alien.buffer as the argument of MODULEINFO's LPVOID lpBaseOfDll. And buffers are described as byte arrays, which can be indexed using this notation: buf[1], buf[2], etc. Additionally, buffers go by bytes, so this would ideally solve all problems. (If I am understanding this correctly).
Unfortunately, I can not find any examples of this anywhere (not in the docs, stackoverflow, Google, etc), so I am have no idea how to do this. I've tried a few variations of syntax, but nearly every one gives a runtime error (others simply does not work as expected).
Any insight on how I might be able to go byte-by-byte (C char-by-char) across the mod.lpBaseOfDll through dereferences and pointer arithmetic?
I need to go byte-by-byte, and there is no alien.tochar function.
Sounds like alien.tostring has you covered:
alien.tostring takes a userdata (usually returned from a function that has a pointer return value), casts it to char*, and returns a Lua string. You can supply an optional size argument (if you don’t Alien calls strlen on the buffer first).
Lua strings can contain arbitrary byte values, including 0 (i.e. they aren't null-terminated like C strings), so as long as you pass a size argument to alien.tostring you can get back data as a byte buffer, aka Lua string, and do whatever you please with the bytes.
It sounds like you can't tell it to start at an arbitrary offset from the given pointer address. The easiest way to tell for sure, if the documentation doesn't tell you, is to look at the source. It would probably be trivial to add an offset parameter.
I saw this question:
Fortran dynamic objects
and the accepted answer made me question if I wrote the following function safely (without allowing a memory leak)
function getValues3D(this) result(vals3D)
implicit none
type(allBCs),intent(in) :: this
real(dpn),dimension(:,:,:),pointer :: vals3D
integer,dimension(3) :: s
if (this%TF3D) then
s = shape(this%vals3D)
if (associated(this%vals3D)) then
stop "possible memory leak - p was associated"
endif
allocate(vals3D(s(1),s(2),s(3)))
vals3D = this%vals3D
else; call propertyNotAssigned('vals3D','getValues3D')
endif
end function
This warning shows up when I run my code, but shouldn't my this%vals3D be associated if it was previously (to this function) set? I'm currently running into memory errors, and they started showing up when I introduced a new module with this function in it.
Any help is greatly appreciated.
I think I wasn't specific enough. I would like to make the following class, and know how to implement the class, safely in terms of memory. That is:
module vectorField_mod
use constants_mod
implicit none
type vecField1D
private
real(dpn),dimension(:),pointer :: x
logical :: TFx = .false.
end type
contains
subroutine setX(this,x)
implicit none
type(vecField1D),intent(inout) :: this
real(dpn),dimension(:),target :: x
allocate(this%x(size(x)))
this%x = x
this%TFx = .true.
end subroutine
function getX(this) result(res)
implicit none
real(dpn),dimension(:),pointer :: res
type(vecField1D),intent(in) :: this
nullify(res)
allocate(res(size(this%x)))
if (this%TFx) then
res = this%x
endif
end function
end module
Where the following code tests this module
program testVectorField
use constants_mod
use vectorField_mod
implicit none
integer,parameter :: Nx = 150
real(dpn),parameter :: x_0 = 0.0
real(dpn),parameter :: x_N = 1.0
real(dpn),parameter :: dx = (x_N - x_0)/dble(Nx-1)
real(dpn),dimension(Nx) :: x = (/(x_0+dble(i)*dx,i=0,Nx-1)/)
real(dpn),dimension(Nx) :: f
real(dpn),dimension(:),pointer :: fp
type(vecField1D) :: f1
integer :: i
do i=1,Nx
f(i) = sin(x(i))
enddo
do i=1,10**5
call setX(f1,f) !
f = getX(f1) ! Should I use this?
fp = getX(f1) ! Or this?
fp => getX(f1) ! Or even this?
enddo
end program
Currently, I'm running on windows. When I CTR-ALT-DLT, and view performance, the "physical memory usage histery" increases with every loop iteration. This is why I assume that I have a memory leak.
So I would like to repose my question: Is this a memory leak? (The memory increases with every one of the above cases). If so, is there a way I avoid the memory leak while still using pointers? If not, then what is happening, should I be concerned and is there a way to reduce the severity of this behavior?
Sorry for the initial vague question. I hope this is more to the point.
Are you really restricted to Fortran 90? In Fortran 2003 you would use an allocatable function result for this. This is much safer. Using pointer function results, whether you have a memory leak with this code or not depends on how you reference the function, which you don't show. If you must return a pointer from a procedure, it is much safer to return it via a subroutine argument.
BUT...
This function is pointless. There's no point testing the association status of this%vals3D` after you've referenced it as the argument to SHAPE in the previous line. If the pointer component is disassocated (or has undefined pointer association status), then you are not permitted to reference it.
Further, if the pointer component is associated, all you do is call stop!
Perhaps you have transcribed the code to the question incorrectly?
If you simply delete the entire if construct starting with if (associated(this%vals3D))... then your code may make sense.
BUT...
if this%TF3D is true, then this%vals3D must be associated.
when you reference the function, you must use pointer assignment
array_ptr => getValues3D(foo)
! ^
! |
! + this little character is very important.
Forget that little character and you are using normal assignment. Syntactically valid, difficult to pick the difference when reading code and, in this case, potentially a source of memory corruption or leaks that might go undetected until the worst possible moment, in addition to the usual pitfalls of using pointers (e.g. you need to DEALLOCATE array_ptr before you reuse it or it goes out of scope). This is why functions returning pointer results are considered risky.
Your complete code shows several memory leaks. Every time you allocate something that is a POINTER - you need to pretty much guarantee that there will be a matching DEALLOCATE.
You have a loop in your test code. ALLOCATE gets called a lot - in both the setter and the getter. Where are the matching DEALLOCATE statements?
Every time setX is called, any previously allocated memory for the x component of your type will be leaked. Since you call the function 10^5 times, you will waste 100000-1 copies. If you know that the size of this%x will never change, simply check to see if a previous call had already allocated the memory by checking to see if ASSOCIATED(this%x) is true. If it is, skip the allocation and move directly to the assignment statement. If the size does change, then you will first have to deallocate the old copy before allocating new space.
Two other minor comments on setX: The TARGET attribute of the dummy argument x appears superfluous since you never take a pointer of that argument. Second, the TFx component of your type also seems superfluous since you can instead check if x is allocated.
For the function getX, why not skip the allocation completely, and merely set res => this%x? Admittedly, this will return a direct reference to the underlying data, which maybe you want to avoid.
In your loop,
do i=1,10**5
call setX(f1,f) !
f = getX(f1) ! Should I use this?
fp = getX(f1) ! Or this?
fp => getX(f1) ! Or even this?
enddo
fp => getX(f1) will allow you to obtain a pointer to the underlying x component of your type (if you adopt my change above). The other two use assignment operators and will copy data from the result of getX into either f, or (if it is previously allocated) fp. If fp is not allocated, the code will crash.
If you do not want to grant direct access to the underlying data, then I suggest that the return value of getX should be defined as an automatic array with the size determined by this%x. That is, you can write the function as
function getX(this) result(res)
implicit none
type(vecField1D),intent(in) :: this
real(dpn),dimension(size(this%x,1)) :: res
res = this%x
end function
I'm getting this unhandled exception when I exit my program:
Unhandled exception at 0x102fe274 (msvcr100d.dll) in Parameters.exe: 0xC0000005: Access violation reading location 0x00000005.
The debugger stops in a module called crtdll.c on this line:
onexitbegin_new = (_PVFV *) DecodePointer(__onexitbegin);
The top line on the call stack reads:
msvcr100d.dll!__clean_type_info_names_internal(__type_info_node * p_type_info_root_node=0x04a6506c) Line 359 + 0x3 bytes C++
The program then remains in memory until I close down the IDE.
I'm more used to developing with managed languages so I expect I'm doing something wrong with my code maintenance. The code itself reads a memory mapped file and assoiciates it with pointers:
SUBROUTINE READ_MMF ()
USE IFWIN
USE, INTRINSIC :: iso_c_binding
USE, INTRINSIC :: iso_fortran_env
INTEGER(HANDLE) file_mapping_handle
INTEGER(LPVOID) memory_location
TYPE(C_PTR) memory_location_cptr
INTEGER memory_size
INTEGER (HANDLE) file_map
CHARACTER(5) :: map_name
TYPE(C_PTR) :: cdata
integer :: n = 3
integer(4), POINTER :: A, C
real(8), POINTER :: B
TYPE STRUCT
integer(4) :: A
real(8) :: B
integer(4) :: C
END TYPE STRUCT
TYPE(STRUCT), pointer :: STRUCT_PTR
memory_size = 100000
map_name = 'myMMF'
file_map = CreateFileMapping(INVALID_HANDLE_VALUE,
+ NULL,
+ PAGE_READWRITE,
+ 0,
+ memory_size,
+ map_name // C_NULL_CHAR )
memory_location = MapViewOfFile(file_map,
+IOR(FILE_MAP_WRITE, FILE_MAP_READ),
+ 0, 0, 0 )
cdata = TRANSFER(memory_location, memory_location_cptr)
call c_f_pointer(cdata, STRUCT_PTR, [n])
A => STRUCT_PTR%A
B => STRUCT_PTR%B
C => STRUCT_PTR%C
RETURN
END
Am I supposed to deallocate the c-pointers when I'm finished with them? I looked into that but can't see how I do it in Fortran...
Thanks for any help!
The nature of the access violation (during runtime library cleanup) suggests that your program is corrupting memory in some way. There are a number of programming errors that can lead to that - and the error or errors responsible could be anywhere in your program. The usual "compile and run with all diagnostic and debugging options enabled" approach may help identify these.
That said, there is a programming error in the code example shown. The C_F_POINTER procedure from the ISO_C_BINDING intrinsic module can operate on either scalar or array Fortran pointers (the second argument). If the Fortran pointer is a scalar then the third "shape" argument must not be present (it must be present if the Fortran pointer is an array).
Your code breaks this requirement - the Fortran pointer STRUCT_PTR in your code is a scalar, but yet you provide the third shape argument (as [n]). It is quite plausible that this error will result in memory corruption - typically the implementation of C_F_POINTER would try and populate a descriptor in memory for the Fortran pointer, and the descriptor for a pointer to an array may be very different from a pointer to a scalar.
Subsequent references to STRUCT_PTR may further the corruption.
While it is not required by the standard to diagnose this situation, I am a little surprised that the compiler does not issue a diagnostic (assuming you example code is what you actually are compiling). If you reported this to your compiler's vendor (Intel, presumably given IFWIN etc) I suspect they would regard it as a deficiency in their compiler.
To release the memory associated with the file mapping you use the UnmapViewOfFile and CloseHandle API's. To use these you should "store" (your program needs to remember in some way) the base address (memory_location, which can also be obtained by calling C_LOC on STRUCT_PTR once the problem above is fixed) returned by MapViewOfFile, and the handle to the mapping (file_map) returned by CreateFileMapping; respectively.
I've only ever done this with Cray Pointers: not with the ISO bindings and I know it does work with Cray Pointers.
What you don't say is whether this is happening the first time or second time the routine is being called. If it is called more than once, then there is a problem in the coding in that Create/OpenFileMapping should only be called once to get a handle.
You don't need to deallocate memory because the memory is not yours to deallocate: you need to call UnmapViewOfFile(memory_location). After you have called this, memory_location, memory_location_cptr and possibly cdata are no longer valid.
The way this works is with two or more programs:
One program calls CreateFileMapping, the others calls OpenFileMapping to obtain a handle to the data. This only needs to be called once at the start of the program: not every time you need to access the file. Multiple calls to Create/OpenFileMapping without a corresponding close can cause crashes.
They then call MapViewOfFile to map the file into memory. Note that only one program can do this at a time. When the program is finished with the memory file, it calls UnmapViewOfFile. The other program can now get to the file. There is a blocking mechanism. If you do not call UnmapViewOfFile, other programs using MapViewOfFile will be blocked.
When all is done, call close on the handle created by Create/OpenFileMapping.