Related
I'm writing a HDF5 wrapper subroutine that will read/write a double precision array of any shape from/to a dataset inside a HDF5 file. To achieve this, I use some C pointer trickery such that the subroutine takes in only the first element of the array as val, but it actually reads/writes the whole array using the temporary buffer buf(1:sz_buf).
So far I have the following for the read subroutine (after removing error checks to keep it concise):
SUBROUTINE hdf5_read_array_d( fname, path, name, val, dims )
USE ISO_C_BINDING, ONLY: C_SIZE_T, C_LOC, C_F_POINTER
! Input arguments
CHARACTER(LEN=*), INTENT(IN) :: fname, path, name
REAL(KIND(1.D0)), TARGET, INTENT(OUT) :: val
INTEGER, DIMENSION(:), INTENT(IN) :: dims
! Internal variables
INTEGER(KIND=HID_T) :: h5root, h5path, h5dset
INTEGER(KIND=HSIZE_T), DIMENSION(SIZE(dims)) :: h5dims
REAL(KIND(1.D0)), DIMENSION(:), POINTER :: buf
INTEGER(KIND=C_SIZE_T) :: sz_buf
INTEGER :: dim
! Open the file in read-only mode
CALL h5fopen_f( TRIM(fname), H5F_ACC_RDONLY_F, h5root, ierr )
! Open the pre-existing path in the file as a group
CALL h5gopen_f( h5root, TRIM(path), h5path, ierr )
! Open the dataset
CALL h5dopen_f( h5path, TRIM(name), h5dset, ierr )
! Convert dims to HSIZE_T
h5dims(:) = dims(:)
! C pointer trickery: cast double -> void* -> double*
sz_buf = PRODUCT(dims)
ALLOCATE( buf( sz_buf ) )
CALL C_F_POINTER( C_LOC(val), buf, (/sz_buf/) )
! Read data from dataset through buffer
CALL h5dread_f( h5dset, H5T_NATIVE_DOUBLE, buf, h5dims, ierr )
! Clean up and close HDF5 file
NULLIFY(buf)
CALL h5dclose_f( h5dset, ierr )
CALL h5gclose_f( h5path, ierr )
CALL h5fclose_f( h5root, ierr )
RETURN
END SUBROUTINE hdf5_read_array_d
Now, the question is, do I need to also put in DEALLOCATE(buf) in addition to / in place of the NULLIFY(buf)?
Any help would be appreciated.
Note: I am aware that Fortran 2018 includes assumed-rank arrays val(..) that will elegantly solve this problem. But again, it's a newer feature that might not be implemented by all compilers yet.
Edit: On C_F_POINTER(), here's a screenshot of Metcalf, Reid, and Cohen (4th Edition, not the newest one that has Fortran 2018 stuff):
You can use C-style pointer trickery to do what you want, but you have some things to address in your approach:
you have a memory leak with allocate(buf)
you are (subtly) lying about the scalar nature of val
you'll horribly confuse anyone reading your code
The reason why this is horribly confusing, is because you don't need to do this trickery. That's also why I won't show you how to do it, or to address the question "do I need to deallocate as well as nullify?".
You know that you have an array val to stuff n values in, in a contiguous lump. You worry that that can't do that because you (without using an assumed-rank dummy) have to match array shape. Worry not.
integer :: a(2,2,2,2), b(4,2,2), c(4,4)
are all arrays with 16 elements. So is
integer :: d(16)
You can associate actual arguments a, b and c with dummy argument d. Let's see that in action:
implicit none
integer :: a(2,2,2,2), b(4,2,2), c(4,4)
call set_them(a, SHAPE(a))
call set_them(b, SHAPE(b))
call set_them(c, SHAPE(c))
print '(16I3)', a, b, c
contains
subroutine set_them(d, dims)
integer, intent(in) :: dims(:)
integer, intent(out) :: d(PRODUCT(dims))
integer i
d=[(i,i=1,SIZE(d))]
end subroutine
end program
You can even associate array sections in this way to define portions.
You can see several other questions around here about this sequence association, in particular looking at changing shapes of arrays. This answer is more of a motivation of what to look for when tempted to do something complicated instead.
complex to real+imag part
Given a complex array we can assign pointers to its real and imaginary part following the answer in https://stackoverflow.com/a/54819542
program main
use iso_c_binding
implicit none
complex, target :: z(2) = [(1,2), (3,4)]
real, pointer :: re(:), im(:), buf(:)
call c_f_pointer(c_loc(z), buf, shape=[size(z)*2])
re(1:2) => buf(1::2)
im(1:2) => buf(2::2)
print *, 'z', z
print *, 're', re
print *, 'im', im
end program
real+imag part to complex
My question is how to do it the other way around?
Given two real arrays re,im. How can one assign a complex pointer z where the real part is given by re and imaginary part by im? Something similar to
program main
implicit none
real, target :: re(2)=[1,3], im(2)=[2,4]
complex, pointer :: z(:)
z => cmplx(re, im) ! doesnt work as cmplx doesnt return a pointer result
end program
Is it even possible as re,im are not always contiguous in memory?
I want to define a pointer to a subarray. For a simple range this is easily done by pointer => array(i:j), but I can't figure out how to do this for a map like k=[k1,k2,k3]. If I would define another array I could use a loop like array2=[(array1(k(j)),j=1,size(k,1))]. But it isn't possible to assign a pointer in a similar way (pointer => [(array1(k(j)),j=1,size(k,1))]) since the r.h.s. of the expression seems to define another variabel which then not even has the target attribute. For simple tasks, a trick around this, is to first assign a pointer to the total array an to use the map on the readout. But in my case this doesn't seem to be possible.
I will attach to examples: The first one shows what I described above. The second one is a more complicated example, where the trick doesn't work anymore. And in addition a two dimensional map is required.
Minimal example:
program test
integer, parameter :: n=10,n_k=3
real,target :: a(1:n)
real :: b(1:n_k)
integer :: k(1:n_k)
integer :: j
real,pointer :: p(:)
! fill array a and define map k:
a=[(real(j),j=1,n)]
k=[((j+1)*2,j=1,n_k)]
! can be used to print the arrays:
!write(*,*) a
!write(*,*) k
! can be used to write only the part of a defined by k:
!write(*,*) (a(k(j)),j=1,n_k)
! this an similar things didn't work:
!p(1:n_k) => [(a(k(j)),j=1,n_k)]
! works, but not generally:
p => a
write(*,*) (p(k(j)),j=1,n_k)
! works, only for arrays:
b=(/(a(k(j)),j=1,n_k)/)
write(*,*) b
end program
More complicated (but also kind of minimal) example which shows (hopefully) the problem I really have. For an easy understanding some explanation leads through it. There are plenty of write commands to print the arrays. I appreciate for the amount of code, but I really don't see how to make a shorter and understandable working example:
module mod1
type base
real :: a
end type
type,extends(base) :: type1
end type
type,extends(base) :: type2
type(type1),allocatable :: b(:)
end type
type(type2),allocatable,target :: c(:)
contains
subroutine printer(z)
class(*),pointer,dimension(:) :: z
integer :: j,a_z,n_z
character(len=40) :: f,ff='(F10.2,1x))',form_z
! define format for printing:
a_z=lbound(z,1)
n_z=ubound(z,1)
write(f,'(I0)') (n_z-a_z+1)
form_z="("//trim(adjustl(f))//ff
! writing:
select type(z)
class is (base)
write(*,form_z) (z(j)%a,j=a_z,n_z)
end select
end subroutine
end module
program test
use mod1
integer,parameter :: n_b=8,n_c=6,n_js=3,n_ls=2
integer :: js(1:n_js),ls(1:n_ls)
integer :: j,l
class(*),pointer :: p(:)
character(len=40) :: f,ff='(F10.2,1x))',form_c,form_b
! define format for printing:
write(f,'(I0)') n_b
form_b="("//trim(adjustl(f))//ff
write(f,'(I0)') n_c
form_c="("//trim(adjustl(f))//ff
! creating and filling the arrays:
allocate(c(n_c))
c%a=[(2d0*real(j),j=1,n_c)]
do j=1,n_c
allocate(c(j)%b(n_b))
c(j)%b%a=[(real(l)*1d1**(j-1),l=1,n_b)]
end do
! write arrays to compare later:
write(*,form_c) c%a
write(*,*)
write(*,form_b) (c(j)%b%a,j=1,n_c)
write(*,*)
! denfining two maps (size and entries will be input in the final program):
js=[1,4,6]
ls=[2,7]
! using the maps to print only the desired entries:
write(*,*) (c(js(j))%a,j=1,n_js)
write(*,*)
write(*,*) ((c(js(j))%b(ls(l))%a,j=1,n_js),l=1,n_ls)
write(*,*)
! !!! here I want to use the maps as well, but so far I only know how to use ranges:
p => c(1:4)
call printer(p)
write(*,*)
p => c(2)%b(3:6)
call printer(p)
write(*,*)
end program
Edit:
Just for the record, I solved the problem now by using arrays of derived types including pointers and slightly changing the calling subroutines.
You cannot do this with pointer association (e.g. pointer1 => array1(vector_subscript). Section 7.2.2.2 of the Fortran 2008 standard that disallows this is:
R733 pointer-assignment-stmt is data-pointer-object [ (bounds-spec-list) ] => data-target
There are two other forms, but they do not match your use, nor would they change the outcome. Reading further:
R737 data-target is variable
C724 (R737) A variable shall have either the TARGET or POINTER attribute, and shall not be an array section with a vector subscript.
This is why you cannot perform the pointer association your are attempting. You can however work around this and with pointer allocation. See this code:
n_k = 3
k = [((j+1)*2,j=1,n_k)] ! a vector subscript
p => a(k) ! NOT OK. Violates C724
allocate(p(n_k)) ! Associate your pointer this way
p = a(k) ! This is OK.
write(*,*) p
Which yields (wrapped in your example program):
% ./ptrtest
4.00000000 6.00000000 8.00000000
This allocates p to be the proper size and then assigns from a with a vector subscript. This gets around the issue of directly associating p with a map of a. This snippet assumes the variables are declared and initialized per your example code. This shows that you can assign a vector subscript of an array to a pointer, but only one that is already associated, not during the association.
As noted in a comment to your Q, if you have a regular stride, you can make the pointer association directly. For your first test case, this would be equivalent and work:
p => a(4:2:8) ! Allocation to a strided array is allowed
If however, you have an irregular vector subscript then the method in this answer will be what you need to use to accomplish the pointer association.
Another workaround you can use is passing a pointer and the map to a procedure. Consider the following code:
program test
implicit none
integer, parameter :: nx = 10, nx_m = 3
integer,dimension(nx_m) :: x_map
integer :: i
real, dimension(nx),target :: a
real, dimension(:), pointer :: p
! initialize array
a = [(real(i*2),i=1,10)]
write (*,'(10(f5.1 x))') a
!define a map
x_map = [1, 9, 4]
! associate pointer
p => a
call print_map(p, x_map)
contains
subroutine print_map(apointer, map)
implicit none
real, dimension(:), pointer :: apointer
integer, dimension(:) :: map
write (*,*) apointer(map)
end subroutine print_map
end program test
In this case, p "knows" about a and the map of elements in a can be calculated in the caller. Rather than associating (=>) p as a map of a (which cannot be done), p is associated to a and the map passed along with it.
This code produces the output:
% ./ptrtest3
2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0
2.00000000 18.0000000 8.00000000
There are basically two ways to pass arrays to a subroutine in Fortran 90/95:
PROGRAM ARRAY
INTEGER, ALLOCATABLE :: A(:,:)
INTEGER :: N
ALLOCATE(A(N,N))
CALL ARRAY_EXPLICIT(A,N)
! or
CALL ARRAY_ASSUMED(A)
END PROGRAM ARRAY
SUBROUTINE ARRAY_EXPLICIT(A,N)
INTEGER :: N
INTEGER :: A(N,N)
! bla bla
END SUBROUTINE ARRAY_EXPLICIT
SUBROUTINE ARRAY_ASSUMED(A)
INTEGER, ALLOCATABLE :: A(:,:)
N=SIZE(A,1)
! bla bla
END SUBROUTINE ARRAY_ASSUMED
where you need an explicit interface for the second, usually through the use of a module.
From FORTRAN77, I'm used to the first alternative, and I read this is also the most efficient if you pass the whole array.
The nice thing with the explicit shape is that I can also call a subroutine and treat the array as a vector instead of a matrix:
SUBROUTINE ARRAY_EXPLICIT(A,N)
INTEGER :: N
INTEGER :: A(N**2)
! bla bla
END SUBROUTINE ARRAY_EXPLICIT
I wondered if there is a nice way to do that kind of thing using the second, assumed shape interface, without copying it.
See the RESHAPE intrinsic, e.g.
http://gcc.gnu.org/onlinedocs/gfortran/RESHAPE.html
Alternatively, if you want to avoid the copy (in some cases an optimizing compiler might be able to do a reshape without copying, e.g. if the RHS array is not used afterwards, but I wouldn't count on it), as of Fortran 2003 you can assign pointers to targets of different rank, using bounds remapping. E.g. something like
program ptrtest
real, pointer :: a(:)
real, pointer :: b(:,:)
integer :: n = 10
allocate(a(n**2))
a = 42
b (1:n, 1:n) => a
end program ptrtest
I was looking to do the same thing and came across this discussion. None of the solutions suited my purposes, but I found that there is a way to reshape an array without copying the data using iso_c_binding if you are using the fortran 2003 standard which current fortran 90/95 compilers tend to support. I know the discussion is old, but I figured I would add what I came up with for the benefit of others with this question.
The key is to use the function C_LOC to convert an array to an array pointer, and then use C_F_POINTER to convert this back into a fortran array pointer with the desired shape. One challenge with using C_LOC is that C_LOC only works for array that have a directly specified shape. This is because arrays in fortran with an incomplete size specification (i.e., that use a : for some dimension) include an array descriptor along with the array data. C_LOC does not give you the memory location of the array data, but the location of the descriptor. So an allocatable array or a pointer array don't work with C_LOC (unless you want the location of the compiler specific array descriptor data structure). The solution is to create a subroutine or function that receives the array as an array of fixed size (the size really doesn't matter). This causes the array variable in the function (or subroutine) to point to the location of the array data rather than the location of the array descriptor. You then use C_LOC to get a pointer to the array data location and C_F_POINTER to convert this pointer back into an array with the desired shape. The desired shape must be passed into this function to be used with C_F_POINTER. Below is an example:
program arrayresize
implicit none
integer, allocatable :: array1(:)
integer, pointer :: array2(:,:)
! allocate and initialize array1
allocate(array1(6))
array1 = (/1,2,3,4,5,6/)
! This starts out initialized to 2
print *, 'array1(2) = ', array1(2)
! Point array2 to same data as array1. The shape of array2
! is passed in as an array of intergers because C_F_POINTER
! uses and array of intergers as a SIZE parameter.
array2 => getArray(array1, (/2,3/))
! Change the value at array2(2,1) (same as array1(2))
array2(2,1) = 5
! Show that data in array1(2) was modified by changing
! array2(2,1)
print *, 'array(2,1) = array1(2) = ', array1(2)
contains
function getArray(array, shape_) result(aptr)
use iso_c_binding, only: C_LOC, C_F_POINTER
! Pass in the array as an array of fixed size so that there
! is no array descriptor associated with it. This means we
! can get a pointer to the location of the data using C_LOC
integer, target :: array(1)
integer :: shape_(:)
integer, pointer :: aptr(:,:)
! Use C_LOC to get the start location of the array data, and
! use C_F_POINTER to turn this into a fortran pointer (aptr).
! Note that we need to specify the shape of the pointer using an
! integer array.
call C_F_POINTER(C_LOC(array), aptr, shape_)
end function
end program
#janneb has already answered re RESHAPE. RESHAPE is a function -- usually used in an assignment statement so there will be a copy operation. Perhaps it can be done without copying using pointers. Unless the array is huge, it is probably better to use RESHAPE.
I'm skeptical that the explicit shape array is more efficient than the assumed shape, in terms of runtime. My inclination is to use the features of the Fortran >=90 language and use assumed shape declarations ... that way you don't have to bother passing the dimensions.
EDIT:
I tested the sample program of #janneb with ifort 11, gfortran 4.5 and gfortran 4.6. Of these three, it only works in gfortran 4.6. Interestingly, to go the other direction and connect a 1-D array to an existing 2-D array requires another new feature of Fortran 2008, the "contiguous" attribute -- at least according to gfortran 4.6.0 20110318. Without this attribute in the declaration, there is a compile time error.
program test_ptrs
implicit none
integer :: i, j
real, dimension (:,:), pointer, contiguous :: array_twod
real, dimension (:), pointer :: array_oned
allocate ( array_twod (2,2) )
do i=1,2
do j=1,2
array_twod (i,j) = i*j
end do
end do
array_oned (1:4) => array_twod
write (*, *) array_oned
stop
end program test_ptrs
You can use assumed-size arrays, but it can mean multiple layers of wrapper
routines:
program test
implicit none
integer :: test_array(10,2)
test_array(:,1) = (/1, 2, 3, 4, 5, 6, 7, 8, 9, 10/)
test_array(:,2) = (/11, 12, 13, 14, 15, 16, 17, 18, 19, 20/)
write(*,*) "Original array:"
call print_a(test_array)
write(*,*) "Reshaped array:"
call print_reshaped(test_array, size(test_array))
contains
subroutine print_reshaped(a, n)
integer, intent(in) :: a(*)
integer, intent(in) :: n
call print_two_dim(a, 2, n/2)
end subroutine
subroutine print_two_dim(a, n1, n2)
integer, intent(in) :: a(1:n1,1:*)
integer, intent(in) :: n1, n2
call print_a(a(1:n1,1:n2))
end subroutine
subroutine print_a(a)
integer, intent(in) :: a(:,:)
integer :: i
write(*,*) "shape:", shape(a)
do i = 1, size(a(1,:))
write(*,*) a(:,i)
end do
end subroutine
end program test
I am using ifort 14.0.3 and 2D to 1D conversion, I could use an allocatable array for 2D array and a pointer array for 1D:
integer,allocatable,target :: A(:,:)
integer,pointer :: AP(:)
allocate(A(3,N))
AP(1:3*N) => A
As #M.S.B mentioned, in case both A and AP have the pointer attribute, I had to use contiguous attribute for A to guarantee the consistency of the conversion.
Gfortran is a bit paranoid with interfaces. It not only wants to know the type, kind, rank and number of arguments, but also the shape, the target attribute and the intent (although I agree with the intent part). I encountered a similar problem.
With gfortran, there are three different dimension definition:
1. Fixed
2. Variable
3. Assumed-size
With ifort, categories 1 and 2 are considered the same, so you can do just define any dimension size as 0 in the interface and it works.
program test
implicit none
integer, dimension(:), allocatable :: ownlist
interface
subroutine blueprint(sz,arr)
integer, intent(in) :: sz
integer, dimension(0), intent(in) :: arr
! This zero means that the size does not matter,
! as long as it is a one-dimensional integer array.
end subroutine blueprint
end interface
procedure(blueprint), pointer :: ptr
allocate(ownlist(3))
ownlist = (/3,4,5/)
ptr => rout1
call ptr(3,ownlist)
deallocate(ownlist)
allocate(ownlist(0:10))
ownlist = (/3,4,5,6,7,8,9,0,1,2,3/)
ptr => rout2
call ptr(3,ownlist)
deallocate(ownlist)
contains
! This one has a dimension size as input.
subroutine rout1(sz,arr)
implicit none
integer, intent(in) :: sz
integer, dimension(sz), intent(in) :: arr
write(*,*) arr
write(*,*) arr(1)
end subroutine rout1
! This one has a fixed dimension size.
subroutine rout2(sz,arr)
implicit none
integer, intent(in) :: sz
integer, dimension(0:10), intent(in) :: arr
write(*,*) "Ignored integer: ",sz
write(*,*) arr
write(*,*) arr(1)
end subroutine rout2
end program test
Gfortran complains about the interface. Changing the 0 into 'sz' solves the problem four 'rout1', but not for 'rout2'.
However, you can fool gfortran around and say dimension(0:10+0*sz) instead of dimension(0:10) and gfortran compiles and gives the same
result as ifort.
This is a stupid trick and it relies on the existence of the integer 'sz' that may not be there. Another program:
program difficult_test
implicit none
integer, dimension(:), allocatable :: ownlist
interface
subroutine blueprint(arr)
integer, dimension(0), intent(in) :: arr
end subroutine blueprint
end interface
procedure(blueprint), pointer :: ptr
allocate(ownlist(3))
ownlist = (/3,4,5/)
ptr => rout1
call ptr(ownlist)
deallocate(ownlist)
allocate(ownlist(0:10))
ownlist = (/3,4,5,6,7,8,9,0,1,2,3/)
ptr => rout2
call ptr(ownlist)
deallocate(ownlist)
contains
subroutine rout1(arr)
implicit none
integer, dimension(3), intent(in) :: arr
write(*,*) arr
write(*,*) arr(1)
end subroutine rout1
subroutine rout2(arr)
implicit none
integer, dimension(0:10), intent(in) :: arr
write(*,*) arr
write(*,*) arr(1)
end subroutine rout2
end program difficult_test
This works under ifort for the same reasons as the previous example, but gfortran complains about the interface. I do not know how I can fix it.
The only thing I want to tell gfortran is 'I do not know the dimension size yet, but we will fix it.'. But this needs a spare integer arguemnt (or something else that we can turn into an integer) to fool gfortran around.
Context
The toy Fortran code posted below calls two pointer functions. That is, both functions return a pointer. In fact, they're both array pointers. They both attempt to do the same thing, which is to return an integer array pointer referencing an integer array having three elements, 1, 2, and 3. The first function uses the pointer assignment operator (=>) to point the function pointer to an allocatable array that holds the data. The second function allocates a block of dynamic memory directly, via the pointer, for storing the data. The calling program just prints the elements of the returned array(s).
Here's what I find odd.
If I point a to the result of function1, the results are not correct. The first element of a appears to be "clobbered": a has 0, 2, 3.
If I point b to the result of function2, the results are correct. b gets 1, 2, 3.
Stranger still, pointing b to the result of function2 after pointing a to function1 changes a such that it becomes correct. a then has 1, 2, 3.
Question
Why does this occur? More precisely, why does a pointer function that returns a pointer to an allocatable array clobber the first element of that array for the caller? More precisely still, why does pointing one pointer (b) produce a side-effect on another pointer (a), where the targets come from different functions that are written so as not to interact with each other at all?
Caveats
I get this behavior using the GNU Fortran compiler v.4.3.3, running an Intel laptop with Ubuntu (Jaunty). Your results may vary, which might be more interesting still. Finally, as always it could be operator error on my part, which would be interesting to me at least.
Code
program main
implicit none
integer, dimension(:), pointer :: a, b
integer :: i
a => function1()
b => function2()
do i = 1, 3
print *, a(i)
end do
! do i = 1, 3
! print *, b(i)
! end do
contains
function function1 ()
integer, dimension(:), allocatable, target :: array
integer, dimension(:), pointer :: function1
allocate(array(3))
array(1) = 1
array(2) = 2
array(3) = 3
function1 => array
end function function1
function function2 ()
integer, dimension(:), pointer :: function2
allocate(function2(3))
function2(1) = 1
function2(2) = 2
function2(3) = 3
end function function2
end program main
Variable array of function1 is a local variable -- because it is declared without the "save" attribute, it is not persistent and is undefined when the function exits. You assign the address of array to function1, "keeping" this address, but the address isn't meaningful once the variable becomes undefined after exiting from the function. A likely implementation is that array of function1 will be placed on the stack, and that area of the stack will be freed for other uses when function1 returns. But this is just a guess at a likely implementation -- the key point is that you aren't allowed to use the pointer value after the variable becomes undefined. Allocatable variables are automatically deallocated when they go out of scope unless you declare them with the "save" attribute.