I'm trying to integrate two Fortran 9x codes which contain data arrays with opposite array ordering. One code (I'll call it the old code) has an established library of subroutines and I am trying to take advantage of these with the other (new) code as efficiently as possible (i.e. not having to create temporary arrays just to reorder an array and pass it to a subroutine and then have to replace the old array with the new reordered result). For example,
Old code:
oldarray(1:n,1) -> variable 1 for n elements
oldarray(1:n,2) -> variable 2 for n elements
.. and so on
new code:
newarray(1,1:n) -> variable 1 for n elements
newarray(1,1:n) -> variable 2 for n elements
.. and so on
The variable indices do not necessarily relate between the two codes. If I only need one variable to pass to a procedure, I just pass newarray(1,1:n) and the procedure doesn't know the difference. However, if a procedure from the old code requires variables 1-6 of oldarray which might correspond to variables 2,6,8,1,4,3 (I just picked arbitrary numbers) of newarray, is it possible to create a pointer that I could pass to the procedure?
On a simpler side, would it be possible to just create pointer for the tranpose of the new array? As an example, pointer(1000,6) points to newarray(6,1000).
Note: It is not possible to rewrite the new code to use the same array ordering because both codes use an array ordering that best suits its loop structures which cannot be changed.
Also, I have very little experience with pointers. I know I can create a derived datatype which consists of an array of pointers but I don't think I would be able to pass that to a procedure in the manner required (I could be wrong as I also have very little experience with derived datatypes). The reference book I have (Fortran 95/2003 for Scientists and Engineers) only explores advanced applications of pointers in terms of linked lists and trees. I have also found little Fortran pointer information outside of what is covered in this book on the internet.
Thank you for your help.
I think the answer is no, you can't do this, and it wouldn't help anyway.
You can do all sorts of super-cool things with array pointers, with strides across arrays, etc, but I don't see on the face of it how you can change the order of the data.
So I could be wrong on this and it is possible, but then the question is: how would it help you? Presumably you want to user pointers to re-arrange the data without copying; but when you're passing such a thing around, the compiler is allowed to do copy-in, copy-out; eg, create a temporary array, copy the data in, pass it to the subroutine, and copy the data out upon return. And in fact that would almost certainly be the right thing to do in this case, performance-wise; that way the old code could be accessing memory in the fast order, and the transpose-copy could be done in a fast way, as well.
So I suspect the right way to treat this problem is to do the copy-in/copy-out approach yourself explicitly.
Related
From the Golang source code, they seem to follow a pretty standard implementation of hash tables (ie array of buckets). Based on this it seems that iteration should be deterministic for an unchanged map (ie iterate the array in order, then iterate within the buckets in order). Why do they make the iteration random?
TL;DR; They intentionally made it random starting with Go 1 to make developers not rely on it (to not rely on a specific iteration order which order may change from release-to-relase, from platform-to-platform, or may even change during a single runtime of an app when map internals change due to accommodating more elements).
The Go Blog: Go maps in action: Iteration order:
When iterating over a map with a range loop, the iteration order is not specified and is not guaranteed to be the same from one iteration to the next. Since the release of Go 1.0, the runtime has randomized map iteration order. Programmers had begun to rely on the stable iteration order of early versions of Go, which varied between implementations, leading to portability bugs. If you require a stable iteration order you must maintain a separate data structure that specifies that order.
Also Go 1 Release Notes: Iterating in maps:
The old language specification did not define the order of iteration for maps, and in practice it differed across hardware platforms. This caused tests that iterated over maps to be fragile and non-portable, with the unpleasant property that a test might always pass on one machine but break on another.
In Go 1, the order in which elements are visited when iterating over a map using a for range statement is defined to be unpredictable, even if the same loop is run multiple times with the same map. Code should not assume that the elements are visited in any particular order.
This change means that code that depends on iteration order is very likely to break early and be fixed long before it becomes a problem. Just as important, it allows the map implementation to ensure better map balancing even when programs are using range loops to select an element from a map.
Notable exceptions
Please note that the "random" order applies when ranging over the map using for range.
For reproducible outputs (for easy testing and other conveniences it brings) the standard lib sorts map keys in numerous places:
1. encoding/json
The json package marshals maps using sorted keys. Quoting from json.Marshal():
Map values encode as JSON objects. The map's key type must either be a string, an integer type, or implement encoding.TextMarshaler. The map keys are sorted and used as JSON object keys by applying the following rules, subject to the UTF-8 coercion described for string values above:
keys of any string type are used directly
encoding.TextMarshalers are marshaled
integer keys are converted to strings
2. fmt package
Starting with Go 1.12 the fmt package prints maps using sorted keys. Quoting from the release notes:
Maps are now printed in key-sorted order to ease testing. The ordering rules are:
When applicable, nil compares low
ints, floats, and strings order by <
NaN compares less than non-NaN floats
bool compares false before true
Complex compares real, then imaginary
Pointers compare by machine address
Channel values compare by machine address
Structs compare each field in turn
Arrays compare each element in turn
Interface values compare first by reflect.Type describing the concrete > - type and then by concrete value as described in the previous rules.
3. Go templates
The {{range}} action of text/template and html/template packages also visit elements in sorted keys order. Quoting from package doc of text/template:
{{range pipeline}} T1 {{end}}
The value of the pipeline must be an array, slice, map, or channel.
If the value of the pipeline has length zero, nothing is output;
otherwise, dot is set to the successive elements of the array,
slice, or map and T1 is executed. If the value is a map and the
keys are of basic type with a defined order, the elements will be
visited in sorted key order.
This is important for security, among other things.
There are lots of resources talking about this online -- see this post for example
Sorry if my question seems stupid. My background is in PHP, Ruby, Python, Lua and similar languages, and I have no understanding of pointers in real-life scenarios.
From what I've read on the Internet and what I've got as responses in a question I asked (When is a pointer idiomatic?), I have understood that:
Pointers should be used when copying large data. Instead of getting the whole object hierarchy, receive its address and access it.
Pointers have to be used when you have a function on a struct that modifies it.
So, pointers seem like a great thing: I should just always get them as function arguments because they are so lightweight, and it's okay if I somehow end up not needing to modify anything on the struct.
However, looking at that statement intuitively, I can feel that it sounds very creepy, and yet I don't know why.
So, as someone who is designing a struct and its related functions, or just functions, when should I receive a pointer? When should I receive a value, and why?
In other words, when should my NewAuthor method return &Author{ ... }, and when should it return Author{ ... }? When should my function get a pointer to an author as an argument, and when should it just get the value (a copy) of type Author?
There's tradeoffs for both pointers and values.
Generally speaking, pointers will point to some other region of memory in the system. Be it the stack of the function that wants to pass a pointer to a local variable or some place on the heap.
func A() {
i := 25
B(&i) // A sets up stack frame to call B,
// it copies the address of i so B can look it up later.
// At this point, i is equal to 30
}
func B(i *int){
// Here, i points to A's stack frame.
// For this to execute, I look at my variable "i",
// see the memory address it points to, then look at that to get the value of 25.
// That address may be on another page of memory,
// causing me to have to look it up from main memory (which is slow).
println(10 + (*i))
// Since I have the address to A's local variable, I can modify it.
*i = 30
}
Pointers require me to de-reference them constantly whenever I was to see the data it points to. Sometimes you don't care. Other times it matters a lot. It really depends on the application.
If that pointer has to be de-referenced a lot (ie: you pass in a number to use in a bunch of different calcs), then you keep paying the cost.
Compared to using values:
func A() {
i := 25
B(i) // A sets up the stack frame to call B, copying in the value 25
// i is still 25, because A gave B a copy of the value, and not the address.
}
func B(i int){
// Here, i is simply on the stack. I don't have to do anything to use it.
println(10 + i)
// Since i here is a value on B's stack, modifications are not visible outside B's scpe
i = 30
}
Since there's nothing to dereference, it's basically free to use the local variable.
The down side of passing values happens if those values are large because copying data to the stack isn't free.
For an int it's a wash because pointers are "int" sized. For a struct, or an array, you are copying all the data.
Also, large objects on the stack can make the stack extra big. Go handles this well with stack re-allocation, but in high performance scenarios, it may be too much of an impact to performance.
There's a data safety aspect as well (can't modify something I pass by value), but I don't feel that is usually an issue in most code bases.
Basically, if your problem was already solvable by ruby, python or other language without value types, then these performance nuances don't super-matter.
In general, passing structs as pointers will usually do "the right thing" while learning the language.
For all other types, or things that you want to keep as read-only, pass values.
There are exceptions to that rule, but it's best that you learn those as needs arise rather than try to redefine your world all at once. If that makes sense.
Simply you can use pointers anywhere you want, sometimes you don't want to change your data. It may stand for abstract data, and you don't want to explicitly copy the data. Just pass by value and let compiler do its job.
Are using pointers the most memory/processor efficient way to pass structs for modification like below or is there a better way?
Basically there are 2 ways to pass parameters to a function or method: by value and by address (pointer).
Passing a parameter by value makes a copy of the passed value and so if you would modify it, it would modify the copy and not the original value. So if you want to modify the original value, that leaves you only the passing by address option.
Notes:
Note that you could also pass by value and return the modified copy and assign the returned modified value to the variable, but obviously this is less efficient - especially if the struct is large (contains many fields).
In rare cases there might be more efficient ways to pass a value for modification, but I would rather name these cases "denote" rather than pass. Let's assume you have a global variable, being a slice of structs. In this case you could just pass the index of the value in the slice you want to modify. And your function could just do the modification of the field of the element denoted by the passed index value. If you just want to modify 1 field, this may be faster and on 32-bit architecture the size of the index value may be smaller than the pointer, and this way you could spare the address taking and dereferencing operations (needs benchmark). But the usability of the function would drop drastically, so I don't recommend this.
That leaves passing by pointer the optimal way in case you need to modify it.
In Go, using pointers to pass structs for modification is the idiomatic way to do it. There are no reference variables as there are in other languages. So passing a pointer is the most efficient way to do it now and since it is idiomatic, will most likely be so in future versions of Go as well.
Do you have any multidimensional arrays tasks. I need some to practice.
My teacher used in some cases multidimensional array in dynamic programming. To store the temporary result it needs an array. It is a characteristic of the problem: the grade? and that requires the same level of multidimensional array. For eg if grade is 3 than you need to bypass an int[][][] to a resoveProblem() function.
If I would be teacher ever... I would search for a dynamic programming needed task and I would present the solution with multi array. Hopefully the students will realize the execution speed difference and will solve, exercise the matrices problems too.
Task2: you can invent a text why is needed to be stored the data in 1 dimension of array, for eg need to save data to file, than a bit imagination why need 2 dimensional array to use the loaded data in a virtual chessboard. Than they must do the 1 dimension to 2 dimension conversion
multiply matrices
Oops! Your answer couldn't be submitted because:
body must be at least 30 characters; you entered 17
I see a lot of functional programming related topics mention destructive updates. I understand that it is something similar to mutation, so I understand the update part. But what is the destructive part? Or am I just over-thinking it?
You're probably overthinking it a bit. Mutability is all there is to it; the only thing being "destroyed" is the previous value of whatever you mutated.
Say you're using some sort of search tree to store values, and you want to insert a new one. After finding the location where the new values goes, you have two options:
With an immutable tree, you construct new nodes along the path from the new value's location up to the root. Subtrees not along the path are reused in the new tree, and if you still have a reference to the original tree's root you can use both, with the common subtrees shared between them. This economizes on space with no extra effort if you have lots of slightly-different copies floating around, and of course you have all the usual benefits of immutable data structures.
With a mutable tree, you attach the new value where it belongs and that's that; nothing else has to be changed. This is almost always faster, and economizes on memory allocation if you only ever have one copy around, but anything that had a reference to the "old" tree now has a reference to the new one. The original has been destroyed; it's gone forever. If you need to keep the original around, you have to go to the expense of creating an entirely new copy of the whole thing before changing it.
If "destruction" seems an unnecessarily harsh way to describe a simple in-place update, then you've probably not spent as much time as I have debugging code in order to figure out where on Earth some value is being changed behind your back.
The imperative programming languages allow variables to be redefined, e.g
x = 1
x = 2
So x first has the value 1 then, later, it has the value 2. The second operation is an destructive update, because x looses its initial definition as being equal to 1.
This is not how definition is handled in common mathematics. Once defined, a variable keeps its value.
The above, seen as system of equations, would allow to subtract the first from the second equation, which would give
x - x = 2 - 1 <=> 0 = 1
which is a false statement. It is assumed that once introduced, x is the same.
A familiar statement like
x = x + 1
would lead to the same conclusion.
The functional languages have the same use of variables, once they are defined it is not possible to reassign them. The above statement would turn into
x2 = x + 1
and we would have no for or while loop but rather recursion or some higher order function.