Related
I'm an amateur playing with discrete math. This isn't a
homework problem though I am doing it at home.
I want to solve ax + by = c for natural numbers, with a, b and c
given and x and y to be computed. I want to find all x, y pairs
that will satisfy the equation.
This has a similar structure to Bezout's identity for integers
where there are multiple (infinite?) solution pairs. I thought
the similarity might mean that the extended Euclidian algorithm
could help here. Below are two implementations of the EEA that
seem to work; they're both adapted from code found on the net.
Could these be adapted to the task, or perhaps can someone
find a more promising avenue?
typedef long int Int;
#ifdef RECURSIVE_EEA
Int // returns the GCD of a and b and finds x and y
// such that ax + by == GCD(a,b), recursively
eea(Int a, Int b, Int &x, Int &y) {
if (0==a) {
x = 0;
y = 1;
return b;
}
Int x1; x1=0;
Int y1; y1=0;
Int gcd = eea(b%a, a, x1, y1);
x = y1 - b/a*x1;
y = x1;
return gcd;
}
#endif
#ifdef ITERATIVE_EEA
Int // returns the GCD of a and b and finds x and y
// such that ax + by == GCD(a,b), iteratively
eea(Int a, Int b, Int &x, Int &y) {
x = 0;
y = 1;
Int u; u=1;
Int v; v=0; // does this need initialising?
Int q; // quotient
Int r; // remainder
Int m;
Int n;
while (0!=a) {
q = b/a; // quotient
r = b%a; // remainder
m = x - u*q; // ?? what are the invariants?
n = y - v*q; // ?? When does this overflow?
b = a; // A candidate for the gcd - a's last nonzero value.
a = r; // a becomes the remainder - it shrinks each time.
// When a hits zero, the u and v that are written out
// are final values and the gcd is a's previous value.
x = u; // Here we have u and v shuffling values out
y = v; // via x and y. If a has gone to zero, they're final.
u = m; // ... and getting new values
v = n; // from m and n
}
return b;
}
#endif
If we slightly change the equation form:
ax + by = c
by = c - ax
y = (c - ax)/b
Then we can loop x through all numbers in its range (a*x <= c) and compute if viable natural y exists. So no there is not infinite number of solutions the limit is min(c/a,c/b) ... Here small C++ example of naive solution:
int a=123,b=321,c=987654321;
int x,y,ax;
for (x=1,ax=a;ax<=c;x++,ax+=a)
{
y = (c-ax)/b;
if (ax+(b*y)==c) here output x,y solution somewhere;
}
If you want to speed this up then just iterate y too and just check if c-ax is divisible by b Something like this:
int a=123,b=321,c=987654321;
int x,y,ax,cax,by;
for (x=1,ax=a,y=(c/b),by=b*y;ax<=c;x++,ax+=a)
{
cax=c-ax;
while (by>cax){ by-=b; y--; if (!y) break; }
if (by==cax) here output x,y solution somewhere;
}
As you can see now both x,y are iterated in opposite directions in the same loop and no division or multiplication is present inside loop anymore so its much faster here first few results:
method1 method2
[ 78.707 ms] | [ 21.277 ms] // time needed for computation
75044 | 75044 // found solutions
-------------------------------
75,3076776 | 75,3076776 // first few solutions in x,y order
182,3076735 | 182,3076735
289,3076694 | 289,3076694
396,3076653 | 396,3076653
503,3076612 | 503,3076612
610,3076571 | 610,3076571
717,3076530 | 717,3076530
824,3076489 | 824,3076489
931,3076448 | 931,3076448
1038,3076407 | 1038,3076407
1145,3076366 | 1145,3076366
I expect that for really huge c and small a,b numbers this
while (by>cax){ by-=b; y--; if (!y) break; }
might be slower than actual division using GCD ...
UPDATE3: Problem is solved but I'm leaving the code here as-is for future reference--I've posted an answer below with the final state of the code in case people wanted to see the final product.
UPDATE2: Refactored to use R_alloc instead of calloc for automated cleanup. Unfortunately the problem persists.
UPDATE: If I add this line right before UNPROTECT(1):
Rprintf("%p %p %p", (void *)rans, (void *)fm, (void *)corrs);
then the function executes with no corrupted heap error. Maybe there's a background garbage collection call that corrupts one of the pointers prior to execution finishing, resulting in a write to a garbage pointer? Important to note here that if I don't print out all three of the pointer addresses, the error comes back.
Also I'm running this on an M1 Mac and compiling with clang via R CMD SHLIB, in case Apple silicon is to blame.
I'm at my wits end trying to debug this issue, and I figured I'd turn to SO for help. I'm writing a function in C to optimize some parts of my R code, and I'm getting a Heap Corruption Error when running the function many times. The function trimCovar() is called from R using the .Call("trimCovar", ...) interface.
I'm having a lot of difficulty debugging this for a few reasons:
I'm on OSX, so I can't use Valgrind
C function depends on inputs from R, so I can't debug the C code on its own
Heap corruption only occurs when calling the function many times within an R function
(just running .Call directly a bunch of times has no errors)
Error point is inconsistent
I start with two sets of vectors, and I condense them into a frequency matrix, where each column is a position in the vector set, and each row is a particular character that appears. I concatenate them into one matrix prior to passing in because it makes pre-processing easier. An toy example of the frequency matrix would be:
INPUT:
v1_1 = 101
v1_2 = 011
v2_1 = 111
v2_2 = 110
Frequency Matrix:
position: | 1_1 | 1_2 | 1_3 | 2_1 | 2_2 | 2_3 |
0: 0.5 0.5 0.0 0.0 0.0 0.5
1: 0.5 0.5 1.0 1.0 1.0 0.5
The goal is to find the NV highest correlated positions across the vector sets, which I do by calculating pairwise KL divergence of positions. These are stored in a linked list sorted in ascending order, and at the end I take the positions corresponding to the first NV entries. The R code I have can deparse everything else, so I really just need a vector of positions at the end (duplicates are allowed).
The function takes in 5 arguments:
fMAT: a frequency matrix (RObject, so gets read in as a flat vector)
fSP : columns in matrix corresponding to positions from the first vector set
sSP : same as fSP but for second vector set
NV : Number of values to return
NR : Number of columns in fMAT
The error returned is:
R(95564,0x104858580) malloc: Heap corruption detected, free list is damaged at 0x600000f10040
*** Incorrect guard value: 4626885667169763328
R(95564,0x104858580) malloc: *** set a breakpoint in malloc_error_break to debug
This only happens when I run an R function that calls this 10+ times, so I'm assuming that I'm just missing one or two small hanging pointers corrupting a memory reference. I've tried running this with gc() called in R immediately after each call, but it doesn't fix the problem. I'm not really sure what else to do at this point, I've tried using lldb but I'm not really sure how to use that program. From running lots of print statements I've determined that it usually crashes in the main loop (identified in code below), but it's inconsistent on when it crashes. I've also tried saving off erroneous inputs--I can rerun them individually with no issues, so it must be something relatively small that only appears over many runs.
Happy to provide more details if it would help. Code is listed at the bottom.
The only thing being allocated here are linked list nodes, and I thought I had free()'d them all prior to returning. I've also double checked the input values, so I'm 99.99% sure that I'm never referencing out of bounds on firstSeqPos, secondSeqPos, ans, or fm. I've also triple checked the R code surrounding this and can confidently say it is not the source of this error.
I haven't coded in C in a long time so I feel like I'm missing something obvious. If I really have to I can try to get ahold of a Linux box to run valgrind, but if there's another option I'd prefer it. Thanks in advance!
Code:
#include <R.h>
#include <Rdefines.h>
#include <Rinternals.h>
#include <math.h>
#include <stdlib.h>
#include <stdbool.h>
typedef struct node {
double data;
int i1;
int i2;
struct node *next;
} node;
// Linked list
// data is the correlation value,
// i1 the position from first vector set,
// i2 the position from second vector set
node *makeNewNode(double data, int i1, int i2){
node *newNode;
newNode = (node *)R_alloc(1, sizeof(node));
newNode->data = data;
newNode->i1 = i1;
newNode->i2 = i2;
newNode->next = NULL;
return(newNode);
}
//insert link in sorted order (ascending)
void insertSorted(node **head, node *toInsert, int maxSize) {
int ctr = 0;
if ((*head) == NULL || (*head)->data >= toInsert->data){
toInsert->next = *head;
*head = toInsert;
} else {
node *temp = *head;
while (temp->next != NULL && temp->next->data < toInsert->data){
temp = temp->next;
if (ctr == maxSize){
// Performance optimization, if we aren't inserting in the first NR
// positions then we can just skip since we only care about the NR
// lowest scores overall
return;
}
ctr += 1;
}
toInsert->next = temp->next;
temp->next = toInsert;
}
}
// MAIN FUNCTION CALLED FROM R
// (This is the one that crashes)
SEXP trimCovar(SEXP fMAT, SEXP fSP, SEXP sSP, SEXP NV, SEXP NR){
// Converting input SEXPs into C-compatible values
int nv = asInteger(NV);
int nr = asInteger(NR);
int sp1l = length(fSP);
int sp2l = length(sSP);
int *firstSeqPos = INTEGER(coerceVector(fSP, INTSXP));
int *secondSeqPos = INTEGER(coerceVector(sSP, INTSXP));
double *fm = REAL(fMAT);
int colv1, colv2;
// Using a linked list for efficient insert
node *corrs = NULL;
int cv1, cv2;
double p1, p2, score=0;
// USUALLY FAILS IN THIS LOOP
for ( int i=0; i<sp1l; i++ ){
cv1 = firstSeqPos[i];
colv1 = (cv1 - 1) * nr;
for ( int j=0; j<sp2l; j++ ){
cv2 = secondSeqPos[j];
colv2 = (cv2 - 1) * nr;
// KL Divergence
score = 0;
for ( int k=0; k<nr; k++){
p1 = fm[colv1 + k];
p2 = fm[colv2 + k];
if (p1 != 0 && p2 != 0){
score += p1 * log(p1 / p2);
}
}
// Add result into LL
node *newNode = makeNewNode(score, cv1, cv2);
insertSorted(&corrs, newNode, nv);
}
R_CheckUserInterrupt();
}
SEXP ans;
PROTECT(ans = allocVector(INTSXP, 2*nv));
int *rans = INTEGER(ans);
int ctr=0;
int pos1, pos2;
node *ptr = corrs;
for ( int i=0; i<nv; i++){
rans[2*i] = ptr->i1;
rans[2*i+1] = ptr->i2;
ptr = ptr->next;
}
UNPROTECT(1);
return(ans);
}
int *firstSeqPos = INTEGER(coerceVector(fSP, INTSXP));
int *secondSeqPos = INTEGER(coerceVector(sSP, INTSXP));
This is not good. The SEXPs returned by the 2 calls to coerceVector() need to be protected. However it's usually considered better practice to do this coercion at the R level right before entering the .Call entry point. Note that if fSP and sSP are integer matrices, there's no need to coerce them to integer as they are already seen as integer vectors at the C level. This also avoids a possibly expensive copy (as.integer() in R and coerceVector() in C both trigger a full copy of the matrix data).
The question was answered above, but I received a couple messages from people asking for the final code, so I'm going to include it as an answer to preserve the original question. There's a couple optimizations here (thanks to #hpages for help and troubleshooting regarding these):
Original code fails because the output of coerceVector() wasn't protected with PROTECT(). I've refactored the R code to check for integer inputs prior to calling this C function to avoid this function call and be more efficient with memory (see the accepted answer for more details).
Original code uses R_alloc(), which gives responsibility to R to clean up memory at the end of the function call. However, this introduces substantial memory overhead during the runtime of the function, since memory allocated to nodes not inserted into the linked list aren't cleared until the end of the function call.
Allocation with calloc() isn't as simple as switching over and calling free() at the end of the function, since we have to guard the case where the user interrupts execution of the program. If an interrupt signal is thrown prior to the end of the function, we'll never free the memory.
Final C Code:
#include <R.h>
#include <Rdefines.h>
#include <Rinternals.h>
#include <math.h>
#include <stdlib.h>
#include <stdbool.h>
typedef struct node {
double data;
int i1;
int i2;
struct node *next;
} node;
// Defining the head as a static so that we can access it globally
// Important for ensuring clean up in case of interrupt
static node *corrs = NULL;
// Function to clean up memory allocations in case of interrupt
void cleanupFxn(){
node *ptr = corrs;
// Free allocated memory in linked list
while (corrs != NULL){
ptr = corrs;
corrs = corrs->next;
free(ptr);
}
}
node *makeNewNode(double data, int i1, int i2){
node *newNode;
// very important to use calloc here so we have control of when we free it
// R_alloc() memory won't be freed until after function finishes execution
newNode = (node *)calloc(1, sizeof(node));
newNode->data = data;
newNode->i1 = i1;
newNode->i2 = i2;
newNode->next = NULL;
return(newNode);
}
// insert link in sorted order
// returns a bool corresponding to if we inserted
bool insertSorted(node **head, node *toInsert, int maxSize) {
int ctr = 0;
if ((*head) == NULL || (*head)->data >= toInsert->data){
toInsert->next = *head;
*head = toInsert;
return(true);
} else {
node *temp = *head;
while (temp->next != NULL && temp->next->data < toInsert->data){
temp = temp->next;
if (ctr == maxSize){
// Performance optimization, if we aren't inserting in the first NR
// positions then we can just skip since we only care about the NR
// lowest scores overall. Saves a huge amount of time and memory.
return(false);
}
ctr += 1;
}
toInsert->next = temp->next;
temp->next = toInsert;
return(true);
}
}
SEXP trimCovar(SEXP fMAT, SEXP fSP, SEXP sSP, SEXP NV, SEXP NR){
// Converting inputs into C-compatible forms
int nv = asInteger(NV);
int nr = asInteger(NR);
int sp1l = length(fSP);
int sp2l = length(sSP);
// Note here we're not using coerceVector() anymore
// typechecking done on R side
int *firstSeqPos = INTEGER(fSP);
int *secondSeqPos = INTEGER(sSP);
double *fm = REAL(fMAT);
int colv1, colv2;
// Using a linked list for efficient insert
corrs = NULL;
int cv1, cv2;
double p1, p2, score=0;
bool success;
for ( int i=0; i<sp1l; i++ ){
cv1 = firstSeqPos[i];
colv1 = (cv1 - 1) * nr;
for ( int j=0; j<sp2l; j++ ){
cv2 = secondSeqPos[j];
colv2 = (cv2 - 1) * nr;
score = 0;
for ( int k=0; k<nr; k++){
p1 = fm[colv1 + k];
p2 = fm[colv2 + k];
if (p1 != 0 && p2 != 0){
score += p1 * log(p1 / p2);
}
}
node *newNode = makeNewNode(score, cv1, cv2);
success = insertSorted(&corrs, newNode, nv);
// If we don't insert, free the associated memory
// I'm checking for NULL here just out of an abundance of caution
if (!success && newNode != NULL){
free(newNode);
newNode = NULL;
}
}
R_CheckUserInterrupt();
}
SEXP ans;
PROTECT(ans = allocVector(INTSXP, 2*nv));
int *rans = INTEGER(ans);
node *ptr=corrs;
for ( int i=0; i<nv; i++){
rans[2*i] = ptr->i1;
rans[2*i+1] = ptr->i2;
ptr = ptr->next;
}
// Free allocated memory in linked list
cleanupFxn();
UNPROTECT(1);
return(ans);
}
Assuming the C file is named trimCovar.c, we'd compile with R CMD SHLIB trimCovar.c.
R Code to run this function:
dyn.load("trimCovar.so")
# Wrapped into a function with on.exit(...) to ensure cleanup
# in the event the user or system interrupts execution early
CorrComp_C <- function(fm, fsp, ssp, nv, nr){
# type checking to ensure input to C is integer vector
# (could probably do more type checking here, mainly for illustration)
stopifnot(is(fsp, 'integer'))
stopifnot(is(ssp, 'integer'))
on.exit(.C("cleanupFxn"))
a <- .Call('trimCovar', fm, fsp, ssp, nv, nr)
return(a)
}
https://play.golang.org/p/DOhYaiH53Ek
I do not understand the *&p operation nor how a pointer *p which is an int() value NOT a memory address is able to modify its value i.e. valueX = valueY. I know that I must be misunderstanding something, but this code literally appears to be self-conflicting.
//point
package main
import "reflect"
var pt int = 27
func main() {
println(reflect.TypeOf(pt))
println("pt = ", pt) //value of pt
println("&pt = ", &pt) //memory address of pt
updatePointer(&pt)
println("pt = ", pt) //value of pt
}
func updatePointer(p *int) { //requires memory address of an int
println("&p = ", &p) //memory address of p
println("p = ", p) //memory address of pt
println("*p before = ", *p) //value of pt
println(*p == 27) //true
*p = 14 //27 = 14??????????
println("*p =", *p) //value of pt
println(reflect.TypeOf(&pt) == reflect.TypeOf(*&p)) //true!!!?????
println("*&p = ", *&p) //memory address which p's memory address evals to???? 0x800 (p) -> 0x900 (pt) = 0x800 (p)?
}
/*
Why can't I do the following?
func updatePointer(p *int){
p = 14
//OR
&p = 14
//OR
*&p = 14
}
*/
Considering *&p firstly lets simplify the example (removing all of the irrelevant stuff)
var pt int = 27
p := &pt // because &pt is passed as the argument into updatePointer
*p = 14 // This assigns 14 to whatever p points to (i.e. pt)
println(reflect.TypeOf(&pt) == reflect.TypeOf(*&p))
*& does nothing (and I can see no reason to use it in a real application); the statement gets the address of p (the &p bit) and then gets what the result points at (p). So this an be rewritten
println(reflect.TypeOf(&pt) == reflect.TypeOf(p))
p is pointing to pt so p == &pt (by definition this means that *p == pt). This means they are the same thing so will, of course, have the same type.
So why does *p = 14 work? You said that "*p which is an int() value NOT a memory address is able to modify its value" but that is not quite what the spec says:
For an operand x of pointer type *T, the pointer indirection *x
denotes the variable of type T pointed to by x. If x is nil, an
attempt to evaluate *x will cause a run-time panic.
So what *p = 14 is saying is set the variable that p points to to 14.
Now lets look at your second question:
// Why can't I do the following?
func updatePointer(p *int){
p = 14
//OR
&p = 14
//OR
*&p = 14
}
So p is a pointer to an integer (thats what *int means). Saying p = 14 is an attempt to set the pointer (not the int) to the value 14. 14 is an int not a pointer hence the compiler error cannot use 14 (type int) as type *int in assignment.
&p = 14 is saying set the address of p to 14. p is a *int so getting the address will give you a **int (a pointer to a pointer to an int). The compiler error you will get is cannot assign to &p and this is because the result of &p is not addressable. You could get around that by saying
x := &p
x = 14
and this will give you the error you might expect based upon what I said above: cannot use 14 (type int) as type **int in assignment.
*&p = 14 is basically the same as saying p=14 (it gets the address of p and then gets whatever the result points at whis will be p).
Pointers can get quite confusing (especially in contrived examples) and this article may help you understand.
Simplify. For example,
package main
import "fmt"
func f(q *int) {
fmt.Println(*q, q, &q, "f")
*q = 14
fmt.Println(*q, q, &q, "f")
}
func main() {
var i int = 27
var p *int = &i
fmt.Println(*p, p, &p, i, "main")
f(p)
fmt.Println(*p, p, &p, i, "main")
}
Output:
27 0x40e020 0x40c138 27 main
27 0x40e020 0x40c148 f
14 0x40e020 0x40c148 f
14 0x40e020 0x40c138 14 main
Errors:
/*
func g(r *int) {
// cannot use 14 (type int) as type *int in assignment
r = 14
// cannot assign to &r
&r = 14
// cannot use 14 (type int) as type *int in assignment
*(&r) = 14
}
*/
Playground: https://play.golang.org/p/Hwe3anFBTfD
In Go, all arguments are passed by value, as if by assignment. A pointer p to an int i is passed by value (q = p) to function f. The pointer q, a copy of p, is used to modify the value of i, *q = 14, *q dereferences type *int to type int.
The function g compiler error messages explain why the statements are illegal. For example, *&r = 14 is *(&r) = 14 is r = 14, cannot use 14 (type int) as type *int in assignment, r is type *int.
References:
A Tour of Go
The Go Programming Language Specification
struct a{
double array[2][3];
};
struct b{
double array[3][4];
};
void main(){
a x = {{1,2,3,4,5,6}};
b y = {{1,2,3,4,5,6,7,8,9,10,11,12}};
}
I have two structs, inside which there are two dim arrays with different sizes. If I want to define only one function, which can deal with both x and y (one for each time), i.e., the function allows both x.array and y.array to be its argument. How can I define the input argument? I think I should use a pointer.... But **x.array seems not to work.
For example, I want to write a function PrintArray which can print the input array.
void PrintArray( ){}
What should I input into the parenthesis? double ** seems not work for me... (we can let dimension to be the PrintArray's argument as well, telling them its 2*3 array)
Write a function that takes three parameters: a pointer, the number of rows, and the number of columns. When you call the function, reduce the array to a pointer.
void PrintArray(const double *a, int rows, int cols) {
int r, c;
for (r = 0; r < rows; ++r) {
for (c = 0; c < cols; ++c) {
printf("%3.1f ", a[r * cols + c]);
}
printf("\n");
}
}
int main(){
struct a x = {{{1,2,3},{4,5,6}}};
struct b y = {{{1,2,3,4},{5,6,7,8},{9,10,11,12}}};
PrintArray(&x.array[0][0], 2, 3);
PrintArray(&y.array[0][0], 3, 4);
return 0;
}
Im trying to do a merge sort in cpp on a vector called x, which contains x coordinates. As the mergesort sorts the x coordinates, its supposed to move the corresponding elements in a vector called y, containing the y coordinates. the only problem is that i dont know how to (or if i can) return both resulting vectors from the merge function.
alternatively if its easier to implement i could use a slower sort method.
No, you cannot return 2 results from a method like in this example.
vector<int>, vector<int> merge_sort();
What you can do is pass 2 vectors by reference to a function and the resultant mergesorted vector affects the 2 vectors...e.g
void merge_sort(vector<int>& x, vector<int>& y);
Ultimately, you can do what #JoshD mentioned and create a struct called point and merge sort the vector of the point struct instead.
Try something like this:
struct Point {
int x;
int y;
operator <(const Point &rhs) {return x < rhs.x;}
};
vector<Point> my_points.
mergesort(my_points);
Or if you want to sort Points with equal x value by the y cordinate:
Also, I thought I'd add, if you really ever need to, you can alway return a std::pair. A better choice is usually to return through the function parameters.
operator <(const Point &rhs) {return (x < rhs.x || x == rhs.x && y < rhs.y);}
Yes, you can return a tuple, then use structured binding (since C++17).
Here's a full example:
#include <cstdlib>
#include <iostream>
#include <numeric>
#include <tuple>
#include <vector>
using namespace std::string_literals;
auto twoVectors() -> std::tuple<std::vector<int>, std::vector<int>>
{
const std::vector<int> a = { 1, 2, 3 };
const std::vector<int> b = { 4, 5, 6 };
return { a, b };
}
auto main() -> int
{
auto [a, b] = twoVectors();
auto const sum = std::accumulate(a.begin(), a.end(), std::accumulate(b.begin(), b.end(), 0));
std::cout << "sum: "s << sum << std::endl;
return EXIT_SUCCESS;
}
You can have a vector of vectors
=> vector<vector > points = {{a, b}, {c, d}};
now you can return points.
Returning vectors is most probably not what you want, as they are copied for this purpose (which is slow). Have a look at this implementation, for example.