Image processing in MPI - mpi

Question

This is my attempt to code the classical smoothing pixel average algorithm in MPI. I almost got it working but something weird happens with the halo exchange as can see the lines right in the edges. I can't seem to find the bug. Am I properly exchanging halos? What section of the final array should I gather?
https://pastebin.com/4rtFnSJ5
int next = rank + 1;
int prev = rank - 1;
if (next >= size) {
next = MPI_PROC_NULL;
}
if (prev < 0) {
prev = MPI_PROC_NULL;
}
int rows = y / px;
int cols = x;
int d = 1;
for (int iter = 0; iter < TotalIter; iter++) {
for (int i = 0; i < rows + 2; i++)
for (int j = 0; j < cols + 2; j++)
for (int k = 0; k < rgb; k++)
new[i][j * rgb + k] = 0;
for (int i = 1; i < rows + 1; i++) {
int iMin = -min(d, i - 1);
int iMax = min(d, (rows + 1 - i - 1));
for (int j = 1; j < cols + 1; j++) {
int jMin = -min(d, j - 1);
int iMax = min(d, (cols + 1 - j - 1));
int counter = 0;
for (int p = iMin; p <= iMax; p++)
for (int q = jMin; q <= jMax; q++) {
counter = counter + 1;
for (int k = 0; k < rgb; k++) {
new[i][j * rgb + k] += old[i + p][(j + q) * rgb + k];
}
}
for (int k = 0; k < rgb; k++) {
new[i][j * rgb + k] -= old[i][j * rgb + k];
new[i][j * rgb + k] /= (counter - 1);
}
}
}
for (int i = 2; i < rows; i++)
for (int j = 2; j < cols; j++)
for (int k = 0; k < rgb; k++) {
old[i][j * rgb + k] = new[i][j * rgb + k];
}
MPI_Sendrecv(&old[rows][1], cols * rgb, MPI_INT, next, 1, &old[0][1],
cols * rgb, MPI_INT, prev, 1, MPI_COMM_WORLD, &status);
MPI_Sendrecv(&old[1][1], cols * rgb, MPI_INT, prev, 2, &old[rows + 1][1],
cols * rgb, MPI_INT, next, 2, MPI_COMM_WORLD, &status);
}
for (int i = 1; i< rows+1; i++)
for (int j = 1; j< cols+1; j++)
for (int k = 0; k< rgb; k++) {
buf[i-1][(j-1)*rgb+k] = old[i][j*rgb+k] ;
}
MPI_Gather(&buf[0][0], rows *cols *rgb, MPI_INT, &Finalbuffer[0][0],
rows *cols *rgb, MPI_INT, 0, MPI_COMM_WORLD);
The output looks like this when run on 8 MPI processes. I can clearly see delimiting lines. For that reason I thought I was not doing halo exchanges properly.

Answer 1

OK, so there are a bunch of issues here.
First, your code could only ever work with d=1 since you only swap halos of depth 1. If you want to process neighbours of distance d, you need to swap halos of depth d.
Second, you do the first halo swap after your first sweep through the arrays so you are reading junk halo data on iteration 1 - you need to do a halo swap before you start processing your arrays.
Third, when you copy back new to old you start from index 2 : you need to include all the pixels from 1 to lrows and 1 to lcols.
Finally, your logic of Imin, Imax etc seems wrong. You don't want to truncate the range at the edges in the parallel program - you need to go off the edges to pick up the halo data. I just set Imin = -d, Imax = d etc.
With these fixes the code seems to run OK, i.e. there are no obvious halo effects, but it still gives different results on different numbers of processes.
PS I was also flattered to see you used the "arraymalloc2d" code from one of my own MPI examples - http://www.archer.ac.uk/training/course-material/2018/07/intro-epcc/exercises/cfd.tar.gz ; I'm glad to see that these training codes are proving useful to people!