i download the scilab sourcecode, i am interested how the conv2 works and want translate it to c# code, but i don't know what the meaning of "C2F(ddot)" and how it works. if i tranfer the "C2F(ddot)" into c or c# code how i should implement it. here are some piece of source code in scilab
extern double C2F(ddot)(int *n, double *A, int *iA, double *B, int *iB);
/*--------------------------------------------------------------------------*/
void conv2_separable_R(double *R, int nR, double *C, int mC, double *A, int mA, int nA, double *Out, int mOut, int nOut, int edgM, int edgN, double *T)
{
int ai = 0, tj = 0, ci = 0, rj = 0; /*current index over A,T,C and R */
int i = 0, j = 0; /* loop variables*/
int l = 0;
int one = 1, minusone = -1;
for (i = 0; i < mOut; i++ )
{
/*Compute the 1-D conv A(i,:) and C in T */
ai = Max(0, i - edgM) ;
ci = mC - 1 - Max(0, edgM - i);
l = Min(ci + 1, mA - ai);
for (j = 0; j < nA; j++ )
{
T[j] = C2F(ddot)(&l, A + ai + mA * j, &one, C + ci - l + 1, &minusone);
}
/*1-D convolution of T and R */
for (j = 0; j < nOut; j++ )
{
rj = nR - 1 - Max(0, edgN - j);
tj = Max(0, j - edgN) ;
l = Min(rj + 1, nA - tj);
Out[i + j * mOut] = C2F(ddot)(&l, T + tj, &one, R + rj - l + 1, &minusone);
}
}
}
if i want tranform the code:" T[j] = C2F(ddot)(&l, A + ai + mA * j, &one, C + ci - l + 1, &minusone);" into c or c# ,how i should do ?
In scilab's sources, C2F is a macro used to call Fortran function from C.
It's declared in machine.h.
Actually this code probably make a call to the Blass ddot function.
That it, the dot product of two vector is done in Fortran...
Related
im trying to solve a differentiel delay equation system with c++. Im a newbie in terms of coding, so please if you have recommendations, tell me, I would like to improve my writing! What i want to do: initialize the history-array and then start to solve the differential equation by overwriting the history-array. But the problem is, I get the error message:
terminate called after throwing an instance of 'std::out_of_range'
what(): vector::_M_range_check: __n (which is 9999) >= this->size() (which is 9999)
It seems that the history-arrays are out of range. I tried to put a std::cout in after the second if-condition to check if the code is going through the second for-loop, but he isn't. Since im learning c++ by doing right now, the problem isn't really clear to me. I hope someone sees the error. And dont hesitate to improve my code, I would really appreciate!
Thanks for your help!
#include <iostream>
#include <vector>
#include <cmath>
#include <iomanip>
#include <fstream>
const double pi = 3.14159265358979323846;
//delay
int tau = 1;
//initial values
double x = 1.0;
double y = 1.0;
double t = 0.0;
//constants and parameters
double K = 0.25;
double lam = 0.5;
double omega = pi;
double dx, dy;
//step-size
double dt = pow(10.0, -4.0);
//number of steps
int Delta = static_cast<int>(tau/dt);
std::vector<double> hist_x((static_cast<int>(tau/dt) - 1), 0.0);
std::vector<double> hist_y((static_cast<int>(tau/dt) - 1), 0.0);
std::vector<double> t_val;
std::vector<double> x_val;
std::vector<double> y_val;
double euler(double f, double di, double time_step){
f = f + time_step * di;
return f;
}
int main()
{
std::ofstream file_x;
std::ofstream file_y;
std::ofstream file_t;
file_x.open("x_val.txt");
file_y.open("y_val.txt");
file_t.open("t_val.txt");
for(int n = 0; n < 2; n++){
if(n==0){
for(int j; j < Delta; j++){
dx = lam * x + omega * x;
dy = lam * y - omega * x;
x = euler(x, dx, dt);
y = euler(y, dy, dt);
t = t + dt;
x_val.push_back(x);
y_val.push_back(y);
t_val.push_back(t);
hist_x.at(j) = x;
hist_y.at(j) = y;
file_x<<x_val.at(j)<<std::endl;
file_y<<y_val.at(j)<<std::endl;
file_t<<t_val.at(j)<<std::endl;
}
}
if(!(n==0)){
for(int k = 0; k < Delta; k++){
//f1(x,y)
dx = lam * x + omega * x - K * ( x - hist_x.at(k) );
//f2(x,y)
dy = lam * y - omega * x - K * ( y - hist_y.at(k) );
x = euler(x, dx, dt);
y = euler(y, dy, dt);
t = t + dt;
x_val.push_back(x);
y_val.push_back(y);
t_val.push_back(t);
hist_x.at(k) = x;
hist_y.at(k) = y;
file_x<<x_val.at(k + n * Delta)<<std::endl;
file_y<<y_val.at(k + n * Delta)<<std::endl;
file_t<<t_val.at(k + n * Delta)<<std::endl;
}
}
}
file_x.close();
file_y.close();
file_t.close();
}
for(int j; j < Delta; j++){
You forgot to initialize j; you meant:
for (int j = 0; j < Delta; j++)
{
int Delta = static_cast<int>(tau/dt);
std::vector<double> hist_x((static_cast<int>(tau/dt) - 1), 0.0);
std::vector<double> hist_y((static_cast<int>(tau/dt) - 1), 0.0);
You index from 0 to Delta−1, this means the vectors need to have Delta elements, and you allocate one less; correct:
std::vector<double> hist_x(Delta, 0.0);
std::vector<double> hist_y(Delta, 0.0);
In experimenting with PyOpenCL, I noticed my code was running slower than expected. It turned out that it ran faster on CPU than on GPU (running on PyOpenCL in both cases, achieving just 1 GFLOP).
To debug this, I then tried naive matrix multiplication as a comparison, and only see a 2x speedup on GPU vs CPU (~20 GFLOPs vs ~10 GFLOPs). My system is i7 8750H + GTX 1070 Max-Q.
Does anyone have any thoughts they could share about what I might be doing wrong? I know that the code below is not optimal, but I would have expected that with the much increased floating point capability and memory bandwidth of my GPU there would be a bigger difference.
import pyopencl as cl
import pyopencl.array as pycl_array
import numpy as np
import numpy.linalg as la
import time
size = 4000
m1 = np.random.normal(size = [size,size]).astype(np.float32)
m2 = np.random.normal(size = [size,size]).astype(np.float32)
ctx = cl.create_some_context(interactive=True)
queue = cl.CommandQueue(ctx)
a = pycl_array.to_device(queue, m1)
b = pycl_array.to_device(queue, m2)
res = pycl_array.empty_like(a)
prg = cl.Program(ctx, """
__kernel void multiplymatrices(const unsigned int size, __global const float * a,
__global const float * b, __global float * res) {
int i = get_global_id(0);
int j = get_global_id(1);
res[size * i + j] = 0;
for (int k = 0; k < size; k++)
{
res[size * i + j] += a[k + size * j] * b[i + size * k];
}
}
""").build()
t = time.time()
task = prg.multiplymatrices(queue, m1.shape, None, np.int32(size), a.data, b.data, res.data)
task.wait()
tot_time = time.time()-t
print("gflops", 2*size**3/(tot_time*1000**3))
Following the suggestion to use a local register to accumulate the results, I modified my code as follows, getting about 90 gflops at about 360 GB/s of memory bandwidth (which is the maximum bandwidth my GPU is capable of). Improving the gflops would require a more sophisticated matrix multiplication algorithm which reuses the same data stored in cache multiple times, but is outside the scope of this question.
__kernel void multiplymatrices(const unsigned int size, __global const float * a,
__global const float * b, __global float * res) {
int i = get_global_id(0);
int j = get_global_id(1);
float temp = 0;
for (int k = 0; k < size; k++)
{
temp += a[k + size * j] * b[i + size * k];
}
res[size * i + j] = temp;
}
EDIT: For those looking for an example of fast matrix multiplication, which showcases using local memory with workgroups as well as 2D register tiling, I have created the below based on the tutorial here. It gets 1.4 TFLOPs on my GPU.
prg4 = cl.Program(ctx, """
__kernel void multiplymatrices(const unsigned int size, __global const float * A,
__global const float * B, __global float * res) {
int ig = get_group_id(0);
int jg = get_group_id(1);
int il = get_local_id(0);
int jl = get_local_id(1);
const int memtile = 64;
const int regtile = 4;
volatile int il2;
volatile int jl2;
int iglob = memtile*ig + regtile*il;
int jglob = memtile*jg + regtile*jl;
__local float Asub[64][64];
__local float Bsub[64][64];
float acc[4][4];
float Areg;
float Breg[4];
for (int k = 0; k < regtile; k++) {
for (int m = 0; m < regtile; m++) {
acc[k][m] = 0;
}
}
for (int l = 0; l < size/memtile; l++) {
for (int k = 0; k < regtile; k++) {
for (int m = 0; m < regtile; m++) {
il2 = il*regtile + k;
jl2 = jl*regtile + m;
Asub[il2][jl2] = A[size*(iglob + k) + memtile*l + jl2];
Bsub[il2][jl2] = B[size*(memtile*l + il2) + jglob + m];
}
}
barrier(CLK_LOCAL_MEM_FENCE);
for (int k = 0; k < regtile; k++) {
for (int r = 0; r < regtile; r++) {
Breg[r] = Bsub[il*regtile+k][jl*regtile+r];
}
for (int m = 0; m < regtile; m++) {
Areg = Asub[il*regtile+m][jl*regtile+k];
for (int r = 0; r < regtile; r++) {
acc[k][m] += Areg*Breg[r];
}
}
}
}
for (int k = 0; k < regtile; k++) {
for (int m = 0; m < regtile; m++) {
res[size*(iglob+k)+jglob+m] = acc[k][m];
}
}
}
""").build()
t = time.time()
memtile = 64
regtile = 4
wgsize = int(memtile/regtile)
global_size = int(size/regtile)
task = prg4.multiplymatrices(queue, (global_size,global_size), (wgsize,wgsize), np.int32(size), a.data, b.data, res.data)
queue.finish()
tot_time = time.time()-t
print("gflops", 2*size**3/(tot_time*1000**3))
print("GB/s total", 2*4*size**3/(tot_time*1000**3))
print("GB/s global", 2*4*size**3/(memtile*tot_time*1000**3))
I'm new in open cl. And tried as my first work to write code that checks intersection between many polylines to single polygon.
I'm running the code in both cpu and gpu.. and get different results.
First I sent NULL as local parameter when called clEnqueueNDRangeKernel.
clEnqueueNDRangeKernel(command_queue, kIntersect, 1, NULL, &global, null, 2, &evtCalcBounds, &evtKernel);
After trying many things i saw that if i send 1 as local it is working good. and returning the same results for the cpu and gpu.
size_t local = 1;
clEnqueueNDRangeKernel(command_queue, kIntersect, 1, NULL, &global, &local, 2, &evtCalcBounds, &evtKernel);
Played abit more and found that the cpu returns false result when i run the kernel with local 8 or more (for some reason).
I'm not using any local memory, just globals and privates.
I didn't added the code because i think it is irrelevant to the problem (note that for single work group it is working good), and it is long. If it is needed, i will try to simplify it.
The code flow is going like this:
I have polylines coordinates stored in a big buffer. and the single polygon in another. In addition i'm providing another buffer with single int that holds the current results count. All buffers are __global arguments.
In the kernel i'm simply checking intersection between all the lines of the "polyline[get_global(0)]" with the lines of the polygon. If true,
i'm using atomic_inc for the results count. There is no read and write memory from the same buffer, no barriers or mem fences,... the atomic_inc is the only thread safe mechanism i'm using.
-- UPDATE --
Added my code:
I know that i can maybe have better use of open cl functions for calculating some vectors, but for now, i'm simply convert code from my old regular CPU single threaded program to CL. so this is not my concern now.
bool isPointInPolygon(float x, float y, __global float* polygon) {
bool blnInside = false;
uint length = convert_uint(polygon[4]);
int s = 5;
uint j = length - 1;
for (uint i = 0; i < length; j = i++) {
uint realIdx = s + i * 2;
uint realInvIdx = s + j * 2;
if (((polygon[realIdx + 1] > y) != (polygon[realInvIdx + 1] > y)) &&
(x < (polygon[realInvIdx] - polygon[realIdx]) * (y - polygon[realIdx + 1]) / (polygon[realInvIdx + 1] - polygon[realIdx + 1]) + polygon[realIdx]))
blnInside = !blnInside;
}
return blnInside;
}
bool isRectanglesIntersected(float p_dblMinX1, float p_dblMinY1,
float p_dblMaxX1, float p_dblMaxY1,
float p_dblMinX2, float p_dblMinY2,
float p_dblMaxX2, float p_dblMaxY2) {
bool blnResult = true;
if (p_dblMinX1 > p_dblMaxX2 ||
p_dblMaxX1 < p_dblMinX2 ||
p_dblMinY1 > p_dblMaxY2 ||
p_dblMaxY1 < p_dblMinY2) {
blnResult = false;
}
return blnResult;
}
bool isLinesIntersects(
double Ax, double Ay,
double Bx, double By,
double Cx, double Cy,
double Dx, double Dy) {
double distAB, theCos, theSin, newX, ABpos;
// Fail if either line is undefined.
if (Ax == Bx && Ay == By || Cx == Dx && Cy == Dy)
return false;
// (1) Translate the system so that point A is on the origin.
Bx -= Ax; By -= Ay;
Cx -= Ax; Cy -= Ay;
Dx -= Ax; Dy -= Ay;
// Discover the length of segment A-B.
distAB = sqrt(Bx*Bx + By*By);
// (2) Rotate the system so that point B is on the positive X axis.
theCos = Bx / distAB;
theSin = By / distAB;
newX = Cx*theCos + Cy*theSin;
Cy = Cy*theCos - Cx*theSin; Cx = newX;
newX = Dx*theCos + Dy*theSin;
Dy = Dy*theCos - Dx*theSin; Dx = newX;
// Fail if the lines are parallel.
return (Cy != Dy);
}
bool isPolygonInersectsPolyline(__global float* polygon, __global float* polylines, uint startIdx) {
uint polylineLength = convert_uint(polylines[startIdx]);
uint start = startIdx + 1;
float x1 = polylines[start];
float y1 = polylines[start + 1];
float x2;
float y2;
int polygonLength = convert_uint(polygon[4]);
int polygonLength2 = polygonLength * 2;
int startPolygonIdx = 5;
for (int currPolyineIdx = 0; currPolyineIdx < polylineLength - 1; currPolyineIdx++)
{
x2 = polylines[start + (currPolyineIdx*2) + 2];
y2 = polylines[start + (currPolyineIdx*2) + 3];
float polyX1 = polygon[0];
float polyY1 = polygon[1];
for (int currPolygonIdx = 0; currPolygonIdx < polygonLength; ++currPolygonIdx)
{
float polyX2 = polygon[startPolygonIdx + (currPolygonIdx * 2 + 2) % polygonLength2];
float polyY2 = polygon[startPolygonIdx + (currPolygonIdx * 2 + 3) % polygonLength2];
if (isLinesIntersects(x1, y1, x2, y2, polyX1, polyY1, polyX2, polyY2)) {
return true;
}
polyX1 = polyX2;
polyY1 = polyY2;
}
x1 = x2;
y1 = y2;
}
// No intersection found till now so we check containing
return isPointInPolygon(x1, y1, polygon);
}
__kernel void calcIntersections(__global float* polylines, // My flat points array - [pntCount, x,y,x,y,...., pntCount, x,y,... ]
__global float* pBounds, // The rectangle bounds of each polyline - set of 4 values [top, left, bottom, right....]
__global uint* pStarts, // The start index of each polyline in the polylines array
__global float* polygon, // The polygon i want to intersect with - first 4 items are the rectangle bounds [top, left, bottom, right, pntCount, x,y,x,y,x,y....]
__global float* output, // Result array for saving the intersections polylines indices
__global uint* resCount) // The result count
{
int i = get_global_id(0);
uint start = convert_uint(pStarts[i]);
if (isRectanglesIntersected(pBounds[i * 4], pBounds[i * 4 + 1], pBounds[i * 4 + 2], pBounds[i * 4 + 3],
polygon[0], polygon[1], polygon[2], polygon[3])) {
if (isPolygonInersectsPolyline(polygon, polylines, start)){
int oldVal = atomic_inc(resCount);
output[oldVal] = i;
}
}
}
Can anyone explain it to me ?
I am programming in OpenCL using a GeForce GT 610 card in Linux. My CPU and GPU double precision results are not consistent. I can post part of the code here, but I would first like to know whether anyone else has faced this problem. The difference between the GPU and CPU double precision results get pronounced when I run loops with many iterations. There is really nothing special about the code, but I can post it here if anyone is interested. Thanks a lot. Here is my code. Please excuse the __ and bad formatting as I am new here :) As you can see, I have two loops and my CPU code is essentially almost an identical version.
#ifdef cl_khr_fp64
#pragma OPENCL EXTENSION cl_khr_fp64 : enable
#elif defined(cl_amd_fp64)
#pragma OPENCL EXTENSION cl_amd_fp64 : enable
#else
#error "Double precision floating point not supported by OpenCL implementation."
#endif
__kernel void simpar(__global double* fp, __global double* fp1,
__global double* fp3, __global double* fp5,
__global double* fp6, __global double* fp7,
__global double* fp8, __global double* fp8Plus,
__global double* x, __global double* v, __global double* acc,
__global double* keBuf, __global double* peBuf,
unsigned int prntstps, unsigned int nprntstps, double dt
) {
unsigned int m,i,j,k,l,t;
unsigned int chainlngth=100;
double dxi, twodxi, dxipl1, dximn1, fac, fac1, fac2, fac13, fac23;
double ke,pe,tke,tpe,te,dx;
double hdt, hdt2;
double alpha=0.16;
double beta=0.7;
double cmass;
double peTemp;
nprntstps=1001;
dt=0.01;
prntstps=100;
double alphaby4=beta/4.0;
hdt=0.5*dt;
hdt2=dt*0.5*dt;
double Xlocal,Vlocal,Acclocal;
unsigned int global_id=get_global_id(0);
if (global_id<chainlngth){
Xlocal=x[global_id];
Vlocal=v[global_id];
Acclocal=acc[global_id];
for (m=0;m<nprntstps;m++){
for(l=0;l<prntstps;l++){
Xlocal =Xlocal+dt *Vlocal+hdt2*Acclocal;
x[global_id]=Xlocal;
barrier(CLK_LOCAL_MEM_FENCE);
Vlocal =Vlocal+ hdt * Acclocal;
barrier(CLK_LOCAL_MEM_FENCE);
j = global_id - 1;
k = global_id + 1;
if (j == -1) {
dximn1 = 0.0;
} else {
dximn1 = x[j];
}
if (k == chainlngth) {
dxipl1 = 0.0;
} else {
dxipl1 = x[k];
}
dxi = Xlocal;
twodxi = 2.0 * dxi;
fac = dxipl1 + dximn1 - twodxi;
fac1 = dxipl1 - dxi;
fac2 = dxi - dximn1;
fac13 = fac1 * fac1 * fac1;
fac23 = fac2 * fac2 * fac2;
Acclocal = alpha * fac + beta * (fac13 - fac23);
barrier(CLK_GLOBAL_MEM_FENCE);
Vlocal += hdt * Acclocal;
v[global_id]=Vlocal;
acc[global_id]=Acclocal;
barrier(CLK_GLOBAL_MEM_FENCE);
}
barrier(CLK_GLOBAL_MEM_FENCE);
tke = tpe = te = dx = 0.0;
ke=0.5*Vlocal*Vlocal;//Vlocal*Vlocal;
barrier(CLK_GLOBAL_MEM_FENCE);
fp6[(m*100)+global_id]=ke;
keBuf[global_id]=ke;
ke=0.0;
barrier(CLK_GLOBAL_MEM_FENCE);
j = global_id - 1;
k = global_id + 1;
if (j == -1) {
dximn1 = 0.0;
} else {
dximn1 = x[j];
}
if (k == chainlngth) {
dxipl1 = 0.0;
} else {
dxipl1 = x[k];
}
dxi = Xlocal;
twodxi = 2.0 * dxi;
fac = dxipl1 + dximn1 - twodxi;
fac1 = dxipl1 - dxi;
fac2 = dxi - dximn1;
fac13 = fac1 * fac1 * fac1;
fac23 = fac2 * fac2 * fac2;
Acclocal = alpha * fac + beta * (fac13 - fac23);
barrier(CLK_GLOBAL_MEM_FENCE);
Vlocal += hdt * Acclocal;
v[global_id]=Vlocal;
acc[global_id]=Acclocal;
barrier(CLK_GLOBAL_MEM_FENCE);
}
barrier(CLK_GLOBAL_MEM_FENCE);
tke = tpe = te = dx = 0.0;
ke=0.5*Vlocal*Vlocal;//Vlocal*Vlocal;
barrier(CLK_GLOBAL_MEM_FENCE);
fp6[(m*100)+global_id]=ke;
keBuf[global_id]=ke;
ke=0.0;
barrier(CLK_GLOBAL_MEM_FENCE);
j = global_id - 1;
k = global_id + 1;
if (j == -1) {
dximn1 = 0.0;
} else {
dximn1 = x[j];
}
if (k == chainlngth) {
dxipl1 = 0.0;
} else {
dxipl1 = x[k];
}
dxi = Xlocal;
twodxi = 2.0 * dxi;
fac = dxipl1 + dximn1 - twodxi;
fac1 = dxipl1 - dxi;
fac2 = dxi - dximn1;
fac13 = fac1 * fac1 * fac1;
fac23 = fac2 * fac2 * fac2;
Acclocal = alpha * fac + beta * (fac13 - fac23);
barrier(CLK_GLOBAL_MEM_FENCE);
Vlocal += hdt * Acclocal;
v[global_id]=Vlocal;
acc[global_id]=Acclocal;
barrier(CLK_GLOBAL_MEM_FENCE);
}
barrier(CLK_GLOBAL_MEM_FENCE);
tke = tpe = te = dx = 0.0;
ke=0.5*Vlocal*Vlocal;//Vlocal*Vlocal;
barrier(CLK_GLOBAL_MEM_FENCE);
fp6[(m*100)+global_id]=ke;
keBuf[global_id]=ke;
ke=0.0;
barrier(CLK_GLOBAL_MEM_FENCE);
if (global_id ==0){
for(t=0;t<100;t++)
tke+=keBuf[t];
}
barrier(CLK_GLOBAL_MEM_FENCE);
k = global_id-1;
if (k == -1) {
dx = Xlocal;
}else{
dx = Xlocal-x[k];
}
fac = dx * dx;
peTemp = alpha * 0.5 * fac + alphaby4 * fac * fac;
fp8[global_id*m]=peTemp;
if (global_id == 0)
tpe+=peTemp;
barrier(CLK_GLOBAL_MEM_FENCE);
cmass=0.0;
dx = -x[100-1];
fac = dx*dx;
pe=alpha*0.5*fac+alphaby4*fac*fac;
if (global_id==0){
fp8Plus[m]=pe;
tpe+=peBuf[0];
fp5[m*2]=i;
fp5[m*2+1]=cmass;
te=tke+tpe;
fp[m*2]=m;
fp[m*2+1]=te;
}
barrier(CLK_GLOBAL_MEM_FENCE);
//cmass /=100;
fp1[(m*chainlngth)+global_id]=Xlocal-cmass;
// barrier(CLK_GLOBAL_MEM_FENCE);
fp3[(m*chainlngth)+global_id]=Vlocal;
// barrier(CLK_GLOBAL_MEM_FENCE);
fp7[(m*chainlngth)+global_id]=Acclocal;
barrier(CLK_GLOBAL_MEM_FENCE);
}
}
}
This is somewhat expected behavior, actually.
On older x86 CPUs, floating point numbers are 80bits long (Intel's "long double"), and truncated to 64bit only when need be.
When SIMD units/instructions for floating point arithmetics arrived for x86 CPUs, floating point double precision became 64bit by default; however, 80bit is still possible, depending on your compiler settings. There's a lot to read about this out there: Wikipedia: Floating Point.
Check your compiler settings for OpenCL and host code on floating point "magic tricks", to get better agreement of your results. Calculate the absolute and relative error of your values and check if this error margin is safe for your application.
What are the best practices to consider when implementing an error function defined as
using an OpenCL kernel?
A, B and C are 3D float arrays and \delta is the Kronecker delta.
Typical values for (N, M) = (2, 7) or (N, M) = (3, 23).
The naive implementation (given below) is by several orders of magnitude slower than the CPU version.
Thanks,
T.
__kernel void cl_bilinear_alg(
__global float * A,
__global float * B,
__global float * C,
__global const int M,
__global const int N,
__global float * R)
{
int index = get_global_id(0);
int N2 = N * N;
int mat_offset = index * N2 * M;
float s1, s2, err = 0.0f;
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
for (int k = 0; k < N; ++k)
{
for (int l = 0; l < N; ++l)
{
for (int m = 0; m < N; ++m)
{
for (int n = 0; n < N; ++n)
{
s1 = (n == i) * (j == k) * (l == m);
s2 = 0;
for (int r = 0; r < M; ++r)
{
s2 += A[mat_offset + r * N2 + i * N + j] *
B[mat_offset + r * N2 + k * N + l] *
C[mat_offset + r * N2 + m * N + n];
}
err += (s2 - s1) * (s2 - s1);
}
}
}
}
}
}
R[index] = err;
}
UPDATE
The primary target is a Geforce GTX 570, though this could change in the future.
UPDATE2
After vectorizing the code, moving bits to local memory, unrolling some loops and passing precomputed Kronecker products explicitly to the kernel the code looks as follows:
__kernel void cl_bilinear_alg(__global const float * A,
__global const float * B,
__global const float * C,
__global const int N,
__global const int M,
__global const float * kron,
__global float * R)
{
__private int index = get_global_id(0);
__private int cM = ceil(M / 4.0f);
__private int N2 = N*N;
__private int N4 = N2*N2;
__private int mat_offset = index * N2 * M;
__private float s1, s2, err = 0;
__private float4 vzero = (float4) (0.0f, 0.0f, 0.0f, 0.0f);
__local float4 va[54], vb[54], vc[54];
for (int ij = 0, k = 0; ij < N2; ++ij)
{
int r = 0;
for (; r < M / 4; r += 4, ++k)
{
int idx0 = mat_offset + N2 * r + ij;
int idx1 = mat_offset + N2 * (r + 1) + ij;
int idx2 = mat_offset + N2 * (r + 2) + ij;
int idx3 = mat_offset + N2 * (r + 3) + ij;
va[k] = (float4) (A[idx0], A[idx1], A[idx2], A[idx3]);
vb[k] = (float4) (B[idx0], B[idx1], B[idx2], B[idx3]);
vc[k] = (float4) (C[idx0], C[idx1], C[idx2], C[idx3]);
}
if (M % 4)
{
float buffa[4] = {0}, buffb[4] = {0}, buffc[4] = {0};
for (; r < M; ++r)
{
int idx = mat_offset + N2 * r + ij;
buffa[r % 4] = A[idx];
buffb[r % 4] = B[idx];
buffc[r % 4] = C[idx];
}
va[k] = vload4(0, buffa);
vb[k] = vload4(0, buffb);
vc[k++] = vload4(0, buffc);
}
}
for (int ij = 0; ij < N2; ++ij)
{
for (int kl = 0; kl < N2; ++kl)
{
for (int mn = 0; mn < N2; ++mn)
{
s1 = kron[ij * N4 + kl * N2 + mn];
s2 = 0;
for (int r = 0; r < cM; ++r)
s2 += dot(va[cM * ij + r], mad(vb[cM * kl + r], vc[cM * mn + r], vzero));
//the most expensive line
err += (s2 - s1) * (s2 - s1);
}
}
}
R[index] = err;
}
By applying these changes a 4x speed increase was observed compared to the naive implementation. Furthermore, it was revealed that the most expensive line of all is the error update, i.e.
err += (s2 - s1) * (s2 - s1);
Any suggestions?
Typically you'd want to break some of those loops up... a lot...
- the outer loops become split over multiple workgroups, which run on their own compute unit (there are around 16 compute units per GPU, not many)
- the next few loops would be split over different threads within each workgroup
If you try to run all the calculations all at the same time, they will all try to load the data into memory at the same time, and this will simply thrash horribly. GPUs have very limited memory. Sure, the global memory sounds large enough, several gigabytes, but the global GPU memory is slow. You want to get the data into the local memory, which is per compute unit, and is of the order of 32-64KB, not much more than that.
You'd typically want to somehow divide your task into very small tasks, and do the following, for each workgroup:
load a chunk of memory from global memory into local memory
the whole workgroup warp of threads can participate in doing the copy, using coallesced access
do work on this memory, like doing some sums, and so on
write the results back to global memory
then, can either iterate a bit, or simply exit, and leave other workgroups to handle other bits of the work
On the CPU, the mathematical operations tend to be a major bottleneck, but on the GPU, generally the cores are mostly spinning uselessly, whilst waiting for data to gradually get to them, from global memory. Whatever you can do to optimize this process, prevent conflicting demands, and so on, will make the kernel significantly faster.