Global memory for work item cache - opencl

Question

Due to to local memory limitations, I need to use global memory as cache for my work items.
Suppose I have 1000 work groups with 64 work items each. Each item needs 4K cache. Cache doesn't need to persist after work item completes.
I will allocate one single global memory buffer and assign 4K pieces to the work items.
(I am targeting AMD GPUs)
What is the minimum size I would need to guarantee that there would not be
any concurrency issues between work items?
Since AMD has <= 64 CUs, my guess is
64 * 128 * 4000 bytes, and use (global work item ID % (64*128))
to assign a cache chunk to a work item.

Answer 1

If each cache item(accessed by global work item ID % (64*128)) is a struct 4000 bytes-long, and if implementation doesn't force each struct to be aligned on 4096 bytes, and if cache line size is not an exact divisor of 4000 and if global memory banks stride length is not exact divisor of 4000, then it shouldn't be a problem.
Profiled this kernel with codexl,(took 0.5s for 16k workitems):
__kernel void test(__global float * a)
{
int i=get_global_id(0)*4096;
for(int j=0;j<4096;j++)
a[i+j]*=2.0f;
}
and some of the output:
mem units stalled %55
cache hit %45
mem unit busy %99
valu busy %0.05
then changed the kernel to interleaved type(executed in 0.25s):
__kernel void test(__global float * a)
{
int i=get_global_id(0);
for(int j=0;j<4096;j++)
a[i+j*4096*4]*=2.0f;
}
mem units stalled %57
cache hit %47
mem unit busy %84
valu busy %1.5
so interleaved mode puts less strain on mem unit and hits cache a tad more often and ALU parts get fed more often and finishes %50 quicker.
Then tried this:
__kernel void test(__global float * a)
{
int i=get_global_id(0)*4100;
for(int j=0;j<4100;j++)
a[i+j]*=2.0f;
}
this took 0.37s, %30 faster than 4096 version but with higher mem unit stalls(the end point non-alignment must have caused this to waste some cycles on unnecessary data-fetches) and cache hit reduced to %37.
Test GPU is R7-240
Last test with structs:
typedef struct test_struct
{
float test_field[4096];
}strr;
__kernel void test(__global strr * a)
{
int i=get_global_id(0);
for(int j=0;j<4096;j++)
a[i].test_field[j]*=2.0f;
}
this completed in 0.53 seconds and had similar profiling data as strided kernel at the beginning.
empty kernel executes in 0.25 seconds so its not loading whole struct with this. Only needed elements are read.
Profiling for interleaved group-centric global accesses:
typedef struct test_struct
{
float test_field[4096];
}strr;
__kernel void test(__global strr * a)
{
int iLocal=get_local_id(0);
int iGroup=get_group_id(0);
for(int j=0;j<64;j++)
a[iGroup].test_field[iLocal+j*64]*=2.0f;
}
0.25s again so it is as fast as it can be.
Cache hit: %44
Mem unit busy: %82
Mem unit stalled: %67
Valu busy: %0.9
so it has best conditions of all, even without caching.