How do I apply a 2d shader to a webgl canvas in js? - 2d

I hope I'm not taking the wrong approach here but I feel like I'm on the right track and this shouldn't be too complicated. I want to take a simple function of x and y on the screen and return a color applied to each pixel of a webGL canvas.
For example: f(x,y) -> rgb(x/canvasWidth,y/canvasHeight) where x and y are positions on the canvas and color is that of the pixel.
My first thought is to take this example and modify it so that the rectangle fills the screen and the color is as described. I think this is achieved by modifying the vertex shader so that the rectangle covers the canvas and fragment shader to implement the color but I'm not sure how to apply the vertex shader based on window size or get my x and y variables in the context of the fragment shader.
Here's the shader code for the tutorial I'm going off of. I Haven't tried much besides manually changing the constant color in the fragment shader and mutating the square by changing the values in the intitBuffers method.

You will need to pass in the canvasWidth and canvasHeight. Your fragment shader might look like this:
precision mediump float;
// Require resolution (canvas size) as an input
uniform vec3 uResolution;
void main() {
// Calculate relative coordinates (uv)
vec2 uv = gl_FragCoord.xy / uResolution.xy;
gl_FragColor = vec4(uv.x, uv.y, 0., 1.0);
}
And then per #LJ's answer, if you really want the fragment to cover the entire canvas, you could modify your vertex shader to ignore the normal matrix transforms:
void main() {
// Pass through each vertex position without transforming:
gl_Position = aVertexPosition;
}
The runnable example below is mostly copy-pasted from the example you linked, with minor modifications:
const canvas = document.querySelector('#glcanvas');
main();
//
// Start here
//
function main() {
const gl = canvas.getContext('webgl');
// If we don't have a GL context, give up now
if (!gl) {
alert('Unable to initialize WebGL. Your browser or machine may not support it.');
return;
}
// Vertex shader program
const vsSource = `
attribute vec4 aVertexPosition;
uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;
void main() {
// We don't need the projection:
//gl_Position = uProjectionMatrix * uModelViewMatrix * aVertexPosition;
// Instead we pass through each vertex position as-is:
gl_Position = aVertexPosition;
}
`;
// Fragment shader program
const fsSource = `
precision mediump float;
// Require resolution (canvas size) as an input
uniform vec3 uResolution;
void main() {
// Calculate relative coordinates (uv)
vec2 uv = gl_FragCoord.xy / uResolution.xy;
gl_FragColor = vec4(uv.x, uv.y, 0., 1.0);
}
`;
// Initialize a shader program; this is where all the lighting
// for the vertices and so forth is established.
const shaderProgram = initShaderProgram(gl, vsSource, fsSource);
// Collect all the info needed to use the shader program.
// Look up which attribute our shader program is using
// for aVertexPosition and look up uniform locations.
const programInfo = {
program: shaderProgram,
attribLocations: {
vertexPosition: gl.getAttribLocation(shaderProgram, 'aVertexPosition'),
},
uniformLocations: {
projectionMatrix: gl.getUniformLocation(shaderProgram, 'uProjectionMatrix'),
modelViewMatrix: gl.getUniformLocation(shaderProgram, 'uModelViewMatrix'),
resolution: gl.getUniformLocation(shaderProgram, 'uResolution'),
},
};
// Here's where we call the routine that builds all the
// objects we'll be drawing.
const buffers = initBuffers(gl);
// Draw the scene
drawScene(gl, programInfo, buffers);
}
//
// initBuffers
//
// Initialize the buffers we'll need. For this demo, we just
// have one object -- a simple two-dimensional square.
//
function initBuffers(gl) {
// Create a buffer for the square's positions.
const positionBuffer = gl.createBuffer();
// Select the positionBuffer as the one to apply buffer
// operations to from here out.
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
// Now create an array of positions for the square.
const positions = [
1.0, 1.0,
-1.0, 1.0,
1.0, -1.0,
-1.0, -1.0,
];
// Now pass the list of positions into WebGL to build the
// shape. We do this by creating a Float32Array from the
// JavaScript array, then use it to fill the current buffer.
gl.bufferData(gl.ARRAY_BUFFER,
new Float32Array(positions),
gl.STATIC_DRAW);
return {
position: positionBuffer,
};
}
//
// Draw the scene.
//
function drawScene(gl, programInfo, buffers) {
gl.clearColor(0.0, 0.0, 0.0, 1.0); // Clear to black, fully opaque
gl.clearDepth(1.0); // Clear everything
gl.enable(gl.DEPTH_TEST); // Enable depth testing
gl.depthFunc(gl.LEQUAL); // Near things obscure far things
// Clear the canvas before we start drawing on it.
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
// Create a perspective matrix, a special matrix that is
// used to simulate the distortion of perspective in a camera.
// Our field of view is 45 degrees, with a width/height
// ratio that matches the display size of the canvas
// and we only want to see objects between 0.1 units
// and 100 units away from the camera.
const fieldOfView = 45 * Math.PI / 180; // in radians
const aspect = gl.canvas.clientWidth / gl.canvas.clientHeight;
const zNear = 0.1;
const zFar = 100.0;
const projectionMatrix = mat4.create();
// note: glmatrix.js always has the first argument
// as the destination to receive the result.
mat4.perspective(projectionMatrix,
fieldOfView,
aspect,
zNear,
zFar);
// Set the drawing position to the "identity" point, which is
// the center of the scene.
const modelViewMatrix = mat4.create();
// Now move the drawing position a bit to where we want to
// start drawing the square.
mat4.translate(modelViewMatrix, // destination matrix
modelViewMatrix, // matrix to translate
[-0.0, 0.0, -6]); // amount to translate
// Tell WebGL how to pull out the positions from the position
// buffer into the vertexPosition attribute.
{
const numComponents = 2;
const type = gl.FLOAT;
const normalize = false;
const stride = 0;
const offset = 0;
gl.bindBuffer(gl.ARRAY_BUFFER, buffers.position);
gl.vertexAttribPointer(
programInfo.attribLocations.vertexPosition,
numComponents,
type,
normalize,
stride,
offset);
gl.enableVertexAttribArray(
programInfo.attribLocations.vertexPosition);
}
// Tell WebGL to use our program when drawing
gl.useProgram(programInfo.program);
// Set the shader uniforms
gl.uniformMatrix4fv(
programInfo.uniformLocations.projectionMatrix,
false,
projectionMatrix);
gl.uniformMatrix4fv(
programInfo.uniformLocations.modelViewMatrix,
false,
modelViewMatrix);
gl.uniform3f(programInfo.uniformLocations.resolution, canvas.width, canvas.height, 1.0);
{
const offset = 0;
const vertexCount = 4;
gl.drawArrays(gl.TRIANGLE_STRIP, offset, vertexCount);
}
}
//
// Initialize a shader program, so WebGL knows how to draw our data
//
function initShaderProgram(gl, vsSource, fsSource) {
const vertexShader = loadShader(gl, gl.VERTEX_SHADER, vsSource);
const fragmentShader = loadShader(gl, gl.FRAGMENT_SHADER, fsSource);
// Create the shader program
const shaderProgram = gl.createProgram();
gl.attachShader(shaderProgram, vertexShader);
gl.attachShader(shaderProgram, fragmentShader);
gl.linkProgram(shaderProgram);
// If creating the shader program failed, alert
if (!gl.getProgramParameter(shaderProgram, gl.LINK_STATUS)) {
alert('Unable to initialize the shader program: ' + gl.getProgramInfoLog(shaderProgram));
return null;
}
return shaderProgram;
}
//
// creates a shader of the given type, uploads the source and
// compiles it.
//
function loadShader(gl, type, source) {
const shader = gl.createShader(type);
// Send the source to the shader object
gl.shaderSource(shader, source);
// Compile the shader program
gl.compileShader(shader);
// See if it compiled successfully
if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
alert('An error occurred compiling the shaders: ' + gl.getShaderInfoLog(shader));
gl.deleteShader(shader);
return null;
}
return shader;
}
canvas {
border: 2px solid black;
background-color: black;
}
video {
display: none;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/gl-matrix/2.8.1/gl-matrix-min.js"></script>
<body>
<canvas id="glcanvas" width="640" height="480"></canvas>
</body>
Same on glitch:
https://glitch.com/edit/#!/so-example-71499942

You're on the right track, for having a quad fill the screen you can skip all the matrix transforms in the vertex shader and just feed normalized device coordinates (-1 ... +1) directly into gl_Position then use gl_FragCoord to access the position of the pixel in the fragment shader.

Related

How can i transform a recursive function with two calls to itself into iterative? [duplicate]

I am writing a GPU-based real-time raytracing renderer using a GLSL compute shader. So far, it works really well, but I have stumbled into a seemingly unsolvable problem when it comes to having both reflections and refractions simultaneously.
My logic tells me that in order to have reflections and refractions on an object, such as glass, the ray would have to split into two, one ray reflects off the surface, and the other refracts through the surface. The ultimate colours of these rays would then be combined based on some function and ultimately used as the colour of the pixel the ray originated from. The problem I have is that I can't split the rays in shader code, as I would have to use recursion to do so. From my understanding, functions in a shader cannot be recursive because all GLSL functions are like inline functions in C++ due to compatibility issues with older GPU hardware.
Is it possible to simulate or fake recursion in shader code, or can I even achieve reflection and refraction simultaneously without using recursion at all? I can't see how it can happen without recursion, but I might be wrong.
I manage to convert back-raytracing to iterative process suitable for GLSL with the method suggested in my comment. It is far from optimized and I do not have all the physical stuff implemented (no Snell's law etc ...) yet but as a proof of concept it works already. I do all the stuff in fragment shader and CPU side code just send the uniforms constants and scene in form of 32 bit non-clamped float texture GL_LUMINANCE32F_ARB The rendering is just single QUAD covering whole screen.
passing the scene
I decided to store the scene in texture so each ray/fragment has direct access to whole scene. The texture is 2D but it is used as linear list of 32 bit floats. I decided this format:
enum _fac_type_enum
{
_fac_triangles=0, // r,g,b,a, n, triangle count, { x0,y0,z0,x1,y1,z1,x2,y2,z2 }
_fac_spheres, // r,g,b,a, n, sphere count, { x,y,z,r }
};
const GLfloat _n_glass=1.561;
const GLfloat _n_vacuum=1.0;
GLfloat data[]=
{
// r, g, b, a, n, type,count
0.2,0.3,0.5,0.5,_n_glass,_fac_triangles, 4, // tetrahedron
// px, py, pz, r, g, b
-0.5,-0.5,+1.0,
0.0,+0.5,+1.0,
+0.5,-0.5,+1.0,
0.0, 0.0,+0.5,
-0.5,-0.5,+1.0,
0.0,+0.5,+1.0,
0.0, 0.0,+0.5,
0.0,+0.5,+1.0,
+0.5,-0.5,+1.0,
0.0, 0.0,+0.5,
+0.5,-0.5,+1.0,
-0.5,-0.5,+1.0,
};
You can add/change any type of object. This example holds just single semi transparent bluish tetrahedron. You could also add transform matrices more coefficients for material properties etc ...
Architecture
the Vertex shader just initialize corner Rays of the view (start position and direction) which is interpolated so each fragment represents start ray of back ray tracing process.
Iterative back ray tracing
So I created a "static" list of rays and init it with the start ray. The Iteration is done in two steps first the back ray tracing:
Loop through all rays in a list from the first
Find closest intersection with scene...
store the position, surface normal and material properties into ray struct
If intersection found and not last "recursion" layer add reflect/refract rays to list at the end.
also store their indexes to the processed ray struct
Now your rays should hold all the intersection info you need to reconstruct the color. To do that:
loop through all the recursion levels backwards
for each of the rays matching actual recursion layer
compute ray color
so use lighting equations you want. If the ray contains children add their color to the result based on material properties (reflective and refractive coefficients ...)
Now the first ray should contain the color you want to output.
Uniforms used:
tm_eyeview camera matrix
aspectview ys/xs aspect ratio
n0 empty space refraction index (unused yet)
focal_length camera focal length
fac_siz resolution of the scene square texture
fac_num number of floats actually used in the scene texture
fac_txr texture unit for the scene texture
Preview:
The fragment shader contains my debug prints so you will need also the texture if used see the QA:
GLSL debug prints
ToDo:
add matrices for objects, camera etc.
add material properties (shininess, reflection/refraction coefficient)
Snell's law right now the direction of new rays are wrong ...
may be separate R,G,B to 3 start rays and combine at the end
fake SSS Subsurface scattering based on ray lengths
better implement lights (right now they are constants in a code)
implement more primitives (right now only triangles are supported)
[Edit1] code debug and upgrade
I removed old source code to fit inside 30KB limit. If you need it then dig it from edit history. Had some time for more advanced debugging for this and here the result:
this version got resolved some geometrical,accuracy,domain problems and bugs. I got implemented both reflections and refractions as is shown on this debug draw for test ray:
In the debug view only the cube is transparent and last ray that does not hit anything is ignored. So as you can see the ray split ... The ray ended inside cube due to total reflection angle And I disable all reflections inside objects for speed reasons.
The 32bit floats for intersection detection are a bit noisy with distances so you can use 64bit doubles instead but the speed drops considerably in such case. Another option is to rewrite the equation to use relative coordinates which are more precise in this case of use.
Here the float shaders source:
Vertex:
//------------------------------------------------------------------
#version 420 core
//------------------------------------------------------------------
uniform float aspect;
uniform float focal_length;
uniform mat4x4 tm_eye;
layout(location=0) in vec2 pos;
out smooth vec2 txt_pos; // frag position on screen <-1,+1> for debug prints
out smooth vec3 ray_pos; // ray start position
out smooth vec3 ray_dir; // ray start direction
//------------------------------------------------------------------
void main(void)
{
vec4 p;
txt_pos=pos;
// perspective projection
p=tm_eye*vec4(pos.x/aspect,pos.y,0.0,1.0);
ray_pos=p.xyz;
p-=tm_eye*vec4(0.0,0.0,-focal_length,1.0);
ray_dir=normalize(p.xyz);
gl_Position=vec4(pos,0.0,1.0);
}
//------------------------------------------------------------------
Fragment:
//------------------------------------------------------------------
#version 420 core
//------------------------------------------------------------------
// Ray tracer ver: 1.000
//------------------------------------------------------------------
in smooth vec3 ray_pos; // ray start position
in smooth vec3 ray_dir; // ray start direction
uniform float n0; // refractive index of camera origin
uniform int fac_siz; // square texture x,y resolution size
uniform int fac_num; // number of valid floats in texture
uniform sampler2D fac_txr; // scene mesh data texture
out layout(location=0) vec4 frag_col;
//---------------------------------------------------------------------------
//#define _debug_print
#define _reflect
#define _refract
//---------------------------------------------------------------------------
#ifdef _debug_print
in vec2 txt_pos; // frag screen position <-1,+1>
uniform sampler2D txr_font; // ASCII 32x8 characters font texture unit
uniform float txt_fxs,txt_fys; // font/screen resolution ratio
const int _txtsiz=64; // text buffer size
int txt[_txtsiz],txtsiz; // text buffer and its actual size
vec4 txt_col=vec4(0.0,0.0,0.0,1.0); // color interface for txt_print()
bool _txt_col=false; // is txt_col active?
void txt_decimal(vec2 v); // print vec3 into txt
void txt_decimal(vec3 v); // print vec3 into txt
void txt_decimal(vec4 v); // print vec3 into txt
void txt_decimal(float x); // print float x into txt
void txt_decimal(int x); // print int x into txt
void txt_print(float x0,float y0); // print txt at x0,y0 [chars]
#endif
//---------------------------------------------------------------------------
void main(void)
{
const vec3 light_dir=normalize(vec3(0.1,0.1,1.0));
const float light_iamb=0.1; // dot offset
const float light_idir=0.5; // directional light amplitude
const vec3 back_col=vec3(0.2,0.2,0.2); // background color
const float _zero=1e-6; // to avoid intrsection with start point of ray
const int _fac_triangles=0; // r,g,b, refl,refr,n, type, triangle count, { x0,y0,z0,x1,y1,z1,x2,y2,z2 }
const int _fac_spheres =1; // r,g,b, refl,refr,n, type, sphere count, { x,y,z,r }
// ray scene intersection
struct _ray
{
vec3 pos,dir,nor;
vec3 col;
float refl,refr;// reflection,refraction intensity coeficients
float n0,n1,l; // refaction index (start,end) , ray length
int lvl,i0,i1; // recursion level, reflect, refract
};
const int _lvls=5;
const int _rays=(1<<_lvls)-1;
_ray ray[_rays]; int rays;
vec3 v0,v1,v2,pos;
vec3 c,col;
float refr,refl;
float tt,t,n1,a;
int i0,ii,num,id;
// fac texture access
vec2 st; int i,j; float ds=1.0/float(fac_siz-1);
#define fac_get texture(fac_txr,st).r; st.s+=ds; i++; j++; if (j==fac_siz) { j=0; st.s=0.0; st.t+=ds; }
// enque start ray
ray[0].pos=ray_pos;
ray[0].dir=normalize(ray_dir);
ray[0].nor=vec3(0.0,0.0,0.0);
ray[0].refl=0.0;
ray[0].refr=0.0;
ray[0].n0=n0;
ray[0].n1=1.0;
ray[0].l =0.0;
ray[0].lvl=0;
ray[0].i0=-1;
ray[0].i1=-1;
rays=1;
// debug print area
#ifdef _debug_print
bool _dbg=false;
float dbg_x0=45.0;
float dbg_y0= 1.0;
float dbg_xs=12.0;
float dbg_ys=_rays+1.0;
dbg_xs=40.0;
dbg_ys=10;
float x=0.5*(1.0+txt_pos.x)/txt_fxs; x-=dbg_x0;
float y=0.5*(1.0-txt_pos.y)/txt_fys; y-=dbg_y0;
// inside bbox?
if ((x>=0.0)&&(x<=dbg_xs)
&&(y>=0.0)&&(y<=dbg_ys))
{
// prints on
_dbg=true;
// preset debug ray
ray[0].pos=vec3(0.0,0.0,0.0)*2.5;
ray[0].dir=vec3(0.0,0.0,1.0);
}
#endif
// loop all enqued rays
for (i0=0;i0<rays;i0++)
{
// loop through all objects
// find closest forward intersection between them and ray[i0]
// strore it to ray[i0].(nor,col)
// strore it to pos,n1
t=tt=-1.0; ii=1; ray[i0].l=0.0;
ray[i0].col=back_col;
pos=ray[i0].pos; n1=n0;
for (st=vec2(0.0,0.0),i=j=0;i<fac_num;)
{
c.r=fac_get; // RGBA
c.g=fac_get;
c.b=fac_get;
refl=fac_get;
refr=fac_get;
n1=fac_get; // refraction index
a=fac_get; id=int(a); // object type
a=fac_get; num=int(a); // face count
if (id==_fac_triangles)
for (;num>0;num--)
{
v0.x=fac_get; v0.y=fac_get; v0.z=fac_get;
v1.x=fac_get; v1.y=fac_get; v1.z=fac_get;
v2.x=fac_get; v2.y=fac_get; v2.z=fac_get;
vec3 e1,e2,n,p,q,r;
float t,u,v,det,idet;
//compute ray triangle intersection
e1=v1-v0;
e2=v2-v0;
// Calculate planes normal vector
p=cross(ray[i0].dir,e2);
det=dot(e1,p);
// Ray is parallel to plane
if (abs(det)<1e-8) continue;
idet=1.0/det;
r=ray[i0].pos-v0;
u=dot(r,p)*idet;
if ((u<0.0)||(u>1.0)) continue;
q=cross(r,e1);
v=dot(ray[i0].dir,q)*idet;
if ((v<0.0)||(u+v>1.0)) continue;
t=dot(e2,q)*idet;
if ((t>_zero)&&((t<=tt)||(ii!=0)))
{
ii=0; tt=t;
// store color,n ...
ray[i0].col=c;
ray[i0].refl=refl;
ray[i0].refr=refr;
// barycentric interpolate position
t=1.0-u-v;
pos=(v0*t)+(v1*u)+(v2*v);
// compute normal (store as dir for now)
e1=v1-v0;
e2=v2-v1;
ray[i0].nor=cross(e1,e2);
}
}
if (id==_fac_spheres)
for (;num>0;num--)
{
float r;
v0.x=fac_get; v0.y=fac_get; v0.z=fac_get; r=fac_get;
// compute l0 length of ray(p0,dp) to intersection with sphere(v0,r)
// where rr= r^-2
float aa,bb,cc,dd,l0,l1,rr;
vec3 p0,dp;
p0=ray[i0].pos-v0; // set sphere center to (0,0,0)
dp=ray[i0].dir;
rr = 1.0/(r*r);
aa=2.0*rr*dot(dp,dp);
bb=2.0*rr*dot(p0,dp);
cc= rr*dot(p0,p0)-1.0;
dd=((bb*bb)-(2.0*aa*cc));
if (dd<0.0) continue;
dd=sqrt(dd);
l0=(-bb+dd)/aa;
l1=(-bb-dd)/aa;
if (l0<0.0) l0=l1;
if (l1<0.0) l1=l0;
t=min(l0,l1); if (t<=_zero) t=max(l0,l1);
if ((t>_zero)&&((t<=tt)||(ii!=0)))
{
ii=0; tt=t;
// store color,n ...
ray[i0].col=c;
ray[i0].refl=refl;
ray[i0].refr=refr;
// position,normal
pos=ray[i0].pos+(ray[i0].dir*t);
ray[i0].nor=pos-v0;
}
}
}
ray[i0].l=tt;
ray[i0].nor=normalize(ray[i0].nor);
// split ray from pos and ray[i0].nor
if ((ii==0)&&(ray[i0].lvl<_lvls-1))
{
t=dot(ray[i0].dir,ray[i0].nor);
// reflect
#ifdef _reflect
if ((ray[i0].refl>_zero)&&(t<_zero)) // do not reflect inside objects
{
ray[i0].i0=rays;
ray[rays]=ray[i0];
ray[rays].lvl++;
ray[rays].i0=-1;
ray[rays].i1=-1;
ray[rays].pos=pos;
ray[rays].dir=ray[rays].dir-(2.0*t*ray[rays].nor);
ray[rays].n0=ray[i0].n0;
ray[rays].n1=ray[i0].n0;
rays++;
}
#endif
// refract
#ifdef _refract
if (ray[i0].refr>_zero)
{
ray[i0].i1=rays;
ray[rays]=ray[i0];
ray[rays].lvl++;
ray[rays].i0=-1;
ray[rays].i1=-1;
ray[rays].pos=pos;
t=dot(ray[i0].dir,ray[i0].nor);
if (t>0.0) // exit object
{
ray[rays].n0=ray[i0].n0;
ray[rays].n1=n0;
v0=-ray[i0].nor; t=-t;
}
else{ // enter object
ray[rays].n0=n1;
ray[rays].n1=ray[i0].n0;
ray[i0 ].n1=n1;
v0=ray[i0].nor;
}
n1=ray[i0].n0/ray[i0].n1;
tt=1.0-(n1*n1*(1.0-t*t));
if (tt>=0.0)
{
ray[rays].dir=(ray[i0].dir*n1)-(v0*((n1*t)+sqrt(tt)));
rays++;
}
}
#endif
}
else if (i0>0) // ignore last ray if nothing hit
{
ray[i0]=ray[rays-1];
rays--; i0--;
}
}
// back track ray intersections and compute output color col
// lvl is sorted ascending so backtrack from end
for (i0=rays-1;i0>=0;i0--)
{
// directional + ambient light
t=abs(dot(ray[i0].nor,light_dir)*light_idir)+light_iamb;
t*=1.0-ray[i0].refl-ray[i0].refr;
ray[i0].col.rgb*=t;
// reflect
ii=ray[i0].i0;
if (ii>=0) ray[i0].col.rgb+=ray[ii].col.rgb*ray[i0].refl;
// refract
ii=ray[i0].i1;
if (ii>=0) ray[i0].col.rgb+=ray[ii].col.rgb*ray[i0].refr;
}
col=ray[0].col;
// debug prints
#ifdef _debug_print
/*
if (_dbg)
{
txtsiz=0;
txt_decimal(_lvls);
txt[txtsiz]=' '; txtsiz++;
txt_decimal(rays);
txt[txtsiz]=' '; txtsiz++;
txt_decimal(_rays);
txt_print(dbg_x0,dbg_y0);
for (ii=0;ii<rays;ii++)
{
txtsiz=0;
txt_decimal(ray[ii].lvl);
txt_print(dbg_x0,dbg_y0+ii+1);
}
for (ii=0,st=vec2(0.0,0.0),i=j=0;i<fac_num;ii++)
{
c.r=fac_get; // RGBA
txtsiz=0;
txt_decimal(c.r);
txt_print(dbg_x0,dbg_y0+ii+1);
}
if (_txt_col) col=txt_col.rgb;
}
*/
if (_dbg)
{
float x=dbg_x0,y=dbg_y0;
vec3 a=vec3(1.0,2.0,3.0);
vec3 b=vec3(5.0,6.0,7.0);
txtsiz=0; txt_decimal(dot(a,b)); txt_print(x,y); y++;
txtsiz=0; txt_decimal(cross(a,b)); txt_print(x,y); y++;
if (_txt_col) col=txt_col.rgb;
}
#endif
frag_col=vec4(col,1.0);
}
//---------------------------------------------------------------------------
#ifdef _debug_print
//---------------------------------------------------------------------------
void txt_decimal(vec2 v) // print vec2 into txt
{
txt[txtsiz]='('; txtsiz++;
txt_decimal(v.x); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.y); txt[txtsiz]=')'; txtsiz++;
txt[txtsiz]=0; // string terminator
}
//---------------------------------------------------------------------------
void txt_decimal(vec3 v) // print vec3 into txt
{
txt[txtsiz]='('; txtsiz++;
txt_decimal(v.x); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.y); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.z); txt[txtsiz]=')'; txtsiz++;
txt[txtsiz]=0; // string terminator
}
//---------------------------------------------------------------------------
void txt_decimal(vec4 v) // print vec4 into txt
{
txt[txtsiz]='('; txtsiz++;
txt_decimal(v.x); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.y); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.z); txt[txtsiz]=','; txtsiz++;
txt_decimal(v.w); txt[txtsiz]=')'; txtsiz++;
txt[txtsiz]=0; // string terminator
}
//---------------------------------------------------------------------------
void txt_decimal(float x) // print float x into txt
{
int i,j,c; // l is size of string
float y,a;
const float base=10;
// handle sign
if (x<0.0) { txt[txtsiz]='-'; txtsiz++; x=-x; }
else { txt[txtsiz]='+'; txtsiz++; }
// divide to int(x).fract(y) parts of number
y=x; x=floor(x); y-=x;
// handle integer part
i=txtsiz; // start of integer part
for (;txtsiz<_txtsiz;)
{
a=x;
x=floor(x/base);
a-=base*x;
txt[txtsiz]=int(a)+'0'; txtsiz++;
if (x<=0.0) break;
}
j=txtsiz-1; // end of integer part
for (;i<j;i++,j--) // reverse integer digits
{
c=txt[i]; txt[i]=txt[j]; txt[j]=c;
}
// handle fractional part
for (txt[txtsiz]='.',txtsiz++;txtsiz<_txtsiz;)
{
y*=base;
a=floor(y);
y-=a;
txt[txtsiz]=int(a)+'0'; txtsiz++;
if (y<=0.0) break;
}
txt[txtsiz]=0; // string terminator
}
//---------------------------------------------------------------------------
void txt_decimal(int x) // print int x into txt
{
int a,i,j,c; // l is size of string
const int base=10;
// handle sign
if (x<0.0) { txt[txtsiz]='-'; txtsiz++; x=-x; }
else { txt[txtsiz]='+'; txtsiz++; }
// handle integer part
i=txtsiz; // start of integer part
for (;txtsiz<_txtsiz;)
{
a=x;
x/=base;
a-=base*x;
txt[txtsiz]=int(a)+'0'; txtsiz++;
if (x<=0) break;
}
j=txtsiz-1; // end of integer part
for (;i<j;i++,j--) // reverse integer digits
{
c=txt[i]; txt[i]=txt[j]; txt[j]=c;
}
txt[txtsiz]=0; // string terminator
}
//---------------------------------------------------------------------------
void txt_print(float x0,float y0) // print txt at x0,y0 [chars]
{
int i;
float x,y;
// fragment position [chars] relative to x0,y0
x=0.5*(1.0+txt_pos.x)/txt_fxs; x-=x0;
y=0.5*(1.0-txt_pos.y)/txt_fys; y-=y0;
// inside bbox?
if ((x<0.0)||(x>float(txtsiz))||(y<0.0)||(y>1.0)) return;
// get font texture position for target ASCII
i=int(x); // char index in txt
x-=float(i);
i=txt[i];
x+=float(int(i&31));
y+=float(int(i>>5));
x/=32.0; y/=8.0; // offset in char texture
txt_col=texture(txr_font,vec2(x,y));
_txt_col=true;
}
//---------------------------------------------------------------------------
#endif
//---------------------------------------------------------------------------
The code is not optimized yet I wanted to have the physics working correctly first. There are still not Fresnells implemented but refl,refr coefficients of material are used instead.
Also you can ignore the debug prints stuff (they are encapsulated by #define).
I build a small class for the geometry texture so I can easily set up scene objects. This is how the scene was initiated for the preview:
ray.beg();
// r g b rfl rfr n
ray.add_material(1.0,1.0,1.0,0.3,0.0,_n_glass); ray.add_box ( 0.0, 0.0, 6.0,9.0,9.0,0.1);
ray.add_material(1.0,1.0,1.0,0.1,0.8,_n_glass); ray.add_sphere( 0.0, 0.0, 0.5,0.5);
ray.add_material(1.0,0.1,0.1,0.3,0.0,_n_glass); ray.add_sphere( +2.0, 0.0, 2.0,0.5);
ray.add_material(0.1,1.0,0.1,0.3,0.0,_n_glass); ray.add_box ( -2.0, 0.0, 2.0,0.5,0.5,0.5);
ray.add_material(0.1,0.1,1.0,0.3,0.0,_n_glass);
ray.add_tetrahedron
(
0.0, 0.0, 3.0,
-1.0,-1.0, 4.0,
+1.0,-1.0, 4.0,
0.0,+1.0, 4.0
);
ray.end();
It is important so computed normals are facing out of objects because that is used for detecting inside/outside object crossings.
P.S.
If you're interested here is my volumetric 3D back ray tracer:
How to best write a voxel engine in C with performance in mind
here archive for low rep users
Here newer version of this "Mesh" Raytracer supporting hemisphere objects:
Ray tracing a Hemisphere

A best way to draw a lot of independent characters in Qt5?

I'm writing an application that displays a lot of text. It's not words and sentences though, it's binary data displayed in CP437 charset. Current form:
I'm having a problem though with drawing those characters. I need to draw each character one by one, because later I would like to apply different coloring. Those characters should have a transparent background as well, because later I would like to draw sections and ranges with different colors in the background (to group those characters based on some criteria).
The application supports multiple opened files at the same time, but when there are multiple files opened, the drawing starts to be noticeable on fast i7, so it's probably badly written.
What would be the best approach to draw this kind of data in Qt5? Should I just prerender characters to a bitmap and start from there, or it actually is possible to draw lots of characters by using normal Qt functions to draw text?
Edit: I'm using a normal QFrame widget that does drawing in paintEvent, using QPainter. Is this a wrong approach? I've read some docs on QGraphicsScene, from which I've remembered that it's best used in situations where a widget needs to have some control on the objects it draws. I don't need any control on what I draw; I just need to draw it and that's all. I won't reference any particular character after I'll draw it.
The widget has 2000 lines, so I won't paste the whole code, but currently my drawing approach is like this:
First, create a table (cache) with 256 entries, put the iterator counter to i variable,
For each entry, create a QStaticText object that contains drawing information about a character identified by ASCII code taken from i variable,
Later, in the drawing function, for each byte in the input stream (i.e. from the file), draw the data using QStaticText from the cache table. So, to draw ASCII character 0x7A, I'll look up QStaticText from index 0x7a in cache table, and feed this QStaticText object into the QPainter object.
I was also experimenting with a different approach, rendering the whole line in one QPainter::drawText call, and indeed it was faster, but I've lost possibility of coloring each character with different color. I would like to have this possibility.
The use of a QGraphicsScene wouldn't improve things - it's an additional layer on top of a QWidget. You're after raw performance, so you shouldn't be using it.
You could implement a QTextDocument as a viewmodel for the visible section of your memory buffer/file, but painting the fresh QTextDocument each time you scroll wouldn't be any faster than drawing things directly on a QWidget.
Using QStaticText is a step in the right direction, but insufficient: rendering QStaticText still requires the rasterization of the glyph's shape. You can do better and cache the pixmap of each QChar, QColor combination that you wish to render: this will be much faster than rasterizing character outlines, whether using QStaticText or not.
Instead of drawing individual characters, you then draw pixmaps from the cache. This commit demonstrates this approach. The character drawing method is:
void drawChar(const QPointF & pos, QChar ch, QColor color, QPainter & p) {
auto & glyph = m_cache[{ch, color}];
if (glyph.isNull()) {
glyph = QPixmap{m_glyphRect.size().toSize()};
glyph.fill(Qt::white);
QPainter p{&glyph};
p.setPen(color);
p.setFont(m_font);
p.drawText(m_glyphPos, {ch});
}
p.drawPixmap(pos, glyph);
}
You could also cache each (character,foreground,background) tuple. Alas, this gets quickly out of hand when there are many foreground/background combinations.
If all of your backgrounds are of the same color (e.g. white), you'd wish to store a negative mask of the character: the glyph has a white background and a transparent shape. This commit demonstrates this approach. The glyph rectangle is filled with glyph color, then a white mask is applied on top:
void drawChar(const QPointF & pos, QChar ch, QColor color, QPainter & p) {
auto & glyph = m_glyphs[ch];
if (glyph.isNull()) {
glyph = QImage{m_glyphRect.size().toSize(), QImage::Format_ARGB32_Premultiplied};
glyph.fill(Qt::white);
QPainter p{&glyph};
p.setCompositionMode(QPainter::CompositionMode_DestinationOut);
p.setFont(m_font);
p.drawText(m_glyphPos, {ch});
}
auto rect = m_glyphRect;
rect.moveTo(pos);
p.fillRect(rect, color);
p.drawImage(pos, glyph);
}
Instead of storing a fully pre-rendered character of a given color, you could store just the alpha mask and composite them on-demand:
Start with a pre-rendered white glyph on a transparent background (CompositionMode_Source).
Fill the glyph rect with background in CompositionMode_SourceOut: the background will remain with a hole for the character itself.
Fill the glyph rect with foreground in CompositionMode_DestinationOver: the foreground will fill the hole.
(Optional) Draw the composite on the widget, if you're not painting on the widget already.
This turns out to be reasonably fast, and the rendering is fully parallelizable - see the example below.
Note: The pre-rendered glyph could use further premultiplication of the color with alpha to appear less thick.
Yet another approach, with excellent performance, would be to emulate a text-mode display using the GPU. Store the pre-rendered glyph outlines in a texture, store the glyph indices and colors to be rendered in an array, and use OpenGL and two shaders to do the rendering. This example might be a starting point to implement such an approach.
A complete example, using CPU rendering across multiple threads, follows.
We start with the backing store view, used to produce QImages that are views into the backing store for a given widget, and can be used to parallelize painting.
On a 2013 iMac, this code repaints the full-screen widget in about 8ms.
// https://github.com/KubaO/stackoverflown/tree/master/questions/hex-widget-40458515
#include <QtConcurrent>
#include <QtWidgets>
#include <algorithm>
#include <array>
#include <cmath>
struct BackingStoreView {
QImage *dst = {};
uchar *data = {};
const QWidget *widget = {};
explicit BackingStoreView(const QWidget *widget) {
if (!widget || !widget->window()) return;
dst = dynamic_cast<QImage*>(widget->window()->backingStore()->paintDevice());
if (!dst || dst->depth() % 8) return;
auto byteDepth = dst->depth()/8;
auto pos = widget->mapTo(widget->window(), {});
data = const_cast<uchar*>(dst->constScanLine(pos.y()) + byteDepth * pos.x());
this->widget = widget;
}
// A view onto the backing store of a given widget
QImage getView() const {
if (!data) return {};
QImage ret(data, widget->width(), widget->height(), dst->bytesPerLine(), dst->format());
ret.setDevicePixelRatio(widget->devicePixelRatio());
return ret;
}
// Is a given image exactly this view?
bool isAView(const QImage &img) const {
return data && img.bits() == data && img.depth() == dst->depth()
&& img.width() == widget->width() && img.height() == widget->height()
&& img.bytesPerLine() == dst->bytesPerLine() && img.format() == dst->format();
}
};
Then, the CP437 character set:
static auto const CP437 = QStringLiteral(
" ☺☻♥♦♣♠•◘○◙♂♀♪♫☼▶◀↕‼¶§▬↨↑↓→←∟↔▲▼"
"␣!\"#$%&'()*+,-./0123456789:;<=>?"
"#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_"
"`abcdefghijklmnopqrstuvwxyz{|}~ "
"ÇüéâäàåçêëèïîìÄÅÉæÆôöòûùÿÖÜ¢£¥₧ƒ"
"áíóúñѪº¿⌐¬½¼¡«»░▒▓│┤╡╢╖╕╣║╗╝╜╛┐"
"└┴┬├─┼╞╟╚╔╩╦╠═╬╧╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀"
"αßΓπΣσµτΦΘΩδ∞φε∩≡±≥≤⌠⌡÷≈°∙·√ⁿ²■ ");
The HexView widget derives from QAbstractScrollArea and visualizes a memory-mapped chunk of data:
class HexView : public QAbstractScrollArea {
Q_OBJECT
QImage const m_nullImage;
const int m_addressChars = 8;
const int m_dataMargin = 4;
const char * m_data = {};
size_t m_dataSize = 0;
size_t m_dataStart = 0;
QSize m_glyphSize;
QPointF m_glyphPos;
int m_charsPerLine, m_lines;
QMap<QChar, QImage> m_glyphs;
QFont m_font{"Monaco"};
QFontMetricsF m_fm{m_font};
struct DrawUnit { QPoint pos; const QImage *glyph; QColor fg, bg; };
QFutureSynchronizer<void> m_sync;
QVector<DrawUnit> m_chunks;
QVector<QImage> m_stores;
using chunk_it = QVector<DrawUnit>::const_iterator;
using store_it = QVector<QImage>::const_iterator;
static inline QChar decode(char ch) { return CP437[uchar(ch)]; }
inline int xStep() const { return m_glyphSize.width(); }
inline int yStep() const { return m_glyphSize.height(); }
void initData() {
int const width = viewport()->width() - m_addressChars*xStep() - m_dataMargin;
m_charsPerLine = (width > 0) ? width/xStep() : 0;
m_lines = viewport()->height()/yStep();
if (m_charsPerLine && m_lines) {
verticalScrollBar()->setRange(0, m_dataSize/m_charsPerLine);
verticalScrollBar()->setValue(m_dataStart/m_charsPerLine);
} else {
verticalScrollBar()->setRange(0, 0);
}
}
const QImage &glyph(QChar ch) {
auto &glyph = m_glyphs[ch];
if (glyph.isNull()) {
QPointF extent = m_fm.boundingRect(ch).translated(m_glyphPos).bottomRight();
glyph = QImage(m_glyphSize, QImage::Format_ARGB32_Premultiplied);
glyph.fill(Qt::transparent);
QPainter p{&glyph};
p.setPen(Qt::white);
p.setFont(m_font);
p.translate(m_glyphPos);
p.scale(std::min(1.0, (m_glyphSize.width()-1)/extent.x()),
std::min(1.0, (m_glyphSize.height()-1)/extent.y()));
p.drawText(QPointF{}, {ch});
}
return glyph;
}
The parallelized rendering is done in class methods - they don't modify the state of the widget, other than accessing read-only data, and rendering into the backing store. The threads each act on isolated lines in the store.
static void drawChar(const DrawUnit & u, QPainter &p) {
const QRect rect(u.pos, u.glyph->size());
p.setCompositionMode(QPainter::CompositionMode_Source);
p.drawImage(u.pos, *u.glyph);
p.setCompositionMode(QPainter::CompositionMode_SourceOut);
p.fillRect(rect, u.bg);
p.setCompositionMode(QPainter::CompositionMode_DestinationOver);
p.fillRect(rect, u.fg);
}
static QFuture<void> submitChunks(chunk_it begin, chunk_it end, store_it store) {
return QtConcurrent::run([begin, end, store]{
QPainter p(const_cast<QImage*>(&*store));
for (auto it = begin; it != end; it++)
drawChar(*it, p);
});
}
This method distributes the chunks of work between threads:
int processChunks() {
m_stores.resize(QThread::idealThreadCount());
BackingStoreView view(viewport());
if (!view.isAView(m_stores.last()))
std::generate(m_stores.begin(), m_stores.end(), [&view]{ return view.getView(); });
std::ptrdiff_t jobSize = std::max(128, (m_chunks.size() / m_stores.size())+1);
auto const cend = m_chunks.cend();
int refY = 0;
auto store = m_stores.cbegin();
for (auto it = m_chunks.cbegin(); it != cend;) {
auto end = it + std::min(cend-it, jobSize);
while (end != cend && (end->pos.y() == refY || (refY = end->pos.y(), false)))
end++; // break chunks across line boundaries
m_sync.addFuture(submitChunks(it, end, store));
it = end;
store++;
}
m_sync.waitForFinished();
m_sync.clearFutures();
m_chunks.clear();
return store - m_stores.cbegin();
}
The remainder of the implementation is uncontroversial:
protected:
void paintEvent(QPaintEvent *ev) override {
QElapsedTimer time;
time.start();
QPainter p{viewport()};
QPoint pos;
QPoint const step{xStep(), 0};
auto dividerX = m_addressChars*xStep() + m_dataMargin/2.;
p.drawLine(dividerX, 0, dividerX, viewport()->height());
int offset = 0;
QRect rRect = ev->rect();
p.end();
while (offset < m_charsPerLine*m_lines && m_dataStart + offset < m_dataSize) {
const auto address = QString::number(m_dataStart + offset, 16);
pos += step * (m_addressChars - address.size());
for (auto c : address) {
if (QRect(pos, m_glyphSize).intersects(rRect))
m_chunks.push_back({pos, &glyph(c), Qt::black, Qt::white});
pos += step;
}
pos += {m_dataMargin, 0};
auto bytes = std::min(m_dataSize - offset, (size_t)m_charsPerLine);
for (int n = bytes; n; n--) {
if (QRect(pos, m_glyphSize).intersects(rRect))
m_chunks.push_back({pos, &glyph(decode(m_data[m_dataStart + offset])), Qt::red, Qt::white});
pos += step;
offset ++;
}
pos = {0, pos.y() + yStep()};
}
int jobs = processChunks();
newStatus(QStringLiteral("%1ms n=%2").arg(time.nsecsElapsed()/1e6).arg(jobs));
}
void resizeEvent(QResizeEvent *) override {
initData();
}
void scrollContentsBy(int, int dy) override {
m_dataStart = verticalScrollBar()->value() * (size_t)m_charsPerLine;
viewport()->scroll(0, dy * m_glyphSize.height(), viewport()->rect());
}
public:
HexView(QWidget * parent = nullptr) : HexView(nullptr, 0, parent) {}
HexView(const char * data, size_t size, QWidget * parent = nullptr) :
QAbstractScrollArea{parent}, m_data(data), m_dataSize(size)
{
QRectF glyphRectF{0., 0., 1., 1.};
for (int i = 0x20; i < 0xE0; ++i)
glyphRectF = glyphRectF.united(m_fm.boundingRect(CP437[i]));
m_glyphPos = -glyphRectF.topLeft();
m_glyphSize = QSize(std::ceil(glyphRectF.width()), std::ceil(glyphRectF.height()));
initData();
}
void setData(const char * data, size_t size) {
if (data == m_data && size == m_dataSize) return;
m_data = data;
m_dataSize = size;
m_dataStart = 0;
initData();
viewport()->update();
}
Q_SIGNAL void newStatus(const QString &);
};
We leverage modern 64-bit systems and memory-map the source file to be visualized by the widget. For test purposes, a view of the character set is also available:
int main(int argc, char ** argv) {
QApplication app{argc, argv};
QFile file{app.applicationFilePath()};
if (!file.open(QIODevice::ReadOnly)) return 1;
auto *const map = (const char*)file.map(0, file.size(), QFile::MapPrivateOption);
if (!map) return 2;
QWidget ui;
QGridLayout layout{&ui};
HexView view;
QRadioButton exe{"Executable"};
QRadioButton charset{"Character Set"};
QLabel status;
layout.addWidget(&view, 0, 0, 1, 4);
layout.addWidget(&exe, 1, 0);
layout.addWidget(&charset, 1, 1);
layout.addWidget(&status, 1, 2, 1, 2);
QObject::connect(&exe, &QPushButton::clicked, [&]{
view.setData(map, (size_t)file.size());
});
QObject::connect(&charset, &QPushButton::clicked, [&]{
static std::array<char, 256> data;
std::iota(data.begin(), data.end(), char(0));
view.setData(data.data(), data.size());
});
QObject::connect(&view, &HexView::newStatus, &status, &QLabel::setText);
charset.click();
ui.resize(1000, 800);
ui.show();
return app.exec();
}
#include "main.moc"
One solution I sometimes use is to keep a cache of pre-rendered lines. I normally use a doubly-linked LRU list of entries with about twice the lines that can be seen on the screen. Every time a line is used for rendering is moved to the front of the list; when I need to create a new line and the current cache count is past the limit I reuse the last entry in the list.
By storing the final result of individual lines you can repaint the display very quickly as probably in many cases most of the lines will not change from one frame to the next (including when scrolling).
The increased complexity is also reasonably confined in having to invalidate the line when you change the content.

QGraphicsScene & OpenGL Fragment Shader Not Working

I have a very large QGraphicsScene that can contain a very large number of graphics. I'm using a QGLWidget as the viewport so that I can leverage OpenGL to try to improve how some things get rendered. I have created a custom QGraphicsItem that I can use to draw several quads with the same texture in one render call rather than having hundreds or thousands of different QGraphicsItems in the scene that really all get drawn the same way, just in different locations. In my custom QGraphicsItem's paint() method, I called beginNativePainting() and endNativePainting() and do all of my OpenGL calls between them.
I want to use shader programs so that I can manipulate the vertices somewhat within the vertex shader, so I copied Qt's OpenGL Textures Example which uses a shader program to draw 6 textured quads. That example works just fine as is, but when I try to use the same approach within a QGraphicsItem's paint() method, all of my quads just get drawn white. My best guess is that my fragment shader just isn't getting used. I've even tried hardcoding the color within the fragment shader and nothing changes.
Here's the source code of my custom QGraphicsItem class.
class BatchGraphics : public QGraphicsPixmapItem
{
enum {PROGRAM_VERTEX_ATTRIBUTE = 0,
PROGRAM_TEXCOORD_ATTRIBUTE = 1};
public:
BatchGraphics()
: _program(0),
_texture(0),
_dirty(false)
{
}
// Returns the custom bounding rect for this item which encompasses all quads
QRectF boundingRect() const
{
return _boundingRect;
}
// Add a quad to the batch. Only the center point is necessary
void addQuad(int id, float x, float y)
{
_quads.insert(id, QPointF(x, y));
updateBoundingRect();
_dirty = true;
}
// Remove a quad from the batch.
void removeQuad(int id)
{
if (_quads.contains(id))
{
_quads.remove(id);
updateBoundingRect();
_dirty = true;
}
}
// Return the number of quads in the batch
int count() {return _quads.count();}
// Paint the batch using a custom implementation.
void paint(QPainter *painter, const QStyleOptionGraphicsItem *option, QWidget *widget)
{
// If the item is dirty (has been modified, update the geometry)
if (_dirty) {
updateGeometry();
}
if (_program)
{
painter->beginNativePainting();
// Enable GL states
//glEnable(GL_TEXTURE_2D);
// Set the MVP matrix
_program->setUniformValue("matrix", painter->transform());
// Enable and set the vertex and texture attributes
_program->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
_program->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
_program->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, GL_FLOAT, _vertices.constData(), 3, 5*sizeof(GLfloat));
_program->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, GL_FLOAT, _vertices.constData()+2, 2, 5*sizeof(GLfloat));
// Bind the texture
_texture->bind();
// Draw the arrays
glDrawArrays(GL_TRIANGLES, 0, _quads.count()*6); // 6 vertices per quad
painter->endNativePainting();
}
}
private:
// Initialize the shader and texture
void initialize()
{
// Create the OpenGL texture
_texture = new QOpenGLTexture(pixmap().toImage());
// Vertex Shader
QOpenGLShader *vshader = new QOpenGLShader(QOpenGLShader::Vertex);
const char *vsrc =
"attribute highp vec4 vertex;\n"
"attribute mediump vec4 texCoord;\n"
"varying mediump vec4 texc;\n"
"uniform mediump mat4 matrix;\n"
"void main(void)\n"
"{\n"
" gl_Position = matrix * vertex;\n"
" texc = texCoord;\n"
"}\n";
vshader->compileSourceCode(vsrc);
// Fragment Shader
QOpenGLShader *fshader = new QOpenGLShader(QOpenGLShader::Fragment);
const char *fsrc =
"uniform sampler2D texture;\n"
"varying mediump vec4 texc;\n"
"void main(void)\n"
"{\n"
" gl_FragColor = texture2D(texture, texc.st);\n"
"}\n";
fshader->compileSourceCode(fsrc);
// Program
_program = new QOpenGLShaderProgram;
_program->addShader(vshader);
_program->addShader(fshader);
_program->bindAttributeLocation("vertex", PROGRAM_VERTEX_ATTRIBUTE);
_program->bindAttributeLocation("texCoord", PROGRAM_TEXCOORD_ATTRIBUTE);
_program->link();
_program->bind();
_program->setUniformValue("texture", 0);
}
// Update the vertex array. Calls initialize the first time.
void updateGeometry()
{
if (_program == 0) {
initialize();
}
_vertices.clear();
// Half pixmap size
QPointF s = QPointF(pixmap().width()/2, pixmap().height()/2);
// Build vertex data for each quad
foreach (const QPointF& point, _quads)
{
// Top Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 1; // tv
// Top Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 1; // tv
// Bottom Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 0; // tv
// Top Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()-s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 1; // tv
// Bottom Left
_vertices << point.x()-s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 0; // tu
_vertices << 0; // tv
// Bottom Right
_vertices << point.x()+s.x(); // x
_vertices << point.y()+s.y(); // y
_vertices << 1; // z
_vertices << 1; // tu
_vertices << 0; // tv
}
_dirty = false;
}
private:
// Updates the bounding rect based on the quads in the batch.
void updateBoundingRect()
{
prepareGeometryChange();
double left = 9999;
double right = -9999;
double top = 9999;
double bottom = -9999;
double w = pixmap().width()/2;
double h = pixmap().width()/2;
foreach (const QPointF& p, _quads)
{
left = qMin(left, p.x()-w);
right = qMax(right, p.x()+w);
top = qMin(top, p.y()-h);
bottom = qMax(bottom, p.y()+h);
}
_boundingRect = QRectF(left, top, (right-left), (bottom-top));
}
private:
QOpenGLShaderProgram* _program;
QOpenGLTexture* _texture;
QRectF _boundingRect;
QMap<int, QPointF> _quads;
QVector<GLfloat> _vertices;
bool _dirty;
};
I understand the basics of the render pipeline and how to use shaders, but as far as dependencies between things and other OpenGL methods that must be called when using certain features I'm pretty clueless on. I can get the quads to be rendered with the texture using a fixed function pipeline approach, but that's old school and like I said, I want to be able to manipulate the vertices in the vertex shader once I get this working.
I'm not doing anything special when creating the QGLWidget, and its QGLFormat ends up being 2.0. I'v also tried calling glEnable(GL_TEXTURE_2D), but that just makes the quads get rendered black instead of white. I've also tried binding the program each time paint() is called, thinking perhaps Qt is binding a different shader somewhere else behind the scenes, but that just causes NOTHING to appear.
Can anyone provide any help please? I can't figure out why this approach works fine in Qt's Textures example but not when I try to do it inside of a QGraphicsItem.
I finally figured it out after looking at Qt's source code and what happens when beginNativePainting(). First, I DID have to bind my shader each time paint() was called, and second I had to get the correct MVP matrix.
I was trying to just pass the QPainter's transform to my shader to act as the modelview projection matrix, but the transform was only the modelview matrix. I needed to get the projection matrix as well, which Qt sets when beginNativePainting() is called.
I got the project and modelview matrices from OpenGL directly and combined them to pass to my shader after binding my texture and presto! It worked!
Here are the relevant changes I had to make:
painter->beginNativePainting();
// Enable GL states
//glEnable(GL_TEXTURE_2D);
// === Begin New Code ======
// Bind my program
_program->bind();
QMatrix4x4 proj;
glGetFloatv(GL_PROJECTION_MATRIX, proj.data());
QMatrix4x4 model;
glGetFloatv(GL_MODELVIEW_MATRIX, model.data());
// Set the MVP matrix
_program->setUniformValue("matrix", proj * model);
// === End New Code ======
// Enable and set the vertex and texture attributes
_program->enableAttributeArray(PROGRAM_VERTEX_ATTRIBUTE);
_program->enableAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE);
_program->setAttributeArray(PROGRAM_VERTEX_ATTRIBUTE, GL_FLOAT, _vertices.constData(), 3, 5*sizeof(GLfloat));
_program->setAttributeArray(PROGRAM_TEXCOORD_ATTRIBUTE, GL_FLOAT, _vertices.constData()+2, 2, 5*sizeof(GLfloat));

Qt5 OpenGL Texture Sampling

I'm trying to render a QImage using OpenGL wrapper classes of Qt5 and shader programs. I have the following shaders and a 3.3 core context. I'm also using a VAO for the attributes. However, I keep getting a blank red frame (red is the background clear color that I set). I'm not sure if it is a problem with the MVP matrices or something else. Using a fragment shader which sets the output color to a certain fixed color (black) still resulted in a red frame. I'm totally lost here.
EDIT-1: I also noticed that attempting to get the location of texRGB uniform from the QOpenGLShaderProgram results in -1. But I'm not sure if that has anything to do with the problem I'm having. Uniforms defined in the vertex shader for the MVP matrices have the locations 0 and 1.
Vertex Shader
#version 330
layout(location = 0) in vec3 inPosition;
layout(location = 1) in vec2 inTexCoord;
out vec2 vTexCoord;
uniform mat4 projectionMatrix;
uniform mat4 modelViewMatrix;
void main(void)
{
gl_Position = projectionMatrix * modelViewMatrix * vec4(inPosition, 1.0);
// pass the input texture coordinates to fragment shader
vTexCoord = inTexCoord;
}
Fragment Shader
#version 330
uniform sampler2DRect texRGB;
in vec2 vTexCoord;
out vec4 fColor;
void main(void)
{
vec3 rgb = texture2DRect(texRGB, vTexCoord.st).rgb;
fColor = vec4(rgb, 0.0);
}
OGLWindow.h
#include <QOpenGLWindow>
#include <QOpenGLFunctions>
#include <QOpenGLBuffer>
#include <QOpenGLShaderProgram>
#include <QOpenGLVertexArrayObject>
#include <QOpenGLTexture>
#include <QDebug>
#include <QString>
class OGLWindow : public QOpenGLWindow, protected QOpenGLFunctions
{
public:
OGLWindow();
~OGLWindow();
// OpenGL Events
void initializeGL();
void resizeGL(int width, int height);
void paintGL();
// a method for cleanup
void teardownGL();
private:
bool isInitialized;
// OpenGL state information
QOpenGLBuffer m_vbo_position;
QOpenGLBuffer m_vbo_index;
QOpenGLBuffer m_vbo_tex_coord;
QOpenGLVertexArrayObject m_object;
QOpenGLShaderProgram* m_program;
QImage m_image;
QOpenGLTexture* m_texture;
QMatrix4x4 m_projection_matrix;
QMatrix4x4 m_model_view_matrix;
};
OGLWindow.cpp
#include "OGLWindow.h"
// vertex data
static const QVector3D vertextData[] = {
QVector3D(-1.0f, -1.0f, 0.0f),
QVector3D( 1.0f, -1.0f, 0.0f),
QVector3D( 1.0f, 1.0f, 0.0f),
QVector3D(-1.0f, 1.0f, 0.0f)
};
// indices
static const GLushort indices[] = {
0, 1, 2,
0, 2, 3
};
OGLWindow::OGLWindow() :
m_vbo_position (QOpenGLBuffer::VertexBuffer),
m_vbo_tex_coord (QOpenGLBuffer::VertexBuffer),
m_vbo_index (QOpenGLBuffer::IndexBuffer),
m_program (nullptr),
m_texture (nullptr),
isInitialized (false)
{
}
OGLWindow::~OGLWindow()
{
makeCurrent();
teardownGL();
}
void OGLWindow::initializeGL()
{
qDebug() << "initializeGL()";
initializeOpenGLFunctions();
isInitialized = true;
QColor backgroundColor(Qt::red);
glClearColor(backgroundColor.redF(), backgroundColor.greenF(), backgroundColor.blueF(), 1.0f);
// load texture image
m_image = QImage(":/images/cube.png");
m_texture = new QOpenGLTexture(QOpenGLTexture::TargetRectangle);
// set bilinear filtering mode for texture magnification and minification
m_texture->setMinificationFilter(QOpenGLTexture::Nearest);
m_texture->setMagnificationFilter(QOpenGLTexture::Nearest);
// set the wrap mode
m_texture->setWrapMode(QOpenGLTexture::ClampToEdge);
m_texture->setData(m_image.mirrored(), QOpenGLTexture::MipMapGeneration::DontGenerateMipMaps);
int imgWidth = m_image.width();
int imgHeight = m_image.height();
m_projection_matrix.setToIdentity();
m_projection_matrix.ortho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f);
// m_projection_matrix.ortho(0.0, (float) width(), (float) height(), 0.0f, -1.0f, 1.0f);
m_model_view_matrix.setToIdentity();
glViewport(0, 0, width(), height());
m_program = new QOpenGLShaderProgram();
m_program->addShaderFromSourceFile(QOpenGLShader::Vertex, ":/shaders/vshader.glsl");
m_program->addShaderFromSourceFile(QOpenGLShader::Fragment, ":/shaders/fshader.glsl");
m_program->link();
m_program->bind();
// texture coordinates
static const QVector2D textureData[] = {
QVector2D(0.0f, 0.0f),
QVector2D((float) imgWidth, 0.0f),
QVector2D((float) imgWidth, (float) imgHeight),
QVector2D(0.0f, (float) imgHeight)
};
// create Vertex Array Object (VAO)
m_object.create();
m_object.bind();
// create position VBO
m_vbo_position.create();
m_vbo_position.bind();
m_vbo_position.setUsagePattern(QOpenGLBuffer::StaticDraw);
m_vbo_position.allocate(vertextData, 4 * sizeof(QVector3D));
// create texture coordinates VBO
m_vbo_tex_coord.create();
m_vbo_tex_coord.bind();
m_vbo_tex_coord.setUsagePattern(QOpenGLBuffer::StaticDraw);
m_vbo_tex_coord.allocate(textureData, 4 * sizeof(QVector2D));
// create the index buffer
m_vbo_index.create();
m_vbo_index.bind();
m_vbo_index.setUsagePattern(QOpenGLBuffer::StaticDraw);
m_vbo_index.allocate(indices, 6 * sizeof(GLushort));
// enable the two attributes that we have and set their buffers
m_program->enableAttributeArray(0);
m_program->enableAttributeArray(1);
m_program->setAttributeBuffer(0, GL_FLOAT, 0, 3, sizeof(QVector3D));
m_program->setAttributeBuffer(1, GL_FLOAT, 0, 2, sizeof(QVector2D));
// Set modelview-projection matrix
m_program->setUniformValue("projectionMatrix", m_projection_matrix);
m_program->setUniformValue("modelViewMatrix", m_model_view_matrix);
// use texture unit 0 which contains our frame
m_program->setUniformValue("texRGB", 0);
// release (unbind) all
m_object.release();
m_vbo_position.release();
m_vbo_tex_coord.release();
m_vbo_index.release();
m_program->release();
}
void OGLWindow::resizeGL(int width, int height)
{
qDebug() << "resizeGL(): width =" << width << ", height=" << height;
if (isInitialized) {
// avoid division by zero
if (height == 0) {
height = 1;
}
m_projection_matrix.setToIdentity();
m_projection_matrix.perspective(60.0, (float) width / (float) height, -1, 1);
glViewport(0, 0, width, height);
}
}
void OGLWindow::paintGL()
{
// clear
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// render using our shader
m_program->bind();
{
m_texture->bind();
m_object.bind();
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, 0) );
m_object.release();
}
m_program->release();
}
void OGLWindow::teardownGL()
{
// actually destroy our OpenGL information
m_object.destroy();
m_vbo_position.destroy();
m_vbo_color.destroy();
delete m_program;
}
EDIT-2: I'm creating the context as follows:
QSurfaceFormat format;
format.setRenderableType(QSurfaceFormat::OpenGL);
format.setProfile(QSurfaceFormat::CoreProfile);
format.setVersion(3,3);
This line in your fragment shader code is invalid:
vec3 rgb = texture2DRect(texRGB, vTexCoord.st).rgb;
texture2DRect() is not a built-in function.
Since you're using the GLSL 3.30 core profile (core is the default for the version unless compatibility is specified), you should be using the overloaded texture() function, which replaces the older type specific functions like texture2D() in the core profile.
Functions like texture2D() are still supported in GLSL 3.30 core unless a forward compatible core profile context is used. So depending on how the context is created, you can still use those functions.
However, sampler2DRect was only added as a sampler type in GLSL 1.40 as part of adding rectangular textures to the standard in OpenGL 3.1. At the time, the legacy sampling functions were already marked as deprecated, and only the new texture() function was defined for rectangular textures. This means that texture2DRect() does not exist in any GLSL version.
The correct call is:
vec3 rgb = texture(texRGB, vTexCoord.st).rgb;
Another part of your code that can prevent it from rendering anything is this projection matrix:
m_projection_matrix.perspective(60.0, (float) width / (float) height, -1, 1);
The near and far planes for a standard projection matrix both need to be positive. This call will set up a projection transformation with a "camera" on the origin, looking down the negative z-axis. The near and far values are distances from the origin. A valid call could look like this:
m_projection_matrix.perspective(60.0, (float) width / (float) height, 1.0f, 10.0f);
You will then also need to set the model matrix to transform the coordinates of the object into this range on the negative z-axis. You could for example apply a translation by (0.0f, 0.0f, -5.0f).
Or, if you just want to see something, the quad should also become visible if you simply use the identity matrix for the projection.

CgFX geometry shader

How to use geometry shaders with CgFX? Actually how to specify geometry shader within the 'technique'?
The listing is below, vertex and fragment shaders are compiled well. But 'NVIDIA FX Composer 2.5' fires an 'error C3001 no program defined' after I added 'GeometryProgram' to the 'technique'.
float4x4 WorldITXf : WorldInverseTranspose;
float4x4 WorldViewProjXf : WorldViewProjection;
float4x4 WorldXf : World;
struct appdata
{
float4 Position : POSITION;
float4 Color : COLOR0;
};
struct vertexOutput
{
float4 Position : POSITION;
float4 Color : COLOR0;
};
TRIANGLE void gshader1(AttribArray<float4> pos : POSITION,
AttribArray<float4> col : COLOR0)
{
// some code will be here;
}
vertexOutput vshader1(appdata IN)
{
vertexOutput OUT;
float4 Po = float4(IN.Position.xyz,1.0f);
OUT.Position = mul(WorldViewProjXf, Po);
OUT.Color = IN.Color;
return OUT;
}
float4 fshader1(vertexOutput IN) : COLOR
{
return IN.Color;
}
technique Tec1 {
pass p0 {
GeometryProgram = compile glslg gshader1();
VertexProgram = compile glslv vshader1();
FragmentProgram = compile glslf fshader1();
}
}
Solved. Something is wrong with 'glslg' profile, changing it to 'gp4gp' fixes the issue.

Resources