Recently I’ve been working on some functionality on one of my own games. The main problem I’ve been having is gDEBugger is no longer in the Beta stage for the Mac, and so my temporary license has run out. At some point I do plan to buy a license, but for the moment I’m limited to my own OpenGL debugging.
So I got inspiration to make a shader to render my stencil buffer for easier debugging. It works like this, after reading the stencil from my Frame Buffer Object (FBO), I render it to a fullscreen quad. During the Fragment Shader process, I read the pixel from the stencil texture, separating a single component (I choose red), I then use this to sample a pixel from a 1D texture (256 pixels wide since the stencil is 8bit, each with a unique colour).
There are other applications I found for this, such as rendering the depth buffer as well, usually the depth buffer is practically white, and make it difficult to see objects far away. Of course, using many colours to do this, make it almost impossible to understand it. And because its a 1D 256 pixel texture, it doesn’t show the full range of depths since the depth buffer I’m using is 24bit.
It does demonstrate however how the depth buffer gets less accurate the further away an object is. the colours get further and further apart. This can be remedied by increase the number of bits a depth has, or reducing the difference between the near and far plane. Or you could do multiple renders (back to front) rendering the different distant objects, clearing the depth buffer after each render.
Though that is a little overkill. I do love the effect of the stencil buffer shader on the depth buffer though, reminds me of Commodore 64 crashes.