14.4 Compositing Images with Depth

Compositing separate images together is a useful technique for increasing the complexity of a scene [12]. An image can be saved to memory, then drawn to the screen using glDrawPixels(). Both the color and depth buffer contents can be copied into the frame buffer. This is sufficient for 2D style composites, where objects are drawn on top of each other to create the final scene. To do true 3D compositing, it's necessary to use the color and depth values simultaneously, so that depth testing can be used to determine which surfaces are obscured by others.

The stencil buffer can be used for true 3D compositing in a two pass operation. The color buffer is disabled for writing, the stencil buffer is cleared, and the saved depth values are copied into the frame buffer. Depth testing is enabled, insuring that only depth values that are closer to the original can update the depth buffer. glStencilOp() is called to set a stencil buffer bit if the depth test passes.

The stencil buffer now contains a mask of pixels that were closer to the view than the pixels of the original image. The stencil function is changed to accomplish this masking operation, the color buffer is enabled for writing, and the color values of the saved image are drawn to the frame buffer.

This technique works because the fragment operations, in particular the depth test and the stencil test, are part of both the geometry and imaging pipelines in OpenGL. Here is the technique in more detail. It assumes that both the depth and color values of an image have been saved to system memory, and are to be composited using depth testing to an image in the frame buffer:

1. Clear the stencil buffer using glClear(), or'ing in GL_STENCIL_BUFFER_BIT
2. Disable the color buffer for writing with glColorMask()
3. Set stencil values to 1 when the depth test passes by calling glStencilFuncGL_ALWAYS, 1, 1(GL_ALWAYS, 1, 1), and glStencilOpGL_KEEP, GL_KEEP, GL_REPLACE(GL_KEEP, GL_KEEP, GL_REPLACE)
4. Ensure depth testing is set; glEnableGL_DEPTH_TEST(GL_DEPTH_TEST), glDepthFuncGL_LESS(GL_LESS)
5. Draw the depth values to the frame buffer with glDrawPixels(), using GL_DEPTH_COMPONENT for the format argument.
6. Set the stencil buffer to test for stencil values of 1 with glStencilFuncGL_EQUAL, 1, 1(GL_EQUAL, 1, 1) and glStencilOpGL_KEEP, GL_KEEP, GL_KEEP(GL_KEEP, GL_KEEP, GL_KEEP).
7. Disable the depth testing with glDisableGL_DEPTH_TEST(GL_DEPTH_TEST)
8. Draw the color values to the frame buffer with glDrawPixels(), using GL_RGBA as the format argument.

At this point, both the depth and color values will have been merged, using the depth test to control which pixels from the saved image would update the frame buffer. Compositing can still be problematic when merging images with coplanar polygons.

This process can be repeated to merge multiple images. The depth values of the saved image can be manipulated by changing the values of GL_DEPTH_SCALE and GL_DEPTH_BIAS with glPixelTransfer(). This technique could allow you to squeeze the incoming image into a limited range of depth values within the scene.