Dev:Source/Lighting/Shadows/DeepShadowMaps

This page is a review of what I've done in the soc-2007-joeedh DSM branch. The project has suffered from many, many setbacks, horrible bugs that took months to fix, and at times developer fatigue. It is now however finally ready for a review and integration into trunk.

Introduction

Deep Shadow Maps is a (somewhat slow) technique to render filtered, transparent and volumetric shadows via an algorithm akin to traditional shadow buffering. It's most valuable for rendering hair and fur, which is what it was originally designed for.

The algorithm uses a compressed volumetric structure, which has a lot of potential for future research (one example is the physically-correct soft shadow method created by DreamWorks, though that one turned out to not be of much practical benefit).

My Project

My project consists of a DSM implementation, a simple bucketing system I wrote for DSM, and a generic tile disk cache system that is used to control the memory consumption of both. The tile cache system can be used for anything that lives in arrays, in up to three dimensions.

I also modified guardedalloc to be more useful to debugging, by having it print line and file information on any sort of error.

DSM Implementation

The DSM implementation is coded entirely in C. It works similar to envmaps, in that it has to rotate the scene into the lamp's view space, render, then rotate back (this is because otherwise texture coordinates will be invalid). It currently supports polygons and strands, but not volumetrics (which can be tackled once they, too, arrive in trunk). Strand rendering is fully supported, unlike normal shadow buffers.

The Tile Cache System

The header file for the tile cache system can be found here. The system uses the struct-with-function-pointers style of component-oriented design in C. It has a fully-featured IO wrapper library, for supporting both disk and memory-backed files.

To use it, you have to instantiate the ***Type structures with appropriate function pointers. Each tile (or cell) in your buffer must include the TCS_Tile structre at it's head, and the buffer itself must include the TCS_TileBuffer structure. Read the header for more details.

The tile system internally zlib-level-2 compresses the tile data, which not only saves on disk space but also drastically reduces the amount of disk IO needed to keep memory consumption below the cache limit. Because of this, the implementation for fixed-sized tiles isn't quite complete yet, since getting that to work properly is a more complicated problem.

Generic Bucketing System

The generic bucking system is just one of the optimizations I did for DSM. It's a simple tile buffer, whose tiles store lists of polygons and strands whose bounding box intersects that tile. It makes a significant different in speed. It is now, however, used for anything other then DSM at the moment (though it was designed to be used everywhere, eventually).

Other Render Optimizations

Another render optimization I did was to make the transparent zbuffer implementation for dsm use a prepend operation, rather then an append operation, for inserting data into the buffer. This makes quite a different with large numbers of polygons. I have not as yet modified either the normal ZTransp implementation nor the strand ZTransp code to use this same technique.

Soft Shadows=

Soft shadows is planned to be supported, though the original paper I based my code on turned out not to be of much practical use. Instead I'm working on a more traditional PCF approach, basing it off of nvidia's PCSS (Percentage Close Soft Shadows) whitepaper.

User Interface

The user interface for DSM is a little different than that of normal shadow buffers, and probably isn't quite finished yet. It currently reuses the PCF sample number to set the buffer oversampling rate (the original idea being that DSM will use the ray soft shadow settings instead). As such, the Soft value currently has no real meaning (though it's being used at the moment in the experimental, new PCF code I just wrote). There's also an issue where the auto clipstart/end function doesn't always work properly for DSM.

The user can set the cache memory limit for DSM in the System and Opengl info buttons.

Any feedback on the UI would be appreciated.