Soft Edged Shadows?

[sybixsus]

I've been following this article on GameDev ( http://www.gamedev.net/reference/articles/article2193.asp ) to give me soft-edged shadows via shadowmapping, and I think I've got it perfect, but there's a problem which seems to be inherent with this technique. Blurring the shadow map (not the depth map) in screenspace means that you get shadows bleeding into areas they would not have been ( and non-shadowed areas bleed back ) Now possibly I've missed something in the article which prevents this, but I don't think so.

Now at the moment, all my maps ( depth, shadow and blur buffers ) are 1024x1024 but I'm considering using 512x512 maps for the shadow and blur buffers, but that's going to make the bleeding even worse. So I really need to sort the bleeding before I do that. Have I missed something in the article? If not, how do I deal with this? Would I have to do the depth comparison again in the final shader instead of just projecting the shadow map over everything? Or would that cause its own problems?

[Zaknafein]

There's indeed nothing in this article that covers it.
I've never attempted to fix this problem (I think Arius has in his SSAO demo), but the theory is to "multiply" the blur weights by the inverse depth difference in relation to the center tap. You could simply discard the weight if its depth difference (so eye-space Z distance) is bigger than a certain threshold. This implies that the blur shader has access to the depth texture.

Keep us informed on your progress, I'd love to see a working implementation of this!

[WEst]

The bleeding when using postblur in screenspace is a common problem with this technique and it's nearly impossible to overcome this problem without a lot of tweaking (but shadowmapping is always tweaking). The higher the screenresolution and/or the shadowmapresolution the better it gets but it is hard to get it perfect. Also there are some other artifacts with this technique because the blur is bigger on higher distance shadows than on near distance shadows.

A better solution is to use different samples of the depthmap and average them in some way (e.g. gaussian kernel), but this will ofcourse need more performance. There are lots of soft shadowmapping techniques out there but they all have some kind of tradeoff.

Zaknafein did a great article on his Blog (http://www.theinstructionlimit.com/?p=13) about different shadowmapping techniques, maybe the article will help you with your problems.

[sybixsus]

Thank you, both of you. Zak, your explanation of using the inverse depth difference makes perfect sense. It sounds as though it will make the blur very expensive though. I'm already doing 11 taps, so depth sampling for each would mean 22 texture reads per pixel. I suppose in theory, this would be offset by being able to drop the resolution of the blur buffers to 512x512.

West, you're right, of course, about the relationship between distance and blur, but my game has a 3/4 view, much like an isometric 2d game ( only without an isometric perspective ) so there isn't enough variation in depth to affect me. I don't think changing shadowmapping systems completely at this stage of development is sensible for me, but I will have a read and see if it might be sensible for the next game.

[AriusEso]

Well, I've posed the problem before now, indeed Zak, I was going to tie it in with my SSAO map and then blur the two at the same time. I never got around to actually testing it however. Basically, the bleed from my SSAO seems pretty minimal using my current blurring method. So I wanted to test out the shadow map in the same way.

Not sure if it is of any use to you, but here you can see I'm storing the normals in the RGB of the deep map.

Code:

                float3 Norm = tex2D(DeepSample, IN.UV).rgb;
                float  AO   = tex2D(SSAOSample, IN.UV).r;
               
                float  Num  = 1;
                int    Sam  = 32;
               
                for(int i = -Sam/2; i <= Sam/2; i+=1)
                {
                        float2 nUV    = float2(IN.UV + i * Direction.xy);
                       
                        float  Sample = tex2D(SSAOSample, nUV).r;
                        float3 sNorm  = tex2D(DeepSample, nUV).rgb;
                       
                        if(dot(Norm, sNorm) > 0.99)
                        {
                                Num += (Sam/2 - abs(i));
                                AO  += Sample * (Sam/2 - abs(i));
                        }
                }
               
                return AO / Num;

I was essentially going to render the scene as white with just the shadows, then merge that target with the SSAO map creating a single map with both, blur it and then dump them both to the screen through the final SSAO composite shader. So they'd technically be screen-space shadows in the end I guess - they would be kind of ultra SSAO .

Anyway, as I said, dunno if this helps at all. But perhaps it'll give you an idea. I'd be interested in seeing a solution if you work one out - or indeed if this one works. I really should get around to testing it.

Good luck man.