Shadow-Mapping Directional Lights

[Zaknafein]

Better FPS still!!



I found out the biggest slowdown cause : COM indirection.
Using TVMathLibrary for all my vector and matrix operations made everything slower. I made a mini maths library in VB.Net and my FPS pretty much doubled... incredible.

Now what's taking the most time is (and it's logical) shader rendering. Apart from debugging, because stupid texturefonts are slow.
I'm using 1024x1024 depth map with 3x PCF, but if I reduce this the FPS goes way beyond. It's very encouraging to see it all smooth again.
Expect a demo by the end of next week!!

[Hawthorne]

Better FPS still!!



I found out the biggest slowdown cause : COM indirection.
Using TVMathLibrary for all my vector and matrix operations made everything slower. I made a mini maths library in VB.Net and my FPS pretty much doubled... incredible.

Now what's taking the most time is (and it's logical) shader rendering. Apart from debugging, because stupid texturefonts are slow.
I'm using 1024x1024 depth map with 3x PCF, but if I reduce this the FPS goes way beyond. It's very encouraging to see it all smooth again.
Expect a demo by the end of next week!!

COM calls are killers, especially tons of distance checks, threaded or looping. I feel your pain. Good to see progress.  Can't wait to rig it up

-Pat

[nomadsoul]

Can we do the same with 6.2 release ? or we must wait the 6.5 and shader effect for doing this ?

[Zaknafein]

Sorry, that's not doable with 6.2... It really requires shaders to work.

[Brac]

The account I'm posting with here is actually used by two guys - Brac (Christian) and me (Georg). Most idea stuff you get on IRC & MSN is from both of us, too... we're sitting side by side, sooo...

Kudos Zak - keep on keeping on!

G & C

[darqSHADOW]

Wow, simply amazing Zak.  You've definately mastered shader-development with TV3D...  

DS

[Hawthorne]

I will start holding my breath.  :oops:

Simply spectacular. Now lets see how it works out on single chunked small landscapes! muahhaah! muahaahahhaha!!!!!

Great work Z.
-Pat

[fex]

Zak, you continue to rock my world.

[Dragoon]

Zak, seriously your shader rocks my socks!  :shock:

now i need a new pair of socks thnx a lot  :x

[Zaknafein]

Thanks DS, and thanks everyone else... it's appreciated.
The demo is still scheduled for the end of this week, which gives me until Sunday 23:59:59. Heehee.

In recent developments (to keep you updated), I fixed a major problem which made some shadows disappear if the casting object is not in the recieving chunk. It has a kind of crude solution, which makes landscape self-shadowing impossible for the time being : the shadow camera is so far away and the depth limits are so wide that I don't have the needed precision for self-shadows.

I also stripped the VSM support because it was too problematic, and replaced it with "shadowmap-space gaussian blur", which is kind of like screen-space blur applied to the shadowmap. It works VERY well, is faster and smoother than PCF, and it works because the shadowmaps are top-down orthogonal and map proportionally to world units. It's now the default setting for filtering. I'll post shots later...
Also I'm optimizing everything I can (btw, anyone has an easy and fast method to invert a matrix?) and avoiding all the COM calls I can, and soon I'll get to commenting and polishing.

Edit : An interesting comparision.
The scene has 50 trees and 25 teapots on a 4x4 landscape with HIGH precision and LOD down to ULTRA_LOW precision. I compared stencils vs shadowmapping.

Shadow Maps


Stencil Shadows


More than twice the framerate, and about twice as good-looking.

Buuut... it's not that simple really. I cheated at this game :
- The shadow maps are definitely not rendered every frame. At most, the chunk directly under the camera will have its shadowmap updated every 1/30th of a second. Stencils have no choice but to be rendered every frame.
- Far-away chunks might be rendered every 1/7th of a second, which definitely shows when in bird's eye view. This, of course, can be tweaked via parameters if you can afford it.
- Shadow maps use a LOT of memory. Right now all shadow maps are 512x512, so 16 rendersurfaces of this size plus one 1024x1024 depth map, and one 512x512 depth map for farther chunks. Oh, and plus one 512x512 gaussian blur surface. The sizes can be tweaked with parameters as well for lower-end machines.

Still... encouraging results. I'm not even done with optimization!!

[Rynus_Rein]

The difference is amazing. gj _o_

[Hypnotron]

Wow.  I guess that is the nail in the coffin for stencil shadows then.  Sylvain should add this to 6.5    

[patrickrho]

yes,
this should be included!
i really dont care if my shadows are not rendered every frame lol

[Hypnotron]

lol.  well i just look at the pretty pictures and go "oooh."  I only just now read about the memory reqts and the skipped frames for performance.

[fex]

Sylvain should add this to 6.5    

Seconded!

[Zaknafein]

Sylvain should add this to 6.5

lol.  well i just look at the pretty pictures and go "oooh."  I only just now read about the memory reqts and the skipped frames for performance.

Hehe, yeah. The memory requirements are the biggest problem. And for that reason alone, I don't think it should be part of the engine.

To overcome this I'd need to have an LOD algorithm like for the depth maps, but for the shadowmaps i.e. far-away chunks are rendered with a lower precision shadowmap. This is not as simple as it sounds though... how many high-res shadowmaps do you create? how many lo-res ones? Is assigning them to the right chunk every frame too costly?
I don't think I'll do this in the upcoming demo.

Besides... Look, I've made it. Once it's finished, it might fit in a DLL that you just import, initialize, update and render. Why bother Sylvain with it?

[BlindSide]

Sylvain and I had a talk about this actually, adding shadowmapping into TV, waaay back in 2005. He was working on adding projective texture shadows.

It simply will not be done. Not now.. maybe in the long, long future, but not anytime soon. Any shadowmapping we have will need to be custom. The reason for this is that unlike stencil shadows, where the stencil is placed on top of the rendered scene (over-simplification), shadows from the shadowmap must be added during the lighting computations in the pixel shader.

So, in order to add shadowmapping, Sylvain would be required to write a second lighting pipeline. And not only that, but he'd need several versions of every lighting shader - for 2x2 and 3x3 pcf, optimizations, each light type, texture projection, no pcf, etc. 6.5 would never be released =(

However, Zak and I are working pretty hard to wrap this stuff up nice and conciesly, so that you should be able to pretty much just drop a class and the required files into your project, and have it go. It'll be a little more complicated than that, but that's the aim.

[sybixsus]

Besides... Look, I've made it. Once it's finished, it might fit in a DLL that you just import, initialize, update and render. Why bother Sylvain with it?

So long as it's usable for those of us who are not using .Net-based compilers and not really able to use COM objects, OLE and all that, no reason at all. But most of the third party additions ( GUI's for example ) that are around at the moment are completely unusable for myself and others, so that would be one reason.

Of course, we're not getting updates to the engine itself in line with everyone else at the moment either, so even bothering Sylvain with it might not necessarily achieve anything for us.

[Zaknafein]

That is true. If I do a port of this demo it'll be C++ or C#, not Blitz Basic/Delphi/APL or any other.
But if you guys can read VB.Net, nothing stops you from making the conversion yourselves...

[AriusMyst]

As I said in Blinds thread. Top class stuff Zak. Yours an Blinds shader knowledge always impresses me. I too agree that this should be left out of 65, its a question of balance within resource management for individual project devs. An what with you guys being gracious enough to release a demo/dll of your hard work it should be fairly simple to trace your hard trodden footsteps.

Edit: If it comes too it I could always port to Delphi. Though, I haven't used it in 6 months. Like riding a bike .