Shadow Gitches With Disable/Enable

[sybixsus]

I've never seen this on a previous build, so I have to assume this is something which has changed in the very latest version.

My graphics settings menu has an option to disable and enable dynamic ( stencil ) shadows in-game. When this is done, I cycle through all the objects in my level and do the following :

If it's a mesh and it receives shadows, I make a note so that it is not rendered before FinalizeShadows().

If it's a mesh and it casts shadows, I call SetShadowCast(False)

If it's an actor and it casts shadows, I call SetShadowcast(False)

If it's a light and it casts shadows, I call SetLightProperties and disable casting.


Then if I reenable shadows, I do the precise opposite. However, when I do this, I get stencil errors. Some vertices are offset badly from the mesh and leave those long stencil trails, like you get with an unclosed mesh and other general stencil errors. The actual erroring vertices seem to be moving all around as the direction and position of the stencil errors keeps changing.

Here's a pic

How it looks with shadows enabled :

http://www.shadersforgames.com/badshadows1.jpg

How it looks with shadows disabled and renabled :

http://www.shadersforgames.com/badshadows2.jpg



If you quit to the menu and reload the scene, all is fine again.

[SylvainTV]

Well have not changed anything in shadows for a while.
But I remember this bug occured from time to time.

Does it happen more on actors or meshes ?

[sybixsus]

It always happens with actors, I've never seen it happen with a mesh, and I have several meshes casting shadows in the same scene, so if it was happening, I would see it.

[sybixsus]

I know you said you haven't changed anything in shadows for a while, but I've found another, unrelated, bug with stencil shadows which was not happening until recently. Now I'm finding that mesh shadows are disappearing when the camera is in a certain position. To make it even more odd, the shadows reappear without moving the camera if an actor ( which also casts shadows ) in a completely different area of the map begins to move.

At first I thought it was only happening when the camera was between the light and the shadow caster, but I've since proven this to be false. It is only at certain angles and positions though. Once the object is moved, I can't get it to happen again at any angle. Similarly, once the actor ( which is genuinely nowhere near ) moves, I can't get the shadows to disappear on the mesh again.

One other strange little bug, completely unrelated to stencil shadows, but since it happened at the same time, I noticed that a custom shader was being displayed incorrectly once. It only happened once, and I could never get it to happen again. It happened while I was testing some resolution changing code, so perhaps the shader is not always reloaded when the device is lost? I've tried it over and over again and I can't get this to reoccur, but it just did it this once. It loaded ok, and the diffuse texture was displayed correctly too, but the normal mapping and specular highlights were not displayed. So perhaps it was actually the normal map which was not reloaded correctly? I'm just guessing here. If I find a way to reproduce it reliably, you'll be the first to know.

I only mention this in the same thread on the ( unlikely ) possibility that there is some kind of connection between these issues and it makes more sense to you than it does to me. All I know is that I was not getting any of these issues before the latest bulld and now suddenly, I am, and I don't really have much to go on.  Hopefully it makes a bit more sense to you than it does to me.

[sybixsus]

I've been playing more with this today and I cannot under any circumstance get the shader bug to reoccur, so I'll leave that aside for now.

I can get the shadow bug to happen consistently in my game, but it's very strange. I have one mesh and one actor casting shadows and one mesh receiving shadows. Shadows are always rendered for the actor, but not always for the mesh. The mesh shadow is not cast when the camera is in a certain position, and that position seems to be roughly between the light source and the mesh, but not precisely. Moreover, if the actor is in a certain position, the mesh will cast a shadow properly, regardless of where the camera is.

So the position of the actor definitely affects whether or not the mesh has it's shadow cast correctly, which makes no sense to me. Why would the actor affect the mesh? Unless.. I thought the actor was an enclosed mesh. If it were not, if there were a hole or an unwelded vertex somewhere, could this lead to the sort of problem I'm reporting with a completely unrelated mesh not casting a shadow when the camera is in certain positions? If not, what other possible explanation could there be for the actor position affecting the shadow of an unrelated mesh?

[SylvainTV]

Well there is shadow culling using the shadow volume bounding boxes.
That probably explains why a shadow can suddently disappear when you move the camera, the mesh, or the light.

Shadows are treated in the same way for actors and meshes, so the fact that a mesh disables an actor shadow is possible because it uses the same buffer then.

However this kind of bug is very difficult to reproduce here unfortunately and it doesn't happen every time either

There was another bug with a problem with shadows not appearing when using one mesh with bumpmapping lighting and one mesh without. I'm probably going to test & fix that first and who knows it might fix also other problems