Search Home Members Contacts
About Us
Products
Downloads
Community
Support
Pages: [1]
  Print  
Author Topic: Using TVLandscape.SetSplattingMode breaks the Tex3D lookup in external shader!  (Read 620 times)
Lenn
Customers
Community Member
*****
Posts: 725

+/-


« on: February 14, 2009, 03:57:08 PM »

Fixed!


--------------------------------


After around 14 hours of painstaking bug hunting, literally getting a pulsating headache, and fairly dull step-by-step code removal and analysis, I have finaly found the cause and narrowed it down for a very odd bug that can only be caused by TV3D itself!



In short:
-If I initialize a TVLandscape with splatting and set it's splatting mode (.SetSplattingMode) to use the PS2 shader, it breaks the Tex3D lookup in a completely unrelated, external shader.


Why this is the worst TV bug I've ever encountered in my 2 years of using it:
-Because unlike all bugs so far, there is no clear way for me to work around it. If this doesn't get fixed I will either have to scrap the ocean shader or landscape splatting for my project, neither of which is acceptable. I specifically need a water shader of this quality, and this was in fact one of the top 10 reasons we decided to use TV for this project. Trying to rewrite a new, slightly different shader which just MIGHT work, carries the risk that it won't work in the end, plus wasting too much time.



A way to test it:
-I have discovered this while converting Zaknafien's ocean shader to VB.Net. You can download my sample solution from:
http://www.johnmell.com/tv3d/TheOcean.zip (14 MB)

Then, all you need to do is add these lines at the end of the InitEnvironment sub (line 225):

Code:
        land.SetSplattingEnable(True)
        land.SetSplattingMode(True, True)

What that does is it causes the 6th line in the PS() (in the WaterShader.fx file), the tex3D function, to not accept the "animation" variable if it isn't in the range from 0 - 1, and generally just return wrong values:
Code:
sampledNormal = tex3D(samp3DNormal, float3(texCoord, animation));
And it screws up the UVs, the texture resolution, or something, or everything, i'm not really even sure what happens myself. But you can see the


Visual example of what happens:

UsePS2Shader = False


UsePS2Shader = True






What I have tried so far:

-The second variable in SetSplattingMode is UseTextureAlphaAsSpecularMap and setting it to true or false makes no difference.

-Splatting has to be enabled (obviously?) for this to break, using .SetSplattingEnable(True)

-When the UsePS2Shader = True, I tried setting the animation variable manually (not making it dependent on TIME), I have found out it only accepts values 0-1.

-Tried changeing most of the variable names in the shader, in case TV3D somehow switched their indexes and the splatting shader had the same variable names which were wrongly being set in the external shader. Huh But it didn't make any difference. However, I didn't change absolutely all of the variable names, just the ones that were crucial for the wrong results, but i thought that was enough testing that theory.

-Also, another obvious clue that it screws with UVs is, something strange happens to the volume map as you can see in the screenshot. It's as if it looses resolution and gets stretched on certain parts, see the strange lines? But it seems to tile as it should, or at least it sort of tiles.

-The range of the animation variable 0-1 doesn't allow the texture to be sampled entirely when it is bugged, so even setting it manually (if i ignore the screwed up resolution) isn't an option. However, the range 0-1 covers the whole texture sample when it isn't bugged (when UsePS2Shader = False).

-Tried setting AddressU and AddressV to all the different values in the    sampler3D samp3DNormal = sampler_state {}. Nothing helped.

-Tried forcing it to compile in SM2 by removing the SM3 technique. Same thing happens.

-Initializing the shader before the landscape does not help.

-Tried not using any other TVLandscape functions, only setting the splatting after generating the landscape breaks it. So that is the only culprit.
« Last Edit: February 15, 2009, 11:15:58 AM by Lenn » Logged

Zaknafein
Customers
Community Member
*****
Posts: 2929


WWW
« Reply #1 on: February 14, 2009, 08:22:55 PM »

I didn't read the whole post (sorry!) but it looks pretty clearly that the problem is in the sampling of the detail texture, NOT the volume normalmap. So you should start from there.
Logged

Lenn
Customers
Community Member
*****
Posts: 725

+/-


« Reply #2 on: February 15, 2009, 10:50:34 AM »

Read it whole, please Smiley

OK, maybe i've drawn a wrong conclusion. Maybe the tex2d sampler is getting bugged as well then? But the volume normalmap sampler is surely getting wrong results, why would values higher than 1 be acceptable when not bugged (ps2 shader = false) as opposed to when it is bugged. That's what I focused on, but you're right about the detail map being wrong too. So there's a problem with more than just 1 sampler then.

Conclusion: The samplers return wrong results when ps2 shader = true
Logged

Lenn
Customers
Community Member
*****
Posts: 725

+/-


« Reply #3 on: February 15, 2009, 11:14:42 AM »

Fixed: Solution was that the landscape PS2 shader changes the AddressW clamping default value so it had to be changed to AddressW = Wrap; once again in the shader. However, Sylvain said he removed the renderstate change so in future people shouldn't have this problem.

So it was in part due to me not knowing enough about shaders and that default renderstates can remain set after a shader changes it (ie the landscape shader), and the next shader (ie the water one) doesn't specify it.
Logged

Pages: [1]
  Print  
 
Jump to:  

Powered by SMF 1.1.3 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks