Shader FAQ

[BlindSide]

The MX series does not support shaders. The cards that do are:

Geforce 3, ps1.0, vs1.0
Geforce 4 TI, ps1.1, vs1.1

FX 5200-5900 ps2.0,vs2.0
Geforce 6800+ ps3.0,vs2.0 (not sure if there is a vs3.0, but if there is, these cards would support it).

Shader performance on anything less than a 5900 sucks, so if you get a new card, get a 6800 (AGP + Nvidia), 7800 (SLI + Nvidia, fastest card out), or x800 (AGP + ATI), or x850 (AGP + ATI, fastest AGP card out).

[Zaknafein]

How do you shading gurus calculate the vector between the current vertex and the camera eye?
Best I could do was :

Code:

float4x4 worldMatrix : WORLDMATRIX;
float3 viewPositionVector : VIEWPOSITION;

...

OUT.vecLookAt = -(normalize(viewPositionVector - mul(float4(IN.posVertex, 1.0), worldMatrix).xyz));

...but that looks a little GPU-intensive for such a simple operation...
I read in a GLSL book that they just normalize the vertex position in "eye space", but in HLSL I can't manage it. Or is my way the only way?

[Zaknafein]

A little tip for those into shading...
If you use the vertex normals with the NORMAL semantic, be sure to transform them using the WORLDIT/WORLDINVERSETRANSPOSE float4x4 matrix before working with them, or else all the matrix transformations on your mesh will not affect the normals!

The values in the NORMAL semantic are in object-space, and a simple mul(IN.normal, worldMatrix) will not work, you need to mul them with the inverse-transpose matrix... because the up-scaling you apply to a vertex for example, will down-scale its normal! (or something like that)

I learned that reading http://www.gignews.com/realtime020100.htm

[gg]

Hi guys,
Some examples of fx effects that can be usefull for your projects
http://download.nvidia.com/developer/SDK/Individual_Samples/effects.html
gg

[Zaknafein]

Nice PDF presentation about the differences between shader models 2.0, 2.0a, 2.0b and 3.0.

http://www.ati.com/developer/gdc/D3DTutorial_ShaderModels.pdf

I believe the ATI Radeon X800 series does 2.0b and the GeForce FX 5000 series do 2.0a. The most recent cards (GF 6000 series and Radeon X1000 series) do 3.0, and the older ones (GF 4000 series and Radeon 9000 series) do 2.0. Before that it's 1.4.

Edit : Even better stuff on wikipedia...
http://en.wikipedia.org/wiki/High_Level_Shader_Language

[PhunkeyMonkey]

thanks to Sylvain I found out that if you put semantic TEXTURE0 (or 1,2,3) for texture in your shader, you can change those textures from TV with SetTextureEx.

Hi GD,

Please could you explain in a little more detail how I put that semantic in the shader. This is my first time using shaders and  I'm trying to get cell shading working on textured meshes and actors.

Here is the shader I'm using:

Code:

//**************************************************************//
//  Effect File exported by RenderMonkey 1.6
//
//  - Although many improvements were made to RenderMonkey FX
//    file export, there are still situations that may cause  
//    compilation problems once the file is exported, such as
//    occasional naming conflicts for methods, since FX format
//    does not support any notions of name spaces. You need to
//    try to create workspaces in such a way as to minimize  
//    potential naming conflicts on export.                  
//  
//  - Note that to minimize resulting name collisions in the FX
//    file, RenderMonkey will mangle names for passes, shaders
//    and function names as necessary to reduce name conflicts.
//**************************************************************//

//--------------------------------------------------------------//
// ToonWithDynamicSpecular
//--------------------------------------------------------------//
//--------------------------------------------------------------//
// Single Pass
//--------------------------------------------------------------//

float4x4 world_view_projection : WorldViewProjection;
float4x4 view_matrix : View;
float3 lightDir : LIGHTDIR1_DIRECTION = { 0, 0, 1 };
float4 diffuseMaterial : DIFFUSE = { .51f, .6f, 1, 0 };

struct VS_INPUT
{
   float4 Pos    : POSITION0;
   float3 Normal : NORMAL0;

};

struct VS_OUTPUT
{
   float4 Pos      : POSITION0;
   float2 TexCoord : TEXCOORD0;

};

VS_OUTPUT NPR_ToonWithDynamicSpecular_Single_Pass_Vertex_Shader_vs_main( VS_INPUT In )
{
   VS_OUTPUT Out;

   Out.Pos = mul(In.Pos, world_view_projection);

   float3 posW    = mul( In.Pos, view_matrix );
   float3 normalW = mul( In.Normal, (float3x3)view_matrix );

   float diffuse = max(0,dot(normalW, -lightDir));
   Out.TexCoord.x = diffuse;
   Out.TexCoord.y = 0.0f;

   return Out;
}

texture ToonShaderTexture_Tex
<
   string ResourceName = "Toon.bmp";
>;
sampler ToonShaderTexture = sampler_state
{
   Texture = (ToonShaderTexture_Tex);
};
struct PS_INPUT
{
   float2 TexCoord : TEXCOORD0;
   float4 Color : COLOR0;
};

struct PS_OUTPUT
{
   float4 Color    : COLOR0;
};

PS_OUTPUT NPR_ToonWithDynamicSpecular_Single_Pass_Pixel_Shader_ps_main( PS_INPUT In )
{
    PS_OUTPUT Out;
 
    Out.Color = tex2D(ToonShaderTexture,In.TexCoord);
    Out.Color *= diffuseMaterial;
    return Out;
}

//--------------------------------------------------------------//
// Technique Section for Effect Workspace.NPR.ToonWithDynamicSpecular
//--------------------------------------------------------------//
technique ToonWithDynamicSpecular
{
   pass Single_Pass
   {
      VertexShader = compile vs_1_1 NPR_ToonWithDynamicSpecular_Single_Pass_Vertex_Shader_vs_main();
      PixelShader = compile ps_1_4 NPR_ToonWithDynamicSpecular_Single_Pass_Pixel_Shader_ps_main();
   }

}

Dayne Rathbone

[billythekid]

You get four layers for custom shaders. In the shader you index them with the semantic TEXTURE0-3. In TV code you index them using CONST_TV_LAYER.

Change

Code:

texture ToonShaderTexture_Tex
<
   string ResourceName = "Toon.bmp";
>;

To

Code:

texture ToonShaderTexture_Tex : TEXTURE0
<
   string ResourceName = "Toon.bmp";
>;

Then in your TV code, apply the texture to layer 0 like this:

Code:

textureID = TextureFactory.LoadTexture("Toon.bmp");
mesh.SetTextureEx((int)CONST_TV_LAYER.LAYER0, textureID)

The code probably isnt correct, but hopefully you get the idea.

[PhunkeyMonkey]

Thanks billythekid. That makes sense, but when I tried implementing the code I get no shading on my actor at all.

My ultimate goal is to apply the shader to a textured actor, but at the moment the shader only works on untextured meshes and actors.

The "Toon.bmp" is a simple 1D texture like this:

[billythekid]

Did you set the actor mode to CPU? If you dont, TV wont let you apply any shader to it.

[PhunkeyMonkey]

Yea I did.

The shader works fine on untextured actors. It just uses the color of the mesh to applying the shader to. The problem only occurs when I apply the shader to a textured actor. In this case neither the texture or the shader works properly and just the untextured and unshaded mesh is displayed.

[billythekid]

Well the UVs could be messed up on the model. You are using tex2D in the pixel shader instead of tex1D. I would create a sphere using TV code then apply a texture and shader to that. The UVs on the sphere will be correct. If it still doesnt work then you know it has something to do with your shader.

[PhunkeyMonkey]

Code:

CTVMesh* pMesh;
pMesh = pScene->CreateMeshBuilder();
pMesh->CreateSphere(200);
pMesh->SetShader(CelShader::GetInstance()->getShader());

Applying the shader to a standard mesh created by TV results in the mesh not displaying at all! I had only tested the shader on meshes and actors loaded from TVM and TVAs (with some success) and I was very supprised that it didnt work at all with a default sphere.

Loading a texture on the mesh and then applying the shader has no visible effect - the mesh is still not displayed.

The shader I am using comes from the RenderMonkey install, and the only changes I have made to it have been to make it compatible with TV. Is there another shader that might work better you could reccommend?

Thanks,
Dayne

PS: Changing tex2D to tex1D in the shader had no visible difference in the TVMs that do display.

[billythekid]

Whats your complete TV code for loading the shader, texture, mesh and applying the texture and shader to the mesh?

[PhunkeyMonkey]

Loading the shader:

Code:

void CelShader::Init()
{
pShader = GameLogic::GetInstance()->GetScene()->CreateShader("CelShader");
pShader->CreateFromEffectFile("Cel Shader EDIT.fx");
}

Loading texture and mesh, and applying texture and shader to mesh:

Code:

CTVMesh* pMesh;
pMesh = pScene->CreateMeshBuilder();
pMesh->CreateSphere(200);
int textureID = pTFactory->LoadTexture("./tests/nmcdonald_test_spottexture.jpg");

pMesh->SetTexture(textureID);
pMesh->SetShader(CelShader::GetInstance()->getShader());

Note that the texture mesh displays fine without the shader applied, and the shader works ok on untextured meshes loaded from TVMs.

[billythekid]

Change

Code:

pMesh->SetTexture(textureID);

To

Code:

pMesh->SetTextureEx((int)cTV_LAYER_0, textureID);

Make sure you have the semantic TEXTURE0 in the shader. Also make sure you include tv_types.h.

P.S. If an object with a shader applied to it doesnt show up at all, that means there is something wrong with the shader 99% of the time.

[PhunkeyMonkey]

Yea, I did try put the TEXTURE0 semantic in my shader before (as you advised), but for some reason when I have it there my shader doesn't work at all. Even when i dont set the texture in TV.

The textured mesh doesnt render at all, and the TVAs that were shaded without the TEXTURE0 semantic now become unshaded.

[billythekid]

Try removing the annotations in the shader then, the stuff in the <> brackets. TV doesnt need those.

EDIT:
Looking more closely at your shader. Why do you calculate the UVs in the vertex shader? Normally you would just let TV send the UV data to the vertex shader and simply pass that data to the pixel shader.

[PhunkeyMonkey]

The only brackets left in my shader are here:

Code:

texture ToonShaderTexture_Tex : TEXTURE0
<
   string ResourceName = "Toon.bmp";
>;

This is required to link to the Toon.bmp texture used in the cell shading - not to texture the mesh in the usual way.

To be honest I'm not exactly sure how the shader works. This is my first time using shaders and I was hoping to implement an existing cel shader into TV without having to learn to code my own from scratch in the middle of our game project.

In short, I dont know why the UVs are calculated in the vertex shader.  :cry:

[billythekid]

Well using TV, you should be loading Toon.bmp as your texture (via TextureFactory) and applying it to layer 0. TV cant read annotations, so it is useless to have them for TV. Try that first, then we can talk about the other stuff.

[PhunkeyMonkey]

TV must be using this code:

Code:

<
   string ResourceName = "Toon.bmp";
>;

because when I remove it or chage the file (Toon.bmp) the shader does not work in TV or SoulShader at all.

Toon.bmp is just the texture used by the shader (somehow) to apply the cel-shading look, and it appers to be linking and working correctly within the shader. What I want to do is apply my own texture to the mesh (or use a textured TVM) to give it a more detailed look, but also have it cel-shaded.

I appreciate all the advice billythekid, but my shader simply refuses to work at all when I add the TEXTURE0 sematic (regardless of whether I apply the texture to layer0 in TV or not).

I'm sure you are correct and that the problem is in the shader itself, but at the moment I just dont know enough about HLSL to understand the code fully. I'll read up about HLSL this weekend and hopefully figure out what my shader is doing.