Shader Changes Actor Orientation

[Jode]

I'm new to both TV3D and Shader programming so this question is probably pretty noobish.

When I load my TVActor (an X file) it loads and looks fine.  When I apply the simple shader from Part 2 of Zaknafein's "Programming HLSL Shaders" wiki article (code shown below), my model is now rotated by 90 deg on the x-axis, and is many times larger.  I have tried other simple shaders and get the same effect, so I'm wondering if I'm missing something fundamental (like a step in the process).  Can anyone help?  Thanks.

Code:

// The combined world-view-projection matrix
float4x4 matWorldViewProj : WORLDVIEWPROJECTION;
// The mesh's diffuse material color
float3 diffuseCol : DIFFUSE;
 
// The vertex shader input structure
struct VS_INPUT {
float4 position : POSITION;
float2 texCoord : TEXCOORD0;
};
// Since the VS output and PS input structures are exactly the same in this case,
// defining aliases is faster (and possible in HLSL, noteworthy!)
#define VS_OUTPUT VS_INPUT
#define PS_INPUT  VS_INPUT
 
// Vertex shader function
VS_OUTPUT VS(VS_INPUT IN) {
// Define an "instance" of the output structure
VS_OUTPUT OUT;
// Set the screen-space position
OUT.position = mul(IN.position, matWorldViewProj);
// And the texture coordinate
OUT.texCoord = IN.texCoord;

// Return this instance
return OUT;
}
 
// Between the vertices and the pixels, the texture coordinates will be interpolated
 
// Pixel shader function
float4 PS(PS_INPUT IN) : COLOR {
// Construct a float3 RGB color with the texture coordinate's interpolated value
        // and multiply the result by the material's diffuse color
float3 rgbColor = float3(IN.texCoord, 1) * diffuseCol;

// Return that RGB color with fully opaque alpha (1.0f)
return float4(rgbColor, 1);
}
 
// Simplest technique/pass section
technique TSM1 {
    pass P0 {
        VertexShader = compile vs_1_1 VS();
        PixelShader  = compile ps_1_1 PS();
    }
}

[sybixsus]

Actors are tricky for shaders. The custom actor shader template can be downloaded here, and is good to use a reference.

http://www.truevision3d.com/downloads/files.php?cat=15

Also try setting the actor mode to CPU, and see if anything changes.

EDIT: It doesn't look as though you're handling any of the animation stuff in the shader. Are you setting the actor mode to CPU already?

[Jode]

No, I wasn't using the TVActor in CPU mode.  I tried, but it didn't change anything.  I'm not using a skinned mesh, if that makes it any clearer.  It's a test vehicle with a number of animations.

I don't know if it's significant, but if I show the Actor's bounding box, the box shows up where it is supposed to, including the correct size, but the mesh is rendered as I described before.

I found the following error in the debug log:

Code:

VIEWPORT MANAGER ERROR : Viewport_OnReset : Couldn't create rendering surfaces, unknown DirectX error. Maybe Out of video memory. dx error : -2005530516

but I always get that error, and my test models render OK.

[sybixsus]

Aha, I see. So these "animations" you mention.. they are just basic hierarchical keyframe animations? IE: Wheels are separate objects from body of vehicle and move/rotate around in the animation?

I might be wrong, but the first thing that springs to mind is whether the scale is keyframed as well as the position and rotation. I guess you could test this by creating a quick test animation where one of the object's scale changes over time. If scale is keyframed, it's something you may have to account for in the shader. Just reading the WorldViewProjection matrix and using that to transform your vertices might not be enough. Again, the custom actor shader template I referenced above might be useful for this, but I have no experience of using non-skinned animation with a custom shader. I have written a few custom shaders for actors, but all the actors are skinned, so I could follow the custom actor shader template quite closely.

If that's what it turns out to be, you'd need one of the ahader gurus to help.

[Jode]

How would I go about testing your theory?  I don't quite follow your point about a "quick test animation".  Thanks for your help.

[sybixsus]

Create an animation which is 30 keyframes long. Set a keyframe at zero with nothing changed, and then scale the model up to twice it's normal size and set that at keyframe 29. Then export it.

When you play that animation, does the scale of the object change?

[Jode]

Well, other than the size of the flat spot on my forehead, not much has changed.  I downloaded the trial version of fragMOTION to try what you suggested, but without much success.

One thing that I did notice, howvever was that when I imported the x-file, it was larger than it should have been and in the wrong orientation (it was upside down).  When I selected "AnimaitonState0", the model snapped to its proper orientation and size.  Because the simple shader that I have been using only manipulates colours and does not use bones, I realize that this has been my first mistake.  Therefore, I tried to use the CustomActorShader but my model wouldn't show up in the scene.  I tried loading the shader in SFxE but it wouldn't build - complained of a refefinition of "ActorShader" in the technique name.  I changed the name to "technique0" but the build failed again with two warnings and and error.  They were:


On Line 80 (the for loop), two warnings:

Code:

warning X3557: "Loop only executes for 0 iteration(s), forcing loop to unroll"
warning X3557: "Loop only executes for 1 iteration(s), forcing loop to unroll"

and on line 91, the error:

Code:

norm += mul(modelNorm, (float3x3)mat) * lastweitht;

causes

error X5467: Dest register type for LRP must be temp (r#).
 

I realize now that I should have started with CustomActorShader, but I can't get it to complile.  Presumably others have gotten it to work so I'm not sure what I'm doing wrong.  Any suggestions?   

[sybixsus]

Have you modified the shader? Because I can see a typo that doesn't exist in the original. The original is here:

http://www.truevision3d.com/downloads/uploads/CustomActorShader.fx

Your version has

Code:

norm += mul(modelNorm, (float3x3)mat) * lastweitht;

with "lastweight" missspelled as "lastweitht" but in the original, I'm seeing

Code:

norm += mul(modelNorm, (float3x3)mat ) *  lastweight;

[Jode]

Sorry, fat fingers (I typed the error, rather than cut/paste from SFxE).  The shader has not been modified at all.

Can you get it to compile with SFxE? (please say no, please say no, please say no...)

[jviper]

it compiles

[sybixsus]

I don't have SFxE but it compiles for me in everything else. Out of interest, what videocard do you have? Perhaps it's an issue related to a particular shader model.

[Jode]

Radeon HD 2600.

I do get a warning that 1_ are no longer supported, using 2, and I assumed that was due to the video card.  The vertex shader is being compiled under vs 2 (in the fx file) so I'm not sure why it wouldn't work.  The basic shader that I started with compiled OK.

[Dimple]

 

Hi Jode,

I got your shader to build ok. How I found the problem?  Well I selected all of the code you put in your first post and pasted it into a word document and noticed some thing odd.  A slight colored background that shouldn't have been there, so I highlighted just the text and copied it again. Deleted that document and pasted the code into SFxE - Simple FX Editor and build worked fine.  Here's the code for you to try:

Code:

// The combined world-view-projection matrix
float4x4 matWorldViewProj : WORLDVIEWPROJECTION;
// The mesh's diffuse material color
float3 diffuseCol : DIFFUSE;
 
// The vertex shader input structure
struct VS_INPUT {
float4 position : POSITION;
float2 texCoord : TEXCOORD0;
};
// Since the VS output and PS input structures are exactly the same in this case,
// defining aliases is faster (and possible in HLSL, noteworthy!)
#define VS_OUTPUT VS_INPUT
#define PS_INPUT  VS_INPUT
 
// Vertex shader function
VS_OUTPUT VS(VS_INPUT IN) {
// Define an "instance" of the output structure
VS_OUTPUT OUT;
// Set the screen-space position
OUT.position = mul(IN.position, matWorldViewProj);
// And the texture coordinate
OUT.texCoord = IN.texCoord;

// Return this instance
return OUT;
}
 
// Between the vertices and the pixels, the texture coordinates will be interpolated
 
// Pixel shader function
float4 PS(PS_INPUT IN) : COLOR {
// Construct a float3 RGB color with the texture coordinate's interpolated value
        // and multiply the result by the material's diffuse color
float3 rgbColor = float3(IN.texCoord, 1) * diffuseCol;

// Return that RGB color with fully opaque alpha (1.0f)
return float4(rgbColor, 1);
}
 
// Simplest technique/pass section
technique TSM1 {
    pass P0 {
        VertexShader = compile vs_1_1 VS();
        PixelShader  = compile ps_1_1 PS();
    }
}

[sybixsus]

Radeon HD 2600.

I do get a warning that 1_ are no longer supported, using 2, and I assumed that was due to the video card.  The vertex shader is being compiled under vs 2 (in the fx file) so I'm not sure why it wouldn't work.  The basic shader that I started with compiled OK.

Hmm, a shader model 4 videocard, so more than ample, and it's compiling under SM 2.0 which is the highest it does, and the same mine does. Nope, I guess that's not it then. Sorry. Hopefully Dimple's rewrite helps.

[Jode]

Hi Dimple,

I am also able to compile that shader (it's Zaknafein's).  My problem with that shader was that the Actor was rendered in the wrong orientation.  Because my model is animated, with bones, I think that I should have been using the CustomActorShader.  It is the CustomActorShader that I cannot get to compile with SFxE.  According to jviper and sybixsus, howver, it does compile so I'm not sure what I could be doing wrong.  I'm using the shader as downloaded.  I tried your method of cut and paste, but that didn't work either.  If others can get it to compile, it must be something stupid that I'm missing.  Thanks for the suggestion, though.

[Dimple]

 

Oops! My bad you switched to the CustomActorShader, which doesn't build for me either.  I was on IRC earlier and chatted with Aion and Toaster about the problem with the current CustomActorShader. Aion has a Neverdaunt Custom Actor Shader which Sylvain, one of the developer's helped him get working.  I have made changes to the original CustomActorShader (Build successful) and it may work for you, as I haven't tested it yet.  If what I will post below doesn't work, tell me so in another post and I will post the Neverdaunt Custom Actor Shader for you to try.

Code:

///////////////////////////////////////////////////
// Example file for custom actor shaders using GPU
///////////////////////////////////////////////////

//
// Reminder : Custom Shaders are possible via CPU actor mode
//       but they are quite slow because it must transform every vertex of the actor on CPU
//
// Here we will use a faster technique made possible recently for TVActor
//
//

// The important thing is the vertex structure used by TV.


// here is the non bumpmapped version, so there is no Tangent nor Binormal vector :
struct VS_INPUT_NONBUMP
{
float3 position : POSITION ;    // vertex position in the default pose
    float3 normal : NORMAL;   // vertex normal in the default pose
    float4 blendindices : BLENDINDICES;  // indices of the 4 bone indices used in the bone matrix array
    float4 blendweights : BLENDWEIGHT; // weights of each bone.
    float2 tex1 : TEXCOORD0; // standard texture coordinates.
};

// here is the bumpmapped version, with tangents :
struct VS_INPUT_BUMP
{
float3 position : POSITION;
float3 normal : NORMAL; //
float3 tangent : TANGENT; //  this forms the tangent matrix.
        float3 binormal : BINORMAL; //
float4 blendindices : BLENDINDICES;
float4 blendweights : BLENDWEIGHT;
float2 tex1 : TEXCOORD0;
};


// another important thing is how are stored the matrices.

// Warning : there are two possible versions of the skinning in the TVActor
// - a vertex shader 1.1 version, allowing max 16 bones in a draw call.
// - a vertex shader 2.0 version, allowing max 52 bones in a draw call.
//    These are constant values, you can't change them.

// the easiest way is to work only with VS2.0 vertex shader mode and always allocate 52 bone matrices.

// a matrix here is only using 3 rows (float4x3) so we will use an array of float4x3, it MUST have the semantic BONES
float4x3 boneMatrices[52] : BONES;

// here is the number of bones used per vertex
int iNumBonePerVertex : BONESPERVERTEX; 


// a simple world matrix is used when there is no bone influence (it must have the semantic WORLD);
float4x4 worldMat : WORLD;

// usual viewprojection to complete the transformation
float4x4 ViewProj : VIEWPROJECTION;

 
float3 dirLightDir = (0.0f,1.0f,0.0f); //: LIGHTDIR;  this is broken :so you'll have to set it from TV
float3 materialEmissive : EMISSIVE;
float3 materialAmbient : AMBIENT;
float4 materialDiffuse : DIFFUSE;
float3 materialSpecular : SPECULAR;
float materialPower : SPECULARPOWER;
float3 dirLightColor = (1,1,1); // : LIGHTDIR0_COLOR; also broken, set it manualy
float3 viewdir;// : VIEWPOS; again.. borked.. you'll need to set it every frame :\par
float2 scale = (1,1);
 
float glowpower = 3; //this changes how much backglow you have
float4 glowcolor = (1,1,1,1);//and its color
 
// here is the standard skinning function of TV to skin the position and normal
// there must be 5 versions : one for each case (0 bone blending, 1 bone blending, 2 bones blending, .. ,4 bones blending)
void TransPositionNormal( uniform int iNumBones, float4 modelPos, float3 modelNorm,
                   float4 boneWeights, float4 fBoneIndices, out float3 pos, out float3 norm)
{

if(iNumBones == 0)
{
         pos = mul(modelPos, worldMat);
norm = mul(modelNorm, (float3x3)worldMat);
}
else
{
        int4 ibone = D3DCOLORtoUBYTE4(fBoneIndices);
       int   IndexArray[4]        = (int[4])ibone;

pos = 0;
norm = 0;
float lastweight = 0.0f;
for(int i = 0; i < iNumBones - 1; i++)
{
          float4x3 mat = boneMatrices[IndexArray[i]];
  lastweight = lastweight + boneWeights[i];
  pos += mul(modelPos, mat) *  boneWeights[i];
  norm += mul(modelNorm, (float3x3)mat ) *  boneWeights[i];
}

lastweight = 1 - lastweight;
    float4x3 mat = boneMatrices[IndexArray[iNumBones-1]];
        pos += mul(modelPos, mat) * lastweight;
norm += mul(modelNorm, (float3x3)mat ) *  lastweight;
}
return;

} 


// simple sample that does nothing fancy.
struct VS_OUTPUT
{
   float4 position : POSITION;
   float4 diffuse : COLOR0;
   float2 tex : TEXCOORD0;
   float3 ldir : TEXCOORD1;
   float3 normal : TEXCOORD2;
   float3 view : TEXCOORD3;
   float3 dlc : TEXCOORD4;
};

VS_OUTPUT ActorShader(uniform int iNumBones, VS_INPUT_NONBUMP inp)
{
   VS_OUTPUT o = (VS_OUTPUT)0;
   float3 pos, norm;
   
   // use our skinning method
   TransPositionNormal( iNumBones, float4(inp.position, 1.0f), inp.normal, inp.blendweights, inp.blendindices, pos, norm);
   
   // then let's transform the position into clip space using view proj matrix
   float4 p = mul(float4(pos,1.0f), ViewProj);
   o.position = p;
   float3 n = normalize(norm);
   o.normal = n;
   o.tex = inp.tex1 * scale;
   o.view = viewdir;
   o.ldir = normalize(dirLightDir);
   o.diffuse = float4(normalize(dirLightDir), 1.0f);  // some weird const lighting computation
   return o;
}

texture texTexture : TEXTURE0; // tv will automatically set current texture to TEXTURE0
sampler sampTexture = sampler_state {Texture = <texTexture>;};

// simple pixel shader that just output the pixel
float4 ActorPixelShader(VS_OUTPUT inp) : COLOR0
{
   float3 light = normalize(-inp.ldir);
   float3 lcolor =  saturate(inp.dlc * (1, 1, 1));
   float3 view = normalize(inp.view);
   float3 inview = normalize(-view);
   float2 tcoor = inp.tex;
   float4 tColor = tex2D(sampTexture, tcoor);
   float3 ambient =  materialAmbient;
   float3 normish = normalize(inp.normal);
   float3 smooth = lerp(light,normish,0.3f);
   float3 halfway = lerp(light, view,0.3f);
   float3 specular = pow(saturate(dot(normish, halfway)), materialPower) * materialSpecular;
   float3 emissive = materialEmissive;
 
   float3 glowy = lerp(inview, normish,0.3f);
   float3 glow = pow(saturate(dot(glowy / 1.2f, inview)), glowpower) * glowcolor.rgb;
 
   float3 diffuse = pow(saturate(dot(smooth , light)) * materialDiffuse.rgb, 1.5);
 
   float3 c = (saturate(ambient + diffuse ) * tColor.rgb + specular) * lcolor + emissive + (glow * 0.8f);
 
   float alpha = materialDiffuse.a * tColor.a;
   return float4(c, alpha);
}

VertexShader VSArray[5] = { compile vs_2_0 ActorShader(0),
   compile vs_2_0 ActorShader(1),
   compile vs_2_0 ActorShader(2),
       compile vs_2_0 ActorShader(3),
   compile vs_2_0 ActorShader(4)
  };




technique ActShader
{
    pass pass0
    {
       VertexShader = ( VSArray[iNumBonePerVertex] );
       PixelShader = compile ps_2_0 ActorPixelShader();
    }
}

 

[Jode]

Thanks, Dimple.

It compiles now, and when I use it in my app, my Actor is now properly aligned and sized.  There is no texture, however, on the model.  Based on my (very) limited experience with shaders, it looks like the texture should show up.  If it's not supposed to, then I'll take a crack at adding that to the shader.  If it is supposed to show up, do you have any idea why it isn't?  Thanks again for your help so far.