Need more shader help

[serial]

Well I started a a terrain shader a few months ago.  I back to working on it now.  I found a few problems with it.

The problem that I am having is my alpha texture isn't covering the entire terrain.  the edges are tiling. 

In the attached screen shot the black area should be black all of the way to the edges.  However it isn't you can see a thin line on the very outside each.  Its actually tiling.
I was thinking it had something to do with filtering.  but adjust that didn't change anything.

I know I've seen suggestions for these issues before but can't seem to find them. 

www.serialcrusher.net/images/ss.png

Code:

float4x4 matWorldViewProj : WORLDVIEWPROJECTION;

float L1Scale = 10;
float L2Scale = 1;
float L3Scale = 1;
float L4Scale = 1;



texture alpha : TEXTURE0;
texture layer1 : TEXTURE1;
texture layer2 : TEXTURE2;
texture layer3 : TEXTURE3;
texture layer4 : TEXTURE4;


sampler2D alphaSamp = sampler_state {
Texture = (alpha);
AddressU = Wrap;
AddressV = Wrap;

};

sampler2D layerOne = sampler_state {
Texture = (layer1);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerTwo = sampler_state {
Texture = (layer2);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerThree = sampler_state {
Texture = (layer3);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerFour = sampler_state {
Texture = (layer4);
AddressU = Wrap;
AddressV = Wrap;
};


struct VS_INPUT {
float4 position : POSITION;
float2 AlphaTC : TEXCOORD0;
};


#define VS_OUTPUT VS_INPUT
#define PS_INPUT  VS_INPUT
 
VS_OUTPUT VS(VS_INPUT IN) {
VS_OUTPUT OUT;
OUT.position = mul(IN.position, matWorldViewProj);
OUT.AlphaTC = IN.AlphaTC;
return OUT;
}

float4 PS(PS_INPUT IN) : COLOR {

         
float4 l1 = tex2D(alphaSamp, IN.AlphaTC).r * tex2D(layerOne, IN.AlphaTC * L1Scale);
float4 l2 = tex2D(alphaSamp, IN.AlphaTC).g * tex2D(layerTwo, IN.AlphaTC * L2Scale);
float4 l3 = tex2D(alphaSamp, IN.AlphaTC).b * tex2D(layerThree, IN.AlphaTC * L3Scale);
float4 l4 = tex2D(alphaSamp, IN.AlphaTC).a * tex2D(layerFour, IN.AlphaTC * L4Scale);
float4 final = l1 + l2 + l3; 
return final;

}


technique TSM1 {
    pass P0 {
        VertexShader = compile vs_2_0 VS();
        PixelShader  = compile ps_2_0 PS();
    }
}

[JukkaKevät]

I believe what you are looking for is texture clamping.

Try changing the AdressU and AdressV in the corresponding texture sampler into CLAMP (you currently have WRAP in each of them). Clamping will repeat the edge pixel for each lookup which is over 1 or below 0 on the given direction.

[SylvainTV]

I second that

[serial]

Hrmmph...Well I messed around with clamping a lot before I posted that.  I spent most of today messing with the shader.  Then I went back and set only the sampler for the alpha texture to clamp and it started to work the way I wanted.

I plan on adding specular to my shader next.  I was thinking about putting the specular info in the alpha channel of each texture. 

I have other ideas.  I will keep you guys posted as they are implemented. 

I am currently working on a class to go along with the shader that will make it easier to implement.

Code:

float4x4 matWorldViewProj : WORLDVIEWPROJECTION;

float L1Scale = 6;
float L2Scale = 6;
float L3Scale = 6;
float L4Scale = 6;



texture alpha : TEXTURE0;
texture layer1 : TEXTURE1;
texture layer2 : TEXTURE2;
texture layer3 : TEXTURE3;
texture layer4 : TEXTURE4;


sampler2D alphaSamp = sampler_state {
Texture = (alpha);
AddressU = Clamp;
AddressV = Clamp;

};

sampler2D layerOne = sampler_state {
Texture = (layer1);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerTwo = sampler_state {
Texture = (layer2);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerThree = sampler_state {
Texture = (layer3);
AddressU = Wrap;
AddressV = Wrap;
};

sampler2D layerFour = sampler_state {
Texture = (layer4);
AddressU = Wrap;
AddressV = Wrap;
};


struct VS_INPUT {
float4 position : POSITION;
float2 AlphaTC : TEXCOORD0;
};


#define VS_OUTPUT VS_INPUT
#define PS_INPUT  VS_INPUT
 
VS_OUTPUT VS(VS_INPUT IN) {
VS_OUTPUT OUT;
OUT.position = mul(IN.position, matWorldViewProj);
OUT.AlphaTC = IN.AlphaTC;
return OUT;
}

float4 PS(PS_INPUT IN) : COLOR {

         
float4 l1 = tex2D(alphaSamp, IN.AlphaTC).r * tex2D(layerOne, IN.AlphaTC * L1Scale);
float4 l2 = tex2D(alphaSamp, IN.AlphaTC).g * tex2D(layerTwo, IN.AlphaTC * L2Scale);
float4 l3 = tex2D(alphaSamp, IN.AlphaTC).b * tex2D(layerThree, IN.AlphaTC * L3Scale);
float4 l4 = tex2D(alphaSamp, IN.AlphaTC).a * tex2D(layerFour, IN.AlphaTC * L4Scale);
float4 final = l1 + l2 + l3; 
return final;

}


technique TSM1 {
    pass P0 {
        VertexShader = compile vs_2_0 VS();
        PixelShader  = compile ps_2_0 PS();
    }
}

[Raine]

I'm interested in this, keep us updated

[petrus]

Ok let me help you out And I know this works because I use it.
You could do it the long way and work with the uv coordinates or you could simply use the very neat TV3D function ExpandTexture, you kan find it in the Landscape class.
like so:
Landscape.ExpandTexture getTex("landtex"),0,0
you will need to manually send these landscape textures to your shader:
LandscapeShader.setEffectParamTexture "layer1", getTex("landtex")
Then in your shader do this:
-Do nothing, don't mess with the UVs.
-I'm using wrap not clamp.
I really hope this helps you
Anyway if you want to mesh with UVs you will need to generate UVs for every chunk. I think you need to say something like:(Vertex Shader)
OUT.UV = IN.UV/(ammount of chunks)
and then you will need to add an chunk offset.
The problem is that the UVs represent only one chunk, so you need to stretch them across the landscape and then calculate which UVs represent your current working chunk. I had this method working bevore, but using the expandTexture function will do basically the same thing.

[serial]

Thats what I've been using is the expandtexture function. 

Thanks for bringing this back up though. 

The problem I was having didn't involve tiling across the terrain like the shader does if you don't use expandtextures.  It involved the very outside edge of the terrain basically averaging the outside edge of the alphamap.  That is why only the Alpha needs to be clamped.  If you aren't tiling terrains then no biggy. But if you are it will make the edges no tile correctly.