Screen-Space Ambient Occlusion( SSAO )

[Omnicrash]

I get around 58fps.
- AMD X2 6000+
- GeForce 8800GTS 640MB
- 4GB DDR2 800
- Vista Ultimate x64

[jviper]

Frame Rate 5-9 FPS
-Intel Doul Core T2300 1.666 Ghz
-ATI Radeon Mobility X1300
-1.0 GB RAM
-Windows XP Service Pack 3

When I reduce all of your (1024 x 1024 !!!!!!) Rendersurfaces to 256x256, I get up to 13-21 FPS, then 29-38 FPS When everything stops moving.

[AriusEso]

Yea JV, it was all test stuff... But maybe I got the algo wrong or something, I get a good solid 70fps( is only an x1950 pro atm ) when all the physics dies down. I dunno. Sorry guys, but maybe you can learn from the HLSL at least.

[pizzayoyo]

Nice job on this Arius, it looks really good

I get a solid 76 fps.
- Intel Core2 Duo 3Ghz
- nVidia GeForce 8800Gtx
- 4 Gb RAM
- Vista

I'm sure you'll figure out optimizations sooner or later to make it faster on slower cards.

[Shadowsong]

I can't get it to work.
If I just execute the .exe, it says "Cannot find 'TV3D65.DLL'!".
When I open the project in VS 2008 it says "fatal error C1083: Cannot open include file: 'tvall.h': No such file or directory".
I then changed #include <tvall.h> to #include "tvall.h" and added the .h manually, it then says "fatal error C1083: Cannot open include file: 'CTVActor.h': No such file or directory"

[AriusEso]

I can't get it to work.
If I just execute the .exe, it says "Cannot find 'TV3D65.DLL'!".
When I open the project in VS 2008 it says "fatal error C1083: Cannot open include file: 'tvall.h': No such file or directory".
I then changed #include <tvall.h> to #include "tvall.h" and added the .h manually, it then says "fatal error C1083: Cannot open include file: 'CTVActor.h': No such file or directory"

You have to make sure the solution can find the headers.

Lookit: http://azazeldev.org/here.jpg

[newborn]

Could people please post their FPS rate and hardware settings.

The exe file gives me "This application failed to start because the application configuration is incorrect"

Having to download the whole C++ IDE to see this project in action is a bit out of whack for me.

In any case, avi looks cool

[AriusEso]

Are you running x64?

( I forgot to change it from CPU_ALL in the compile options )

[nullsquared]

Hey, good job   I'm the same agi_shi from Gamedev and the Newton forums (and the same nullsquared over from Ogre).  I love SSAO, I think your's is coming along very nicely.  Did you get to trying that bias I mentioned?  I pretty much skimped over the whole thread, but it seems that your perpendicular surfaces (walls) still get that bad self-occlusion artifact.  Trust me, bias really helps (there was even a YouTube video that showed SSAO with various bias coefficients, but I forgot its name)

Anyways, I think you're the same guy that posted that HDR and physics demo thing on the Newton forums, right?  In that case, I've been checking out your progress for quite a bit now, and it looks really cool!

[AriusEso]

Yea, I am the same guy. It's nice to see you in this little corner of the interweb. I did try to get the bias working several times, but I couldn't work it out. I understand the theory( I think ), but I couldn't work out exactly how to implement it. I spent a while playing with all kinds of variables in the shader . I will get it eventually, I just thought I'd release it as-is for now. I also want to squeeze it down to SM2( Even at 2 samples I'm still above the SM2 limit by about 30 instructions  ).

[Shadowsong]

Yeah ok I fixed it, works fine now!

In fullscreen with 1024x768 and nothing moving:

64 FPS

- Intel Core2Duo E6750 2x 2.66 GHz
- GeForce 8800 GTS 512 MB
- 4 GB RAM
- Vista x64

[nullsquared]

Yea, I am the same guy. It's nice to see you in this little corner of the interweb. I did try to get the bias working several times, but I couldn't work it out. I understand the theory( I think ), but I couldn't work out exactly how to implement it. I spent a while playing with all kinds of variables in the shader

Well, to begin with, do you have any normal information besides the depth?

If yes: Is it in world space (unlikely and unfavorable, but can be transformed into the desired view space with a single matrix mul instruction), view space (desired), or screen-space (barely desired as post-projection space is not linear)?  However you have it, you want it to be in view-space.  Now, pretty much all you want, is to sample this normal at the same position as your original depth (so, basically, the current point being shaded).

If no: Even here, you can still "get away with it".  Compute your view space position using the depth (you already have it, since this is what you add the random direction to such that you get the new sample position in the loop).  Then just get the two derivatives of this 3D vector, normalize them, and cross them for your normal:

Code:

float3 viewSpacePos = /* compute your view space position */;
// the following are all also in view space
float3 tangent = ddy(viewSpacePos);
float3 binormal = ddx(viewSpacePos);
// you might want to flip the order here, depending on your
// target coordinate system (left/right handed)
float3 normal = cross(normalize(tangent), normalize(binormal));

Another choice is to encode the normal information directly with the depth:

Code:

// 128-bit texture
// 32-bit floating point for all 4 channels
float4 outColour = float4(depth, normal.xyz);

// or

// 96-bit texture
// 32-bit floating point for all 3 channels
float4 outColour = float4(depth, normal.xy, 0);

// reconstruct the Z portion of the normal,
// considering it should be normalized when stored
float3 normal = sqrt(1.0 - dot(inNormal.xy, inNormal.xy));

// note that you don't need the sign of the Z component
// since you only need it when normal mapping
// normal mapping is not respected within the depth information
// nor do you want it for the SSAO, so you know what the sign will be

// you can use a 48-bit or 64-bit texture even, sacrificing some
// precision to go from 32 bits per channel to 16 bits per channel

Anyways, now that you have the normal, all you do is add this normal to your random sampling direction by some adjustable coefficient:

Code:

const float BIAS_COEF = 1.0;
float3 randomSamplingDirection = randomSamples[i] + (BIAS_COEF * normal);

// then just the usual (pseudo-code)
float3 randomSamplePos = viewSpacePosition + (RADIUS * randomSamplingDirection);
float4 texCoords = mul(projection, float4(randomSamplePos, 1));
// save the following instruction, do it for free by using tex2Dproj()
texCoords.xy /= texCoords.w;

I will get it eventually, I just thought I'd release it as-is for now. I also want to squeeze it down to SM2( Even at 2 samples I'm still above the SM2 limit by about 30 instructions  ).

There's many things to do to get it to fit in shader model 2 (and it's possible, as shown by one of Iñigo Quilez's demos):
- use tex2Dproj() for the free divide by .w
- get rid of most temporaries that you don't really "need" (hard-code them)
- use the fastest possible reconstruction of the position from depth, which would be two mul instructions:

Code:

// in client-code (non-shader)
float normalizeFactor = 1.0 / farClipDistance; // do the divide on the CPU
// in shader code
// directionToFarCorner is NOT normalized - it is very simply the position
// of the far-plane corner of the frustum, interpolated from the 4 vertices
// down to the current fragment (normalization is taken care by the normalizeFactor,
// since we already know the length of the far corner position on the CPU)
float3 viewSpacePosition = directionToFarCorner * unclampedDepth * normalizeFactor;
// you want unclampedDepth to be the full view-space length of the vector
// in the depth generation
float outDepth = length(viewSpacePosition.xyz); // don't normalize the depth itself

- use the same unclamped depth trick such that you don't need any explicit "depth-unpacking"
- turn off any dynamic branching in SM 2.0: using a quadratic fall off is usually good enough even without a manual "if depth difference is too big then forget about occlusion" check as in SM 3.0
- lots of other small "tricks"

I have my same SSAO shader running in both SM 2.b and SM 3.0, it doesn't do any dynamic branching.  Not sure about vanilla SM 2.0, but I'm pretty sure you can eventually "squeeze it in".

[AriusEso]

I have the normals in the depth.

I've tried doing this, which still results in black walls at higher resolutions.

Code:

                float3 Normal      = tex2D(DeepSample, nUV).rgb;
//------
                        float3 Test   = Samples[i].xyz + (1.0f * Normal);

                        float3 Ray    = reflect(Samples[i].xyz, Norm) * (SampleRadius * Test);

[nullsquared]

I have the normals in the depth.

I've tried doing this, which still results in black walls at higher resolutions.

Code:

                float3 Normal      = tex2D(DeepSample, nUV).rgb;
//------
                        float3 Test   = Samples[i].xyz + (1.0f * Normal);

                        float3 Ray    = reflect(Samples[i].xyz, Norm) * (SampleRadius * Test);

Hold up.  What does "Normal" look like (screen shot ), is it for dithering (random noise texture), or your view-space normals?  The two are different - you want to bias by the view space normal, but dither by some noisy texture.  It seems like you're using the same "Normal" for both things.

Ok, I just took a look at your current code (ssao.shade).  Seems like DeepSample[rgb] = view space normals, and RandSample[rgb] = random dithering normals, right?  So, what you want, is this:

Code:

[...]

for(int i = 0; i < 16; i++)
{
    // this is your "random direction"
    // don't scale it by the radius *just yet*
    float3 Ray    = reflect(Samples[i].xyz, Norm);// * SampleRadius;

    // bias by some factor
    const float BIAS_COEF = 1.25; // I use just about 1.5
    Ray += VSNs * BIAS_COEF; // "VSNs" is your view space normals from above the loop

    // now you can scale by the radius
    Ray *= SampleRadius;

    [...]

[AriusEso]

I swear I have some kind of newb block on this. Sorry Agi.

Using this code:

Code:

        float4x4 Projection: PROJECTION;

        texture  Deepa;
        texture  Rand;
        float    SampleRadius  = 0.84;
        float    DistanceScale = 3.5;
        float    OccludeDist   = 7.5;
        float3   Direction;

        sampler2D DeepSample = sampler_state
        {
       Texture   = (Deepa);
       MIPFILTER = LINEAR;
       MAGFILTER = LINEAR;
       MINFILTER = LINEAR;
        };

        sampler2D RandSample = sampler_state
        {
       Texture   = (Rand);
       MIPFILTER = LINEAR;
       MAGFILTER = LINEAR;
       MINFILTER = LINEAR;
        };

        struct app2vp
        {
                float4 Position: POSITION;
                float2 UV      : TEXCOORD0;
        };

        struct vp2fp
        {
                float4 Position: POSITION;
                float2 UV      : TEXCOORD0;
                float3 Corner  : TEXCOORD1;
        };

        struct fp2s
        {
       float4 Colour  : COLOR0;
        };

        void vp( in app2vp IN, out vp2fp OUT )
        {
                OUT.Position     = IN.Position;
                OUT.UV           = IN.UV;
               
                float2 ScreenPos = IN.UV;
                ScreenPos.y      = 1.0f - ScreenPos.y;
                ScreenPos        = ScreenPos * 2 - 1;
                OUT.Corner       = float3(IN.Position.xy, 1.0f) * Direction;
        }

        void fp( in vp2fp IN, out fp2s OUT )
        {
                float4 Samples[16] =
                {
                        float4(0.355512,   -0.709318, -0.102371,  0.0 ),
                        float4(0.534186,    0.71511, -0.115167,  0.0 ),
                        float4(-0.87866,    0.157139, -0.115167,  0.0 ),
                        float4(0.140679,   -0.475516, -0.0639818, 0.0 ),
                        float4(-0.0796121,  0.158842, -0.677075,  0.0 ),
                        float4(-0.0759516, -0.101676, -0.483625,  0.0 ),
                        float4(0.12493,    -0.0223423, -0.483625,  0.0 ),
                        float4(-0.0720074,  0.243395, -0.967251,  0.0 ),
                        float4(-0.207641,   0.414286, 0.187755,  0.0 ),
                        float4(-0.277332,  -0.371262, 0.187755,  0.0 ),
                        float4(0.63864,    -0.114214, 0.262857,  0.0 ),
                        float4(-0.184051,   0.622119, 0.262857,  0.0 ),
                        float4(0.110007,   -0.219486, 0.435574,  0.0 ),
                        float4(0.235085,    0.314707,    0.696918,  0.0 ),
                        float4(-0.290012,   0.0518654,   0.522688,  0.0 ),
                        float4(0.0975089,  -0.329594,    0.609803,  0.0 )
                };
               
                float  fColour     = 0.0f;
               
                float2 nUV         = IN.UV;

                float  Deep        = tex2D(DeepSample, nUV).a;
                float3 Norm        = tex2D(RandSample, IN.UV * 200).rgb;
                float3 VSNs        = tex2D(DeepSample, nUV).rgb;

                float3 SE          = (normalize(IN.Corner) * Deep);
               
                for(int i = 0; i < 16; i++)
                {
                        float3 Ray    = reflect(Samples[i].xyz, Norm);
                       
                        const float BIAS = 1.5;
                        Ray += VSNs * BIAS;
                       
                        Ray *= SampleRadius;
                       
                        float4 Sample = float4(Ray + SE, 1);
                        float4 SS     = mul(Sample, Projection);
                       
                        float2 sUV    = 0.5 * (SS.xy/SS.w) + float2(0.5, 0.5);
                        sUV.y         = 1-sUV.y;
                       
                        float sDeep   = tex2D(DeepSample, sUV).a;

                        if(length(Deep - sDeep) > OccludeDist)
                        {
                                fColour++;
                        }
                        else
                        {
                                        float Occlude = DistanceScale * max(Deep - sDeep, 0);
                                        fColour      += 1 / (1 + (Occlude * Occlude) * 0.1);
                        }
                }

                fColour           = fColour/16;
                OUT.Colour        = float4(fColour, fColour, fColour, 1);
        }

        technique null
        {
                pass Pass0
                {
                        VertexShader = compile vs_3_0 vp();
                        PixelShader = compile ps_3_0 fp();
                }
        }

I get this result at 1280x1024:

[nullsquared]

I swear I have some kind of newb block on this. Sorry Agi.

Using this code:

Code:

        float4x4 Projection: PROJECTION;

        texture  Deepa;
        texture  Rand;
        float    SampleRadius  = 0.84;
        float    DistanceScale = 3.5;
        float    OccludeDist   = 7.5;
        float3   Direction;

        sampler2D DeepSample = sampler_state
        {
       Texture   = (Deepa);
       MIPFILTER = LINEAR;
       MAGFILTER = LINEAR;
       MINFILTER = LINEAR;
        };

        sampler2D RandSample = sampler_state
        {
       Texture   = (Rand);
       MIPFILTER = LINEAR;
       MAGFILTER = LINEAR;
       MINFILTER = LINEAR;
        };

        struct app2vp
        {
                float4 Position: POSITION;
                float2 UV      : TEXCOORD0;
        };

        struct vp2fp
        {
                float4 Position: POSITION;
                float2 UV      : TEXCOORD0;
                float3 Corner  : TEXCOORD1;
        };

        struct fp2s
        {
       float4 Colour  : COLOR0;
        };

        void vp( in app2vp IN, out vp2fp OUT )
        {
                OUT.Position     = IN.Position;
                OUT.UV           = IN.UV;
               
                float2 ScreenPos = IN.UV;
                ScreenPos.y      = 1.0f - ScreenPos.y;
                ScreenPos        = ScreenPos * 2 - 1;
                OUT.Corner       = float3(IN.Position.xy, 1.0f) * Direction;
        }

        void fp( in vp2fp IN, out fp2s OUT )
        {
                float4 Samples[16] =
                {
                        float4(0.355512,   -0.709318, -0.102371,  0.0 ),
                        float4(0.534186,    0.71511, -0.115167,  0.0 ),
                        float4(-0.87866,    0.157139, -0.115167,  0.0 ),
                        float4(0.140679,   -0.475516, -0.0639818, 0.0 ),
                        float4(-0.0796121,  0.158842, -0.677075,  0.0 ),
                        float4(-0.0759516, -0.101676, -0.483625,  0.0 ),
                        float4(0.12493,    -0.0223423, -0.483625,  0.0 ),
                        float4(-0.0720074,  0.243395, -0.967251,  0.0 ),
                        float4(-0.207641,   0.414286, 0.187755,  0.0 ),
                        float4(-0.277332,  -0.371262, 0.187755,  0.0 ),
                        float4(0.63864,    -0.114214, 0.262857,  0.0 ),
                        float4(-0.184051,   0.622119, 0.262857,  0.0 ),
                        float4(0.110007,   -0.219486, 0.435574,  0.0 ),
                        float4(0.235085,    0.314707,    0.696918,  0.0 ),
                        float4(-0.290012,   0.0518654,   0.522688,  0.0 ),
                        float4(0.0975089,  -0.329594,    0.609803,  0.0 )
                };
               
                float  fColour     = 0.0f;
               
                float2 nUV         = IN.UV;

                float  Deep        = tex2D(DeepSample, nUV).a;
                float3 Norm        = tex2D(RandSample, IN.UV * 200).rgb;
                float3 VSNs        = tex2D(DeepSample, nUV).rgb;

                float3 SE          = (normalize(IN.Corner) * Deep);
               
                for(int i = 0; i < 16; i++)
                {
                        float3 Ray    = reflect(Samples[i].xyz, Norm);
                       
                        const float BIAS = 1.5;
                        Ray += VSNs * BIAS;
                       
                        Ray *= SampleRadius;
                       
                        float4 Sample = float4(Ray + SE, 1);
                        float4 SS     = mul(Sample, Projection);
                       
                        float2 sUV    = 0.5 * (SS.xy/SS.w) + float2(0.5, 0.5);
                        sUV.y         = 1-sUV.y;
                       
                        float sDeep   = tex2D(DeepSample, sUV).a;

                        if(length(Deep - sDeep) > OccludeDist)
                        {
                                fColour++;
                        }
                        else
                        {
                                        float Occlude = DistanceScale * max(Deep - sDeep, 0);
                                        fColour      += 1 / (1 + (Occlude * Occlude) * 0.1);
                        }
                }

                fColour           = fColour/16;
                OUT.Colour        = float4(fColour, fColour, fColour, 1);
        }

        technique null
        {
                pass Pass0
                {
                        VertexShader = compile vs_3_0 vp();
                        PixelShader = compile ps_3_0 fp();
                }
        }

I get this result at 1280x1024:

Hm (this smiley looks a bit sad, where is the plain :| smiley ?)

Your code looks just about right now.  Seems like you lost the dithering, so now it started to band.  If you remove the "Ray += ..." line (just comment out the bias), does it look like before?  If it looks like it did before, then play with the bias, 1.5 is probably too strong for your scene so the addition of the normal cancels out the dithering (it's completely scene-dependent, I use meters).  If it doesn't look as it did before (without the bias), then you've obviously made some other change that caused this

Anyways, even with the banding and no dithering, walls look forked either way.  If you look straight at the walls, do they fix themselves?  Basically, are all perpendicular surfaces affected, or is it isolated to those side walls?  Because it seems a bit isolated - meaning, it's probably some depth/normal issue on those two faces and no where else.

Other than that, I've noticed some improvements you can make to moves towards you pixel shader 2.0 goal:

1) do the move/scale by 0.5 trick on the CPU:

Code:

// in CPU
// pseudo matrix class, since I've never used TV3D
matrix4x4 perspectiveToTextureSpace(
    0.5,    0,    0,  0.5,
    0,   -0.5,    0,  0.5,
    0,      0,    1,    0,
    0,      0,    0,    1
);

matrix4x4 finalProjectionMatrix = perspectiveToTextureSpace * projectionMatrix;

// and then bind this to the shader however you do it
shaderParameters->setNamedConstant("Projection", finalProjectionMatrix);

// in the shader, this is not needed, handled by fixed projection matrix
//float2 sUV    = 0.5 * (SS.xy/SS.w) + float2(0.5, 0.5);
//sUV.y         = 1-sUV.y;

// you can even take out the divide by .w if you use tex2Dproj instead
// (tex2Dproj does the divide by .w for free)

This eliminates a few instructions.

2) You sample DeepSample twice, once for the RGB and once for the A.  Just sample it once, and grab the info from that:

Code:

float4 geometry = tex2D(DeepSample, uv);
float deep = geometry.w;
float3 normal = geometry.xyz;

3) In shader model 3.0, you might find speed improvements using tex2Dlod() with a mip-map specification of 0:

Code:

float4 geometry = tex2Dlod(DeepSample, float4(uv, 0, 0)); // .w = mip map, .z is not used

This avoids some calculations on the GPU since you know that a screen-aligned texture will not use mip mapping.

EDIT: Also, technically, SSAO should be resolution independent as long as your sampling radius is constant.  Meaning, unless you base your radius on the texture size, results should be identical at 640x480 as at 1440x900, for example (minus the obvious resolution differences, of course).

[AriusEso]

Well, this seems better on the left hand wall .

[nullsquared]

Well, this seems better on the left hand wall .

Is that with a different bias coefficient, or without the bias altogether?

The walls seem like an isolated issue, unrelated to the bias or SSAO itself.  Can I have a screen shot of what your normals look like? (just output VSNs.xyz as the final SSAO)

[AriusEso]

Yeap, you're right I think. 2 walls are just black, the other 2 are ok.

http://azazeldev.org/ssao/norms.jpg

[nullsquared]

Yeap, you're right I think. 2 walls are just black, the other 2 are ok.

http://azazeldev.org/ssao/norms.jpg

I think we've found your problem.  Can I see the code you use to output the normals?  Even without seeing the code, I can tell you're using the object space normals.  Notice how differently rotated balls have different normals?  You want the view space normals for this bias to work right.  When storing the normals, just do:

Code:

float3 outNormal = mul(worldViewMatrix, normal);
// you'll want to normalize it at this point
outNormal = normalize(outNormal);

There's no way to retrieve per-object world matrix information once you render the normals, so you need to do at least the world space transformation before rendering the normals.  Though since you can do the view space transformation along with this all in one go, that's the best solution. (otherwise you'd need to do the view matrix multiplication in the SSAO shader itself, which is wasteful)