Convert TextureSpace to WorldSpace Coords?

[sybixsus]

Hi,

I'm trying to convert texture space coordinates into world space coordinates ( for lightmapping, as per my thread in off topic on lighting. )

I've found a little documentation online, and had a stab at it, but it's not working. I'm hoping some kind soul can point me in the right direction because I'm sure I can't see what's wrong here.

To ( hopefully ) make it a little easier to help, I've cut out all the code which isn't relevant ( lighting calculations and point-in-tri checking ) because A) I know they're right, and B) They're not involved in the calculation at hand.  All the important functions are included though.

Code:

' MAIN CODE - THIS IS THE ESSENCE OF THE LIGHTMAPPER. IT CHECKS IF EACH POINT IS IN A TRI
' AND IF IT IS, IT FINDS THE WORLD COORDINATES WITHIN THAT TRI THAT THE PIXEL COORDINATES MAP TO

PrecalculateTriangles(Mesh,TexWidth,TexHeight,UVChannel)

For Local TX:Int=0 To TexWidth-1
For Local TY:Int=0 To TexHeight-1

For Local T:LightmapTriangle=EachIn LightmapTriangle.List

SearchU=(TX+0.5)/Float(TexWidth) ' TURN PIXEL COORDINATES INTO UV COORDINATES
SearchV=(TY+0.5)/Float(TexHeight)

Local PIT:Int=False

Select UVChannel

Case 0
If PointInTriangle(SearchU,SearchV,T.TA1.X,T.TA1.Y,T.TA2.X,T.TA2.Y,T.TA3.X,T.TA3.Y)
PIT=True
End If

Case 1
If PointInTriangle(SearchU,SearchV,T.TB1.X,T.TB1.Y,T.TB2.X,T.TB2.Y,T.TB3.X,T.TB3.Y)
PIT=True
End If

End Select

If PIT=True

Local DeltaUAdd:cTV_3DVECTOR=TVMathLibrary.VMultiply(T.DeltaU,SearchU)
Local DeltaVAdd:cTV_3DVECTOR=TVMathLibrary.VMultiply(T.DeltaV,SearchV)

TVMathLibrary.TVVec3Add(PointOut1,T.OriginWS3,DeltaUAdd)
TVMathLibrary.TVVec3Add(PointOut2,PointOut1,DeltaVAdd)

' WORLD COORDINATES SHOULD NOW BE IN POINT OUT 2!!

' LIGHTING CALCULATION AND SETPIXEL'ING GOES HERE
End If

Next

Next
Next

' END OF LIGHTMAPPING CODE

' FUNCTIONS USED BY THE MAIN CODE

Function PrecalculateTriangles(Mesh:cTVMesh,TexWidth:Int,TexHeight:Int,UVChannel:Int)

For Local TCount:Int=0 To Mesh.GetTriangleCount()-1

Local T:LightmapTriangle=New LightmapTriangle

T.Face=TCount

Mesh.GetTriangleInfo(TCount,T.V1,T.V2,T.V3,T.Group)

Mesh.GetVertex(T.V1,T.P1.x,T.P1.y,T.P1.z,T.N1.x,T.N1.y,T.N1.z,T.TA1.X,T.TA1.Y,T.TB1.X,T.TB1.Y,T.Col1)
Mesh.GetVertex(T.V2,T.P2.x,T.P2.y,T.P2.z,T.N2.x,T.N2.y,T.N2.z,T.TA2.X,T.TA2.Y,T.TB2.X,T.TB2.Y,T.Col2)
Mesh.GetVertex(T.V3,T.P3.x,T.P3.y,T.P3.z,T.N3.x,T.N3.y,T.N3.z,T.TA3.X,T.TA3.Y,T.TB3.X,T.TB3.Y,T.Col3)

Select UVChannel
Case 0
ComputeTangentBasis(T.P1,T.P2,T.P3,T.TA1,T.TA2,T.TA3,T.Tangent,T.BiTangent)
Case 1
ComputeTangentBasis(T.P1,T.P2,T.P3,T.TB1,T.TB2,T.TB3,T.Tangent,T.BiTangent)
End Select

T.DeltaU=TVMathLibrary.VMultiply(T.Tangent,1/Float(TexWidth))
T.DeltaV=TVMathLibrary.VMultiply(T.BiTangent,1/Float(TexHeight))

T.OriginWS=cTV_3DVECTOR.create(T.P1.x,T.P1.y,T.P1.z)

Select UVChannel
Case 0
T.XMult=T.TA1.X*TexWidth-0.5
T.YMult=T.TA1.Y*TexHeight-0.5
Case 1
T.XMult=T.TB1.X*TexWidth-0.5
T.YMult=T.TB1.Y*TexHeight-0.5
End Select

T.DeltaUSubtract=TVMathLibrary.VMultiply(T.DeltaU,T.XMult)
T.DeltaVSubtract=TVMathLibrary.VMultiply(T.DeltaV,T.YMult)

TVMathLibrary.TVVec3Subtract(T.OriginWS2,T.OriginWS,T.DeltaUSubtract)
TVMathLibrary.TVVec3Subtract(T.OriginWS3,T.OriginWS2,T.DeltaVSubtract)

' CALCULATE THE SURFACE NORMAL
TVMathLibrary.TVVec3Subtract(T.P1P2,T.P2,T.P1)
TVMathLibrary.TVVec3Subtract(T.P1P3,T.P3,T.P1)
TVMathLibrary.TVVec3Cross(T.SurfaceNormal,T.P1P2,T.P1P3)
T.SurfaceNormal.Normalize()

Next

End Function

Function ComputeTangentBasis(P1:cTV_3DVECTOR,P2:cTV_3DVECTOR,P3:cTV_3DVECTOR,UV1:cTV_2DVECTOR,UV2:cTV_2DVECTOR,UV3:cTV_2DVECTOR,Tangent:cTV_3DVECTOR,Bitangent:cTV_3DVECTOR)

Local Edge1:cTV_3DVECTOR=cTV_3DVECTOR.create(0,0,0)
TVMathLibrary.TVVec3Subtract(Edge1,P2,P1)
Local Edge2:cTV_3DVECTOR=cTV_3DVECTOR.create(0,0,0)
TVMathLibrary.TVVec3Subtract(Edge2,P3,P1)
Local Edge1UV:cTV_2DVECTOR=cTV_2DVECTOR.create(0,0)
TVMathLibrary.TVVec2Subtract(Edge1UV,UV2,UV1)
Local Edge2UV:cTV_2DVECTOR=cTV_2DVECTOR.create(0,0)
TVMathLibrary.TVVec2Subtract(Edge2UV,UV3,UV1)

Local CP:Float=(Edge1UV.Y*Edge2UV.X)-(Edge1UV.X*Edge2UV.Y)

If CP<>0

Local Mul:Float=1.0/CP

Local TangentEdgeMult1:cTV_3DVECTOR=TVMathLibrary.VMultiply(Edge1,-Edge2UV.Y)
Local TangentEdgeMult2:cTV_3DVECTOR=TVMathLibrary.VMultiply(Edge2,Edge1UV.Y)
Local BiTangentEdgeMult1:cTV_3DVECTOR=TVMathLibrary.VMultiply(Edge1,-Edge2UV.X)
Local BiTangentEdgeMult2:cTV_3DVECTOR=TVMathLibrary.VMultiply(Edge2,Edge1UV.X)

Local TangentTotal:cTV_3DVECTOR=cTV_3DVECTOR.create(0,0,0)
TVMathLibrary.TVVec3Add(TangentTotal,TangentEdgeMult1,TangentEdgeMult2)
Local BiTangentTotal:cTV_3DVECTOR=cTV_3DVECTOR.create(0,0,0)
TVMathLibrary.TVVec3Add(BiTangentTotal,BiTangentEdgeMult1,BiTangentEdgeMult2)

Local VTangent:cTV_3DVECTOR=TVMathLibrary.VMultiply(TangentTotal,Mul)
Local VBiTangent:cTV_3DVECTOR=TVMathLibrary.VMultiply(BiTangentTotal,Mul)

Tangent.x=VTangent.x
Tangent.y=VTangent.y
Tangent.z=VTangent.z

BiTangent.x=VBiTangent.x
BiTangent.y=VBiTangent.y
BiTangent.z=VBiTangent.z

End If



End Function

EDIT: More code to post. It submitted before I was ready

[JukkaKevät]

Heh, I was going to need something like this myself so I made it according to this document here:

http://www.3dkingdoms.com/weekly/weekly.php?a=39

The functions are written in VB.net. I have modified them a little (I dropped the tex height and width as it was somewhat weird imo, and as you are probably going to put the needle per pixel yourself, unnecessary).

Click for full version

Code:

    Private Function TextureToWorldSpace(ByVal VertPos0 As TV_3DVECTOR, ByVal VertPos1 As TV_3DVECTOR, ByVal VertPos2 As TV_3DVECTOR, ByVal UV0 As TV_2DVECTOR, ByVal UV1 As TV_2DVECTOR, ByVal UV2 As TV_2DVECTOR, ByVal u As Single, ByVal v As Single) As TV_3DVECTOR

        Dim DeltaU, DeltaV, originWS As New TV_3DVECTOR

        ' Compute the world space position delta along u and v
        DeltaU = ComputeTangentVecs(VertPos0, VertPos1, VertPos2, UV0, UV1, UV2, False, False)  ' Calculate tangent, don't normalize
        DeltaV = ComputeTangentVecs(VertPos0, VertPos1, VertPos2, UV0, UV1, UV2, True, False)   ' Calculate bitangent, don't normalize

        ' Compute the world space origin from the position and uv of vertex 0
        originWS = VertPos0
        originWS -= DeltaU * UV0.x
        originWS -= DeltaV * (-UV0.y + 1) ' Flip V coordinate.

        ' This is how you compute the world space position for any uv on this triangle
        Return originWS + DeltaU * u + DeltaV * v

    End Function

    Private Function ComputeTangentVecs(ByVal P1 As TV_3DVECTOR, ByVal P2 As TV_3DVECTOR, ByVal P3 As TV_3DVECTOR, ByVal UV1 As TV_2DVECTOR, ByVal UV2 As TV_2DVECTOR, ByVal UV3 As TV_2DVECTOR, ByVal calcbi As Boolean, Optional ByVal wantunit As Boolean = True) As TV_3DVECTOR

        Dim Edge1, Edge2, tangent, bitangent As New TV_3DVECTOR
        Dim Edge1uv, Edge2uv As New TV_2DVECTOR

        Edge1 = P2 - P1
        Edge2 = P3 - P1
        Edge1uv = UV2 - UV1
        Edge2uv = UV3 - UV1

        Dim cp As Single = Edge1uv.y * Edge2uv.x - Edge1uv.x * Edge2uv.y

        If cp <> 0.0 Then
            Dim mul As Single = 1.0 / cp

            If calcbi = True Then
                bitangent = (Edge1 * -Edge2uv.x + Edge2 * Edge1uv.x) * mul
                If wantunit = True Then
                    Math.TVVec3Normalize(bitangent, bitangent)
                End If
                Return bitangent
            Else
                tangent = (Edge1 * -Edge2uv.y + Edge2 * Edge1uv.y) * mul
                If wantunit = True Then
                    Math.TVVec3Normalize(tangent, tangent)
                End If
                Return tangent
            End If
        End If

    End Function

And example of usage

Code:

            Dim newpos As New TV_3DVECTOR
            Dim tmpmtx As New TV_3DMATRIX
            Dim v0UV, v1UV, v2UV As New TV_2DVECTOR
            Dim v0pos, v1pos, v2pos As New TV_3DVECTOR

            tmpmtx = Triangle.GetMatrix()

            Triangle.GetVertex(0, v0pos.x, v0pos.y, v0pos.z, 0, 0, 0, v0UV.x, v0UV.y, 0, 0, 0)
            Triangle.GetVertex(1, v1pos.x, v1pos.y, v1pos.z, 0, 0, 0, v1UV.x, v1UV.y, 0, 0, 0)
            Triangle.GetVertex(2, v2pos.x, v2pos.y, v2pos.z, 0, 0, 0, v2UV.x, v2UV.y, 0, 0, 0)

            Math.TVVec3TransformCoord(v0pos, v0pos, tmpmtx) ' Transform coordinates from local to world
            Math.TVVec3TransformCoord(v1pos, v1pos, tmpmtx)
            Math.TVVec3TransformCoord(v2pos, v2pos, tmpmtx)

            newpos = TextureToWorldSpace(v0pos, v1pos, v2pos, v0UV, v1UV, v2UV, needleU, needleV)
            Cube.SetPosition(newpos.x, newpos.y, newpos.z)

Sorry for the removal of the post for a while, there was couple of really bad errors

------
www.studiohorisen.com

[sybixsus]

Hi Jukka,

Thanks so much for your code. I was actually using the same article, it looks as though the article is not 100% correct, but I've spotted where you've adapted things and I think I've spotted what I need to change. It looks to me as though I need to use pixel coordinates instead of UV coordinates ( because I'm converting to pixel coordinates in the transform and  - as you said - you didn't do that because it seemed weird.

Also, it looks as though you've found that you need to flip the V coordinate when calculating the origin. I think I'll have to flip the V coordinate in two different places since I'm using pixel coordinates.

So yeah, I've made those changes and it seems to be working on a simple cube. I'll have to try some more advanced models and see if it's completely working or not but if not, I'll come back and have another look at your code to see if I can spot any more differences.

Thanks again for your help, it's really made a difference.

[JukkaKevät]

Good to hear that you got it working also.

I'm currently trying to solve on how to calculate world space surface normal on a per pixel basis. This is a must for any lightmapper to work correctly on smooth surfaces.

Have to check if I can find an article about that too tomorrow.

-----------
www.studiohorisen.com

[sybixsus]

Heh, you're one step ahead of me all the way. Calculating per-pixel normals is next on my to-do list, because otherwise - as you say - we're going to get hard edges on all our smooth surfaces. I haven't had a chance to do any serious reading yet because I wanted to get the texture coordinate -> world space conversion working properly first, but I believe the subject we're going to need to read up on is Phong Lighting. It's my understanding that phong lighting works by interpolating vertex normals to get a pixel normal.

If I get on to reading up on that before you do, I'll post what I find. I have a couple of books on 3d maths for game programming, so I'm hoping that one of them will cover Phong lighting.

[Zaknafein]

Phong lighting works by taking all the surrounding vertices' vertex normals, interpolating between them (can't help you on that part, the GPU does all the work for me) and then re-normalizing the interpolated vector per-pixel.

I believe SLERPing the vectors instead of LERP + normalize would give a slightly more accurate result, if you manage that.

[jmaddox]

This Gamasutra article was helpful to me.  The math in the 3DKingdoms tangent space article is wrong.

Note that the Gamasutra article does leave out one step.  If you are transforming light and view vectors to tangent space in the shader, you need to transpose the TBN matrix before using it.

Jim

[sybixsus]

Phong lighting works by taking all the surrounding vertices' vertex normals, interpolating between them (can't help you on that part, the GPU does all the work for me) and then re-normalizing the interpolated vector per-pixel.

I believe SLERPing the vectors instead of LERP + normalize would give a slightly more accurate result, if you manage that.

Aha, thanks. Yes, shaders do seem to take a lot of the strain out of some of these things, don't they? Thanks for the info about LERP and SLERP, I'll definitely bear that in mind and give both a try if and when I can find info.

The math in the 3DKingdoms tangent space article is wrong.

In what way? I put together a test mesh, which I carefully UV mapped so that I knew exactly what the world coordinates should be for a few sample pixel locations and they were perfect. I don't get want to get bitten on the ass later if there's some kind of problem though.

This Gamasutra article was helpful to me.

I read that, and couldn't make head or tail of it, to be honest. I'm comfortable with vectors and reasonably comfortable with quaternions, but I'm nowhere with matrices.

[jmaddox]

Perhaps I spoke too harshly.  I quadruple-checked the code and couldn't get it working until I ripped everything out and replaced it with code based on the Gamasutra article's equations.  Some differences came to light: signs were reversed, binormal wasn't being calculated orthogonally to the tangent and normal, etc.

But if it's working for you, maybe I just had one of those bugs that's hard to spot, or maybe it just wasn't compatible with my shader code.  Since I have everything working now, maybe if I have time I'll replace the Gamasutra math with the 3DKingdoms math again just to see if it works.

Jim