Search Home Members Contacts
About Us
Products
Downloads
Community
Support
Pages: [1]
  Print  
Author Topic: Post Processing Sample (+ Component/Service Architecture)  (Read 63316 times)
Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« on: June 26, 2008, 02:05:30 PM »

Heya, it's been a while since I posted something here. Cheesy

http://www.theinstructionlimit.com/wp-content/uploads/2008/06/FullScreenShaders.zip (C# 3.5, Visual Studio 2008 - 8.67 Mb)

This sample showcases different post-processing shaders with different-sized input surfaces (including the straight Main Buffer). It's really simple in terms of visuals, and it doesn't reinvent anything at all, but my intent is to show how to properly use texture coordinate offsets to do pixel-perfect sampling with post processing shaders.
Also it shows how to accumulate different effects in any order without using more than two surfaces, sometimes even a single one!

The Vignette shader was heavily inspired (i.e. stolen) from Arius' HDR demo. Smiley

I realize that the shaders and sample code have no documentation at all, so here's a little explanation.

When using the main buffer, a single RenderSurface is needed to do as many (single-input & single-output) post-process shaders as you want. I call it the Post surface. So if you want to apply a Grayscale effect, after TV.RenderToScreen(), you :
- Blit the main buffer unto the Post surface
- Draw_FullscreenQuadWithShader from (0, 0) to (1, 1) while passing Post as input texture
- In the vertex shader, offset the texture coordinates by half the texel size of what you sample. That means a resolution of 1024x768 will need a positive offset of (0.5/1024, 0.5/768) to have pixel-perfectness.

When using a smaller/larger intermediary surface, you actually need two surfaces; a Temp surface and the Post surface. So if you want to apply a Grayscale effect...

Before TV.Clear() : (once per effect)
- Blit the Post surface onto the Temp surface (this will be read by the shader)
- StartRender on the Post surface
- Draw_FullscreenQuadWithShader from (0, 0) to (1, 1) while passing Temp as input texture
- In the vertex shader, offset the texture coordinates by half the texel size of what you sample. That means a resolution of 1024x768 downscaled by a factor of two will need a positive offset of (0.5 / 1024 / 2, 0.5 / 768 / 2) to have pixel-perfectness. It entirely and solely depends on the size of the texture you sample.
- EndRender on the Post surface

After TV.RenderToScreen() : (once for all effects)
- Blit the main buffer onto the Temp surface (this will be used for the next frame's effects)
- Draw_Texture using the Post surface (which contains the accumulated effects), making sure that you upscale/downscale the coordinates to fill the entire screen
- Blit the Temp surface onto the Post surface (this will be used for the next frame's effects)

If you followed this right, you'll notice that my 2nd method (the one with 2 RSes) is "one frame late" at all times. But this is pretty much unavoidable, I think. And it never shows at 30fps+.



But that's only part of what this sample's about. You'll also find a satellite project called ComponentFramework which is my proposal for a basic, over-simplifying yet very functional component/service architecture for TV3D.

As you may know, I've been working with XNA on Fez for almost a year now, and when coming back to TV3D for some tests or prototypes, I find it hard not to work with game components like XNA provides.
So I've decided to take what I liked from XNA's component framework, add in what I already extended in Fez, and even some more stuff to keep the game code as lean and simple as possible.

I have documented my ComponentFramework entirely with XML comments so it should be understandable, and the sample serves as an example usage.

A couple of things to note :
- Components marked at [AutoLoad] and Services which are not marked [Disable] will be automatically loaded by reflection when calling Core.Run()
- Service injection is used when using the [ServiceDependency] attribute on properties with public setters. This is available inside services and components.
- The "Controllers/Effects/Hosts" namespaces I made in the sample are not necessary or important, it's a design choice. You can do whatever you want, you can even not use namespaces at all.

Enjoy!
« Last Edit: June 26, 2008, 03:00:49 PM by Zaknafein » Logged

Raine
Customers
Community Member
*****
Posts: 1212


« Reply #1 on: June 26, 2008, 03:57:26 PM »

mmm yes the Zak is back
Logged

Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #2 on: June 26, 2008, 07:35:18 PM »

For bonus points, spot the 300 meme reference.
Logged

JohnB
Customers
Community Member
*****
Posts: 183


« Reply #3 on: June 26, 2008, 11:24:48 PM »

Thanks for the samples Zak - top notch work as always!

John B.
Logged
Raine
Customers
Community Member
*****
Posts: 1212


« Reply #4 on: June 27, 2008, 01:40:28 AM »

"is this madness? THIS, IS, REFLECTION!"

I think I won Tongue
Logged

Mietze
Community Member
*
Posts: 415

Pleeease, don't let it crash!


WWW
« Reply #5 on: June 27, 2008, 09:23:34 AM »

Just wanted to test this out, but I get a MissingMethodException right after:

06-27-2008 16:15:09 | SHADER MANAGER : Shader 'VignetteView' successfully created.

This occurs cause of the new TextureFont_Create(), which actually takes 8 (for cleartype) instead of 7 parameters. Recompiling it with the new DLL works - just to mention it.

Overall it shows the correct way to combine post process shaders, which is a nice contribute to the community.

I'd love to see this stickied! Great job!
Logged

Check out my blog at www.e-studioz.de - The finest in gross trash Wink
Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #6 on: June 27, 2008, 09:57:15 AM »

That's odd, I thought I was using the latest DLL. Mine is dated May 15th, and in the download section all packs are dated May 15th as well.
Maybe you're using some private updated DLL?
Logged

Mietze
Community Member
*
Posts: 415

Pleeease, don't let it crash!


WWW
« Reply #7 on: June 27, 2008, 10:37:18 AM »

That's odd, I thought I was using the latest DLL. Mine is dated May 15th, and in the download section all packs are dated May 15th as well.
Maybe you're using some private updated DLL?
Hm.. weird one. My one has the timestamp from 5th May Oo

Could you please check that this fits your DLL?

public int TextureFont_Create(string sUserFont, string sFontName, int iSize, bool bBold, bool bUnderlined, bool bItalic, bool bInternational, bool bClearType)
Logged

Check out my blog at www.e-studioz.de - The finest in gross trash Wink
jviper
Community Member
*
Posts: 2131

Discipline in training


« Reply #8 on: June 27, 2008, 10:52:42 AM »

Finally a sample of Zaks that runs above 50 FPS!!!  Grin
Anyway, when I downloaded your sample, there was no TV3D DLL (which I think is a good idea anyway). I referenced mine.
Logged

JAbstract.....Don't just imagine, make it happen!
AriusEso
Customers
Community Member
*****
Posts: 940

Esoteric


« Reply #9 on: June 27, 2008, 11:01:00 AM »

The Vignette shader was heavily inspired (i.e. stolen) from Arius' HDR demo. Smiley

I bet you added awesome. Smiley

Anyways, I just installed XP. Once I have AoC and then VS installed I'll take a look see. My post stuff is sloppy at best, as you know. Wink
Logged

-...-

Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #10 on: June 27, 2008, 11:21:40 AM »

Could you please check that this fits your DLL?

public int TextureFont_Create(string sUserFont, string sFontName, int iSize, bool bBold, bool bUnderlined, bool bItalic, bool bInternational, bool bClearType)

My DLL has :
Code:
public int TextureFont_Create(string sUserFont, string sFontName, int iSize, bool bBold, bool bUnderlined, bool bItalic, bool bInternational)
and
Code:
public unsafe int TextureFont_Create(string sUserFont, string sFontName, int iSize, bool bBold, bool bUnderlined, bool bItalic, bool bInternational, bool bUseClearTypeAntialiasing)
So aside from the parameter name, the 2nd signature fits...

And thanks everyone, glad you all like it. Smiley

I'm working on an update for this sample. For a couple of months I was working on-and-off on the math and theory behind the Kaiser downsampling filter, now I'm trying to actually implement it. So instead of blitting to a lower-resolution surface, you can also use a Tent shader or a Kaiser shader.
Logged

Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #11 on: June 28, 2008, 05:47:09 PM »

I should've posted this in "In Development",... because it's still highly WIP.

I've already added :
- A 2-pass Kaiser 2x Downsampling shader that works really well with anti-aliasing, and pre-computed weights for 4, 8, 16, 32, 64 and even 128 taps. But anything higher than 16 is too slow and quite ugly, there are heavy ringing artifacts like Jon Blow mentioned in his article.
- A single-Pass Tent/Triangular 4x Downsampling shader that also works great with anti-aliasing, and uses hardware filtering to do only 4 texture samples.
- A double-blit 4x downsampling technique that uses an intermediary surface to lose less pixels than a single blit, it still looks like crap compared to my Tent shader.
- A used memory counter to compare the memory usage of the different settings.

I'm currently adding/will add :
- A fast Gaussian Blur shader that uses hardware filtering to have the same visual results with half the texture samples. (up to 17-tap filtering at the cost of 9-tap!)
- A class that generates weights for the Kaiser window function
- A cleaned .NET 3.5 class for generation of the Gaussian Distribution (like I had in my Bloom and original Gaussian Blur samples)
- A 2-pass Bicubic Interpolation shader as an alternative to blitting (I haven't actually started it, but I really want to try making one).

I had forgotten how fun TV3D programming is. Shocked
« Last Edit: June 28, 2008, 05:53:43 PM by Zaknafein » Logged

Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #12 on: July 01, 2008, 01:46:28 PM »

Alright, here's an update! And I'll stop losing all of my waking hours on this silly experiment...

http://www.theinstructionlimit.com/wp-content/uploads/2008/06/FulscreenShaders.01072008.zip (C# 3.5, Visual Studio 2008 - 8.9 Mb)




I've implemented the following downsampling modes :

- Blit; this is just a call to BlitFromMainBuffer from a smaller-sized rendersurface. For 2x downscaling, it's bilinear sampling, but for 4x it just doesn't sample enough and becomes similar to nearest-neighbour.

- Double Blit; this is only available for 4x downscaling and is just two calls to BlitFromMainBuffer, each being a 2x downscale. Oddly enough, it still looks worse than real bilinear filtering, and requires an additional rendersurface.

- Bicubic; this should be exactly like Photoshop's bicubic downsampling filter. I know that it works great for 2x but I'm not sure it works as expected for 4x. It sharpens the image a tiny bit.

- Tent/Triangular (4x Bilinear); this is a true 4x bilinear downsampling filter. It uses hardware filtering to do it one pass and only 4 texture samples. This setting produces the "smoothest" 4x downsampling, but it doesn't preserve fine details.

- Sinc with Kaiser window; this is the theoretically ideal downsampling filter, but I'm having mixed success with it. I'm not even sure if I'm calculating the weights right... But it looks relatively OK. But like my primary reference says, the more taps you use, the more "rippling" you get, especially for big source images. It does a fair amount of sharpening though and keeps a lot of detail.

I tried for a full day to implement the bicubic interpolation shader for upscaling, but I failed miserably. I have to re-think my approach, and I should really get to work on something more useful. Shocked
So the code is there, but it doesn't work, and I've disabled the radio-button.

Also I've made a big number of improvements on my Gaussian Blur implementation :

- "Fast Gaussian"; this mode uses hardware filtering and custom tap offsets to nearly halve the number of texture samples. For instance, a 9-tap blur usually takes 9 samples per pass, and with in "fast" mode it takes only 5!
I thought that this would double the speed, but hardware filtering has its cost, and usually you win only 10-15 FPS on my hardware. But what it does allow is filtering up to 17-tap in a single pass, and with very few instructions.

- Standard Deviation tweaking; in my past demos, the standard deviation for 'N' taps was fixed, I thought I had found the ideal value for each tap count. But I realized when playing with 15- and 17-tap filters that past a certain threshold, since the weights are normalized in order not to lose any light, the kernel becomes an averaging Box filter and stops widening.
So depending on the amount of "lost light" of the gaussian approximation, the bell curve becomes an horizontal line and the filter stops looking like a Gaussian Blur!
I then calculated the standard deviation at which the kernel loses ~0 light and set that as the ideal st-dev. And found out that this ideal st-dev is very low, and allows only a very minute amount of blurring...
I just needed a method of measuring how "box-like" the gaussian kernel was for a st-dev, and find at which percentage of "likeness" it becomes too much. I used a correlation coeffeicient calculation between the current gaussian kernel and a box kernel (and some normalization stuff to keep it between 0 and 1), which shows up as "Box-Likeness" under the slider.
My conclusion is that at 70%-75% is the highest you can go with a pretty believable filter, but it's really up to you to decide what looks best.

Whew. I'm going to rewrite this more coherently in a blog entry and with more references/images. I have pretty huge Excel documents that back all this up.

I've also made a handful of fixes to my Component Framework, but not much worth mentioning. They're XML-documented.

Comments/critique appreciated!
« Last Edit: July 01, 2008, 02:04:40 PM by Zaknafein » Logged

SylvainTV
Administrator
Community Member
*****
Posts: 4971


WWW
« Reply #13 on: July 01, 2008, 02:23:43 PM »

Really nice !

Don't worry it's really worth the time you're putting on it Smiley
Logged

Regards

Sylvain Dupont
TrueVision3D Developer
sylvain@truevision3d.com

TV3D IRC at http://chat.truevision3d.com or on server irc.truevision3d.com #Truevision3D. Come talk with us !
Zaknafein
Customers
Community Member
*****
Posts: 2940


WWW
« Reply #14 on: July 03, 2008, 12:10:18 AM »

Sorry for bumping, but if you're interested in what I did with the Gaussian blur in my latest sample, I'm currently making a series about my findings on my blog. First part is here : http://www.theinstructionlimit.com/?p=40
Logged

Pages: [1]
  Print  
 
Jump to:  

Powered by SMF 1.1.3 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks