After reading a little about SSAO (Screen-Space Ambient Occlusion), I came up with the intriguing idea of using the same method to also compute secondary light bounces a la photon mapping/path tracing in screen space on rasterization hardware. It really does seem possible now that I think about it. In addition to the ambient occlusion contribution, a light bleeding contribution will be composited with the Phong direct-lighting rendering. You're, of course, going to face some inaccuracies because you're using the depth buffer, which does not give you a complete view of the surrounding geometry, but I'd imagine for simplistic environments like the familiar Cornell box scene, this really won't matter as much (at least aesthetically). You're also going to only have a limited number of samples (8-32 depending on hardware capabilities) to work with for evaluating your illumination integral (bidirectional reflectance distribution function (BRDF)) over a hemisphere Ω via Monte Carlo. So you're coming up with a crude approximation to the illumination integral on rasterization hardware. You're going to have to definitely use the color buffer as well for fetching samples for the rays of light intersecting the scene geometry. Since SSAO is a multi-pass technique, you have the directly-lit color buffer readily available. I'd imagine, although not perfect, this method might yield some pretty convincing results. I think some pictures here might illustrate what I'm talking about (Wikipedia "Rendering equation", Boreskoff "Screen-Space Ambient Occlusion", 2008):

Computing illumination over a hemispherical region via rays

Sampling the depth buffer for crude ray-geometry intersections

Hemispherical illumination integral (f_r() - bidirectional reflectance distribution function (BRDF); L_i,r() - light incident, reflected; ω_i,r - incident, reflected light directions)

I apologize for the lack of MathML here with the rendering equation. I'm definitely going to give this more thought since I'm pretty sure nobody else has really explored this idea yet. Maybe I haven't really looked hard enough yet, or I haven't realized a pressing limitation that could prevent this from actually being feasible. Also, I've been reading about some other promising real-time global illumination methods such as placing additional point lights "ad hoc" within the scene where the secondary light bounces are occurring, and I'm really curious to find out how this method might stack up against them. I am working on an OpenGL demo right now of this technique because if it works, it could really be a huge step for improving the realism of rasterized 3D graphics (we're talking light bleeding for indoor scenes!). I'd imagine NVidia/ATI are looking into finding a way to provide developers with some sort of "geometry buffer" so that a full view of the surrounding scene geometry can be utilized for "screen-space" sampling (as opposed to the more "incomplete" depth buffer).

EDIT: Hmm, looks like someone beat me to it, nullsquared on the Ogre forums. Back to the drawing board. There's probably some more cool stuff worth looking into here, though. I'll see what I can come up with in my own implementation.

I finally got HDR lighting working (took only a day). I definitely took inspiration from the Mirror's Edge, where the bloom effect lends a pristine quality to the dystopian city environments. Like any over-enthusiastic game developer, I over-exaggerated the bloom highlights to the point where it's unrealistic, but that's the point. Here are the obligitary screen shots (yes, I just got Windows 7 RC1):

Glowing Character Model

Blooming Light Sources

More Blooming Light Sources

Specular Highlights

Overbright Behavior

Really, it was just a matter of importing already-written code. The FBO stuff I threw into my Scene class though really sets the stage for some shadow mapping. I think I'm going to go ahead and implement variance shadow maps or at least percentage closer filtering. After that, there won't be much major graphics coding left for now so I'll finally be able to work on some gameplay stuff (net code, scripting, game logic, etc.). It took me a while to get familiar with my code again. I forgot what a lot of stuff did and where a lot of things were at, but I'm starting to get the hang of things again.

I could probably experiment with different ways of tone-mapping and blurring. Right now I'm using a Gaussian blur with radius eight that takes advantage of the linear separability of the filter kernel matrix. HDR textures are now supported, too, thanks to Greg Ward's RGBE file format. The guy happens to be from my University, too, which is pretty neat. I saw some stuff about auto-exposure correction (the human eye does this already), and I definitely saw it in action in Mirror's Edge so it will be interesting to try to implement so I can avoid tirelessly tuning the values in the shader for each environment.

You're probably wondering why the frame rate is so low. Barring my sub-par NVidia Quadro FX laptop GPU, my main excuse is the horribly inefficient one pass per light rendering path. I really need to work on deferred shading before I go any further. Also, this article I found in GPU Gems on Managing Visibility for Per-Pixel Lighting is going to be really helpful since it analyzes lit, visible batches in terms of set theory. In other news, I've been reading up on atmospheric scattering (Rayleigh and Mie scattering along with aeral perspective) to try to take my flight simulator's graphics out of the N64 era. I've gotta give credit to both GPU Gems (Sean O'Neil) and Ralf Stokholm Nielsen for doing excellent research on the topic.

I uploaded a You Tube video of the 3D engine that's been sitting around on my hard drive for a while. Eventually I'm going to reformat all of my videos into 720p high definition now that You Tube finally made the screen dimensions of its videos larger than a postage stamp. If anything, it's worth watching because of the Origin Unknown/MC Dynamite track.

It really hasn't changed much since November. I added in smooth camera path-following interpolation and began to work on improving collision response (by adding wall sliding and gravity like in Quake). I might take a break from working on the graphics side of things and work more on the functionality for a while, but first I have to survive finals week.

Well there's a bit of a lull between exams here at school so I figured I'd quickly write an update for the site. I remembered I was working on a GLSL water shader demo a while ago, but I didn't get around to posting screenshots. This demo uses reflection/refraction complete with Snell's Law and the Fresnel equation. I was having some coordinate system problems but I'm sure when I get around to taking linear algebra in a few semesters, I'll be able to work things out. Oh, and Happy Halloween! Afrika Bambaataa is coming here to perform and talk about old school hip hop, too, which will be a nice break after my killer math prelim. Without further ado:

3D Water Demo

Update: I got Afrika Bambaataa and GW Theodore (the inventor of record scratching) to sign my MP3 player. Unfortunately, the sharpie smudged a bit while it was drying in both signatures, but it's still legible.

Autographs

Finals are finally done, along with high school, so I got back into working on the 3D engine a bit. I wrote code to parse the BSP30 light entities. Then I found out that there were only eight light entities concentrated in one room -- the rest of the map was left dark.

Light Entities

Well I found out that some of the textures can act as emissive lights, and now the scene is fully lit. I displace the lights a bit along the normal vectors of the textured polygons so that more of the scene is lit. I had to guess on what some of the numerical values meant in the .RAD file. The light sources are all attenuated point (area) lights, but I'd like to add spot lights and flickering lights. The nice thing about area lights is that I optimized them by culling out all of the leafs outside the spherical area of each light. I also performed visibility culling with the light itself when the bounding sphere is outside the view frustum. In order to minimize lit triangles, I also cut down on the visible leaves by performing frustum testing with the leaf bounding boxes. There's still some work to do with the stencil shadows though. I'd like to try some of the GL extensions used for two-sided stencil depth testing.

Texture Lights

The pictures were taken before optimization where FPS was in the 10-70 range. Now with all of the leaf culling, it's in the 60-120 range. I'd like to get it up even higher so that HDR lighting and stencil shadows are still viable. Maybe I should try per-face culling for each visible leaf since unnecessary faces really do seem to slow things down a lot. I disabled stencil shadows for now because there are around 30 lights in the entire scene and that would require 30 shadow passes for each object until I start optimizing the shadow code as well. Later on, the level geometry is going to have to self-shadow itself because right now, only MD2 models act as shadow casters.

I added in ray-polygon and sphere-polygon collision detection. It was a little tricky trying to perform intersection testing with polygons, rather than just triangles. If the point of intersection is inside the polygon, the sum of the angles between the point and adjacent vertices is 2pi. Then tests need to be done for the edges and points as well. Collision response still needs sliding, but I added in some gravity to stop falling player entities from hitting the ground. Now player models have their own ellipsoid bounding volumes. I followed the paper "Improved Collision Detection and Response". I'm having some trouble with sliding right now, but I think it has to do with the recursive function I'm using.

Oh, and I updated the Running page.

All right... here's a much-needed pre-finals week update on the FPS project. First, I added in multiple per-pixel light support with stencil shadows that mask out the diffuse light from each source.

Multiple Per-Pixel Lights

From what I've read, pretty much the only way you can handle multiple lights with stencil shadows is if you render one pass per light. I also render an ambient/depth pass for the shadows so that depth calculations don't need to be performed for each pass. Still, it's a lot slower than I expected because there's a huge FPS drop when I try to enable stencil shadows. I'm also getting weird stencil buffer/depth issues when I try z-fail shadows so it's z-pass for now. I might move on to shadow maps (maybe soft shadows) a bit later but I diverted my work elsewhere:

BSP30 Loading

Yep! BSP30 loading, complete with textures and PVS leaf culling.

Per-pixel BSP30

To make things more exciting, I hacked in two per-pixel point lights into the scene for some real-time lighting that rivals precomputed lightmaps. Later, I will parse the light entities from the BSP file itself and render the actual level's lights. Here's a second shot:

More Per-pixel BSP30

I also added in XML scripting support to make my engine more game engine-like. Right now, you can define custom materials (lighting properties, shaders, normal maps, etc.) in scripts for each texture in the BSP. I then threw in some normal map material scripts to make the rendering appear more like Doom 3:

Normal Mapped BSP30

Well, not quite. The textures in the BSP30 maps I could find are all small, as in 128x128 pixels or less. I'm going to have to find some replacements. Then I'm going to have to generate normal maps for all of them, and add shadows back in. Should look pretty awesome after that. And finally, a mock gameplay concept screenshot:

Gameplay Concept

Here's some more of my long ramblings about what's going on. I formulated the TBN basis vectors as a linear system of equations, following Christopher Cameron's method which I read about in an online undergraduate graphics course at CMU. I still think there's a better way of doing this, though, since the TBN matrix always has to use the face normals now. This means I can't use the precomputed normals provided by the artist (well, generated by the 3D modelling app). As a result, I switched to using the weighted averages method for calculating the vertex normals in order to eliminate the blatant blockiness of the models, but this limits the artist's freedom to modify the smoothness of each surface. At least all of this work will allow me to have animated 3D models with normal mapping in my engine. Here's a lovely picture of the new-fangled tangent space coordinate system vectors:

Tangent Space Vectors

I really need to try to optimize this stuff, too. I'm doing a ton of matrix math every frame because of the interpolation required by the keyframe animation code. I'm almost certain some of the repetitive number crunching can either be done lazily or precomputed. Maybe I should look into skeletal animation while I'm at it (in other words, derive a Quake 3 or Doom 3 model loader from my object base class). Come to think of it, the shadow code is doing a lot of repeated work, too... I still want to stick to my goal of having the final product run on a Geforce 6 or better.

In other news, I quickly hacked together some parallax mapping in a pixel shader. It's pretty simple stuff once you realize that all you really need to do is distort the texture coordinates of the normal and the texture maps based on the view vector and the height of the surface. This is about as exciting as a 10 polygon room gets unless I venture into the realm of relief mapping shaders.

Parallax Mapping

Another shot of parallax mapping

I have a lot of boring infrastructure stuff to throw in now since there's no real scene graph yet. I'm embarrassed to say this, but I still need to coalesce all of my shadow rendering into one pass. On the plus side, having a scene graph will allow me to optimize things, clean up a lot of code, add in camera support (complete with my quaternion class), and load up some interesting scenery (read: Quake 1,2,3 levels). I will finally be able to integrate my HDR lighting code from my other project into the engine, too, so things will start looking as good as Oblivion (wishful thinking). Actual gameplay is still a long way off, but I haven't decided yet whether to work on some of the game logic before or after the scene graph.

I added in the bump mapping code from the teapot demo. I made a point of using that cliche brick texture that every bump mapping demo on the planet uses. I still need to decide on how I'm going to precompute the tangent vectors for every surface on the Quake 2 models though. That will give me a chance to do away with the Quake 2 normal lookup table. Here's what I've got so far:

Bump Mapping

Parallax mapping, relief mapping, and HDR lighting are on their way.

I finally snuck in some more coding time. I got around to writing a Quake 2 MD2 loader for my 3D engine, and I reorganized a lot of the code. I also implemented stencil shadows complete with self-shadowing! Eventually, I could do psuedo-soft shadows by jittering the light and rendering a bunch of extra passes, but right now, the hard shadows look fine. Quake 2 models have extremely low polygon counts, of course, and the normal vectors are pulled out of a single lookup table that every Quake 2 model in the world uses (yikes). Maybe I'll have to try computing weighted average vertex normals instead because there's visible discontinuities along the surfaces.

Quake 2 Player Model

What's left? More model formats, multiple lights, multiple shadow casters, more code reorganization, and tangent space vector calculation for the normal maps. I've been struggling to get a water shader going with normal mapping and environment mapping because of tangent space coordinate system issues. I will probably revisit the problem sometime later on. I have a few workaround ideas. Lastly, the environment needs to be more interesting than a brick box. I'll be looking into loading vintage Quake maps and rendering them using normal mapping and stencil shadows like Tenebrae does. Basically, Doom 3 graphics with decade-old artwork. You probably noticed that there isn't any HDR lighting in this screenshot. I'm still cleaning up the code for that. My rigid body dynamics engine is temporarily in hiatus until I replace the hackish impulse-based contact forces with an analytical LCP solver.

Quake 2 Weapon

I've been slowly but surely working on a rewrite of the OO3D engine. I've removed the ODE support, and I have integrated my rigid body dynamics engine from the flight simulation. Also, I've started reading papers about some of the more recent graphics technologies and have incorporated them in my engine. A lot of new features have been added, especially with GPU programming. I'm still sticking with OpenGL for now because it's multiplatform.

Teapot Picture

Here is the venerable Utah Teapot rendered using HDR lighting, cubic environment mapping, refraction, reflection, chromatic aberration, bump mapping, parallax mapping, and per-pixel lighting. The HDR lighting uses floating point FBOs, performs luminance tone mapping, and loads HDR light probes in the OpenEXR/RADIANCE format. Right now I'm using the common "Campus at Sunset" light probe along with an inflated exposure time setting in order to exaggerate the Gaussian bloom effect. For the sake of the tech demo, I threw in every single feature I could. It gets about 30-40 FPS on my Geforce 7950 GT. I haven't really optimized anything yet, and there's still some very prominent aliasing around the edges. There's no shadows yet because I haven't decided whether I want to go with stencil shadows, shadow mapping, or some sort of soft shadowing. I also need to work on loading 3D models and displaying environments such as terrain or BSPs. The mathematics behind realistic water rendering seem fascinating as well, and I can't help but look at Crysis and wonder how it works. Hopefully, I'll be up to speed with some of the newest rendering technologies soon.

Everyone deserves the opportunity to receive a first-rate education and to freely pursue their own intellectual interests. With respect to this ideal, the One Laptop Per Child project sets its sights on providing $100 Linux-based notebook computers to children in third world countries. The project emphasizes affordability by using economical hardware and software components. It also promotes connectivity via mesh networking.

I was lucky enough to have been provided with a computer and Internet access when I was a child, and it pretty much changed my life. I suppose my Nintendo DS work can be compared to this project because I envisioned transforming the device into a functional $200 computer with wireless Internet access. Spread the word; this is a good cause, and computers could have the same effect on others' lives as they did on mine.

Be sure to share your support for Wintermute and his library that makes NDS homebrew possible. Nominate Devkit Pro for the Sourceforge Community Choice Awards. Everyone seems to be picking "Best Tool or Utility for Developers" for the category:

In other news, I posted a new happy hardcore mix of Rank 1 - Airwave here. I've been working on it off and on for a few weeks with my M-Audio Axiom keyboard.

Also, I've been experimenting with web-based 3D graphics using Java. If you have the latest JRE, check out my demo. It might take a while to load because I'm hosting the JOGL and Gluegen library files on my broadband line at home. I have some pretty big plans for online Java applications. In the meanwhile, convince Sun to release a 64-bit browser plug-in!

EDIT: I better move the Java rant to another section. :P

All right, here's a much-needed StellaDS Atari 2600 emulator update before the major overhaul begins. Changes include:

• DLDI and libfat support for all of the new flash cartridges
• Updated to latest Devkit Pro tool chain
• Long-awaited streaming sound without any crackling and popping noises
• Code optimizations (code in the ITCM and the new armv5te compiler optimizations)
• Faster speed but still not 100%
• Usability enhancements with the menus (key repeat, no more bogus dot entries)

So now the sound finally works as it should, and many more people with micro SD cards and the like can enjoy StellaDS. Look forward for more features in the future! Wifi multi-player Warlords or Pong anyone? How about 100% speed via an assembly-optimized CPU emulator? I can't make any guarantees, but I'm definitely trying to implement these features. Grab the new release from the StellaDS Download page. Enjoy!