Polygon Pi

Lighting is what makes a 3D scene look real.  You used to hear talk of the number of polyons or the resolution of the textures in a game, but these days GPUs are powerful enough that we’re rarely limited by these things in any meaningful sense.  The focus now is on the lighting.  The most detailed 3D model will look terrible if it’s unlit, and the simplest scene can look photorealistic if rendered with good lighting.

I’ll first cover the most basic lighting solutions that have been used in games for years, and later talk about more advanced and realistic methods.  This will cover simple ambient and diffuse lighting.  But first of all I’ll explain a bit of the theory about lighting in general and how real objects are lit.

A bit of light physics

Eyes and cameras capture images by sensing photons of light.  Photons originate from light sources, for example the sun or a light bulb.  They then travel in a straight line until they are bounced, scattered, or absorbed and re-emitted by objects in the world.  Finally some of these photons end up inside the eye or the camera, where they are sensed.

Not all photons are the same – they come in different wavelengths which correspond to different colours.  Our eyes have three types of colour-sensitive cells which are sensitive to photons of different wavelengths.  We can see many colours because our brains combine the responses from each of the cell types, for example if both the ‘green’ and ‘red’ cells are stimulated equally we’ll see yellow.

Televisions and monitors emit light at just three wavelengths, corresponding to the red, green and blue wavelengths that our eyes are sensitive to.  By emitting different ratios of each colour, almost any colour can be reproduced in the eye.  This is a simplification of the real world because sunlight is made up of photons with wavelengths all across the visible spectrum, and our cones are sensitive across a range of wavelengths.

Objects look coloured because they absorb different amounts of different frequencies of light.  For example, if a material absorbs most of the green and red light that hits it, but reflects or re-emits most of the blue light, it will look blue.  Black materials absorbs most of the light across all frequencies.  White materials absorb very little.

The rendering equation

Feel free to skip this section if you don’t get it.  The rendering equation is a big complicated equation that states exactly how much light will reach your eye from any point in the scene.  Accurately solving the rendering equation will give a photorealistic image.  Unfortunately it’s too complicated to solve it exactly, so lighting in computer graphics is about finding better and better approximations to it.  The full equation looks something like this:

Don’t worry about understanding it.  What it says is that the light leaving a point towards your eye is made up of the light directly emitted by that point, added to all of the other light that hits that point and is subsequently deflected towards your eye.

If you zoom in closer and closer to any material (right down to the microscopic level), at some point it will be effectively ‘flat’.  Hence we only need to consider light over a hemisphere, because light coming from underneath a surface would never reach it.  Here is a slightly easier to understand picture of what the rendering equation represents:

The reason that the rendering equation is so hard to solve is because you have to consider all of the incoming light onto each point.  However, all of that light is being reflected from an infinite number of other points.  And all the light reflected from those points is coming from an infinite number of other points again.  So it’s basically impossible to solve the rendering equation except in very special cases.

Ambient light

Hopefully you’re still with me!  Let’s leave the complicated equations behind and look at how this relates to simple lighting in games.

As the rendering equation suggests, there is loads of light bouncing all over the place in the world that doesn’t appear to come from any specific direction.  We call this the ambient lighting.  At its simplest, lighting can be represented as a single colour.  Here is my simple test scene (a box and a sphere on a flat floor plane) lit with white ambient light:

To work out what the final colour of a surface on the screen is, we take the lighting colour for each channel (red, green, blue) and multiply it with the surface colour for that channel.  So under white light, represented in (Red, Green, Blue) as (1, 1, 1), the final colour is just the original surface colour.  So this is what my test world would look like if an equal amount of light hit every point.  As you can see, it’s not very realistic.  Constant ambient light on its own is a very bad approximation to world lighting, but we have to start somewhere.

As a quick aside, I mentioned earlier that we only use three wavelengths of light when simulating lighting, as a simplification of the real world where all wavelengths are present.  In some cases this can cause problems.  The colour of the floor plane in my world is (0, 0.5, 0), meaning the material is purely green and has no interaction with red or blue light.  Now we can try lighting the scene with pure magenta light, which is (1, 0, 1), thus it contains no green light.  This is what we get:

As you can see, the ground plane has gone completely black.  This is because when multiplying the light by the surface colour, all components come out to zero.  Usually this isn’t a problem as scenes are rarely lit with such lights, but it’s something to be aware of.

Directional light and normals

The first thing we can do to improve the lighting in our scene is to add a directional light.  If we are outside on a sunny day then the scene is directly lit by one very bright light – the sun.  In this case the ambient light is made up of sunlight that has bounced off other objects, or the sky.  We know how to apply basic ambient, so we can now look at adding the directional light.

It’s time to introduce another concept in rendering, which is the normal vector.  The normal vector is a direction that is perpendicular to a surface, i.e. it points directly away from it.  For example, the normal vector for the floor plane is directly upwards.  For the sphere, the normal is different at every position on the surface and points directly away from the centre.

So where do we get this normal vector from?  Last time, I introduced the concept of vertex attributes for polygon rendering, where each vertex contains information on the position, colour and texture coordinates.  Well, the normal is just another vertex attribute, and is usually calculated by whichever tool you used to make your model.  Normals consist of three numbers, (x, y, z), representing a direction in space (think of it as moving from the origin to that coordinate in space).  The length of a normal should always be 1, and we call such vectors unit vectors).  Hence the normal vector for the floor plane is (0, 1, 0), which is length 1 and pointing directly up.  Normals can then be interpolated across polygons in the same way as texture coordinates or colours, and this is needed to get smooth lighting across surface made up of multiple polygons.

We also need another vector, the light direction.  This is another unit vector that points towards the light source.  In the case of distant light sources such as the sun, the light vector is effectively constant across the whole scene.  This keeps things simple.

Diffuse lighting

There are two ways that light can interact with an object and reach your eye.  We call these diffuse and specular lighting.  Diffuse lighting is when a photon is absorbed by a material, and then re-emitted in a random direction.  Specular lighting is when photons are reflected off the surface of a material like a mirror.  A completely matt material would only have diffuse lighting, while shiny material look shiny because they have specular lighting.

For now we will only use diffuse lighting because it is simpler.  Because diffuse lighting is re-emitted is all directions it doesn’t matter where you look at a surface from, it will always look the same.  With specular lighting, materials look different when viewed from different angles.

The amount of light received by a surface is affected by where the light is relative to a surface – things that face the sun are lighter than things that face away from it.  This is because when an object is at an angle, the same amount of light is spread over a greater surface area (the same reason as why the earth is cold at the poles).  We now have the surface normal and the light angle, so we can use these to work out the light intensity.  Intuitively, a surface will be brighter the closer together the normal and the light vectors are.

The exact relation is that the intensity of the light is proportional to the cosine of the angle between them.  There is a really easy and quick way to work this out, which is to use the dot product.  Conveniently enough the dot product is a simple mathematical function of two vectors that happens to give you the cosine of the angle between them, which is what we want.  So, give two vectors A and B, the dot product is just this:

(A.x * B.x) + (A.y * B.y) + (A.z * B.z)

To get the diffuse lighting at a pixel, take the dot product of the normal vector and the light vector.  This will give a negative value if the surface is pointing away from the light, but because you don’t get negative light we clamp it to zero.  Then you just add on your ambient light and multiply with the surface colour, and you can now make out the shapes of the sphere and the box:

Better ambient light

At this point I’ll bring in a really simple improvement to the ambient light.  The ambient represents the light that has bounced around the scene, but it’s not all the same colour.  In my test scene, the sky is blue, and therefore the light coming from the sky is also blue.  Similarly, the light bouncing off the green floor is going to be green.  When you can make generalisations like this (and you often can, especially with outdoor scenes), we may as well make use of this information to improve the accuracy of the ambient light.

In this case we can simply have two ambient light colours – a light blue for the sky and a green for the ground.  Then we can use the surface normal to blend between these colours.  A normal pointing up will receive mostly blue light from the sky, and a normal pointing down will receive mostly green light from the floor.  We can then apply a blend based on the Y component of the normal.  This is the scene using just the coloured ambient:

And then add the diffuse lighting back on:

Now just combine the lighting with some texture mapping and you’re up to the technology levels of the first generation of 3D polygonal games!

That’s more than enough for this post, so next time I’ll talk about specular lighting.

We saw in part 3 how to move the camera around a wireframe world.  Now it’s time to move onto proper solid 3D rendering.  For this we need to introduce a few new concepts: basic shading, vertex attributes, textures and the depth buffer.

Solid rendering

Moving from wireframe to solid rendering can be as simple as filling in between the lines.  There are a load of algorithms for efficient scanline rasterisation of triangles, but these days you don’t have to worry about it because your graphics hardware deals with it – simply give it three points and tell it to draw a triangle.  This is a screenshot of a solid-rendered triangle, which is pretty much the most basic thing you can draw in D3D or OpenGL:

That’s not very exciting, but that’s about all you can draw with only positional data.  In case you wonder why I choose a triangle, it’s because a triangle is the simplest type of polygon and all rendering eventually breaks down to drawing triangles (for example a square will be cut in half diagonally to give two triangles).

Vertex attributes and interpolation

I’ll just clarify a bit of terminology.  A vertex is a single point in space, usually used to define the corners of a shape, so a triangle has three vertices.  An edge is a line between two vertices, so in wireframe drawing we just draw the edges.  A polygon is the whole filled in shape, such as the triangle above.

Vertices don’t just have to contain positional information, they can have other attributes.  One example of a simple attribute is a colour.  If all the vertices are the same colour then the whole polygon could just be drawn the same colour, but if the vertices are different colours then the values can be interpolated between the vertices.  Interpolation simply means to blend smoothly from one value to a different value – for example, exactly half way between the two vertices the colour will be an even mix of each.  Because triangles have three vertices, a slightly more complex interpolation is actually used that blends smoothly between three values.  Here is an example of the same triangle with coloured vertices:

Texture mapping

The other common vertex attributes are texture coordinates.  A texture is just a picture, usually stored as an image file (although they can be generated algorithmically at runtime).  Textures can be applied to polygons by ‘stretching the picture’ across the polygon.  You can think of a 2D picture as having coordinates like on a graph – an X coordinate running horizontally, and a Y coordinate running vertically.  These coordinates range from 0 to 1 across the image, and the X and Y coordinates are usually called U and V in the case of textures.

Textures are applied to polygons by specifying a U and V coordinate at each vertex.  These coordinates (together referred to as UVs) are interpolated across the polygon when it is drawn, and instead of directly drawing a colour for each pixel, the coordinates are used to specify a point in the texture to read from.  The colour of the texture at the point is drawn instead.  This has the effect of stretching some part of the texture across the polygon that is being drawn.

As an example, here is a texture and a screenshot of part of it mapped onto our triangle:

   

This is just a really quick introduction to the basics of polygon rendering.  There are a lot more clever and interesting things that can be done which I will talk about in later sections.

A question of depth

The problem with rendering solid geometry is that often two pieces of geometry will overlap on the screen, and one will be in front of the other.  The two options for dealing with this are either to make sure that you draw everything in back-to-front order, or to keep track of the depth of each pixel that you’ve rendered so that you don’t draw more distant objects in front of closer ones.

Rendering back to front will give the correct screen output but is more expensive – a lot of rendering work will be done to draw objects that will later be drawn over (called overdraw), and additional work is required to sort the geometry in the first place.  For this reason it is more efficient in almost all cases to use a depth buffer.

Screen buffers

First lets talk about how a computer stores data during rendering.  It has a number of buffers (areas of memory) which store information for each pixel on the screen.  It renders into these buffers before copying the contents to the display.  The size of the buffers depends on the rendering resolution and the quality.  The resolution is how many pixels you want to draw, and as an example if you want to ouput a 720p picture you need to render 1280×720 pixels.

The colour buffer stores the colour of each pixel.  For a colour image you need at least one byte of storage (which can represent 256 intensity levels) for each of the three colour channels, red, green and blue.  This gives a total of 256x256x256 = 16.7 million colours, and so each pixel required three bytes of storage (but due to the way memory is organised there is a fourth spare byte for each pixel, which can be used for other things).  These days a lot of rendering techniques require higher precision but I’ll be writing more about this later on.

The second type of buffer is the depth buffer.  This is the same resolution as the colour buffer but instead stores the distance of the pixel along the camera’s Z axis.  This is often stored as a 24-bit floating point number, meaning it can represent basically any number (to varying degrees of accuracy).  Here is an example courtesy of Wikipedia of the colour and depth buffers for a scene, where you can see that pixels in the distance have a lighter colour in the depth buffer, meaning that they are a larger value:

Depth buffer rendering

Using a depth buffer to render is conceptually very simple.  At the beginning of the frame, every pixel in the buffer is reset to the most distant value.  For every pixel of every polygon that is about to be rendered to the screen, the depth of that pixel is compared with the depth value stored for that pixel in the depth buffer.  If it’s closer, the pixel is drawn and the depth value updated.  Otherwise, the pixel is skipped.  If a pixel is skipped then no calculations have to be done for that pixel, for example doing lighting calculations or reading textures from memory.  Therefore it’s more efficient to render object from front to back so that objects behind closer ones don’t have to be drawn.

That’s all that needs to be said about depth buffers for now.  Next time I’ll be talking about lighting.