# shaders

## Functy Updates – Shadows and UI Improvements

The last two weeks of summer break have given me the opportunity to give Functy some long-overdue attention. As I mentioned in my previous post, one of the improvements I’ve been working on has been dynamic shadows, and these are now fully implemented. The result is a big improvement in the visual fidelity and sense of depth in the visualisation. You can see the difference in the two screenshots below and I’ll include a proper comparison in a future post.

The next step will be to test out ambient occlusion, as kindly suggested by Andrea Bernabei (@faenil). I’m not sure how well this will work with the type of functional objects rendered by Functy, but it’s worth investigating. Ambient occlusion produces the sort of shadow effects more likely to occur indoors where the light tends to be more scattered, as compared to directed lights or outside direct sunlight, as the current version produces. However, research has shown that ambient occlusion gives a heightened sense of depth as compared to direct shadows, so it’ll be interesting to see the results.

The second major change has been to the user interface. This has been completely rebuilt from the ground up to fit with a single window workbench-style interface, as opposed to the previous multiple-window toolbox-style interface.

Personally I’m a big fan of multi-window interfaces; it should be the job of the window manager to allow you to arrange and configure your windows however you want. Sadly most window managers seem to be lacking in the flexibility department, and managing multiple windows becomes an exercise in hide-and-seek, trying to find windows that got lost behind others like a stack of papers on a desk. So, even though it’s not the perfect solution, I’ve converted Functy so that everything is visible and available in a single window.

I have to admit that the result does look more professional and I think it’s a positive change. I’m not quite sure what will happen if things get more complicated, but it works well for the timebeing.

There’s still a fair bit of work to be done before this can be given a full binary release, such as updating everything to work on Windows, and switching from libglade to GtkBuilder (this may be for a future time). Feel free to test out the version from source (it’s very easy to build, honest!) in the meantime.

## Shadow mapping

One of the great benefits of teaching on computer game development modules at the university is that I have a justifiable reason for looking into interesting graphical effects for my work. It’s a real perk as far as I’m concerned. Last semester I covered shadow rendering as a new topic, which gave me the opportunity to find out about the subject in some depth.

Now it’s the summer and I have more time to experiment, so it only seemed right to apply some of these techniques to Functy; I’m hoping the function rendering will look a lot more realistic and solid once accurate shadows are being cast throughout the environment. Given the functions are all generated on the GPU, texture-based shadows seemed the most appropriate choice compared to stencil shadows. Texture-based shadows also strike me as more flexible and efficient in the long run. Below you can see the depth texture (greyscale) rendered alongside the existing function render (colour). Although they look similar, if you look closely you can see that they’re actually rendered from slightly different angles. That’s because the depth texture is rendered from the perspective of the light, rather than the camera.

The idea with texture-based shadow mapping is that by rendering the depth map from the perspective of the light, you get to see the parts of the world that the light is shining on. This is what the greyscale image in the screenshot above shows. The depth map is then used when rendering the functions to decide whether or not a given pixel is lit (visible from the light) or in shade (occluded from the light by another object). Put simply, if the distance of the pixel from the light is greater than the distance of the pixel transformed onto a point on the depth map, then the pixel is in shadow; otherwise it’s in light.

Once the shadow depth map has been rendered, this therefore needs to be fed into the fragment shaders used for rendering the functions. A quick texture lookup and a distance comparison are then baked into the shader to complete the process. Although it’s a really simple technique in theory, getting it to work in practice is turning out to be… intricate!

Sadly it’s been quite some time since I last had the chance to properly update the Functy codebase. Hopefully in time some of the latent improvements that haven’t yet made it into the release build can be rolled out, along with additions like this shadow rendering. It’s not quite there yet, but soon!

## Less focus for more clarity

One of the difficulties with 3D visualisations - and network visualisations seem to suffer especially badly from this - is that while they’re a great way to represent complex data, they rely on animation for much of their clarity. This stems from the fact that the third dimension is illusory. The fact is we’re still largely stuck with 2D screens that our 3D images get projected onto. Until we move to proper volumetric displays, even the current attempts at 3D (stereoscopic) displays won’t fix this.

For the moment then, the best way for us to understand the lost depth component is by shifting our perspective, or in other words, by moving things around.

However, there are other ways to approach this and in the past I’ve been incredibly impressed by the way blur can be used to improve the perception of depth. This can give a really nice effect of camera focus, where different parts of the image have different levels of blur applied depending on whether they’re in or out of focus.

Testing this on some of the network graphs that I’m currently working with has produced some really quite nice effects. Ironically, by making parts of the image less clear, the overall coherence and clarity of the image is improved substantially. Here are a couple of screenshots of a random network with focus blur applied, both from up close and from a further distance.

The effect is achieved by rendering the whole image to a texture framebuffer. A very simple fullscreen shader is then applied, which increases or decreases the level of blur for a given pixel depending on the z-buffer depth value at that point. It’s a really very simple technique, but in my opinion produces some quite effective result.

This is one of the reasons why shaders are so phenomenally powerful. The depth blur is a very short program, but it needs to be executed for every single pixel of the image. That’s a lot of pixels, and a lot of computing power is needed to do this. Applying a shader program in parallel across the whole image is hugely more efficient than getting the CPU to do it. This allows it to be applied in realtime for each rendered frame, without stretching resources, even on my relatively underpowered laptop.

Check out Lighthouse3d.com and open.gl for some nice tutorials about using framebuffers for applying full screen shaders.

## Comparing implicit and parametric functions

In an earlier post I talked about Sederberg *et al.’s* paper that uses Elimination Theory to demonstrate how parametric curves can be represented in implicit form. Reading through the literature it quickly becomes clear that this is important work if you’re interested in rendering parametric curves or surfaces. Unfortunately it can be difficult to get to grips with the theory without also being able to play around with the functions themselves. Consequently I’d expected to spend much of my summer writing software to render the different types of curves for exploring them and play around with their different representations.

That was, until I realised Functy was quite capable of doing it already. Functy’s parametric curves are already perfectly suited to the rendering of parametric equations. This part might be obvious. Less obvious for me was that the colouring of a flat Cartesian surface is perfect for the rendering of the implicit form.

Above are a couple of screenshots showing the two types of function. These are both taken from the example in another paper by Sederberg, Anderson and Goldman about “Implicitization, Inversion, and Intersection of Planar Rational Cubic Curves” (available from ScienceDirect). The curve is a quartic monoid which can be expressed parametrically and implicitly as follows.

Implicitly:

(*x*^{4} - 2*x*^{3}*y* + 3*x*^{2}*y*^{2} - *xy*^{3} +*y*^{4}) + (2*x*^{3} - *x*^{2}*y* + *xy*^{2} + 3*y*^{3}) = 0

Parametrically:

*x* = - (3*t*^{3} + *t*^{2} - *t* + 2) / (*t*^{4} - *t*^{3} + 3*t*^{2} - 2*t* + 1)

*y* = - (3*t*^{4} + *t*^{3} - *t*^{2} + 2*t*) / (*t*^{4} - *t*^{3} + 3*t*^{2} - 2*t* + 1)

In the screenshots the red line is the parametric version of the curve for *t* in the interval (0, 1). The other colours on the surface represent the values of the implicit function. Note that the implicit function actually lies at the boundary of the yellow and blue areas. You can see this slightly better in the 3D version, where the height represents the value of the implicit function. The actual curve occurs only where this is zero - in other words where the surface cuts through the plane *z* = 0.

It was reassuring to see that the parametric curve matches the implicit version. It’s also interesting to note that the implicit version is rendered entirely using the shaders in a resolution-independent way. It’s possible to zoom in as much as you like without getting pixelisation. This is exciting for me since, although it’s not what I’m really trying to achieve (that would be *too* easy!), it hints at the possibility.

## Rendering tubes with shaders

One of the main aims with Functy has always been to allow functions to be rendered using shaders on the GPU and using the function derivative to generate normals. This should be faster than rendering on the CPU. Defining the normals mathematically should also give more accurate results, and since we have the functions to play around with, it just seems like the sensible thing to do.

This presented a bit of a challenge for the new *curve* functions though. As is so often the case when using shaders, the problem is one of parallelism. If you have a function, the position of each vector in the model should be independent of the others and therefore a prime target for parallelism. With a curve you have the path of the curve and the radius at a particular point determined mathematically as long as you have the position along the curve, *s* and the rotation around the curve *p* given to you. However, what isn’t necessarily pre-determined is the *orientation* of the curve.

To explain this a bit further, consider the curve in the diagram below. Notice how the vectors perpendicular to the curve change direction as you move along the curve. These vectors are used to define the thickness of the curve at a particular point. In two-dimensions this is fine, as there’s no ambiguity about which direction these vectors should be pointing in.

However, lets now consider this in 3D. Suddenly these vectors can rotate around the axis of the curve, and while the vector must always lie within the plane perpendicular to the curve, there’s still an infinite number of possible directions that the vector can point.

In Functy, we use all of these directions, because the *p* variable defines a full rotation around the curve, so that it becomes a tube (rather than a line). But we still need to decide which direction the zero angle should point.

Some how or other a choice has to be made for this. There are a number of possibilities. We could set it randomly. However, this means there will be no consistency from one piece of the curve to the next, and if the cross section isn’t a circle, will result in a random twisting of the curve. This would look rubbish, so it’s not an option.

In my 3D Celtic Knot program I came up against exactly the same problem. Since it was essential for the start and end of a curve to match up exactly, I used an iterative approach there. For each piece, the rotation between the previous and next point on the curve is calculated and the perpendicular vector transformed by this in order to establish its new position. At the end of the curve an adjustment is made to ensure pairs of curves will always fit perfectly together. This is possible because the maximum adjustment needed will never be more than 2π/*x* radians where *x* is the number of segments that make up the cross section of the curve (called the Radial Accuracy in Functy).

However, unfortunately this technique can’t be easily parallelised since the orientation of each piece depends on the last, meaning that it couldn’t be translated easily into shader code. For Functy I therefore needed a different solution.

Luckily for me this isn’t a new problem, and the solution came in the form of Frenet Frames. A Frenet Frame is an orthogonal set of axes that’s defined based on the curvature and torsion of the curve at a particular point. Since it’s (in general) canonically defined at each point on the curve, it can be calculated independently from the other points, by calculating the derivatives and second-derivatives of the curve. More specifically, it requires that the tangent, normal and binormal vectors of the curve be calculated. There’s a decent explanation in the Wikipedia section “Other expressions of the frame”, and there’s also a neat Wolfram Demonstration too.

Since these three vectors can be calculated using the derivative of the curve, there’s no need to iterate along the curve, which makes it perfect for calculation using shaders. This is now implemented in Functy, and it seems to work pretty well. On my laptop, which has a decent but not mindblowing graphics card, animating a Frenet curve on the GPU using shader code is considerably faster than using the CPU.

The only problem is that this method has a tendency to generate curves with twists in. That is, the axis can make sudden rotations around the direction of the curve. In general this isn’t a problem, but can cause the curve to ‘pinch’ if the resolution of the pieces is too low. Below is a particularly extreme example.

Usually it’s not as bad as this, but it’s a shame nonetheless. For the benefit to be had from parallelisation I’m willing to live with it.