# Coding

## Functy Updates – Shadows and UI Improvements

The last two weeks of summer break have given me the opportunity to give Functy some long-overdue attention. As I mentioned in my previous post, one of the improvements I’ve been working on has been dynamic shadows, and these are now fully implemented. The result is a big improvement in the visual fidelity and sense of depth in the visualisation. You can see the difference in the two screenshots below and I’ll include a proper comparison in a future post.

The next step will be to test out ambient occlusion, as kindly suggested by Andrea Bernabei (@faenil). I’m not sure how well this will work with the type of functional objects rendered by Functy, but it’s worth investigating. Ambient occlusion produces the sort of shadow effects more likely to occur indoors where the light tends to be more scattered, as compared to directed lights or outside direct sunlight, as the current version produces. However, research has shown that ambient occlusion gives a heightened sense of depth as compared to direct shadows, so it’ll be interesting to see the results.

The second major change has been to the user interface. This has been completely rebuilt from the ground up to fit with a single window workbench-style interface, as opposed to the previous multiple-window toolbox-style interface.

Personally I’m a big fan of multi-window interfaces; it should be the job of the window manager to allow you to arrange and configure your windows however you want. Sadly most window managers seem to be lacking in the flexibility department, and managing multiple windows becomes an exercise in hide-and-seek, trying to find windows that got lost behind others like a stack of papers on a desk. So, even though it’s not the perfect solution, I’ve converted Functy so that everything is visible and available in a single window.

I have to admit that the result does look more professional and I think it’s a positive change. I’m not quite sure what will happen if things get more complicated, but it works well for the timebeing.

There’s still a fair bit of work to be done before this can be given a full binary release, such as updating everything to work on Windows, and switching from libglade to GtkBuilder (this may be for a future time). Feel free to test out the version from source (it’s very easy to build, honest!) in the meantime.

## Shadow mapping

One of the great benefits of teaching on computer game development modules at the university is that I have a justifiable reason for looking into interesting graphical effects for my work. It’s a real perk as far as I’m concerned. Last semester I covered shadow rendering as a new topic, which gave me the opportunity to find out about the subject in some depth.

Now it’s the summer and I have more time to experiment, so it only seemed right to apply some of these techniques to Functy; I’m hoping the function rendering will look a lot more realistic and solid once accurate shadows are being cast throughout the environment. Given the functions are all generated on the GPU, texture-based shadows seemed the most appropriate choice compared to stencil shadows. Texture-based shadows also strike me as more flexible and efficient in the long run. Below you can see the depth texture (greyscale) rendered alongside the existing function render (colour). Although they look similar, if you look closely you can see that they’re actually rendered from slightly different angles. That’s because the depth texture is rendered from the perspective of the light, rather than the camera.

The idea with texture-based shadow mapping is that by rendering the depth map from the perspective of the light, you get to see the parts of the world that the light is shining on. This is what the greyscale image in the screenshot above shows. The depth map is then used when rendering the functions to decide whether or not a given pixel is lit (visible from the light) or in shade (occluded from the light by another object). Put simply, if the distance of the pixel from the light is greater than the distance of the pixel transformed onto a point on the depth map, then the pixel is in shadow; otherwise it’s in light.

Once the shadow depth map has been rendered, this therefore needs to be fed into the fragment shaders used for rendering the functions. A quick texture lookup and a distance comparison are then baked into the shader to complete the process. Although it’s a really simple technique in theory, getting it to work in practice is turning out to be… intricate!

Sadly it’s been quite some time since I last had the chance to properly update the Functy codebase. Hopefully in time some of the latent improvements that haven’t yet made it into the release build can be rolled out, along with additions like this shadow rendering. It’s not quite there yet, but soon!

## Network Video

I finally managed to get the video to record properly, so here’s a video version of the shader-based depth-of-field focus-blur.

P.S. This post was actually made using Ubuntu phone. The browser’s still a bit “experimental” so it’s been a bit tricky, but it does seem to work!

## Network structures

As part of an experimental game project I’ve been trying to use the Functy rendering routines to visualise network structures. At the moment it’s at a very early stage, but has - I think - already generated some interesting results.

The screenshot below shows a network of 60 nodes, each one rendered as a spherical co-ordinate function, joined together using links rendered as curves. I just plucked some simple functions out of the air to see what the results would be like but am hoping to extend it with more interesting shapes as things progress.

The various parts of the network are a little hard to discern with a static image, but when I tried to capture a video the result was a mess of fuzzy artefacts (I think there must be something going wrong with my screen capture software), so I gave up on that.

The next step, after neatening up the code, is to arrange better animation of the nodes and links, with dynamic movement based on things like the forces between the nodes. I’m hoping this will produce some really nice effects, and if anything comes of it I’ll put a bit more effort into getting a successful video capture.

## 3D printed Functy rings

A parcel arrived from Shapeways recently containing some of the 3D printed ring prototypes I generated using Functy. The models were exported directly from Functy and converted into STY format before being directly uploaded to Shapeways for printing. All based on sine/cosine curves, there’s a flat version, a slightly bulging version and an irregular version. Since Shapeways did such a brilliant job printing the prototypes, the next step is to get them to print them in silver. Click on the links if you fancy having your own printed!

The Functy function files for all of these rings are up in the repository and will be included as example files in the next full release.

## Rendering tubes with shaders

One of the main aims with Functy has always been to allow functions to be rendered using shaders on the GPU and using the function derivative to generate normals. This should be faster than rendering on the CPU. Defining the normals mathematically should also give more accurate results, and since we have the functions to play around with, it just seems like the sensible thing to do.

This presented a bit of a challenge for the new *curve* functions though. As is so often the case when using shaders, the problem is one of parallelism. If you have a function, the position of each vector in the model should be independent of the others and therefore a prime target for parallelism. With a curve you have the path of the curve and the radius at a particular point determined mathematically as long as you have the position along the curve, *s* and the rotation around the curve *p* given to you. However, what isn’t necessarily pre-determined is the *orientation* of the curve.

To explain this a bit further, consider the curve in the diagram below. Notice how the vectors perpendicular to the curve change direction as you move along the curve. These vectors are used to define the thickness of the curve at a particular point. In two-dimensions this is fine, as there’s no ambiguity about which direction these vectors should be pointing in.

However, lets now consider this in 3D. Suddenly these vectors can rotate around the axis of the curve, and while the vector must always lie within the plane perpendicular to the curve, there’s still an infinite number of possible directions that the vector can point.

In Functy, we use all of these directions, because the *p* variable defines a full rotation around the curve, so that it becomes a tube (rather than a line). But we still need to decide which direction the zero angle should point.

Some how or other a choice has to be made for this. There are a number of possibilities. We could set it randomly. However, this means there will be no consistency from one piece of the curve to the next, and if the cross section isn’t a circle, will result in a random twisting of the curve. This would look rubbish, so it’s not an option.

In my 3D Celtic Knot program I came up against exactly the same problem. Since it was essential for the start and end of a curve to match up exactly, I used an iterative approach there. For each piece, the rotation between the previous and next point on the curve is calculated and the perpendicular vector transformed by this in order to establish its new position. At the end of the curve an adjustment is made to ensure pairs of curves will always fit perfectly together. This is possible because the maximum adjustment needed will never be more than 2π/*x* radians where *x* is the number of segments that make up the cross section of the curve (called the Radial Accuracy in Functy).

However, unfortunately this technique can’t be easily parallelised since the orientation of each piece depends on the last, meaning that it couldn’t be translated easily into shader code. For Functy I therefore needed a different solution.

Luckily for me this isn’t a new problem, and the solution came in the form of Frenet Frames. A Frenet Frame is an orthogonal set of axes that’s defined based on the curvature and torsion of the curve at a particular point. Since it’s (in general) canonically defined at each point on the curve, it can be calculated independently from the other points, by calculating the derivatives and second-derivatives of the curve. More specifically, it requires that the tangent, normal and binormal vectors of the curve be calculated. There’s a decent explanation in the Wikipedia section “Other expressions of the frame”, and there’s also a neat Wolfram Demonstration too.

Since these three vectors can be calculated using the derivative of the curve, there’s no need to iterate along the curve, which makes it perfect for calculation using shaders. This is now implemented in Functy, and it seems to work pretty well. On my laptop, which has a decent but not mindblowing graphics card, animating a Frenet curve on the GPU using shader code is considerably faster than using the CPU.

The only problem is that this method has a tendency to generate curves with twists in. That is, the axis can make sudden rotations around the direction of the curve. In general this isn’t a problem, but can cause the curve to ‘pinch’ if the resolution of the pieces is too low. Below is a particularly extreme example.

Usually it’s not as bad as this, but it’s a shame nonetheless. For the benefit to be had from parallelisation I’m willing to live with it.

## Working Curves

The latest version of Functy in the repository now has a (roughly) working version of the curve rendering code. This allows a cylindrical coordinate cross section to be extruded along the length of a parametrically defined curve. In other words, something like a tube.

The code is fairly incomplete. Trying to define functional colours will cause a crash, and vertices are all positioned on the CPU, so that animation isn’t particularly efficient. Both of these should be fixed soon, including GPU rendering of the entire curve.

Here’s a brief video demo of this early code to give a flavour of how it can be used.

## The Art of Good Design?

Designing a really good library is a difficulty task. Since around August last year I’ve been writing a program called Knot, which I also recently open sourced. This makes extensive use of 3D Bezier tubes to render Celtic Knots, and having written the code for this, it seemed sensible to try to integrate it into Functy, to allow arbitrary tubes to be rendered using curved cylindrical coordinates. So, that made it a perfect candidate - or so I thought - for implementing as a library. This way, improvements from the Knot code could be easily merged into Functy and *vice versa*.

Designing the library turned out to be tricky, especially the task of managing and animating all of the curves. I figured that if I put the effort in to getting the library right, it would make the process of integrating into Functy that much easier.

I was wrong. After painstakingly implementing everything needed in the library, I tried to integrate it into Functy, only to realise that Functy already has all of the management and animation functionality needed. Given that I wrote the Functy code, you’d have thought I might have know this.

The consequence is that I’ve ended up dropping all of the curve library methods and cut and paste the relevant code straight in to Functy. If I hadn’t done this Functy would have ended up with two totally separate lists of functions, two separate animation steps and two separate update steps. It would have been a mess.

It’s taken longer to modify the curve code for Functy as a result, but the final structure is much better. Unfortunately it also means the library is now redundant. I’ll have to find some other use for it.

There’s a design moral to this story I’m sure, but I’m not exactly sure what it is. I’ve ended up with a nice clean library implementation that doesn’t work for what it was designed for, a new implementation that doesn’t use this library, and what seems like a lot of wasted time. It feels like somewhere something went a bit wrong. At least Functy can now render curves, so even if it was a circuitous route, maybe the journey wasn’t entirely wasted after all.

## Curvy Screenshots

The Curves library is currently being integrated into Functy in order to provide parametric tubes to be added to function scenes. To give an idea about the sort of results this might allow, here are a few screenshots taken from the development version.

In all of these images the tubes follow a path constructed from cubic Bezier curves:

((((1 - *a*)^{3}) × *P1*) + (3 × ((1 - *a*)^{2}) × *a* × *D1*) + (3 × (1 - *a*) × (*a*^{2}) × *D2*) + ((*a*^{3}) × *P2*))

where *a* is the parametric variable, *P1*, *P2*, *D1* and *D2* are the control points. It’s a bit crazy using Bezier curves for this since the library could handle a spiral far more easily, but it was just a product of the way the demo developed. For the first image the function used for the radius was the following.

((2 - cos((((*s* × *a*) + *o*) × 6 × π)) + (sin ((12 × *p*)) / 5)) / 2).

Here *a* and *p* are the parametric variables for the position along the curve and angle around the radius respectively. The values *s* and *o* are offsets and scaling factors that are changed for each quarter-circle segment of the spring.

The intention is to allow these functions to be entered directly into Functy to generate curves such as these. There’s still quite a bit of work to be done. Although the Curves library is coming along, this still needs to be integrated with Functy, and the shader code for these parametric tubes also needs to be implemented (it’s certainly possible, but I’m anticipating problems!).

## Spherical co-ordinates

So far Functy has worked only using Cartesian co-ordinates, but I’m currently working on code for the next release to allow functions to be defined using spherical co-ordinates too.

Spherical function plotting is already working as you can see in the screenshot. It seems to produce some nice effects that are quite different from those you generally get using Cartesian functions.

The tricky part seems to be getting the code for both co-ordinate types integrated nicely together, and also sorting out a suitable user interface to allow them both to be added seamlessly.

The reason the code is turning out to be tricky is that I’m trying to develop using object oriented techniques, but using C rather than C++. I know it’s perfectly possible to do this (and do it well; although not necessarily by me!), and so far there hasn’t been much problem. However, I’m now in a situation where inheritance would seem to be the perfect technique to use. Most of the methods needed for the two co-ordinate systems are the same, so in C++ it would make sense to use some virtual interfaces to define them and inherit these from a generic function type. Unfortunately I’m not sure if this is going to work well in C.

It seems like a silly thing to get stuck on and it would be nice to get the spherical functionality into the next build. I’m hoping the best method will become clearer as things progress.