Unit Tests - A Cautionary Tale

I'll start by saying I'm not a big fan of unit tests. I've been forced to write them before, and I always find them tedious and seldom useful. However, there are cases where unit tests are useful and can save a lot of time, headache and debugging. So even if you're like me, you should still consider when it might be a good idea to add unit tests.

First off, what are unit tests? Unit tests are where you literally write code to test a section or function of your code. As a quick example, the following function:

function square(x){return x*x;}

Might have the following unit tests:

assert square(5) == 25; //Normal case, 5 squared should be 25.
assert square(-4) == 16; //Negatives, -4 squared should be 16.
assert square(0) == 0; //Edge case, 0 squared should be 0.

This tests the function with a few inputs to ensure it has the right output. If all of the unit tests are successful, than you can assume the function is more or less correct. Later on in development, you can run the tests again to make sure that it is still correct (and that nobody messed up that bit of code).

Now you might see why I hate unit tests. They can be very tedious, and typically only work for small, deterministic functions where it's pretty obvious that it works anyways. However, the idea of using a bit of code to test a much larger and more complex section of code is pretty appealing. Instead of having to test and debug output by hand countless times to make sure it's not broken or to track down a bug, you can simply run the test and hopefully find bugs before you even release the game to testers.

Of course, as I hate unit tests, I didn't originally write any for my current game, I Can't Escape: Darkness. However, a month or so ago I made a small change to the procedural generation code of the game to add a new feature. The change was small and I didn't think it would affect any of the other code. Checking the procedurally generated layout by hand many times, everything seemed to be in working order. But that's one of the downsides of procedural generation - since it is random, there can be bugs that don't appear in the majority of generations. This is exactly what that small change did - it broke an assumption from an earlier bit of the generation code that, in a very rare case, would allow a section of the map that should've been reachable to be attached to an unreachable section (instead of attaching it to a reachable section so that the player could actually get there). A month later, a team member was playing the game, and noticed that he couldn't find an essential button he needed to progress in the game. He was stuck, and looking at the generated layout, it turned out that the button could not be pressed as it was spawned down an unreachable corridor. Oops!

I'm sure you've all encountered problems and bugs like this, and some of them can be very game-breaking like this one was. Worried that other game-breaking bugs might exist and tired of manually checking generated layouts by hand, I finally decided to add a unit test. However, I didn't add small unit tests checking each function to make sure that all the parts of the whole work. What I did was one large unit test that checked the integrity of the procedural generation as a whole (and the procedural generation section in I Can't Escape: Darkness is big - about 40 classes and 200 functions). That is the kind of unit test I view as beneficial in game development - one that checks a large section that IS changed a lot, and takes less time to code than to manually check.

The unit test I have now does higher level checks like this:

assert canEnter("vwm", "lr3"); //You should be able to enter vwm from lr3.
assert canEnter ("cr3", "vwm"); //You should be able to enter cr3 from vwm.
assert !canEnter ("cr3", "lr3"); //You should NOT be able to enter cr3 from lr3 without first going through vwm.

Where cr3, lr3 and vwm are codenames for rooms, events and areas that are generated (which I will not explain so as not to spoil what you can find in I Can't Escape: Darkness).

These checks use basic pathfinding, and there are a lot of them that do the same kinds of checks I would do by eye. It ensures that all of the buttons, events and rooms are reachable, and that there are no holes in the maze that let you reach sections that you shouldn't be able to reach (without first hitting a button to open a door or exiting an event). Yes, this took some time to write and is more complex than "normal" unit tests, but it is infinitely more useful. Now, I can run this test 1,000 times or 10,000 times, and even though some bugs may be rare with procedural generation, this test has a much higher chance to catch bugs than I do by hand. This does what unit tests were really meant to - ensure that everything is working properly and saving time.

As a quick caution, I'll mention that you should be careful your unit testing code is simple to limit the chance that it itself has bugs. There's nothing more useless than a buggy unit test that misses errors, and nothing sillier than having to write unit tests for your unit tests! The test I wrote uses hand-written checks that only relies on a pathfinder that I've used many times before and am very confident that it works without bugs. Don't write a unit test that is so complex it breaks more often than it correctly checks your code.

To conclude, I think that you should start considering where in your code a unit test would be beneficial, preferably before someone else finds a game breaking bug. You'll still need to do plenty of testing and debugging by hand, but a good unit test can save you (and others) a lot of time and headache. Whenever you find yourself having to check something often (like the generated map layouts in I Can't Escape: Darkness), ask yourself whether you could automate the process with a unit test. If you can automate it, it's not too complex and it will save you time, then it might be a good idea to write a unit test. Unit tests can be useful - just don't waste your time writing unit tests for functions like square!

Don't Get Caught in the "Last 10%" Trap!

You may have heard the expression "the last 10% of development takes 50% of the time" or some variant of it, and while in a sense it is true, in my opinion this belief is one of the biggest problems in current game development. As a game developer, this phrase makes a lot of sense to me - I am well acquainted with the last little bit taking a lot more time than expected. However, the question really is: why do I believe that polish, testing and details are "the last little bit" of development? Why do so many people value half of a game's development time as only 10% of the project? I myself have been caught up in this common misconception, and I'm writing this blog post partially as a reminder to myself that the "last little bit" is NOT little, it is as important as the rest of game development, maybe even more so.

(original image: http://alvares.in/mario/f/2010/12/project-management-90-90-rule-400x258.jpg )

First off, what is this "last 10%" that is so under-valued? While this varies from developer to developer, for me, I like to think of "feature-complete" (meaning all of the features are in, and the game runs) as "almost done." This leaves polishing, tweaking, balancing, play-testing and minor details to that final 10% phase. However, from experience, I am starting to realize that this "last little bit" is actually the MOST important part of a game, and certainly worth half the development time, if not more. Thinking of it as "little" not only makes me underestimate how much work is left on a project, but it also makes me frustrated at my progress near the end and even encourages me to rush that final phase of development. But this is a dangerous trap, because no matter how much potential a game has, if this essential last phase is rushed, the game will likely turn out poorly.

A good example I saw a while ago about the importance of polishing games is Juice it or lose it: https://www.youtube.com/watch?v=Fy0aCDmgnxg . While this video focuses more on the visual side of polishing, all of the details added in the final phase of development help make a game feel alive. Every little nuance and special touch makes a difference, even the ones that you think most people won't notice or care about. And without thorough testing and revision, you can't know how players will react to the game - where they get stuck, what parts are too easy or boring, what parts they miss or don't learn, what parts are exploitable or even buggy. No matter how good the game's mechanics are, without multiple cycles of testing and revision, players won't understand them or won't care. And no matter how amazing and original the core concept of the game is, if it's not polished and revised, it won't shine through. The "last 10%" is what brings a game to life, making it immersive and fun.

An example of a game that worked well because of the polish is Antichamber. It's mechanics, art and music are (in my opinion) very simple. I'm usually not even a fan of that style of gameplay. And yet, the game drew me in and had me playing for hours. A lot of attention to detail, content and great pacing made the game enjoyable. And, watching this video: http://indiegames.com/2014/04/video_breaking_down_the_seven-.html , I realized just how much effort was put into the fine-tuning of the game. It's really telling how with every conference the developer went to, the average play time increased, from five minutes to over an hour. The game took seven years to release, but that was because he didn't stop at feature complete. He kept fixing, tweaking and improving the game until it really shined. And that's what I think made Antichamber the success it was.

Quite a few games I've played recently have had an interesting concept and a lot of potential, but fall flat somewhere due to lack of polish, often causing me to lose interest half-way through (or in some cases, not really getting started at all). This is also a major flaw with my most recent game, Deity Quest. While it has interesting mechanics and many people enjoyed it, "minor" elements kept it from really shining. Elements like the lack of animations, poor progression within locations and a complicated UI that was not very intuitive. Even though the core of Deity Quest was complete, the game was lacking the "small" elements necessary to make it completely immersive. Almost unanimously players felt that Deity Quest had a lot of "grinding" (even those who enjoyed the game). But it was not a grind because the game had particularly more battles or repetitive events than other games - it was because the pacing and progression wasn't well tested and revised, so the players weren't fully immersed in what they were doing (and without immersion, any task can become a grind). The "last 10%" can seem like just icing on the cake, but it's that icing that can make or break an entire gaming experience.

(original image: http://westkarana.com/wp-content/uploads/2007/11/cake.jpg )

Even some of my all-time favorite games have similar problems. I recently played Lands of Lore: the Throne of Chaos, and the first half of the game was very immersive, with lots of minor details, great balancing and pacing, and it was as fun as I remembered. However, the second half started to lose that polish - even though it still had interesting puzzles and combat, the pacing and balance were off, and those minor details started disappearing (to the point where there wasn't any flavor text at all in the final dungeon). While I still completed and enjoyed the game, it definitely started to lose that magic towards the end that made it one of my favorite games. While I'm sure there were deadlines or crunching that made the developers of LoL focus on the first half (first impressions are important, after all), I would say that it's better to make a game that is well polished throughout, but only half the length.

Realizing the importance of this "last little bit," and how long it takes, will help developers (myself included) design their games and budget their time wisely so they CAN add those all important details throughout the game. Do you really want to put so much time and passion into the game's mechanics, art and music, only to skimp on a part of development that is just as vital? In fact, a part that could have more impact than any other on whether players love your game or not?

The last thing I'll mention is that the "last 10%" doesn't actually have to be the last thing you do. During development it can be nice to switch between tasks so that you don't get tired of doing the same thing every day. Polishing, tweaking and adding details is not as onerous when interspersed with other elements of development - instead it can become a fun change of pace. In my current project, I Can't Escape: Darkness, even though I'm early in development, I've already added flavor text and small details like screen shakes or wall eyes squinting when you throw a mushroom at them. Additionally, getting testing and feedback early can be very beneficial; the earlier you notice changes that need to be made, the easier it is to make those tweaks. For example, even though I know the location pacing is a major flaw in Deity Quest now, at this point it would require a large effort to fix it - I'd need to restructure and perhaps completely rewrite the location API. If I had gotten this feedback during early development, it would have been much simpler to modify.

Takeaways

• Polishing, tweaking, balancing and play-testing are not little in terms of amount of work OR impact on the quality of your final game. Get that idea out of your head if it's there.
• Believing the above are "minor" creates misconceptions that are detrimental to game development.
• It is better to make a smaller, well polished game than a larger, poorly polished game that has trouble immersing players.
• Polishing your game does not have to be put off to the end, it can be done during development as well. In fact, it is beneficial to get feedback and revise as early as possible (instead of just focusing on 'getting the game done' in a mad rush).

In short, a "feature complete" game is only half the battle - don't fall into the trap of underestimating or undervaluing the other half. No matter how much potential a game has or how much passion you put into it, it will fall flat without polish, like a sketch that could've been amazing if only it had been finished.

Indirect Lighting in I Can't Escape: Darkness

In I Can't Escape: Darkness (ICED), lighting plays an important role in the gameplay. I had several game-specific requirements I needed to meet, however the resulting algorithm could easily have wider applications. In this post I'll go into some detail about my lighting approach, the steps I took to reach it, and how to implement it.

So here's the problem I faced: I won't explain the game mechanics (no spoilers!), but some areas of the map needed to be pitch black (no ambient light) for the game to work. However, direct lighting alone wouldn't be enough, as that would make the levels too dark (everything not directly lit would be pure black). Additionally, adding ambient light in an area of effect around light sources would lead to the light bleeding through walls (which would break gameplay, lighting areas that should be pitch black). 

Obviously, I needed something like global illumination with visibility, but for those of you who have done any graphics research, you'd know that this is an open problem, with lots of research and very few algorithms that would work in this case (without requiring high end graphics card). Lightmapping also doesn't work, as some of the light sources are dynamic (like the flashlight).

Knowing that true global illumination is expensive to render, I decided to use a physically inspired, but not physically accurate approach. For those hoping for an accurate, real time (even on integrated cards) global illumination algorithm, I'm sorry, I'm not a magician. However, this approach is very useful, especially for 2D games or cell based games (like ICED or Minecraft). In fact, while I don't know exactly what Minecraft does, their approach looks visually similar to mine (light does not bleed through blocks, but doesn't appear to correctly bounce off walls).

Final direct+indirect lighting in I Can't Escape: Darkness.

The Idea

The idea is actually inspired by heat transfer. Instead of bouncing light off walls in a very complicated computation, heat transfer distributes heat along a surface in an embarrassingly parallel way (well suited for graphics cards). From the heating elements (the light sources), heat spreads and attenuates over distance like light, and by varying the conduction along the surface, we can make blockers (walls) that the heat has to travel around instead of through, satisfying my "no bleeding through walls" requirement. Lets look at an image of this in action:

The top left image is the conduction of my map, where white is open space, and black is walls. I also removed a one pixel border around the walls so that the walls could receive indirect light. Gray pixels could be added to have semi-transparent cells (like partially open doors or windows). This is in 2D as I didn't care about light transfer between levels, but in a Minecraft style game, you could easily augment this algorithm as a 3D volume instead of a 2D texture.

After the conduction mask is created, I place the lights as points, shown in the top middle image. Then, by iteratively spreading out the heat using the conduction mask to block heat transfer through walls (I'll show the algorithm for this later), you get the top right image after 5 iterations, the bottom left image after 10 iterations, and the bottom right image after 20 iterations. 20 iterations was enough for my purposes, and given how fast each iteration is and how small the map is, this renders realtime even on integrated graphics cards. The bottom middle image is also 20 iterations, but has a door closed so you can see how light is blocked from the bottom left room when the door is closed, and can enter when the door opens. Note that this image has been brightened so that you can see the transfer more clearly.

If your game is 2D, simply use the resulting texture (bottom right) as the indirect lighting. Blocking cells won't allow light to leak through, and indirect light will spread throughout your scene. In ICE, by indexing this texture using the x/y coordinates of the geometry, I can get the indirect light at any given point. This is why the 1 pixel border around the walls is needed (otherwise, walls would index a blocker pixel and have no indirect lighting. Since the walls have thickness despite the border, they still correctly block light and stop bleeding). The result is as follows:

With both indirect and direct lighting. There are strong shadows from direct lighting, but instead of being pitch black, there is a little indirect light to lighten the scene. This image has been brightened to make it clearer.

With only indirect illumination. The light appears to come out of the gap, and the right wall is very dark despite the torch being towards the right (you can see the torch position from the previous image with direct lighting) . This has also been brightened.

Here is direct lighting only - as you can see, the walls and room outside are pure black, and there is a pure black shadow where the wall's corner blocks the light. The direct lighting includes dust, another lighting element of ICED that is not mentioned in this blog post. This scene has also been brightened.

As for how realistic this approach is, the answer is not very. It doesn't take height into account at all (although you could with a 3D volume texture), and the walls facing the camera are brighter than they should be, as they would only receive minor illumination from the floor and ceiling. However, it is similar to indirect illumination, and it meets all of my criteria (attenuating as it travels away from the light source, and not able to bleed through walls, it has to wrap around them).

Some interesting notes about this algorithm: in 2D, it's about as close of an approximation you can get to indirect lighting without some knowledge of the depth of the scene (as there's no way to know how light would bounce). Also, this can be extended into a pathfinding algorithm: by "hill climbing" (always traveling to the brightest cell), you could cheaply get a horde of units to travel around walls and reach the closest point (within the number of iterations).

The Implementation

Hopefully you understand the idea at this point and have a vague idea of how it works (and whether you'd want to add it to your game). Obviously, any games with complex geometry that can't be well defined in a 2D or 3D grid for the conduction mask wont work, but for blocky tile-based games, this works very well and even runs realtime on a seven year old computer (with an Nvidia GeForce 9200M GS graphics card, so not top of the line even during it's time)!

The first step is to create the conduction mask. This can be done on the CPU if the scene is relatively static, or by rendering simplified geometry to the texture. In ICED, the conduction mask is four times the size of the levels (so 128x128 for a 32x32 tile level) to allow me to remove a 1 pixel border while still having at least a 2 pixel thickness to the walls.

Next, create two floating point textures of the same size as the conduction mask - the front and back indirect lighting buffers. Render the lights as points to the front buffer (with additive blending enabled in case two points overlap). I actually rendered each light as four points, so that if a light was between cells, it would still add the correct amount of light to nearest four cells. This is the only step that is dependent on the number of lights, so even if your scene has THOUSANDS of lights, rendering thousands of points is no trouble at all (even for old computers), so this algorithm runs quickly almost regardless of how many lights in your scene. Hint: this works well with deferred rendering, as that allows you to add many direct light sources, and all you need to do is render the indirect lighting with additive blending to the direct lighting.

Finally, for each iteration, first swap the front/back buffers. Then render into the front buffer the weighted average of all adjacent pixels. This is similar to a 1 pixel radius gaussian blur, except the pixels have to also be weighted by the conduction mask to determine if light can transfer between the two pixels. Also note that this is NOT a separable kernel, because of the conduction mask. Here is the GLSL shader I wrote that computes each iteration: http://pastebin.com/pTGsx4vf

The shader adds the weighted pixels from the previous pass (and keeps track of the weights), then divides by the sum of the weights. Yes, this could've been done with a double for loop, but I unrolled it by hand (uglier, but faster).

For those who like terminology, this is the laplace operator that computes the heat transfer. When your last iteration is done, the front buffer texture contains the indirect lighting that you can index by position to get the indirect light for that point in your scene! Pretty easy huh? You can also modify the weights to determine how the light spreads (in the above shader, I gave higher weights to points further from the center to get it to spread more with fewer iterations - you wouldn't want to do this if your applications was pathfinding). You can also take the square root of the indirect lighting value to slow its dropoff. But, this is the basic algorithm, and you can tweak it to work for your setup.

Side Note: Direct Lighting

I just thought I'd make a brief mention of direct lighting, since in ICED it actually uses the conduction mask for visibility! The direct lighting is pretty standard deferred rendering, but I didn't want to render a shadow map for each light source (would be slow). Instead, to compute visibility, I decided to raytrace through the conduction map since it already tells me the scene's (simplified) visibility! All I did was multiply the direct lighting by eight evenly spaced pixels in the conduction mask between the light position and the scene position. Depending on the size of each pixel in the mask, eight is probably enough to ensure that the samples will not miss a wall until the direct lighting is dark enough that visibility doesn't matter. Not a bad way to add visibility to all light sources for only eight texture reads, huh?

Deity Quest, Havencall and the future of Fancy Fish Games

So, Deity Quest has been out for a little over a week, and it's been a bit of a slow start. We've had some sales and press, but it's definitely been a lot slower than I had hoped. Part of that was because this was my first commercial game release, and I made some mistakes with the launch (like not giving storefronts enough time to approve Deity Quest before release). But, things are just getting started - we're getting new press articles and are scheduled to release on Desura tomorrow. It's been a rough road, but now we're finally almost launched on all our storefronts. You can already buy Deity Quest on Desura using this widget:


Hopefully, as Deity Quest continues to get in the hands of players and they (hopefully) enjoy the game, word of Deity Quest will spread and we'll do a little better. However, as we start to run out of funds, and I calculate what kind of sales we'd need to stay afloat, I can't help but get a little depressed. For a first release, I should be happy for the sales and positive feedback I've already gotten and am still likely to get, but I've also finally realized that even a good number of sales isn't enough to live off of, and that I'll likely never make enough on games to make Fancy Fish Games my full time job (a dream I've had for a while now). Perhaps it would be possible if I made different kinds of games, or simple games like match 3, but I'm not willing to compromise there - if I'm not making what I want, then it's truly a job and I'd be better off working elsewhere where I'd make a lot more for my efforts.

So where does that leave Fancy Fish Games? A part-time job that doesn't bring in much money? A hobby which is subsidized by a full time job? Honestly, I don't know - the only thing I know for certain is that I'm going to finish the projects I've already started - as it'd leave a bad taste if I quit halfway. This means yes, I'm going to finish Havencall and the new space game (Codename: Pioneer). I may even make I Can't Escape: Darkness, even though I've only written a design document for it so far. But all my other great ideas? Especially These Falling Stars which I've been mulling over for years but have never had the resources to really start? Honestly, I don't know. Maybe if I get lucky, or if I save up enough from a full time job, I'll consider doing Fancy Fish Games full time in the future.

As for Havencall, we're planning to release late this year, and are working hard to complete the first world in time to submit it to the late submission deadline of IndieCade. We've been working on Havencall for a while now, and we've both improved a lot - as you can easily see from Natalie's art:

Havencall may be one of our last games, which is pretty sad, but we're going to do it right, not cutting any corners. Even if game development has to take a back seat for me in the future, I want to be able to look back on what I accomplished with pride.

On my side of Havencall, I've been setting up the scenes (the hometown and space time ruins are almost done), and adding in the UI and Dialogue systems. Havencall is definitely more of an artist's game than a programmer's game, but there's still plenty to do - especially with some features like the boss battles. I've also been working on tools that make integrating art and setting up scenes a lot easier - so that will speed up production of the second and third worlds.

Well, we'll see how things turn out - and I doubt I'm going to stop making games, although I may slow down or work on smaller games if I end up with less time for game development. While this is kind of sad - I'm happy with the games I was able to create. If Fancy Fish fails, it's not due to lack of quality, it's just that it's so hard to get noticed and succeed in today's world.

As a side note, I am now considering doing freelance work for game development. While I'm not sure I'll be able to make enough as a freelancer, I'm certainly open to offers if you like my work. And, at the end of the day, I'd prefer working on games than something else, even if the pay is less, as long as it's livable.

Deity Quest Complete!

It's been quite the year finishing up Deity Quest. I posted that Deity Quest was feature-complete Feb 28 of this year, and since then while the game was "done" and playable, there was so much to do. Polishing, debugging, balancing, the BETA test, marketing and tying up loose ends. I've been on that final stretch for months, and it's been rough, but the end is finally in sight! Last year, when I posted the IndieDB page for Deity Quest, I made an estimate that the release would be May 1st - which was my expected date of completion plus two months padding. It turns out I needed those two months padding, but the good news is that we're on schedule to release the game to the public on May 1st!

Whatever happens after the release date, Deity Quest has been a great experience. As my first commercial game, I had to do a lot of research and figure out things like payment processing and even entered the game for Steam Greenlight, which you can view (and vote for) here:



I also created a much improved trailer for the game, using Adobe After Effects instead of Windows Movie Maker, so I had a lot of new options to make it much better than the old trailer (like zoom, pan, etc):



Right now, I'm finishing up multiplayer battles, which was something I had contemplated adding for a while. This has really been a great learning experience for me, and should help a lot with my future commercial games, like Havencall!

Speaking of Havencall, those who have been hoping for news about Havencall for a while will be glad to hear that we are only about a month away from finishing the first world of the game. Havencall is broken up into three worlds, so that means we'll be 100% done with the first third of the game, and we already have a lot of assets for the second and third world. The expected release date for Havencall (with padding) is now October of this year - over a year beyond our initial release date, but at least we're getting there!

---

As for other projects I have planned to start this year, I have one pretty epic first person, 3D space game. I already have a team on board for it, but because of the scope, I'll probably end up trying to run a Kickstarter for it. It'll have great graphics, so look forward to that!

I also have plans for a sequel to I Can't Escape - so those of you who like horror and the style of I Can't Escape should look forward to that! While the game will have a similar atmosphere and mood, it will have a bunch new features, and of course many new twists & tricks. Those of you who played the original I Can't Escape should still be quite surprised!

I'll keep you posted on updates of all my projects, and get ready for the release of Deity Quest! It's been a crazy road finishing Deity Quest, and there were times I wanted to hit myself for trying a project so ambitious after only a few One Game a Months, but now that it's almost over, I feel quite satisfied, and am ready for new projects - perhaps even more ambitious in scale, more awesome and more of a challenge!