The Artistic Computer

Ever since college, I've been fascinated with machine learning, specifically, teaching computers to create art, music, and potentially even entire games. Is it possible for a computer to do what we do - be creative? Could an advanced statistical learning model, with enough data, produce art, music, stories, or even design gameplay and write code? If you're worried about machines becoming independent of humans and taking over the world, I can assure you that we're a long way away from that. However, every now and then I come back to this idea, thinking about how cool it would be if I could create a game that generated itself from an "intelligent" computer program. Once I can teach a computer to generate at least one game-element satisfactorily, I plan to build a full game around that ability.

This blog post is about my most recent attempt to teach computers to be creative. The idea was to create a program that would draw graphics by learning the probability distribution of a color given the pixel above it, to the left of it, and the pixel in the previous mipmap level of the input data (graphics made by humans). Creating a new graphic then involves drawing from the distributions from lowest detail mipmap to the highest, left to right, and top to bottom, ensuring that we always have the inputs for the distributions while generating. I liked the idea, because it mimicked what artists do - first creating the basic shape of an object, and then adding in details. Similarly, the program progresses from low detail to high detail with each mipmap level.

The Data

The first 493 pokemon used as input data for the program.

I wanted to generate something fairly retro, and I needed a lot of consistent sprites to teach the model. So, I chose to use sprites from Pokemon as my data set as they were easy to find and there were a lot of them. The question of whether a program taught using pokemon sprites generates copyrighted material or not is tricky, but this was just for fun/research. Since the model depends on the number of colors in the image (the more colors, the harder it is to learn and the more data it needs), I decided to convert the image to just four grayscale colors. I also downsized the pokemon to 32x32 pixels, all to make the data smaller and easier to learn from.

The Results

Ideally the program would be able to learn how to create infinite reasonable looking original pokemon (imagine catching them all!). Instead, the program generated what I've termed "inkblots," some of which don't look anything at all like pokemon, some of which have interesting pokemon-ish shapes if you look for them, and some of which are actually decent (if you squint a little). The quality of the results is completely subjective, so I'll let you be the judge:

36 randomly generated results. Didn't quite have the coherence and structure I had hoped, but not outright terrible either.

You can also test the program yourself here:

For those who are curious:
Color Version: http://fancyfishgames.com/Procedural/pokemon_color.html
High Res Version: http://fancyfishgames.com/Procedural/pokemon_col_res.html

Older Experiments

Back in college, I attempted to teach a computer how to compose music by feeding midi data to a Hierarchical Hidden Markov Model. The results for that were not that great, but I still have one midi file generated by it that I thought sounded nice (from a set of anime midi files): http://fancyfishgames.com/Procedural/sample.mid, and here's a mp3 version for those who have trouble playing midi files: http://fancyfishgames.com/Procedural/sample.mp3. However, this was one result from many that were less coherent, so take it with a grain of salt.

Conclusion

I still haven't made a program that can teach computers to make games, or even acceptable pixel art yet, but that doesn't mean it won't be possible one day. For now, the results are interesting given that this was just a short project, so I figured I would share. Feel free to share any experiments you have done with machine learning and games. Until my next experiment!

On Starting a New Game Project

When starting development on my newest project, Havencall, I did a lot of things differently than for games I worked on in the past, learning from my mistakes. In this post, I explain what changes I made, and why.

1. The Simplest Possible Version

Between 2008 and 2010, I worked on Aero Empire (AE), which was a massive project, combining 4 completely different styles of gameplay and an overwhelming amount of content. Our team had many ideas of what we wanted the game to be, and we did not want to give up any one of them. Aero Empire was never completed and is now on indefinite hold.

In comparison, Havencall is a much smaller, more focused project. But that is not because I had less ideas for the project to begin with. In fact, I have notes for it going all the way back to 2009, describing an epic RPG not so much smaller than AE. The first thing I did when designing the final version of Havencall was ask myself what the simplest, most compact version of the game design would be that still had the same feel and "heart" of the original. The answer was to make it a point and click game focused entirely on the protagonist and her adventure through the three worlds, rather than a complex RPG with many characters and storylines. All of the worlds themselves were also cut to their minimal form - the first world was meant to be filled with tons of space-time bubbles, each containing its own little town. Now the first world only has two space time bubbles with towns inside them, including Aura's hometown.

You might think that making all those cuts would ruin the game and its story, but actually the opposite happened. After really thinking about what made the core of the game and cutting everything else, I ended up with something much stronger, more focused and even more exciting. Looking back, a lot of the stuff I cut was actually just detracting from what I really wanted the game to be.

2. A Group of Friends

In today's internet age, it is incredibly easy to recruit strangers to work on your projects. That is exactly what I did with Aero Empire, ending up working mostly with people I had never met or even spoken to before the project. While AE had a lot of talented people involved, the team was not very efficient or focused. We would often lose or gain people, causing loss of unfinished work and requiring time to get new people up to speed on the project. Consistency was also a mess, with almost every asset created by a different team member. And when the project started falling apart, it was only those who I had a strong connection with who stuck with me until the very end. It was also those close team members who really got each other motivated and helped everyone else stay excited.

To keep Havencall consistent and organized, I decided to make it with a team of three people I am very close with - I do all of the programming work, my wife does all of the artwork, and my sister composes all of the music. Even though our team is small, a lot of the time it seems like we are much more efficient than AE's large team - mainly because there is no lost work, no stepping on each other's toes, and we all trust each other. The bottom line is, a team that feels like a group of friends works much more efficiently and smoothly. And never underestimate the power of being able to meet and talk about a project face to face. More can get communicated in a few minutes of real conversation than you could imagine.

3. Planned From Start To Finish

There is a strong desire to just jump into a project once you have the basic idea down. All the details can just be figured out as you go along right? This is what we did with Aero Empire. The design document was rudimentary at best, and we often added or removed features without ever modifying the design document. This was very bad for two reasons: first, the design of the game was never fully understood - this made it difficult to make the game flow well and be consistent. The stages with their vastly different gameplay styles just seemed slapped together, without much thought about how they interacted with each other.

We also did not realize how big of an undertaking AE was - we knew it was big, but it turned out much bigger than we imagined. This was largely because there were gaps in the design document that we overlooked because we did not fully understand the design. The other reason to have a strong design document is to keep everyone on the same page, with the same vision. Not everyone working on AE agreed on what the details of the game were, causing confusion, inconsistency and discord. And to make matters worse, when a new team member would join, the design document was not effective at helping them understand the project.

In contrast, Havencall has been designed from start to finish, with no gaps. All of the cutscenes, puzzles, and interactions are fully detailed. This keeps the game's design focused for all team members, and allows us to create something with tightly knit art, music, and gameplay, and a unified vision.

Announcing Havencall!

EDIT: Havencall is now on IndieDB

We are very excited to officially announce Havencall, our first game using the VIDE animation toolkit! Havencall will be a short (2-4 hour) point and click adventure game which tells the story of a strange girl named Aura, and her journey between worlds to find the haven of her dreams. The original idea for this game dates back to 2009, but it wasn't until recently that everything came together: the technology, the story, and the design for what is in many ways our dream game - both to play and create.

Aura being beckoned into another world by a strange glowing figure.

The game's story and progression have been written, all of the puzzles have been designed, and we're busy working on concept art and hammering out the details. We plan to release Havencall sometime in 2013, for PC, Mac, Linux, and potentially mobile devices as well. The gameplay will revolve around interacting with items and people, solving puzzles, awakening powerful abilities, and even fighting magical devices.

My hope is for the game to be a stunning audio/visual experience that brings the strange and varied worlds of Aura's universe to life, with unusual puzzles and challenges along the way. I don't want to reveal too much of the story, so I will only say that it will be a mystical, magical and even philosophical journey.

Sneak Peek: Concept art of Aura's hometown, a small primitive village completely isolated inside a space-time bubble.

Our trajectory now is to solidify the art style, music style, and prototype the puzzles to make sure we have all of the details ironed out, and then we plan to try to fund the project on Kickstarter. It will be our first time using Kickstarter, which should be an interesting experience in itself, but hopefully a good one as well.

Stay tuned for more updates on Havencall, but they may be sparse as I don't want to spoil the story! I have also set up a website for the game here: http://havencall.com/, which we will update with more content as development progresses.

Four Tricks to Improve Game Balance

Balancing a game's difficulty can be tough. Different players will enter the game at different skill levels depending on whether they've played similar games or not. Their learning curves during the game will be varied as well, making it tricky to decide how difficult to make the game without making the game too difficult (frustrating), or too easy (boring).

Above is an approximate graph of balance zones based on the player's skill and the game's difficulty. As player skill increases, the difficulty must also increase to keep a balance. The balance zones are as follows:

• Frustrating - Too difficult to be fun.
• Hardcore Fun - Really tough, but some people like that.
• Challenging Fun - For people who like to overcome challenges.
• Balanced Fun - The goldilocks zone (not too tough, not too easy).
• Casual Fun - Nice and easy, never a challenge, but not mindless either.
• Mindless Fun - They just want to play, they don't want to think.
• Boring - I could play this in my sleep... in fact, I'd rather sleep.

While it requires testing, balance and player feedback to really balance a game, this article will cover four tips and tricks for designing game difficulty, which I've learned through my game development experiences.

1. Know your audience.

Knowing your audience is important in almost every aspect of game development, and is also important for game balance. Who do you expect to play your game? What games will they have played before yours (and how similar are those games to yours)? Knowing the answer to these two questions will help you guess what skill level the players will start with, and which balance zone they prefer. A casual game should assume that the average player has a low skill level, and doesn't want to be particularly challenged. A niche game should assume that the average player enjoys that niche and has played many similar games before, and so has a high skill level and enjoys a challenge. Having a good read on your target audience gives you a starting point to balance the game, and will make your initial balancing more accurate.

The takeaway point here is that the better you understand your audience, the more you can cater the game to that audience - and that applies to a lot more than just the balance of the game.

2. Underestimate the player's learning curve.

The player's skill will increase throughout the course of the game, and so the difficulty of the game has to increase to compensate. However, overestimating the player's learning curve is worse than underestimating it (and most developers tend to overestimate their players - not everyone is as good as you!). If you overestimate the player's learning curve, players who learn quickly may get a good balance, but the rest of the players will not be able to keep up with the curve and the game will continue to get harder and harder until they can't continue. Whereas if you underestimate the player's learning curve, players who learn quickly will still enjoy the game even if it's not as challenging for them (they will simply feel that they are awesome), while the rest of the players will still be able to keep up with the game difficulty.

You've probably played a game you liked a lot in the beginning, but then it became so difficult that by the end it was no longer fun to play. The final boss was impossibly frustrating, and you probably resorted to walkthroughs or outright gave up. This is a situation you want to avoid at all costs. A player is far less likely to quit because a game is too easy.

The takeaway point here is that it's easier to lose players by making a game too hard than by making a game too easy. So, when in doubt, underestimate the player's learning curve (actually, it's good practice in general to underestimate your players).

3. Don't reward skilled players by making the game easier!

There are a lot of games that reward their players for doing well by giving them more upgrades. But what this is basically doing is making the game easier for players who already found the game easy, while giving nothing to the players who are struggling. A lot of these games try to compensate for these upgrades by increasing the difficulty. While this may balance the game for skilled players, it makes the game even more difficult for the players who were struggling and didn't even get the upgrades. This is a very fast way to lose players. Really, you should "reward" players who do well by making the game more difficult, and "punish" the players who do poorly by making the game easier, in essence dynamically changing the difficulty to suit the player. While this seems like an oxy-moron, there are ways to make higher difficulty feel like a reward, and lower difficulty feel like a punishment. For example, I've seen games that, if you do well enough, reward you by giving you access to a second ending. The gameplay to get the second ending is a lot more difficult than the first ending, but the reward is that you get the second, perhaps better, ending.

You can also hide the fact that the reward is making the game more difficult. For example, you could give the player upgrades if they do well, but increase the difficulty even more than the benefit of the upgrades (and don't change the difficulty for those who didn't get the upgrade). While this seems like cheating the player, most games that give upgrades increase the difficulty to compensate - this is the same idea, simply limiting the increased difficulty to the players who got the upgrades.

The takeaway point here is that while it's important to give rewards to players, making the game easier to a player who is already doing well is not really a reward in the long run.

4. Allow players to change the game's difficulty.

It's impossible to balance a game perfectly for every potential player. So, giving the player a choice on how difficult they want the game can help widen the audience. Players who want casual fun can lower the difficulty, and players who want a challenging experience can raise the difficulty. If the player can adjust the difficulty in the middle of the game, then they can even compensate for their learning curve. Just be certain to never punish a player for lowering the difficulty. It is a choice they are making to improve their gameplay experience. They may already feel bad about having to lower the difficulty, you don't need to rub it in their face with a punishment. If you do anything, reward players who increase the difficulty.


The takeaway point here is that players (sometimes) know themselves best, so letting them choose the difficulty can help balance the game to suit their personal needs.

Closing Thoughts

Testing and tweaking are still the most important methods of balancing a game. No matter how well you balance the game yourself, unless you are the sole audience of the game, you will need to know what it's like for others. Getting friends to play and comment on what they found easy and difficult is a great first step. A beta test that gets comments from the actual target audience of the game is even better. But these four tricks can improve the balance early on, and in doing so help focus the design of the game.

VIDE Player

EDIT:  VIDE is now on IndieDB with some new images/animations.

In celebration of the completion of vide, and to learn how to use the program, my wife made a quick sword swing animation (this was just a quick test, which is why the hand is not grasping the sword), which you can see below in the VIDE flash player (this animation also works on my android device):

The program was easy to use, and the results are quite nice. Being able to animate images makes it easy to make high quality animations. Our next game project, Shattered Worlds (working title), is now cleared to be made in VIDE, so look forward to updates on that.

EDIT: A screenshot of the VIDE Editor for those who are interested:

VIDE is complete!

This will be a short post, but I wanted to inform everyone that the VIDE editor's alpha version is finished. It is feature-complete, and so far seems to be working without bugs, but that wont really be tested until the first game is made using it. It also includes some additional features that I wasn't even thinking of adding originally: the ability to import, add bones and animate vector art, motion paths, and the ability to attach layers to bones (useful for attaching a weapon to a hand, and having the weapon animate with the hand).

I wont be releasing VIDE to the public until I finish a game or two using it (and remove any potential bugs or problems that may arise), but this is big news for Fancy Fish Games. Many games we want to make are dependent on VIDE and were waiting for its completion. Now that it is done, we can begin work on those games. Additionally, we now have our own animation toolkit, which is tuned for the kinds of games we want to make.

Look forward to some updates on our game projects now that VIDE is done!

Progress Update

Since the last blog post, I have been working mainly on VIDE. On the side, I have also been working on two small flash projects, Nom Nom Defense and Dance Lenin, Dance, which you can read about in the Future Projects page. Shattered Worlds and the Unnamed RPG are confirmed to be made in VIDE, and work on Shattered Worlds (along with more information on that project) will likely start after VIDE is done.

The VIDE API is complete, which allows me to load and run VIDE animations in games. Additionally, the API has been tested and works on the following platforms:

PC (Windows XP and Windows 7)

Linux (Ubuntu)

Mac (Snow Leopard)

Android 2.2

iPhone and iPad (Only in the simulator, as I do not own either of these devices)

Flash

As for the editor for making the animations, it is progressing smoothly. Right now, I have a basic library and layered timeline that can do keyframe animations and drag & drop operations. While its basic functionality is pretty advanced, there is still a lot to do before it will be a fully featured editor, so I expect work on VIDE to continue for another month or so.

A screenshot of the VIDE editor, showing a layered bone animation.

There will probably be few updates until VIDE is complete, as there is a lot of work that has to be done on it that is not very interesting to blog about. But rest assured, Fancy Fish Games is well underway, and we have some very interesting game projects planned for when VIDE is complete.