Loom: Music as part of gameplay

My cousin is a Jazz pianist. Sometimes I envy him. I am sitting the whole day in front of the computer, coming up with elegant code, intricate game design mechanics and laboriously squeezing every pixel in Photoshop to communicate the desired message, and people will still have such hard time recognizing the value of that work. Meanwhile, he can casually walk up to a piano, sit down and spontaneously create a tune which everybody in a room will instantly understand as “art” and something worth spending time listening to. Maybe this is why I am a bit suspicious when it comes to music, especially music in games. Not that I don’t like music, I do! I especially like my cousin’s music too! I just sometimes find myself thinking I am the only sane person in the world, who does not make such a big deal out of it.

My cousin, Marcin Masecki, doing his thing.

Here are three important facts about music and what is can and cannot do in games:

Music is great for setting a mood
Seriously, nothing sets a mood as good as a piece of music does. A painter can create the most striking, disturbing image, a poet can write the most gloomy, dark poem. Then, just play Yakkity Sax in the background and all that angst turns into comedy in an instant. And it works the other way around too! Rammstein vs. Teletubbies. Will it feel bright and cheerful like the Tetubbies? I don’t think so.
I learned that lesson when working on the Twisted Reality project. We had some bizarre videos of flying buildings and we had to make a music video out of it. Some people in the project insisted in this dark and gloomy soundtrack. I resisted because people would immediately interpret those buildings as a threat which would take away the mystery. The other team members were convinced you could counter-balance that with post-processing and cutting. So we kept the gloomy soundtrack and you can see who was right in the end.
This is great news for game designers because you can create awesome experiences with so little effort. Maybe you don’t need to spend so much time in those photorealistic, normal-mapped, anti-aliased, carefully lit, laboriously textured, environments when simple 2D sprites and some great piece of music can do the job. I am reminded of “The Crowd” by Kyle Gabler from the Experimental Gameplay Project. When this weird people start to sing I always get the goosebumps.

The people in “The Crowd” look already a bit creepy, but when they start singing…

Music is a system
That seems like something obvious but it can be (and has been) used by game designers to link music to game mechanics. Music is something quite logical. There is rhythm, there are distinct notes, there are tones, which have a mathematical relationship with each other. By just following some simple rules, you can take any kind of sound and make something out of it which will be perceived as music. Games like Electroplankton or Everyday Shooter are great examples of understanding music as a system. But even Guitar Hero uses the rhythm of music to augment gameplay.

Here is Everyday shooter, blurring the boundary between sound effects and music like it’s nothing.

Music is not a language
You hear it always especially form jazz musicians (take that, dear cousin . “Music is like a language but without words so you can communicate across cultural boundaries”. No it isn’t and no you can’t. This is just poor understanding of what a language is. Put down that instrument and go read a book about linguistics. There has been a tremendous amount of research about language and it is still a phenomenon with lots of unsolved problems. But so far it seems like all languages actually share some common features and although music exhibits some of them, it fails to satisfy the criteria entirely. When you compare music to a language it just shows how ignorant you are.
But sure, I can understand how you can arrive at that conclusion. You get into a jam session with a couple of guys from a different country. You don’t know their language and you can’t speak to them, yet you can do music with them and in the way they play you get a vague sense of what they are like. But then again, that happens on a regular basis in about every international soccer game.

Those guys can play with each other but could the guy ask the girl for her telephone number using only music? Could she answer?

As for communication across cultural boundaries: it is a touchy subject. However music seems to be not as permeable as musicians want to make us believe. Subcultures explicitly use music to SEPARATE themselves from others. Generally speaking, Goths listen to a different music then Punks, Emos, B-Boys, Ravers etc. Here, music helps defining cultural boundaries.

As I mentioned previously, music is great at setting the mood, but this is as far as music can go when it comes to communication. You can communicate something like “despair for the lost of a child”. But you can’t communicate “Honey, would you buy some milk on the way back from work?”. Every musician who tries to do so will drink a lot less milk. But that’s ok, because I probably wouldn’t want to listen to that kind of song anyway. Of course, I talk about music without lyrics. If you add lyrics, music becomes a language but only because language is IN IT.

The consequences for game designers are profound. You can put music in the background to communicate something very general and broad – say, the fast music in Tetris when the rows get very high. But you can’t be as specific with it as you can with text or graphics. So for most games, you can play without music, but you can almost never play them without text or graphics or sound effects. There are some exceptions though, so let’s look at the big picture:

Out of this 3 facts about music, we can create categorize the different uses of music in games.

This categorization was put together very quickly and I’m sure it can be improved upon but right now it does a good job to start the discourse. I set up two broad categories of music usage in games. One being “Connonational”, which means that the music doesn’t directly affect the game play and the game can be played with the music turned off. The other is type of usage is “Systemic” which means that there is a deep linkage between the system of music and the system of gameplay and that you loose vital gameplay elements if you turn off the music.
Further, I defined 5 different types of music usage. 4 of them can be categorized as either being Connotational or Systematic. The fifth one is somewhat a transitional type. This reveals that the whole categorization is a very fuzzy thing and there is always something Connotational about Systemic usage and vice versa. Let me say something about the individual types:

Filling Silence
This is the simplest way of implementing music in a game. Select a tune you like, put it in the background, set it to loop. There you go. I’ve mentioned Street Fighter as an example but that isn’t exactly right. Music is so good at setting the mood that removing it will always result in a different game experience. So even in Street Fighter, the music is quick and lively which fits well to the overall playing style. The “Filling Silence” category might be only theoretical. However there are games, which get very close. Many flash games feature really annoying music you almost immediately turn off voluntarily, changing the character of the game in a positive direction.

Setting Mood
This is about what Music is really great at. That’s why most of the games fall in this category. It is characterized by the use of a very specific piece of music, which fits into the context of the game. Even the simplest games use the music to set the mood in some way. Think of Tetris. It may seem like a case of Filling Silence but there is the special music when you are close to game over and the normal music sounds very Russian which is the overall theme of the game.
I’ve mentioned Myst here because it represents a more sophisticated use of music. Music is very subtle there, used scarcely and in very specific scenes. Myst is generally more a game about mood then Tetris or Street Fighter, so it is a better example.

The well-known holoprojector in Myst. When you enter the room (a dark cave), you suddenly hear some very atmospheric music playing.

Accompaning Gamplay
This is the transitional category. A good example is when things happen on the screen in sync with music as in Gutar Hero, Rez or Lumines. In this case, the gameplay as clearly a systemic relation to the gameplay. Somebody must have sat down and actually counted the BPM’s of a song or even designed a whole sequence of Guitar Hero frets, which mirror to some extent the individual notes of the song. But there is some Connotational function there too: the choice of Songs in Gutar Hero is not random. They all help creating the whole rock star fantasy. Right now, this type of usage is a bit underdeveloped but a large chunk of games, which fit into this category are actually an established genre: the rhythm game genre.

Ouendan is a good example of the plethora of rhythm games, which appeared recently.

Part of Gameplay
And we sail into the undiscovered realm of Systemic use of music. Right now, there are very few games, which work like that and even those that do fit, fall into the category of educational games, which teach kids about music. There is one, shining example, though. It is an old adventure called Loom and it was designed by Brian Moriarty I already mentioned. I will talk about Loom some more at the end of this post. For me, it represents a desirable direction to develop games in.

Instruments
In some new, innovative games the player’s ability to generate music is very well developed. Additionally, those games do not present you clear goals or challenges. As a result, they become more digital instruments or music programs then games. There are very few “games” like that but a good example is Electroplankton which is a fascinating collection of various musical “toys”. It is a great experience, I’d wish I had something like that as a kid because you can do some great stuff with it. This kind of “game” can be only created with a deep understanding of the Systemic properties of music in mind. Also they do not necessarily create a specific mood as they provide no context where that mood would be useful. The different toys in Electroplankton have their unique sounds and styles but this is also true for real instruments and to some extent it is up to the player what sort music comes out.

Jam Sessions being played as an actual instument on stage.

Loom
As already said, for me, Loom is the kind of usage of music I would love to experiment in the future. Right now, there are not many examples of music as part of the gameplay. If you do know a game which fits in here, please share it me!
When music becomes part of the gameplay, players have to make music or listen to music in order to proceed. In the case of Loom, this is done by assigning different meanings to certain tunes which players have to remember and reproduce. By doing so, melodies become symbols and therefore transcend the music’s inherit inability of generating meaning by itself. However in this type of usage, music must be very clear and simple so it somewhat awkward if you want to create mood. Yet, Loom is a very old and experimental game and I do believe that some with some research and development it is possible to add some Connotational value to this type of music usage.
Loom is an old-school 2D Point-and-click Lucasarts adventure. You have a character and you can click on things to interact with them. However, Loom is somewhat an oddball. First, there are no Items to pick up in Loom, which is unusual. But more importantly, in other Lucasarts adventures you had a matrix of buttons for different verbs at the bottom of the screen, which you could use to manipulate the environment. In Loom, you just have a wooden stick. It is a magical stick and actually, it is a simple music instrument. Clicking on the different areas on the stick will produce different tones. The idea is that with this stick, you can produce magical tunes which cast spells of different objects. In a way, it is still like the matrix ob buttons but instead of simply clicking a button you have to recall and play a certain tune. You learn new tunes from the environment. Looking at a hatching egg, you will hear the tune for the spell “Open”. Some of the tunes are randomized at the beginning of the game so you are forced to find and remember the tunes even if you play for a second time.


On of the easiest tasks in loom is to turn hay into gold.

This is already a very cool idea but Loom does two things to spice things up even more. First, at the beginning of the game you can play only a few notes. As you proceed and gain more experience, your stick “levels up” giving you access to spells you previously couldn’t play. This gives you a great sense of progress and makes the first levels much easier at the same time.

The second additional idea is that some spells can be reversed by playing them backwards! So playing the “Open” spell backwards is the “Close” spell. Some spells, like “Night Vision” cannot be reversed, because it would make no sense. Here is the kicker: the spells, which aren’t reversible are Palindromes – melodies, which sound the same if you play them backwards! This is just such a great Idea. The actual properties of the melodies naturally define the way the player can interact with them!

On easier difficulty levels, the game offers some visual help. Here, we are learning the “Twist” spell from a twister.

Loom comes in different difficulty modes. In the easier modes, the stick has different visual cues (colors, notes etc..) , which helps players remember the melodies. You get the full Loom experience in the hard mode, where you have to rely on hearing only. So Loom is another example of how usability works different in games. From the point of view of efficiency, the stick is clearly inferior to the classical button matrix. Players will struggle a lot with remembering and reproducing the different tunes. Yet in the context of the game, it creates an experience no other, efficient interface could.
In Loom, you play a mysterious magician from the guild of Weavers (unique theme!). You wield mystical the power of magic. It would take away all that mystery if a spell was just a button you can press. Instead, you are given a stick, which looks ordinary at first but as you learn the tunes, you discover the great powers that lie within. Using it is hard and difficult but if you make an effort, you can do some great things. One of the first spells in the game turns hay into gold! All the time, the interface stays the same. The the struggle of the player with the interface accurately mirrors the experience of the young magician in the game, learning to utilize the magic staff to its full potential.
Of course, Loom is not perfect and it has its share of shortcomings. The tunes used for spells are very simple and ordinary. This might be because of hardware limitations but also because of sheer usability. Additionally, the tunes themselves have a rather arbitrary connection to what they mean. Although this is a property of a real languages too, it would be interesting if the the spells were a bit more Onomatopoeic. Still, it is a great game with a unique theme, great story, unique use of music and a resulting superb gameplay.