Wednesday, March 11, 2009

Don't you Wonder Sometimes 'bout Sound and Vision

One of the interesting things about first-person shooter games is their marriage of vision to power. If you're out to kill some dudes, your primary task is to look at them directly. The protagonist of a first-person shooter game is essentially a murderous, swirling vision cone. Maybe this is why games in the genre occupy the front lines of the battle for visual supremacy.

And this is why I think the sound design of Halo 3 is so ingenious: it uses sound, rather than vision, to expand your hegemony.

Let me explain. One of the big challenges when playing multiplayer first-person shooters is that it's essential to expand your spatial awareness beyond what's going on withinin the frame in front of you. Even when you get acclimated to the maps, and develop this ingrained lizard-brain consciousness that there is a wall behind you and to your left, you must understand where your enemies are in order to succeed. And this is possible when you learn to map the blips on your radar into your lizard-brain wall-consciousness. Once you have all this under you belt, there's still a last thing to consider, which is what weapon your opponent has. You have a split-second to gauge how you're going to approach this encounter-- whether you're going to charge them, or let them come to you, or whatever. These tactics all turn on how your available weapons match up. Often you have to make these calculations before you even see the person you're about to encounter.

The brilliance of Halo 3 is that you can get some of this information by listening. I play with headphones sometimes so as to avoid waking up the housemates, and one thing I notice all the time is that every significant aspect of Halo's gameplay has a distinct and differentiable sound. Each weapon, each piece of equipment, each vehicle is instantly recognizable. They even have different dynamics; some are loud and some are relatively quiet. It's really remarkable once you notice it. I remember once, when I had been playing Halo for about three months, I heard the tic-tic-tic of a minigun in the distance. And I thought “Holy crap, I don't just know that there's someone using a turret, I know how far away they are from me now. They're on the opposite side of the map but that one gun is louder than the rest.” You can use sound to get spatial information that your eye's can't give you.

On an encounter-by-encounter basis this information is often tactically invaluable. (This is why the game also visually represents sounds using yellow arrows at the edge of your field of vision.) Like, you'll hear that the guy in the room below you has a shotgun, a deadly close-range weapon. Which means: for god's sake, don't just drop in there. Engage from a distance. Or you'll come out of a base and you'll hear a Warthog joyriding around nearby and slaying your teammates well before it appears on your radar. Which means: do some cowering inside the base until you figure out how to take it down.

I think a lot of people in the critical-blogging line don't particularly like what Halo represents. It's a totem for the kind of game (maybe even the type of gamer) us we'd like to see less of. At the very least we'd like to see fewer games attempting to be what Halo is. Hell, even I hate Halo some of the time. But the basic truth is that good design conquers all, and this is where the game shines.


Aler said...

I'm lousy at FPSs, but I've started playing Team Fortress 2. It's another game with good sound design - even though I don't yet know the maps and I don't yet know all the team strategies, I can always find the fight just by following the sound of gunfire.

It's amazing just how well this works.

Nels Anderson said...

On a related note, Valve talks a lot about allow the player to resolve input as quickly as possible. Distinct colours that don't interact with the environment's palette to resolve character's team, readability of silhouettes to resolve class, distinct shape to resolve weapon loadout.

It's interesting to see how they use sound when one of the above isn't viable. The Heavy's Sascha sounds very different from the newest Natasha, as do the Scout's two guns.

Thinking bad to something like the original TF, where nearly everyone looked exactly the same. I wonder how we ever played those games sometimes.

Iroquois Pliskin said...

@sdevet and @nels: This is a great great point about Team Fortress 2. Not only the sound, but if you listen to the commentary or interviews you'll hear the valve guys talk about they made sure the visual silhouettes were immediately distinguishable too.

chade fallstar said...

From what I understand, sound plays a role in WOW PVP also (not PVE so much), insofar as it lets you figure out whether someone is sneaking up behind you.

One thing about the WOW sound system is that although ALMOST all the actions have different sounds, there are like a couple that have incredibly unfortunate overlap. I know there was at least one time during raiding that I almost peed myself because I heard something that I thought was going to kill me but really it was like the doucher rogue pulling out his shigawire.

Uggh one of my friends from illinois is a visual cognition specialist at valve. He did some research on 4D perception as a grad student, who knows what he'll come up with as a corporate tool.

Anonymous said...

Good post Iroquois. Not enough smart people realize (or at least don't talk about) the amount of thought that goes into a game like Halo 3. Often, the answer is about 100 times more thought than went into the creation of some indie gem that makes the job of critical analysis easy for us.

It just feels strange celebrating something to mainstream; however, there's a reason every moron and college professor can rattle off at equal length about the brilliance of New Hollywood works such as The Godfather.

I've been playing a lot more Halo recently because of the new maps (they really know how to rope me back in), and every time I come back to it I remember how much the game rewards perception, forethought, and teamwork more than any other shooter I've played (even Left 4 Dead, to be honest - excepting Expert difficulty and some Versus matches).

Returning to the New Hollywood comparison - the popularity of games like Halo are what make the funding of indie games viable. The expansion of the pop cultural cache of videogames is -the- precondition to the development of the form (we all grew up playing Pac-Man and Mario).

Celebrating and rewarding craftsmanship and design in the mainstream is just as (if not more) important than writing about how thoughtful Rohrer and Blow are. Not to mention the fact that it's a lot more difficult (because it requires technical understand AS WELL as a liberal arts degree). Keep up the good work.

Oh, and you had me at David Bowie.

Anonymous said...

technical understanding*

F. Scott Frazier said...

Completely on board here with this topic. Sound design is so intrinsic in being able to interact with a 3D space, especially 3D spaces that don't actually exist.

And yet, most sound design is relegated to the back burner, with little thought or emphasis on its importance. Here's a fun little test: Name your favorite five games of the last few years. I bet you at least four of those games placed an emphasis on sound, so that at most all times it is informational, interactive and atmospheric.

Some companies always nail their sound. Companies like Monolith are always at the bleeding edge when it comes to sound, both in design and technology. FEAR, Condemned and FEAR 2 have some of the best sound editing and sound mixing in games. They rival high end movies in that regard.

Anonymous said...

Great post, Iroquois (great blog, in general!). A game that really did it for me, sound design-wise, was Far Cry 2.

That game conveys a feeling of actually hunting that I think is really effing cool, and I think that the aural experience is why.

Whenever you do any sneaking in that game, you're effectively blinded (crouched as you are with your head in the jungle foliage), so you have to rely pretty heavily on your ears in order to keep your bearings.

It really makes the whole thing work for me, adding an almost primal level of tension to the game.

And there's no feeling quite like sidestepping a charging enemy because you heard him coming, then tossing a grenade into the thicket where you approximate him to be, based purely on what your ears were telling you.

There were some features online in which the FC2 dev team discussed their trip to Africa to record the game's audio, and I think their dedication paid off.

Anonymous said...

Nice post. I think Valve was really one of the forerunners of caring about these little details.

I remember playing Team Fortress and Counterstrike, and feeling sheer animal terror as I recognized the sound of a grenade pin being pulled nearby. In CS, the footsteps alone were so revealing that a lot of servers started playing with them turned off. It was brutally hard.

It's also very telling to me that the ONLY assets TF2 has in common with TFC is its sounds. The same sounds play when you reload a shotgun, when you pick up health, when you take falling damage, etc. This is a really subtle thing that made the game really feel like home for the tons of TFC players that tentatively tried it out but were willing to go back to the old game if they hated it.

As mentioned WoW is also surprisingly good at this. The first time I instinctively hit Will of the Forsaken (fear immunity buff), because I HEARD someone's offscreen pet succubus casting a seduce on me, it took a few minutes to notice. When I did though, I felt great.

Consistency is a big deal to me, and something I write about a lot. It's tempting to have lots of sound effects or animations and swap between them randomly for variety, but the very thing that enables you to become such an expert at the game is that these elements always behave the same way.


Kirk Battle said...

Hrm...makes you wish games would experiment with sound more often. A power-up that masks ALL noise that you make or maybe just having a multi-player mode where the sound is scrambled and hard to interpret because a generator or something is humming in the back. After all, the quickest way to get someone to care about something is to take it away.

Iroquois Pliskin said...

@laz: actually, the post of being a visual cognition specialist at Valve must be pretty sweet. I get the idea that these guys are really into doing empirical testing, and thought I distrust all empirical work on principle it seems to work pretty well.

@simon: I agree. We praise Valve to the stars for their attention to craft and their mastery of the nuances of game design, I think we give short shrift to Halo, whose design is just as meticulous.

I don't think it's a zero-sum game when it comes to praising the indie gems and the big-budget shooters (Braid is pretty meticulously designed, come to think of it.)

@f.Scott Frazier: this is actually true. I really love games (like everyday shooter and REZ) where music is integrated into the gameplay. Never played the FEAR games tho, I hear they are quite good on this front.

@kirk: hey thanks! I thought the sound in FC2 was pretty great, I know the experience you're talking about where you're hunched in the grass and all you can do is listen for the enemy. My one problem was that they didn't get the dynamics right. Like, it would always sound like some south african guy was whispering directly in your ear even if he was 40 feet away. Anyone else notice this?

@mikedarga: good point about Valve getting these things right. Thought nobody's mentioned it yet, L4D was great on this front too: not only the stuff that the characters say to teach other in order to alert you to significant elements of the environment, but also the way you could hear certain enemies before you could see them.

@LB: surely, someone has done this before, right? the flashbangs in COD4 were kind of like this.

chade fallstar said...

Oh, it's sweet alright, he gets a lot of $$ and plays a lot of video games and doesn't have a boss. And also gets to go partake of ridiculous demos by companies saying that they can tell your emotions from your brainwaves. He's pretty much the model of how a psych phd should sell out.

David Sahlin said...

One of the best indie Mods for Source, Dystopia, has an amazing Sound Wave Triangulator upgrade. It displays the strength and orientation of sound coming to you, and also pins individual sound-icons like footsteps and gunfire to the player's UI for the moment.

Very fun take on using sound in PvP.

Anonymous said...

Ha, yeah, you're totally right - with the enemy chatter, they didn't really nail the aural spectrum, so the distances didn't feel right.

Aaah, well... maybe in Far Cry 3.

In the meantime, it'd be cool to see more games force you into situations in which you're effectively blinded and have to rely on your ears to guide you. Very cool trick, and somewhat underused.

Anonymous said...

Nice article.

However, I feel compelled to point out that your observations about Halo 3 basically hold true for most top tier games.

I, for one, was amazed at the amount of near-subliminal data Gears of War 2 was giving me via in-game sounds. (those sharp clicks when you're a few bullets away from an empty clip spring to mind...)

I guess my point is, probably any recent AAA shooter you'd care to name has poured just as much attention as Halo 3 into its sound design. There's no valid reason to single it out.

Nonetheless, I'm happy to see other people are also noticing these little bits of design genius that make modern gaming such a delight.