Psychochild's Blog

A developer's musings on game development and writing.

25 July, 2006

The vocal ones
Filed under: — Psychochild @ 6:31 PM

“What the customer wants is better products for free,” – Dilbert

No, that isn’t an argument using an advertising-based business model in online games. In the Dilbert comic strip, the PHB claims they are going to start listening to the customer, and asks what do customers really want. Dilbert answers with the line quoted above.

In other words, the customer isn’t always right. Shh! Don’t tell them that; it’ll be our little secret.

But, this leads up to a question that most good developers think about: how can you get good feedback from the playerbase?

The main problem is that you can’t do either extreme: You can’t ignore every customer because some people have good feedback. On the other hand, you can’t listen to everyone because there is simply too much information. Even if you have layers of community managers reading and filtering content, there’s no guarantee that they’ll be able to pick out the truly important bits.

Ryan Shwayder posted about influential people on his blog. These people are also known as social network hubs, people who are in contact and that influence many other people. As Ryan points out, if you can keep these people happy their positive latitudes can influence others. However, if they are unhappy they will not hesitate to tell other people and let them know their grievances. It’s a good idea to keep them happy.

But, the problem is: how do you tell who are your hubs? A great way would be to find people who have done research and hire them on. Social network research is relatively new to online games, but the social network study has been done in the “real world” as well. Applying this information to online games is even easier, since we can track anything that happens in our games.

The problem is that you can easily misidentify the social hubs in your game. Do the people posting on your forum count? What about guild leaders? What about crafters? Unfortunately, there’s no single attribute you can identify to say, “this person influences other people.” Even hard-core socializers might just enjoy chatting and not have opinions on the game, so they may not influence other people in ways that the developers care about.

The other problem is that you really have to process the feedback to be useful. One important thing to consider is that people who like something are not necessarily going to be vocal about liking it. It’s easy to see a lot of complaints and assume something is wrong. If 20% of your playerbase is on the forums complaining about something, you should change it, right? Well, not necessarily. What if 50% of your playerbase actually like the current system and would hate it if changed? My rule of thumb is that for every change, about 25% will love it, 25% will hate it, and 50% won’t care one way or another. You’ll likely only get a fraction of the people that hate a change to actually complain about it, and an even smaller faction of people that like the change to publicly give support. The ratio of people who will become vocal will skew towards the negative side even more if people can apply social pressure to “dissenters”. In Meridian 59, people who supported our decisions often made their support for our changes known directly to us. They didn’t want to be singled out and PKed by the people opposing the decision!

Unfortunately, this leads to more conservatism in game design. If people are complaining about the game, that means they are still playing it. The people complaining aren’t the only people you have to worry about: the harder cases are people that just quit when a new change comes along that they do not like. The ultimate measurement of success in a game is the number of people playing. A change could make people happier… or it could piss off enough people to cause a drop in revenue.

The other big problem is the echo chamber situation. People who want to complain about something will go someplace public to complain. Other people with similar complaints will go to the same place and back up the complaint. Soon it seems that “everyone” is complaining about this problem, reinforcing each other’s complaints, and the developers aren’t doing anything about it. Even if “everyone” is only a dozen people making multiple posts on a single thread on the forum. Online forums, particular game forums, can be a place where opinions are reinforced by others. Keep in mind that only a small fraction of players in a game actually go visit the forums in most games, so relying on the forums for all your information can be problematic.

In addition, sometimes the player doesn’t even know what they want. A good game designer is creative and will come up with creative ideas that surprise the players. Before The Sims came out, how many people wanted a game where you could control how often a virtual character could visit the bathroom? I certainly didn’t think I wanted to micromanage someone peeing, but Will Wright showed that this can be part of a rather fun game. The sales figures back him up: The Sims was incredibly popular, even though no one was specifically asking fo this type of game. My friend and business partner Rob “Q” Ellis II often says that our job as game developers is to give the players what they didn’t even know they wanted. I think that’s wise advice.

You also have the problem that some people just like to complain. Everyone complains about X so you pull a week of all-nighters to fix X. Great! Now Y is the one thing that is wrong with the game and it would be soooo great if only Y were fixed. Another week of all-nighters and you fix it. Wonderful! Of course, there’s this problem Z that has been plaguing the game forever…. Online games are dynamic, and there’s no such thing as a state of perfection (regardless of nostalgia tells some players!). You will always have to change the game in one way or another, and therefore will always have people complaining about something that isn’t exactly as they like it. I think you could call this “structural complaining”, after the concept of structural unemployment. Some amount of the complaining will simply be present because there’s no way to make all the people happy all the time.

And, sometimes the complaints simply cannot be addressed. With the area I live in hitting unseasonable 100+ F temperatures, I’ve been seeking the comfort of restaurant air conditioning. Of course, when my food comes it’s hot, which ruins the cool air for a bit. It sure would be nice if they could cook my food without using heat! While it is possible to fix food without heat, this is perhaps a bit unrealistic to expect from the local In-N-Out. Just because I’m a paying customer doesn’t mean I can demand they find a way to cook my french fries without frying!

Keep in mind that with a PvP-focused game these complaints will only get worse. People want any advantage possible, and sometimes convincing a developer is about the best advantage you can have. Small changes will result in large complaints when people think that they are losing power over others. It’s very important to consider feedback very carefully before doing something knee-jerk.

However, don’t think the entire problem rests with the players. There are quite a few issues that cause developers to gum up the system as well. It’s well known that developers are hesitant to throw out systems even if they aren’t working quite right. Of course, sometimes the system might working just fine, but the players aren’t happy with the results because it restricts some undesirable behavior they would like to engage in. In addition, if a major change doesn’t work out, or in the worst case actually results in players leaving, guess who gets fired? Hint: not the players agitating for the change. So, again, you see some reasons for conservative behavior come in.

Developers also tend to overreact to feedback and make the problem worse. Is X too weak? Well, we’ll double the power and now X is certainly powerful! Well, yes, but way too powerful. So, you have to nerf X again; even if it’s more powerful now than it was before, people will only pay attention to the nerf. The most common way to overreact to feedback is to make 2 changes. Is spell X “useless”? Lower the casting cost or increase the effect, but do not do both! If you do both you will almost certainly find yourself on the receiving end that X is too powerful now and must be nerfed! (Usually by people who don’t use spell X….)

You might also have to accept the fact that not every customer is desirable. If I go into In-N-Out and loudly demand that they cook my hamburger without heat, they will throw me out of the store. They will determine that my custom is not worth the effort they have to go through in order to satisfy me. Likewise with the players of our games. The people who complain about having to spend time preparing before they can PvP may not want to play an online RPG. They might find the online play of FPS or RTS type games to be more satisfying; you have to spend less time preparing for a conflict and more time actually playing the fight out. Trying to change an online game radically into a different genre generally isn’t a good solution. Trying to pander to every single player can lead to more headaches than income. The internet is all about getting people what they really want, exploiting the niche interests. Perhaps it’s time we consider that for our own games?

What do you think? Try to keep the comments polite, or I will edit them. I’ve only had to do that a few time in the past, so don’t make me start now.


Next Post: »





9 Comments »

  1. Often times people won’t say what the underlying problem is.

    One suggestion: Use people’s complaints and comments to build a mental model about what they’re thinking. Use that mental model to decide if you should change your product, not the actual complaint.

    For example: If a child complains that they don’t like their dinner, and then they complain that they can’t go out and play, and then the complain about their brother, maybe the real problem is that they’re tired and all the other complaints are symptomatic of being tired.

    The same advice goes for focus group feedback; understand what the people really want, not what they say they want.

    Comment by Mike Rozak — 25 July, 2006 @ 11:44 PM

  2. Don’t remember where I saw the quote, but it applies to your earlier comment:
    “The customer isn’t always right, but they are still the customer.”

    Comment by Notin — 26 July, 2006 @ 7:33 AM

  3. When I was crazy enough to work a full-time job, then come home and volunteer my time for another full-time job, I got to process all kinds of FUN customer complaints for SOE. Even with the reasonably dense mesh of apprentice guides, guides, senior guides, and server GMs that existed for Everquest 1, there weren’t enough people to process all of the real-time feedback. Moreover, I highly suspect that most guides under my command ditched a lot of the comments and didn’t use the feedback logging tool.

    Granted, I don’t know how much of it REALLY got read, but that could have been senior guide burnout too.

    Dealing with customers is hard. When those customers are protected by the anonymity of the internet, they are even harder to deal with. I agree that it’s a tough challenge, and I also agree that within all the noise, there’s some signal.

    If only there was a way to accurately alter the signal:noise ratio, then maybe it’d be a tad easier.

    At any rate, I think your comments are spot on.

    My only real “change” to what you said is that, in the bizarro future, I can see how having a detached, yet possibly listening presence is smarter than having the super-involved nanny presence when it comes to customer relations and feedback. For all of those idiots who threaten to sue over stuff like nerfs, game outages, etc., someday one of them will follow through. Personally, I think that anyone suing for game changes can wish in one hand and shit in the other — guess which hand fills up first. But it’s not out of the realm of possibility.

    Gah, ok, I won’t get all law nerdy here.

    Comment by Matt Hector — 26 July, 2006 @ 6:51 PM

  4. Matt brings up a good point about filtering: How involved should the people be? Customer Service (and I’ll include Community Management in this) need to have some context for discussing the game. If I say that the blooblefrex destroyed my pristine platinum gambol due to an obvious bug, is this more or less urgent than someone else who accidentally sold the bag of endless turnips? Tough to say.

    But, if you get people that are involved, then you lose that detachment aspect that helps keep a sense of perspective. Developers who have specific favorite classes/character types can be accused of working to help those types of characters. Individuals are selfish and want more power for themselves, and developers are often no different. (Of course, we have the motivation that if we make ourselves too powerful and/or abuse such power, we’re more likely to end up without a paycheck.) In extreme cases, you have people forming cults of personality and pandering to a group of people that say nice things. (I’ve heard rumors that such groups exist: no personal experience, I’m afraid.)

    Another aspect of trying to get good feedback.

    My further thoughts,

    Comment by Psychochild — 27 July, 2006 @ 1:14 AM

  5. It would be great to have a mechanism of ‘listening’ to the general feeling of a server. Sort of how standing outside a noisy room, you can tell if the noise is of cheer or hate.

    Or I wonder if you could do virtual ehthnographic studies. Just stand next to a new ‘access point’ to a feature and actually monitor how many people wander in and use it. Just added a new boss? Make yourself invisible and watch every group that fights it for a week. Maybe have some mini interviews.

    At the end of the day I think developers sometimes just need to stick to their guns and fire some shots. Hopefully they’ll hit more often than they miss. No doubt they’ll hit a much loved bystander and the vocal ones will all start shouting at them but hey.

    Comment by Jpoku — 27 July, 2006 @ 11:22 AM

  6. “The public wants what the public gets?”
    or
    “The public gets what the public wants?”

    ;oD

    Comment by Jpoku — 27 July, 2006 @ 11:24 AM

  7. The devs stuck to their guns. I was honestly concerned that barely two dozen players would make an issue seem big enough that development time would be wasted meeting their needs.

    That’s actually really fascinating, and it’s interesting to see this in another game. I’ve done the same thing in some Meridian 59 threads, and people absolutely refuse to accept the fact that for all the sound and fury they bring up, it really does signify nothing. Only a dozen or so people were posting repeatedly trying to convince readers that “everyone” agreed with them. And, people who actually cared enough to speak up in support did so once. Sometimes you see an individual posting multiple rebuttals, but usually those are the people with something to lose in a PvP scenario.

    These people also like to assume that the “silent majority” agrees with them. I actually play my own game on occasion, so I know what people are talking about. I also log in as an admin, and I get in-game mails that give a better picture. As I said above, some people don’t want to go public with their opinion given that there can be repercussions from disagreeing with some of the more aggressive people in the game. ;)

    My further thoughts,

    Comment by Psychochild — 27 July, 2006 @ 2:24 PM

  8. Question : What are your thoughts about ‘official forums’ for a game? Do they give the developer another point to mine the pulse of (a portion of) the gaming populace, or do they create a spot for people to grandstand and add extra headaches for the Customer Service/Community Manager teams?

    It’s very easy for me to ban an idiot who is trying to make a mountain out of his own personal molehill, and for nobody to notice. Put me in charge of an ‘official’ message board and have the exact same circumstances and suddenly “The Dev’s just want us to shut up and go away! They don’t want to listen!” etc.

    Is it better, in the end, to rely upon third party community sites than to build one of your own?

    Comment by Grimwell — 28 July, 2006 @ 6:55 AM

  9. While you can take steps to help encourage people to visit one forum or another, they will decide where to go. We had official Near Death Studios forums on our site for a while, but they were barely used. I think some people got the wrong impression that the game was dead because the forums had so few posts on them. People were much more active on the in-game newsglobes (“bulletin boards”), and the community outside the game stayed on the forums of the largest fan-site for the game. Of course, Meridian 59 had the unique situation of being a pre-existing game, and 3DO didn’t really have an active forum for M59 most of the time it was there.

    But, this does bring up a third option: having internal discussion boards. This may help get more feedback as the boards are available to every person in the game; they don’t have to go outside the game to post. On the downside, if you limit the boards to the people in the game, they lose access if they cannot access the game. Also, people outside the game would not be able to read them to get a little glimpse of what the game is like. However, given the epic drama on some of the boards, this may be a good thing….

    Really, I don’t think one is obviously superior to the other. There’s advantages and disadvantages to each. If you don’t provide the forum, someone else will. In the end, the people with an ax to grind are going to find a public space with your game’s name on it to complain and claim that you don’t care. It doesn’t matter if it’s a third-party site, your own forums, or in-game forums. People who support your decisions aren’t going to spend significant amount of time tilting against windmills to convince the unconvincable, no matter what the forum. I think that having some level of control is better than having none at all, though, so that you can purge the truly harmful stuff: information about hacks, hate speech, etc. If the main forum is hosted by an unfriendly third-party, you could have some bad stuff associated with your game….

    My thoughts,

    Comment by Psychochild — 28 July, 2006 @ 6:30 PM

Leave a comment

I value your comment and think the discussions are the best part of this blog. However, there's this scourge called comment spam, so I choose to moderate comments rather than giving filthy spammers any advantage.

If this is your first comment, it will be held for moderation and therefore will not show up immediately. I will approve your comment when I can, usually within a day. Comments should eventually be approved if not spam. If your comment doesn't show up and it wasn't spam, send me an email as the spam catchers might have caught it by accident.

Line and paragraph breaks automatic, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Email Subscription

Get posts by email:


Recent Comments

Categories

Search the Blog

Calendar

September 2019
S M T W T F S
« Aug    
1234567
891011121314
15161718192021
22232425262728
2930  

Meta

Archives

Standard Disclaimer

I speak only for myself, not for any company.

My Book





Information

Around the Internet

Game and Online Developers

Game News Sites

Game Ranters and Discussion

Help for Businesses

Other Fun Stuff

Quiet (aka Dead) Sites

Posts Copyright Brian Green, aka Psychochild. Comments belong to their authors.

Support me and my work on