Psychochild's Blog

A developer's musings on game development and writing.

23 July, 2016

The purpose of metrics in a game
Filed under: — Psychochild @ 6:49 PM

Years ago people stumbled across the idea of gathering data about players. A few MMO developers realized they could gather information about what players were actually doing. After all, all player activities had to be verified by the server, so recording that information and storing it for later was a direct process.

Then social network games sprang on the scene. Driven by web development which had been collecting metrics for years, this idea became a no-brainer. Measure everything and draw all sorts of conclusions about what the players really want, with an eye toward maximizing revenues.

So, what has happened to metrics in modern game design? Let’s take a look.

Craft vs. metrics

Chris Bateman wrote a blog post where he talks about the role of metrics in game design. His thesis is that game design is a craft, and trying to define and confine it with metrics misses the point. While he predicted that game designers should try to understand players, he says that he didn’t quite see the rise of metrics.

Chris argues that modern analytic metrics are focus on “analysing where the leaks are in their cashflow, and acting as digital predators to suck spare change out of players’ digital wallets.” He laments that instead of trying to understand the player and deliver a game that meets the player’s needs and desires, a focus on profit has displaced the craft of game design. Some parts of the game industry has replaced a need to understand the players with an focus on understanding how to maximize income for a game.

His conclusion is that this focus on profit has turned what game designers do into less of a craft of creating joyful experiences and more like a soulless pursuit of profit.

Art vs. commerce

In a way, this argument can be seen as a rehash of the old “art vs. commerce” debate. The extreme version of the argument goes that art is polluted by the pursuit of filthy lucre. An artist should create art for the sake of art, and sellouts are those that profit from their work. A more moderate take is that when you do creative work with a commercial focus, you have to compromise on creativity; it’s incredibly difficult if not impossible to both be creatively unrestrained and serve the tastes of the public.

To be clear, I don’t think Chris is really advocating abandoning capitalism and demanding game developers give away their labor for free. Definitely go read his post, as I think he brings up a lot of nuanced arguments. But, I think it is useful to look at his post through the lens of this old debate.

I dislike the absolutist nature of the argument, and prefer the more nuanced version. As a creative person, I still like things like food, a roof, and perhaps air conditioning when the temperature and humidity get high outside. But, I think it is important to realize that there is a decision to be made. One can choose to pure creative energy to create experiences on one extreme, pandering to tastes and maximizing for profit on the other, and a lot of room between the two extremes. And, as much as we might lionize the indie iconoclasts, the reality is that sometimes it takes a lot of work and understanding what people actually want to survive as an indie.

Fun vs. profit

This discussion might seem a bit familiar if you’ve read my blog for a while. I wrote about social games and the focus on metrics before. I’ll draw particular attention to a comment I left later pointing to a talk given by Brenda Romero (née Brathwaite) and Laralyn McWilliams about the balance between metrics and the craft of game design. (Note this is the same Laralyn that Chris references in his post, quoting how she grew disillusioned with “designing for ‘friction’”.)

The lesson here is that metrics are another tool for a game developer. You don’t have the binary condition of either metrics or the craft of game design, but they can both work to build better games. Metrics tell you what is happening, but can’t explain why. It still takes the craft of game design to understand the reasons why players do what they do. Otherwise you get quotes like I put in the blog post, where the people running games have a shallow understanding of game design and think, “if a player repeats something, it’s fun.”

Of course, not having game designers isn’t a sure recipe for failure. People can stumble across good game design, and just as with the traditional game industry, cloning games can get you quite a bit of income from the game playing audience. It’s a game of probabilities: you’re more likely to succeed in the long term if you have someone who knows how to retain players by giving them a joyful experience. The single-minded pursuit of profit via metrics is more likely to turn people off, and can make it harder to follow up an initial success with another one. It’s telling that most social games that focused on metrics are floundering and few companies were able to have more than one notable success (outside of acquiring other companies).

But, again, a good game designer shouldn’t discard metrics just because they can be used poorly. Using metrics to help you understand what players are doing so you can start to understand why players do what they do in your game is too good to ignore. Augmenting the craft of game design with data strengthens the power of good game design.

Metrics vs. the future

Every so quietly, metrics went from a crazy idea to the foundation of some of modern games. It took some outsiders to really show the power of metrics, but it will take experienced game designers to understand how best to use this tool. Chris Bateman is right in saying that metrics have been abused for a while, but game designers need to realize that metrics are a good tool that provide useful information. It can cut through the haze of what players say they want (or, at least, what a vocal minority say they want) compared to what players actually behave while playing. Of course, you still need good community management because those vocal people are likely big supporters of your game that you want to keep happy, even if their demands happen to be in the minority.

What do you think? Game developers: have metrics changed the way you do design? Players: how do you perceive metrics as hurting or helping your experience in games?


« Previous Post:





5 Comments »

  1. One might also have to consider the fact that in most games, players will tend to do those things that the game allows most easily. That doesn’t mean, necessarily, that that’s what they’d *rather* be doing. I’m thinking of things like dailies — lots of people do dailies in WoW, but that’s because so much design has been focused on getting players to do dailies. Would they rather be doing something else? I don’t know, but I’m not sure metrics can tell you that if there aren’t any alternatives.

    Comment by Ysharros — 24 July, 2016 @ 12:09 PM

  2. EDIT — Which isn’t to say at all that I hate metrics. I think measuring things is one of the more objective ways of evaluating whatever it is you’re trying to evaluate. But as you say, it has its limitations.

    Comment by Ysharros — 24 July, 2016 @ 12:10 PM

  3. Metrics are just numbers.

    Sure, you can use them to “analyse where the leaks are in the cash flow”, and from a business perspective, that will make sense.

    But you can also use them to detect which parts of the game aren’t played much, which confuse the player the most, etc, and streamline the design in those areas.

    I dislike the distinction between “craft” and “metrics”, mostly based on the terminology. Craft is an experience-based process, and metrics are experience put into numbers rather than a gut feeling. Neither is wrong, but both feed into the crafting process.

    Comment by unwesen — 27 July, 2016 @ 3:54 AM

  4. Copying over a few comments from the Google+ discussion.

    Michael Birke posted:
    There are “monetization designers” nowadays to make sure that the maximum of money gets extracted out of a player before they quit or ideally the maximum without making them show the game the finger.

    Art vs commerce is an ages old argument, even Leonardo da Vinci had to make compromises.

    But for the video games industry I can pick two examples of games that I loved really much that got monetized into the ground: LOTRO, initially hailed as F2P done right, and STO (Star Trek Online), which got too much driven by buy-me design additions.

    LOTRO’s shop quickly became so expansive that you had to read guides where you buy what to get what, there were bundles, this that editions, what do you get for free etc..
    In the end they even added entire mechanics that encouraged spending money to avoid things like the relic grind for legendary weapons, you might remember that, extended to the WARHORSE. So whole new game mechanics got centered around their shopification and makin people spend money. < - boom, bye LOTRO.

    STO is by now lockbox to lockbox design driven. A game that lives from new ships but either gives them away for free or you have to play lottery or buy them for cash.

    As veteran players people had vast resources, and me and others were a bit the Trumps of the galaxy. ;) You could play the market and make oodles of money, a lot of fun - and buy the new stuff for ingame currency.

    When they stripped that away slowly over time, basically crashed the economy, it must have pissed them off that some people could avoid paying real cash all the time, they basically for some 50-100 EUR for new ships every time something new came along. Bye STO.

    The design that most players pay nothing but some, the "whales", pay a damn lot works best for abusive social media games on Facebook etc.. +Brian Inman has some bad experiences with that.

    It's hard to make a good game and monetize it properly. But if you have a good game, you can extract oodles of money from players, and they won't feel bad about it.

    It's corporate GREED that ruins F2P models. They sooner or later all go down the slippery slope and ruin themselves. That's my personal experience.

    I remember Tomb Raider's reboot: Great game, a feminine hero that wasn't reduced to boobs, etc. etc..

    But commercially, it was profitable but did not meet "expectations". And what happens if expectations don't get met? Check the DLC collection for the rebooted version fo Tomb Raider. Holy cow. A really desperate and half-arsed attempt to extract more money.

    I am afraid Game Designers will have to deal with "expectations" how much money a game will make. What has to stop is forcing game designers to "extract more money, now". That's putting the pistol on their temple and results in all kinds of stupidity, possibly even harming future profits.

    From a player perspective, the gaming world has gone nuts. Example Store Citizen. You can already pay a damn lot for a game that is still not finished or half-finished at all. That is unlikely to meet all expectations.

    But also gamers are to blame for supporting abusive monetization schemes. As long as they work out so nicely, there is little incentive to stop doing so.

    Bart Stewart posted:
    Chris’s article got me thinking that designing for monetization pretty much turns games into slot machines that never pay out. “Playing the game” just means depositing quarters to be able to pull the handle in different ways, and “game design” is little more than managing A/B testing of different handles.

    That’s the inevitable corrosion of game design when the real world — in this case, through money — is allowed to seep past the magic circle. Which means that if you think microtransactions are bad now, just watch the progress of AR games, which already invite the real world in. You ain’t seen nothin’ yet.

    [Edit: while the above thought wasn't directly inspired by Raph Koster's recent cautions on multiplayer pathologies affecting AR games, these two concerns are related.]

    I responded to Michael:
    You bring up two good points. The first is that players have some responsibility here. Most free-to-play games I play I don’t pay for. I’m patient, and I cycle through 3-5 games at one time if there are wait gates. I’ll throw a few bucks at games I think are worth it, though, but I keep my spending in check.

    The other thing is profitability. Again, indies have an advantage here in that they don’t have to maximize profit at every turn. Gives me more hope for this side of things.

    Comment by Psychochild — 30 July, 2016 @ 6:08 PM

  5. A friend of mine pointed me to this blog post, talking about data in the ad industry: https://adcontrarian.blogspot.com/2016/07/tons-of-data-and-not-ounce-of-sense.html

    In essence, data is being collected from online advertising and it’s showing what ad people don’t want to hear, that ads are probably not working that well. But, online ad spending is increasing every year.

    An interesting perspective from outside the game industry.

    Comment by Psychochild — 5 August, 2016 @ 10:58 AM

Leave a comment

I value your comment and think the discussions are the best part of this blog. However, there's this scourge called comment spam, so I choose to moderate comments rather than giving filthy spammers any advantage.

If this is your first comment, it will be held for moderation and therefore will not show up immediately. I will approve your comment when I can, usually within a day. Comments should eventually be approved if not spam. If your comment doesn't show up and it wasn't spam, send me an email as the spam catchers might have caught it by accident.

Line and paragraph breaks automatic, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Email Subscription

Get posts by email:


Recent Comments

Categories

Search the Blog

Calendar

June 2017
S M T W T F S
« Apr    
 123
45678910
11121314151617
18192021222324
252627282930  

Meta

Archives

Standard Disclaimer

I speak only for myself, not for any company.

My Book





Information

Around the Internet

Game and Online Developers

Game News Sites

Game Ranters and Discussion

Help for Businesses

Other Fun Stuff

Quiet (aka Dead) Sites

Posts Copyright Brian Green, aka Psychochild. Comments belong to their authors.

Support me and my work on