13 November, 2015
As I’ve posted before, I play FFXIV as one of my primary MMOs. I’ve been playing the new “Heavensward” expansion recently, and the first major patch lands this week. With the patch comes the first of the new raids.
As I might dip my toe into some of these raids, I was thinking about how to measure raid performance. How do you identify a good raider from a poor one?
Raids in FFXIV
FFXIV does a pretty good job of making raids accessible. You can join a raid via “duty finder” interface and have a decent chance of success. There is usually a progression: you generally have to do earlier raids to open quests to unlock the later raids. Plus, you have gear level requirements (institutionalized gear score) before you can queue for the raid. But, these work fairly well to make sure that people have a minimum amount of competence. But, while you can keep the unqualified away from a raid, you can’t keep the unmotivated; some people just want to be carried.
FFXIV also limits gear progression on the far edge; the most recent raids only allow you to get a reward once per week. At least, until the new raids come out, then the restrictions are usually lifted to allow people to power through intermediate content to get to the far edge. This creates a very interesting situation where you can expect to invest a lot of time in acquiring gear in the most recent raids. This can be especially wearing if you want to gear up multiple classes (since a character can gain levels in every unlocked class.)
I’m not a hard-core raider in FFXIV, I mostly play because a friend of mine does. I dip my toe into the raids I can do via the duty finder, and that’s it. I might go back and do some of the Binding Coils of Bahamut unsynced now. But, I’ll hardly claim to e a raiding expert. But, I am a game designer, and sometimes questions lodge into my mind.
So, let’s say you want to be a good raider; how do you measure performance?
Let me do a bit of philosophical pondering here: why measure raid performance?
There are good reasons and bad. The negative reasons for measurement include exclusion and bragging. Gear score was used early on to measure if someone was going to make a PUG easy or not; people only wanted others with a minimum gear score in order to weed out people being “carried”. Ignoring the actual performance of a person, where a slacker with a higher gear score might do worse than an expert with a more modest score. People will do what gets them their rewards the fastest and easiest, though. And the stories of “meter maids”, people who crow about their DPS meter position, is well known.
Of course, there are also positive reasons. If you don’t measure, you can’t improve. To know if one rotation is better than another, you need to measure the results. This can be hard to do in the heat of battle, and target dummies don’t really give you good feedback on realistic combat situations. So, a meter can be a useful tool for improving your ability to perform well in a raid situation.
So there are many motivations for measuring raid performance. Let’s assume it’s not just for assholes to exclude others or brag about their e-peen, even if this has been one of the more visible aspects in the past.
So, what are we going to measure? Since I started talking about FFXIV, a game that uses the modern trinity of roles, let’s look at those roles.
Measuring DPS is the easiest, because it’s already pretty common. DPS meters were an early UI addition to WoW. Figuring out how to maximize damage done (and often brag about it) has long been a goal of the DPS classes. This makes sense; the faster you can down an enemy, the less time there is for something to go horribly wrong. And, some bosses have an enrage timer, so insufficient DPS will cause failure for the raid.
Of course, there are other measurements that don’t get so much focus. For example, time alive is pretty important. That character that has 10% more DPS than anyone else for the first minute of the raid but who gets killed from not watching the environment is not as useful as someone who doesn’t stand in the fire. Measuring amount of damage is another interesting statistic, as a DPS who takes more damage runs more risk of dying or requiring more attention from the healers. But, a character who takes a lot of damage without diminishing the healers’ ability to keep the tank up isn’t necessarily harming the raid as a whole. Changing mechanics so that a character does reduced damage if they lose a lot of hit points might be an interesting mechanic for raiding. So, some measure of survivability might be important to measure as well.
Measuring tanking is quite a bit harder. There’s an obvious fail condition for the tank: dying. But, in most raiding games this is not the tank’s responsibility alone; the healer is expected to help the tank stay alive. Understanding who is responsible for the tank’s death is a good way to understand how well the tank is doing. A tank who is able to use his or her abilities to avoid lethal damage is better than one who relies solely on healing.
The secondary failure mode is for the tank to lose control of the raid boss and for it to kill other party members, particularly the healer. As such, a tank needs to know how to maintain threat from the boss. A boss that requires a tank swap, where another tank takes over so the first tank can recover from some debilitation, requires more mastery of threat mechanics. But, once again this isn’t necessarily a tank-specific mechanic; a good DPS needs to know when to lay on the damage and when to pull back and let the tank maintain threat, particularly with mechanics like tank swaps.
In FFXIV, the current tank meta for raiding also includes another aspect: damage. A recent /r/ffxiv/ thread on Reddit discusses a lot of interesting ideas about tanking. But, it talks about how important tank DPS is to the current raid setup. Measuring tank damage might also be another way to measure a tank’s ability in some situations.
So, what should you measure? Damage taken is one measurement, although that can be inaccurate since damage sometimes depends on the random number generator for things like critical hits. Measuring damage done to others as a negative measure is potentially interesting, but could penalize a tank with sloppy or lazy members who stand in damage fields or a raid boss’s cleave attack. Lots of options, and I think a lot of them really depend on the encounter.
What about healers? Again, they have a pretty obvious failure conditions: the tank dying. If the tank dies it’s often the healer that gets blamed, although in some cases the tank might also to be blamed. To a lesser extent there’s the failure of letting other raid members dying, but unless you’re the tank, defense and not taking excessive damage is a personal responsibility. The healer drawing too much aggro from healing threat is also a danger, but this is usually caused by a tank taking too much damage and not generating enough threat. Or from DPS taking too much damage requiring use of high yield and high threat healing abilities.
How do you measure the quality of a healer? It’s really hard, because more healing done usually indicates a poor group rather than a good healer. Measuring something like mana efficiency is possible; the healer who knows when to use the right heal is better than one who spams a single heal re
Like tanks, the current FFXIV meta also emphasizes healers being able to do some damage. This is an interesting mechanic because a healer must choose between doing damage and healing; using a global cooldown on an attack spell might delay that healing spell enough to cause problems. Of course, a tank needs to use abilities to reduce damage and other raid members to not take unnecessary damage for the healer to contribute sufficient damage. So, in a way, the DPS of a healer can be a reflection of the quality of the performance of the raid group as a whole if the raid is successful.
This isn’t necessarily important to FFXIV, but still something that interests me; how do you measure someone who doesn’t fit within just one role? When I raided in WoW, I played a Feral Druid that could just as easily fill in as DPS as an off tank. If my DPS was less than optimal because I had some off-tanking options instead of concentrating entirely on DPS abilities, what was an acceptable tradeoff? 80%? 90%? 95%? In a large guild, would I get benched if they didn’t need my off-tanking ability?
What about some utility abilities. Druids could do an in-combat rez, allowing them to get a fallen raider back in during a fight. From a balance point of view, an in-combat rezzer has to be less effective (do less damage, be a lesser tank, do less healing, etc.) otherwise the combat-rezzing class is going to always be preferred to any other class. But, that margin becomes a hindrance if the raid doesn’t need any combat rezes. So, what is the proper tradeoff? Can a designer make a class with some unique utility, like an in-combat rez, more demanding to play in order to make up for that unique utility?
Measuring within a discipline
During the TBC era of WoW raiding, each class within a role usually had a specialization. Warrior tanks were the master of single-target threat and taking large hits. Paladin tanks, on the other hand, were exceptional at tanking groups of enemies with their AoE threat. Druid tanks had more hit points and armor value, but were more susceptible to spike damage from crits. Sometimes people would pick certain tanks for certain encounters based on what the encounters required. Other times, more limited groups had to make due with what they had.
How do you measure performance when different characters in the same roles have very different strengths? How good does an AoE tank have to be against a single target to be useful in general? Is a group on the cutting edge only ever going to want the specialized tank? It gets tough because people will get sidelined if your particular strength isn’t required for a raid or for a particular encounter.
This is complicated by the fact the grass is always greener on the other side. When I did healing as a Druid, my friend who played a Priest was jealous of my instant cast HOTs and in-combat rez. I wanted his mana-efficient quick heals and shields. Even though we had our individual strengths, there was always the suspicion that some other specialization in the same role had it “better”.
Measuring the group
There are tons of individual measurements for each role, but what about measuring the group as a whole? The ultimate measurement is “did the group kill the raid boss?” But, what about comparing performance between groups? Whereas currently groups compete on “world first” or “server first” completions, what if there were a way to measure how well a group did overall and rank them?
You could measure time taken, although this emphasizes DPS over the other roles. Unless the tanks and healers can provide significant damage as well, as in the current FFXIV raid meta. But, this feels like optimizing for one specific stat, and as that Reddit post I linked above states, some tanks want to feel more like they’re protecting everyone else rather than that they’re just another person who the healer prioritizes in healing.
You could measure damage taken, with less damage taken being better. This, on the other hand, emphasizes preparation and stats. A strong tank is better than a moderate tank and DPS focusing on damage.
The right answer is probably some balance. A carefully designed equation that balances all the measurements of the different roles and overall performance. The specific equation is left as an exercise to the reader, or for me when someone is willing to pay me to do that bit of hard work properly. :)
The last thing to consider is how special group mechanics could be used as measurement. For example, a group able to pull off LotRO’s fellowship manoeuvre, particularly the right one to match the situation, is better than a group unable to pull this off. Could the execution of other, similar mechanics be an indication of group performance?
What do you think?
So, how do you measure raid performance? What do you think is a good way to measure individual performance? What about group performance?