Metacritic vs. Modern Warfare 3: The Psychology of Fanboys

Does giving Modern Warfare 3 a zero out of ten really help anybody?

There's certainly nothing wrong with disliking Modern Warfare 3. There's nothing wrong with disliking anything.

There's a vast difference, however, between constructive criticism and what's going on over at Metacritic right now.

Metacritic, if you're not aware, is a website that collates and averages review scores for a variety of entertainment products, including games. Over the years the site has become a hugely influential component of the video game industry. The stocks of video game publishers can rise and fall significantly based on Metacritic scores. Earlier this year investors were spooked by Homefront's 72 out of 100 average on Metacritic; THQ's stock sank by 21 per cent on the first day of the game's release, well before any sales data was available. Some companies have even been reported using Metacritic scores as indicators for additional rewards for developers, working a desired 'Metascore' into publishing deals.

Metacritic differs from sites like GameRankings, however, by featuring a user score. User scores are not factored into the critic Metascore, but they sit alongside it.

The user score system, however, is flawed. The user score, out of a possible ten points, for Modern Warfare 3 at the time of writing is 3.2 on Xbox 360, 2.8 on PS3 and 2.0 on PC. The critical averages (out of 100) for the game are 89, 88 and 81 respectively.


What's of note here isn't that Modern Warfare 3 has attracted negativity. Gamers are permitted to dislike any game you put in front of them, Modern Warfare 3 included. What's noteworthy is just how many gamers are so turned-off by it that they are compelled to give it zero out of ten.

Dr. Daniel King is a research associate at the University of Adelaide's School of Psychology, in South Australia. King is happy to help us examine exactly what's going on here. He took a few moments to read some of the community reviews of Modern Warfare 3 on Metacritic.

"So many zeros!" says King. He's not wrong.

"My skim read of the content of these reviews suggests to me that the number one complaint is the perceived lack of innovation and new content from the previous iteration of the series," he continues. "People aren't necessarily saying the game itself is a 'zero' as an overall package, but are expressing a very strong dislike of the series' overall direction as a yearly series that makes relatively small incremental improvements and changes. I'm a gamer, I know where they're coming from. You expect value for your gaming dollar, especially in Australia – these reviews suggest that there is at least a significant minority of players who feel that the Call of Duty franchise is no longer delivering along those long-held gaming values of originality, innovation, what-have-you."

Many Metacritic users have turned a ten-point scale into a two-point scale; "Loved it! 10!" and "Hated it! Zero!" There's no pressure on users to think about assigning numerical scores in context. Fairness, balance and credibility are shot out of the sky when those upset with one flaw in a title rate it zero out of ten. There's no perspective to these sorts of knee-jerk scores and there's very little analysis. Does a zero really help your fellow consumers? If Modern Warfare 3 is a zero, what is Superman 64? Worse? How? It's not objective criticism; it's a protest.



"I think it's pretty fair to say a lot of the user reviews are rants," says King. "Many user reviews lack obvious criteria and systematic reviewing. Many reviews are typically, 'I love this game' or, 'I hate this game and I hate the corporate greed of Activision'. They are framed in terms of the emotional experience of playing the game, be it in awe of the action and spectacle or feeling ripped off for buying the game despite it seeming similar to past games."

"The emotionally framed reviews are also generally dichotomous in their scoring, a zero or ten, because it's hard to put numbers to our feelings. It's not a finely-grained process for many people. We either feel angry or excited, or we don't. That's not saying there aren't some really good reviews on Metacritic, because there are, it's just that they get lost in the sea of dissenting voices.

"I also suspect not everyone wants objective reviews of video games. Emotionally attached people will gravitate to emotional arguments. It's harder for these people to relate to the more objective analysis of a video game, and actually feel turned off or bored by it."



King has actually recently noticed something similar in his current experiment into how video gaming affects sleep onset latency and sleep quality. As part of the study they have had their participants rate the video game they played along emotional indicators of 'exciting', 'boring', and 'fun', among others. They then rate the game with an overall score out of ten.

"It's not a major part of the study, we are mainly interested in the sleep aspect, but it's been interesting to see how breaking down and quantifying the emotional experience of playing the video game has granted insights into how people evaluate games," says King. "For example, some participants will finish playing the game and say, 'This game sucks'. They'll rant about some part of the game where they got stuck or thought the enemies were unfairly difficult."

"Then you give them the survey where they can systematically rate the game on factors like fun, excitement and boredom and they really have to stop and think about it. We see greater variation in scores – like 7 on fun, and 9 on frustrating – and the overall 'score' based on emotion, 'it sucks', doesn't reflect the more systematic evaluation. Having to think in more focussed way seems to give a more balanced overall view rather than give a knee-jerk, emotion-based judgement. Metacritic could introduce criteria for reviewers, but I expect that agreement on what criteria to use would be impossible by democratic vote and would take a lot of fun out of the community."

Metacritic tightened restrictions to cut down on 'review bombing' in August 2010 by requiring users to make an account (prior to this you just needed a unique email address) but the system is still open to abuse. There are more fundamental issues than simply trolling here, even though the latter is a key factor.
Many Metacritic users fail to rate games what they think they should be rated; they just want to have the maximum effect on the average rating. If a member sees a user score higher than they personally think it should be they'll give the game a zero so their score has a larger impact on lowering the rating.

It works the other way too. It's either two thumbs up or two thumbs down. There is no middle ground. Check out LittleBigPlanet on Metacritic. 4556 user ratings and only 55 of them are in the middle of the spectrum. Only 55 out of 4556 people rated LittleBigPlanet a five or six out of ten.



This black-or-white, hot-or-cold mentality completely undermines any honest, legitimate reviews – negative or positive. How can a regular consumer be expected to sift through piles of childish hyperbole and irresponsible exaggeration for any thoughtful discussions of a game's flaws, or measured praise of its strengths?

It's a problem only exacerbated by the embarrassing amount of trolling. It's boggling just how many gamers seem to have bought into the marketing war between Modern Warfare 3 and Battlefield 3 this holiday season. We're obviously aware of a strong anti-Battlefield/anti-Call of Duty sentiment brewing away in each tribe but it's still surprising as to the degree in which it's manifested. It's depressing, for instance, how many Metacritic user profiles you click on to find this.



Not only has this Metacritic member has found it so difficult to function in a world where Modern Warfare 3 exists he's made sure to rate it zero across three platforms. He's also made doubly sure to show the world which side of his bread is buttered. Battlefield 3? 10. Modern Warfare 3? Zero. It's illogical. It's only natural you may end up preferring one over the other but it's impulsive and immature to deal in such absolutes.

There is one explanation for this sort of single-mindedness.

"There's a concept in social psychology called 'Ingroup Bias'," says King. "This refers to the preferential treatment that people give to those who they perceive as belonging to their own group or follow their way of thinking. Ingroup bias is very common among hardcore gamers."

"Some people become very attached to certain video games, especially if they have invested a great deal of time in that game... and so they come to perceive that game as a defining feature of a group to which they belong. Berating someone's favourite game if it differs from your own view is about asserting perceived ingroup dominance, and ensuring that an opposing view does not threaten one's interests as a member of an opposing group."

Unfortunately, it can be very hard to argue with some people. It's going to be tough to convince a gamer who feels Battlefield 3 is a perfect ten and Modern Warfare 3 isn't fit to earn a single point (and vice versa) that they're not being particularly objective.

"Clinical psychologists often think of people's minds as a duality of rational and experiential thought processes," explains King. "The rational mind is based on objective fact, logic, and reasoned argument. You can change the rational mind quite easily with new information. The experiential mind is less easily changed or swayed. It's based on emotion, instincts, intuition or gut feelings. Logical information won't have much or any effect on the experiential mind."

"There are lots of people out there who engage with video games using their experiential mind. That makes sense because games are an artform. Because some people are so emotionally attached to video games, and certain video game franchises in particular, it's hard for the rational mind to win over the experiential mind. Faced with a new Call of Duty that doesn't live up to expectations, the emotional reaction dictates the behavioural response. In this case, ranting on Metacritic, and probably to anyone who dares to ask for their opinion!"

That Sledgehammer Games' co-founder and chief creative officer Glen Schofield took to Twitter late last week to mention how suspiciously low the user review score was really shouldn't be surprising. It shouldn't come as a shock that Modern Warfare 3's ridiculously low user score has caused someone from the development team to say something.

What was of note was the fact he was earnestly asking for fans to help. It was interesting to see a developer of one of the world's biggest entertainment brands paying mind to something as exceedingly easily hijacked as a Metacritic user score.

The vitriol that's been piling up on Metacritic isn't a new phenomenon – games like Dragon Age II and LittleBigPlanet have suffered similar fates on the site. Supergiant Games and Signal Studios were put in similarly unfortunate positions a few months back after the average user review score for Bastion and Toy Soldiers: Cold War suddenly plummeted overnight. The developers asked fans to submit reviews to Metacritic, which resulted in a slight increase for each, but ultimately Metacritic got involved, identified the users spamming the titles with scores of zero and got rid of the junk scores.

We reached out to Sledgehammer and Activision for feedback but have been informed post-launch vacation schedules are preventing us from receiving a response.

For small studios this kind of sabotage can be especially damaging, but for something like Modern Warfare 3 it's hard to see it being a problem. Activision claims to have sold 6.5 million copies of Modern Warfare 3 in 24 hours, making it the most successful entertainment launch of anything, ever. This is only two territories, the US and the UK – European, Australian and rest of world sales aren't even taken into account.

User reviews can be an excellent way for consumers to supplement the impressions they glean from professional reviewers. It's likely that one can find additional information among fellow punters not covered by writers communicating with broader audiences. Unfortunately, until gamers grow up, all this fanboy nonsense and anti-social skulduggery is inevitably going to cloud genuinely helpful criticism.

"When most people go online they are generally considered more or less equal in terms of their basic privileges," says King. "The guy who posts antagonistic criticism gets as much freedom to voice his opinion as the person with a logical argument, or as the spambot, or as the person posting a picture of the infamous Picard Facepalm."

It's a great big wall of noise, and a confusing one at that.



Big Rigs: Over the Road Racing is a 2003 PC truck racing title regarded by many as the single worst video game of all time. It has the lowest score of any game on Metacritic (and the lowest score of any game on GameRankings). And yet the sheer number of ironic 10 out of 10 user reviews on Metacritic have seen its user average hit 4.1 out of 10, over twice that of Modern Warfare 3 on PC and above the Xbox 360 and PS3 versions too. Big Rigs is a game that was broken out of the box; it had no audio (at all), no clipping (you could drive through every object in the environments) and no opponent AI (they would literally remain stationary on the start line when the race began).

Again, you don't have to like Modern Warfare 3. But think about how this looks to the untrained eye.



"There are always going to be problems of quality control on a website where people have total freedom to publish any view they want without screening or review," says King. "You don't need credentials to post a review, you don't need your review approved for publication in order for it to appear on the website, etcetera. I'm not saying Metacritic should introduce such measures because that's a bit silly, but you have to take user reviews with a grain of salt when it appears on an 'open access' forum like Metacritic."


 Video// You tube