APB is going through some changes:
“Vehicle Handling: We’re already underway on a major overhaul to vehicle handling to make cars more responsive and less slippy overall. You’ll still be able to power slide around corners in stylish fashion, but steering is more responsive overall and easier to get the hang of early on.
Combat: We’re looking at almost every aspect of combat – how it looks, feels and sounds, as well as weapon characteristics and tactics. Weapon changes will be put up on the Public Test World to get some feedback in due course.”
All of this but a few weeks after launch.
An all but too familiar story in the MMO industry.
I wonder if many MMO developers wilfully ignore the feedback they get during the testing phase in the hope that a large percentage of their player base is wrong, or whether they’re simply interested in bug reports and don’t even hear the more wide-ranging complaints with respect to the design of major sections of the game. Look at the vast number of tiny angry claws that it took to nip Activision Blizzard on the toe before the corporate behemoth withdrew the giant foot that it had decided to plunge into the Sea of Selfhood, with respect to RealID.
The lesson I think some MMO developers really need to learn is that, whether they like it or not, when players complain en masse about a design decision, the developer generally should take some note of it there and then, thus saving themselves a lot of time and effort when they have to make the change post-release anyway. These developers may roll their eyes when a large proportion of players complain about one aspect or another, but like it or not, these are the people you need to please with your game, because they are not going to complain vehemently about fundamental aspects of the system and then simply say “Oh well, I’ll just subscribe for a year or so anyway”. They’ll pay the box price, and when the included time comes to an end, they’ll just up and leave.
It’s not tourism, it’s consumerism.
Saying “We’re listening to our players and making sweeping changes” after you have already launched your game is extremely disingenuous: these issues are no different to the ones that were mentioned in beta, it’s just that now the players are able to reinforce their complaints by voting with their wallets, which seems to be the only way to get a developer’s attention, to state categorically “no, really, your game is broken in ways X, Y and Z, whether it hurts you to hear it or not”, and then leave. When enough players do this the forum suddenly lights-up with developer posts with words such as ‘overhaul’, ‘restructuring’ and ‘redesign’, and phrases such as ‘fundamental changes’, ‘a new focus’, ‘looking at all aspects’ and ‘listening to players’ feedback’. Unfortunately by that point the only people reading those posts are the dedicated few who were already committed to playing the game anyway, so the next set of developer posts generally contain ‘server merges’ and ‘free to play’, sometimes followed shortly after by ‘closing’ and ‘goodbye’.
The lesson that still has not been learnt is this: after beta is too late.
MMO developers need to break this cycle of beta testing being the glorified equivalent of demo disks on PC Magazines; they need to drop the NDA secrecy; they need to remove the pedestals from the holier-than-thou would-be-rockstar types in the company who are more interested in bathing in the frothing adulation of the game’s ‘number one fans’ than listening to what impartial observers are telling them; they need to stop pandering to the websites that just want to release exclusive details of the game first in order to generate advertising revenue, and instead perhaps start to foster relationships with the MMO community in general; they need to employ celebrity-blind community representatives: people who can touch and feel their way around what a community is saying and thus filter out the distractions provided by fanboys and trolls which often appeal to the ‘celebrity’ that a community management position can foster, so that the community representative can instead present the precise shape and substance of issues and concerns to the development team – but importantly the developers need to listen to that feedback and, where remotely possible, act upon it.
Or not. You know, it’s your game after all. You’re the big development studio. You know best. Just don’t be surprised when nobody plays it and you’re merging servers after three months, however. And don’t blame it on World of Warcraft, lordy don’t do that, when the game is there for all to see, all the things that work and don’t work, in a game that is fast approaching its sixth year of solid subscriptions; I’m afraid you’ve had more than enough time to get walking on your own again, that old crutch simply can’t be used in any valid way anymore if the genre wants to progress.
APB isn’t failing because they tried something new, it’s because in trying something new they started anew, and thus made all the same mistakes that had already been made in games like Tabula Rasa and WAR. They’re not alone of course, Global Agenda being another recent example. There’s a wealth of information out there, written by people who do so for fun and enjoyment and because they are passionate about these things, and not for a salary or other such remuneration. They write with a wealth of experience in what works and doesn’t work for them as a player, and seemingly too many game studio types ignore such feedback as being the ravings of the clueless, to be ignored by the all-knowing Industry clique.
Yet these people do know something, they know where they’re going to spend their money next.
Instead of trying to convince them to spend their money with you through flashy E3 stands and rhetorically-gifted front men, instead of employing bloggers who are clearly chasing the goal of ‘being in The Industry and lording it over others’ rather than ‘making great MMOs’, instead of talking at players and pumping hype at everyone through a fire hose, instead of trying to tell players why they’re categorically wrong about the things they don’t like, and instead of believing in the illusion of your own superior celebrity, take a step back, stop, and listen. Actually listen, take the Feedback Radio and adjust the frequency regularly, stop listening to the Yes Men show on 104.5 Fanboy FM for a while, maybe try Constructive Criticism hour on 98.4 Radio Impartiality instead. Granted there’s a lot of white noise out there, but the strong signals are there too and easy enough to find, if you would only choose to listen once in a while.
I think it’s not quite so simple. Essentially there are three types of design issues.
1) Oversights. Things that the designers simply didn’t realise or accidentally broke. For example Ysharros complained yesterday that the default font in Eve is really hard to read, a complaint also voiced by Zero Punctuation who expressed it pithily as “bored, bored, bored, eyestrain.” That is the sort of thing that should just be fixed.
2) Balance issues. Early in Warhammer’s life people complained that scenarios weren’t worth doing in terms of exp. They bumped the exp up considerably which moved everyone into them and killed open world rvr and pqs. That’s an example of listening to players being a terrible move. It doesn’t help that all MMOs seem to feel the solution to something being overpowered or underpowered is to lurch to the opposite extreme. (Eg paladin dps in WoW which went from being a joke to being amazing in one patch).
3) Yin yang issues. There are a lot of elements in game design where “fixing” them throws out the baby with the bathwater. Permadeath, instancing, slow travel, waiting for shuttles. All of these issues can seem obviously bad and in need of an obvious fix. However there is good as well as the obvious bad and when these issues are “fixed” the game may change for the worst.
On top of the question of design complexity is the simple matter of holding ones nerve. It seems very nerve-wracking to release a major game, all sorts of frothing maniacs start screaming at you. All too often design teams simply lose the courage of their convictions in the faith of all this outrage.
*face
Gah, typos! My capcha is funting and so am I!
I would suggest that
1) is the developer not listening.
2) is the developer listening but then not doing ‘the job of the developer’ in coming up with a sensible workable alternative. It was (perhaps understandably) a panicked response to a sinking ship, which widened the breach rather than sealing it.
3) is not a case of a problem with the game that feedback should change. If you’ve decided that permadeath is part of your game, for example, then that’s the market you’re aiming for and as long as you understand that it will be a much smaller market than a game which has less harsh penalties, then fair play to you. If, on the other hand, your players are telling you that the game is frustratingly hard, and that there’s no fun to be had from playing a permadeath MMO where you can’t get above level 2, then it’s probably time to tune in and listen.
There’s a hellishly complex balancing act of listening and responding to feedback and at the same time not letting your players dictate your game to you, I agree. I think this is where advancements need to be made. The case has been made many times that a decent community team is a must. Developers either need a community team in which they can trust, or they need to know community management as well as they know game design.
Actually, I think ‘outrageous faith’ is probably a fantastic way to describe MMO communities.
I think that there are several conflicts that studios run into with the specific situation of open beta feedback.
1) The internal view of product cycle. In most programmer views, you have the alpha which is to find out what explodes horrifically. The closed beta, which is to find out what needs to be changed in how the game *plays*. And then the open beta, which is to load test the servers and look for latency issues, etc. Viewing the open beta as purely a load testing and latency focus inhibits them from changing anything significant that doesn’t turn up in earlier alpha and closed beta.
2) Deadlines. By the time they’re in wider betas, a lot of design studios have their release dates mapped out, usually to marketing’s specifications. So big changes, like the driving system and combat balancing, are flagged as “things that would push the deadline back”. And few houses seem to have the purse strings and the guts both to take a “we’ll release it when it’s ready” approach.
3) Budgets. I suspect that many design studios have pre-launch budgets and post-launch budgets, and convincing the powers that be to allocate more funding into the pre-launch budget than has already been allocated may be difficult. Accountants like their money in tidy categories.
I think a lot of studios, for a combination of the above reasons, have a “too far along to change that now” mentality. And sometimes it kills them.
One thing I think some devs could do, when faced with an issue they know they need to change but can’t really do before launch date, is to let folks know that they’re “taking another look at x” but need to get past the release rush so they can turn serious focus on it. Something of the sort. This gives beta people the feeling they were listened to, and may make them willing to go ahead and start playing now with a hope that their changes are coming. I think more than not will take it on good faith that you acknowledged the issue and go ahead and play while they wait if they liked everything else. My allergy to subscriptions aside, I could play APB with the bad driving system knowing that they’re taking steps to fix it. But I think most dev houses are afraid to admit to problems before a release for fear of killing release day sales. Which is why the NDAs get more and more restrictive to prevent anyone else from admitting there are problems before release.
Reminds me of this article on Bio Break:
https://biobreak.wordpress.com/2010/02/03/does-gaming-journalism-avoid-the-tough-questions/
Hear, hear.
It’s a common feature for games to follow this slightly stupid course. There is little in testing if all you want to collect is bug reports rather than feedback. A QA department is there for that job, why ask gamers to test at all.
I have two wonderful examples of this occurring, the first you mentioned already was Tabula Rasa. I was very excited about this game and was rather over the moon when I got the email for the Closed Beta.
You see this was my first proper MMO Beta and I took the role very seriously to the point that I must have reported about 10 issue after my first play session. One of the biggest issues was that the GUI didn’t scale with changing resolution.
My second major experience was during the Planetside testing for BFRs. We hounded them in bug reports, stating that they were too powerful, yet they went live and the player population dropped dramatically overnight.
A few weeks later they patched to remove the powerful variant, but he damage was already done.
I think the problem is the gaming public get asked too late in the process for it to make a difference to the games release build.
Melmoth, you ignorant slut. ;)
As has been pointed out, a lot of times the design has been laid out. By the time complaints come in an open beta, it might be too late. As a developer, I see posts like, “We making this major change a few weeks before launch” as really saying, “We’re fucking with things that can’t possibly be done properly by launch, expect a lot of broken parts in the game.” Usually if you change one thing it’s going to have some effect on some other part of the game; that’s the nature of the beast in MMOs.
Also keep in mind that a lot of games have been doing testing for a lot longer than the open Beta period. Of course, closed betas aren’t always well-advertised. But, I assume that there have been external testers in a game like APB for a lot longer than the complaints have been posted. So, were these issues previously brought up and ignored, or is this some new complaint that the smaller group didn’t have? If this is a new complaint, the developer might be skeptical if it’s really an issue.
Not every complaint is legitimate, either. It’s quite common for one person to decide they really don’t like something and start complaining up a storm. Others join the bandwagon and it becomes a bit of an echo chamber where a small minority posts a lot about a single issue and it looks like “everyone agrees” that there is a problem. It can also be the case that the people who sign up for testing aren’t your intended audience; this is failure in marketing to reach your intended audience, and not necessarily the developer’s fault.
Finally, you have the issue that sometimes players just don’t know what they want. They might complain about something that seems “wrong”, but that once they get used to it it could be a real unique point to the game. Sadly, the most enthusiastic MMO players, the type that are eager to hop into betas, also tend to be the most conservative and reactionary when it comes to features. For a non-MMO example, look at the Wii; the hard-core derided the name, the “waggle”, the lack of computing power, etc. Yet, it was the console that has dominated the current generation so far. If Nintendo had listened to the people giving feedback and changed the Wii, it wouldn’t be the system we have today that introduced new concepts to the mainstream like motion control, etc.
All that said, yeah, sometimes players do bring up legitimate complaints, but there are very valid reasons for not addressing every single one from a development point of view.
I’ve been playing APB, and I rather liked this forum post. The bullet points were spot-on the exact things I wanted to hear the devs were looking at.
I’m not certain just how much can be learned from a closed beta. The very thing you want the developers to listen to – the wealth of information generated by players – doesn’t really come into existence till after the game launches. And the number of players after launch must dwarf the closed, and even open, beta; the same holds for the amount of information and feedback gleaned from players. Given the current paradigm of how MMOs launch and evolve, APB seems to be responsive.
But of course, first appearances can be deceiving.
@Jonathan B: I just wonder if game development studios need to look to other areas of software development to get some ideas as to how to solve the problems you list. Without going into tedious specifics, in the part of the software industry in which I work, we have independent design reviews and focus groups at set stages throughout the development lifecycle. I wonder if this happens regularly in game development, and if so, then whether (due to trying to keep things under wraps) there is enough independence in the review process. Hard to tell, and variable from company to company I’m sure, but there still seems to be something fundamentally wrong with the current MMO development philosophy in most companies.
@Fuzzy: It certainly can’t help that many journalists have to tip toe around issues due to the fact that if they annoy a developer/publisher then they’re likely to lose exclusive preview access to content and the like. I also wonder if the same sort of attitude persists within companies, such that people tasked with reviewing the design are restricted by just how much they can say, due to it being a career limiting move, etc.
@welshtroll: I think open beta has become synonymous with stress testing now, there are so few players it seems who go into beta with the intention of raising bug reports. Which is fine, these game companies have their QA departments as you point out. Sourcing public opinion definitely needs to be done sooner in the development lifecycle, I imagine the main reason this isn’t done is for fear of information leaking to the outside world. I have to wonder if this Super Secret Society nonsense with respect to MMOs needs to be knocked on the head. What are game companies trying to hide? The fact that their mini map is square and not circular? That they use pink exclamation marks over NPC heads rather yellow? If you honest-to-goodness have what you think is a ground-breaking idea, then keep it secret, but that shouldn’t preclude opening your game up to peer review early in the development lifecycle. WAR was touting their Public Quests forever and a day before the game was released, it was a good idea which needed to have been peer reviewed earlier in order to highlight some of the population and balancing issues, and which no other game company has come along, all this time after relase, and done in such a way as to steal subscribers from WAR.
@Brian ‘Psychochild’ Green: How dare you sir, I am only partially ignorant!
I demand satisfaction! (that would be the slut part, I suppose).
Fine points, as always. I still think that there is something fundamentally broken with the current idea of how an MMO should be developed, my main feeling at the moment is that this has something to do with openness. There’s this ‘super secret’ vibe attached to projects in the MMO sphere that rivals some of the projects on which I have worked, which were officially designated TOP SECRET. Yet, I feel, none of the recent MMOs have done anything so extraordinary that they couldn’t have been open with the public about what they were doing. They have their IP to protect, but that’s what copyright and trademark laws are for. Any fundamentally unique technologies can be held back, but I don’t think that would prevent companies opening up sections of their game to critical peer review early enough in the development lifecycle to allow for major changes should they be required.
As for the nature of that feedback, well that’s something that needs to be managed, and for which you need people who can take that feedback and draw sensible conclusions from it – remove the outliers and show the issues pointed to by the consensus of opinion – but this is something that definitely can be done, it just depends on how Dilbert-esque your marketing department is, I suppose.
Although I deeply respect that developing an MMO is a mammoth undertaking and that there are many issues to contend with, I feel that something is still fundamentally broken in the way that MMOs are developed currently, that this is a primary cause for the many failures in the MMO market, and that lessons are just not being learned.
@Adventurer Historian: The bullet points are spot on, you’re right. I also saw those same issues repeated over and over by a number of people playing in the beta, and many of them were put off the game entirely because of them. They won’t be back now, and that’s the issue: Realtime are fixing issues for the population that is already playing the game, so although it’s nice, these people were already willing to play. Anyone else will have been put off by their initial experience or the initial reviews, and although some may return after a suitably VanHemlockian Three Month Rule has been applied, once you’ve released an MMO you’ve had your chance to build up a considerable player base, the damage has been done. As we all know, after release is the most expensive time to fix major problems in software, the sooner an issue is found the cheaper and easier it is to fix because there’s less regression through the development lifecycle to be made.
In software this doesn’t apply simply to bugs in the code, but also and possibly more importantly, bugs in the design.
Melmoth wrote:
…independent design reviews and focus groups at set stages throughout the development lifecycle. I wonder if this happens regularly in game development….
It can. The big problem with a lot of traditional software methods is that games have an uncontrollable factor: fun.
Let’s take the example of vehicles. You can write a specification for how vehicles are supposed to be implemented. You can have an independent reviewer look at the spec and judge it. You can have a focus group come in and test an early prototype. But, it’s still entirely possible that the final implementation just might not be fun within the context of the game. Maybe the global physics values of the game aren’t quite what was anticipated, or maybe level design or quest design puts stresses on the vehicle system the original spec didn’t anticipate. So, even if the system is great and 100% according to spec, it can still end up not being fun through nobody’s specific fault.
As I’ve quipped several times before, there’s no unit test for fun.
There’s this ‘super secret’ vibe attached to projects in the MMO sphere that rivals some of the projects on which I have worked, which were officially designated TOP SECRET.
This is a problem in the game industry as a whole. It’s an interesting duality where officially people keep mum, unless you’re at a conference over beers, etc. It’s something I’m not fond of, myself, because I’ve run into so many instances of “I can’t talk about what I’m working on” when I would be otherwise willing to offer help. My favorite example of this was a middleware provider who I asked for some whitepapers so that maybe I could pass the info along to others I talk to. The rep who contacted me wanted me to sign and NDA….
However, secrecy can be a good thing. Let me give a personal example. Obviously I have an interest in cyberpunk and have wanted to do a game in the setting for a while. If I had started pimping the game back when I first wanted to do such a game, people would have become bored when, years later, it has still not happened yet. People will get tired of following a game after a while.
They have their IP to protect, but that’s what copyright and trademark laws are for.
Speaking from a U.S. perspective, and keeping in mind I’m not a lawyer, the issue is a bit stickier. There is some issue about how copyright might not be enough to protect a computer game. There’s a classic case of Capcom vs. Data East, where Capcom thought the game Fighter’s History was a bit too close in look and feel to the classic Street Fighter II. (Truth be told, the game did feel like a clone.) But, Capcom lost this lawsuit.
There’s another category of IP protection called “trade secret”. This is what most game companies use to protect their IP. A the name would imply, you get this protection by taking reasonable precautions on keeping the information secret. If a competitor finds out the information without it being officially disclosed, it can be illegal for them to act upon that information.
Usually the information a company is trying to protect isn’t something like the shape of the minimap or the color of the quest signifiers, but rather that they’re even working on a certain type of game, or if they’ll even have a minimap or quest indicator. In an industry known for making ripoffs of popular products (most recently seen in all the farm-themed social networking games), sometimes keeping your mouth shut is a smart thing to do. Which is really worse: that you might not get sufficient feedback on a game, or that Zynga looks at your game and decides “we’re going to make that, only faster and with better marketing on Facebook.” A game might still be a success without feedback, but if a giant competitor decides to squash you that’s usually game over.
Some more bits to consider.
Wow. why in blazes can’t more debates online be this calm and decently reasoned?
@Melmoth and @Brian Green….IMHO, you’re *both* right. On the one hand, the industry seems to have a problem with not listening enough, but on the other hand listening *too* much to your player base can be just as bad–there has to be a happy medium between the two. There also needs to be communication in both directions–the devs need to at least acknowledge that they’ve heard the issues and if possible make some kind of response, whether it being that they can’t deal with it at the moment, are working on it, or won’t deal with it because the issue’s a Feature rather than a Bug. Otherwise you end up with the supposed ‘testers’ feeling like they’re yelling into a black hole and wondering why they’re bothering to ‘test’ the game at all.
“Wow. why in blazes can’t more debates online be this calm and decently reasoned?”
I think it boils down to the fact that most people on the Internet don’t have a sentient carnivorous machine-plant Captcha system to which they can feed unreasonable commenters.