Lack of build/design notes => poor testing

#1 - May 4, 2012, 10:39 p.m.

The current situation regarding account-wide mounts, pet and achievements highlights one of my main issues with Blizzard's apparent philosophy regarding beta testing - releasing builds without appropriate build and/or design notes.

Without knowing what the design intention is, it is nearly impossible to effective test any specific feature. And since every programming team I've been involved keeps build notes on an on-going basis, I find Blizz's reluctance to publish those somewhat surprising. And I'm not talking about 'in the weeds' type changes (ie, we're changing the order this function is expecting to receive its arguments) , I am referring to the top level changes (ie, the Pet Journal should be fully (or partially) functional).

The sticky in this forum is helpful, as is all the data-mining done by the various sites, but I don't think they are sufficient. And my primary concern is with the lack of design intentions. The basic goal when testing is to answer the question "Is this doing what it is intended to do"? That question cannot be answered if the design's intentions are not specified.

For example, a couple of questions concerning mount sharing:
1) Are the city mounts shared across faction? (Apparently not, based on statements made months ago outside the beta forums. So, my alliance Panderan's ability to ride around on a goblin trike gets reported as a bug, but my NE's inability to ride a goblin trike is a feature)
2) Are mounts that currently require a specific profession to use shared amongst all your characters or not? (Which is the bug: the fact that my non-tailors are unable to use the various flying carpets or the fact that the flying carpets are being shown in my non-tailors' pet journals in the first place?)
3) What about class specific mounts? Should all your characters have access to the warlock or paladin specific mounts?

Blizzard may have outlined their design goals for MOP since its announcement, but I don't think we should have to search thru months of statements or articles to find that information during the beta test. Especially since there is a good possibility that there have been uncommunicated changes to the intentions over that same period.

Ghostcrawler

Game Designer

#5 - May 5, 2012, 12:01 a.m.

We tend to iterate pretty quickly during beta. It's a luxury we don't have when we're making patches to the live game. We can use beta to try some crazy ideas and then quickly back them out again if we don't like the way they are going.

Closely related, because we want to iterate on beta quickly, we don't all sit down together and agree on when is the best time to make a beta push. Usually we try to stabilize things for a little bit to make the build, but even then, you're going to see a lot of things that are half-implemented, not quite working yet, or in the process of being backed out because someone realized it was a bad idea. :)

We understand our betas can be a confusing, bewildering and sometimes frustrating experience. They aren't for everyone. Betas are emphatically not demos. Unless you're really good at looking past all the baling wire and duct tape, it can be hard to get an impression of how the final product will be. Also keep in mind that it's not just the designers who are testing things out, and sadly testing out new code is even less glamorous than testing out new features.

On the topic of account level mounts, we are still evolving our design philosophy, so it's a great thing upon which to offer feedback. We have heard from some players who are thrilled that they don't need to play their "mounty" character to the exclusion of their other characters. We have heard from others who feel like their accomplishments are watered down since they got so many mounts "for free." We're not sure which mounts will be account-level or not. It's weird to have paladin, warlock and DK mounts shared across classes, so there's a good bet those still be character only. What about the others? Let us know what you think.

That philosophy goes for a lot of features. We're all busy working on the game, so we don't have a ton of bandwidth to continually explain our design intent on all of the various features in the beta (keeping in mind that our intent often changes). We'll try to hit some major philosophical points when we can, but if you need a lot of blue explanation or direction for how to beta test in order to get anything out of it, then I can understand why it might be a frustrating experience.

My advice is just to play and let us know how things feel. If we haven't changed a feature in awhile, it could mean that we're happy with how it is, but it could also mean we're just working on other features at the moment. We'll read all of your feedback, but we probably won't have a lot of opportunities to respond. Rest assured that what you say is really important to us and to the development process.