In-Game Surveys: A better way to get feedback

100 Night Elf Druid
16630
The forums are a good tool for developers but it is not necessarily the best or most efficient way to give and receive feedback.

For one, forum opinion may or may not necessarily reflect the opinion of the playerbase as a whole. I have observed many gameplay changes that have been widely praised/condemned on the forums but experience conflicting or completely opposite opinions in game, within my guild, and within my server community. This does not happen all of the time, but it does happen often enough that I have learned to distinguish from "forum opinion" and "opinion from players I meet in game and talk to".

So what would be a better way to give feedback? In my opinion, in-game surveys that are account-based.

Why surveys?

1. They are accessible to everyone with an active account. Due to a myriad of reasons, not all players can or choose to post on the official forums to voice their opinions (ahem, players with "forum bans" or players who simply do not utilize the forums).

2. Surveys allows players to "vote" once per active account on features they would like to see implemented. No more players jumping on alts to voice their opinion for or against a given feature in forum threads!

3. Surveys tied to accounts allows developers a way to analyze a lot of player opinion data very quickly (who does and doesn't like x feature, their pve/pvp participation, subscription history, etc). This way, Blizzard can see what type of player likes which particular features, and can adjust accordingly.

To encourage survey participation they can award some sort of in game perk, like a unique pet, etc.

For players who have left the game, surveys could be given through battle.net or via email.

Feel free to give your feedback :)
Edited by Celista on 5/9/2013 12:46 PM PDT
Reply Quote
90 Pandaren Warrior
14185
Hell no.

In-game surveys would be an unwanted nuisance.

Lets also not pretend that advocating this is anything but a transparent attempt to further the legitimacy of your own personal grievances.

The forums are good for allowing players to get feedback from other players for their thoughts and opinions, a survey doesn't do that. You can even get feedback from developers on twitter.

All extensive surveying would do is encourage players to keep their opinions uninformed and unchallenged by peer-review.
Reply Quote
100 Night Elf Mage
7135
I think some of that they can tell from what a player does, but that actually sounds like a good idea. Only thing, they can NOT push it in a player's face (will avoid bad timing). An in-game email sent out to let players know about how to access the survey would be sufficient (for most at least). Once done the survey option would disappear for all alts of that player.

Good idea for using those options for inactive accounts. Battle.net already has the functions (tech support feedback) so it wouldn't be hard there. Sending an email asking for them to login to their b.net account to fill out the survey would make it easy.
Reply Quote
100 Night Elf Druid
16630
Hell no.

In-game surveys would be an unwanted nuisance.

Lets also not pretend that advocating this is anything but a transparent attempt to further the legitimacy of your own personal grievances.

The forums are good for allowing players to get feedback from other players for their thoughts and opinions, a survey doesn't do that. You can even get feedback from developers on twitter.

All extensive surveying would do is encourage players to keep their opinions uninformed and unchallenged by peer-review.

1. surveys could be unobtrusive, under the "?" tab. They would not necessarily affect gameplay at all.

2. Players can still discuss opinions and ideas on the forums.

3. I have no personal grievances with the game at the moment, and my grievances in the past have been minor.

4. Surveys are easy to develop, and would be inexpensive to implement.
Edited by Celista on 5/9/2013 1:07 PM PDT
Reply Quote
100 Night Elf Druid
16630
Hell yes.

Have it be a one time quest from a quest giver in Org and SWC.

Have it give 50g (scaled by level) and 10,000 XP and some Honor and VPs to encourage people to participate.

Cheers.

These are good ideas. Giving gold, xp, honor and/or valor would be good (and cheap) incentives.
Reply Quote
90 Pandaren Monk
10790
People will either click through them without thinking about the questions, making the data invalid, or not participate, causing the sample size to not reflect the whole and making the data invalid.

No.
Reply Quote
90 Blood Elf Warlock
11400
Because getting data from multi boxers and bots is smart
Reply Quote
100 Night Elf Druid
16630
People will either click through them without thinking about the questions, making the data invalid, or not participate, causing the sample size to not reflect the whole and making the data invalid.

No.

There are ways to design surveys to discourage this behavior. The sample size will still likely be an improvement over forum sample size, which is a very small segment of the wow playerbase.

Because getting data from multi boxers and bots is smart

Bots would likely not bother responding; multiboxers are entitled to their one vote per one account --why wouldn't they be? They pay approximately $15 per account; for serious multiboxers, that is a $150/month investment (10-11 accounts).

I think people are underestimating how effective surveys can be and how efficient they can be at analyzing data. As a business school graduate who spent a year designing surveys for a living, I can attest that almost all Fortune 500 companies utilize them, and they are highly useful when designed well to minimize bias.
Edited by Celista on 5/9/2013 2:17 PM PDT
Reply Quote
Community Manager
I've taken some cold medicine, and I anticipate some meandering thoughts ahead, so please forgive me. And go easy.


1. They are accessible to everyone with an active account. Due to a myriad of reasons, not all players can or choose to post on the official forums to voice their opinions (ahem, players with "forum bans" or players who simply do not utilize the forums).


Totally agree with you there, the forums, fansites, front page, and even gaming-websites as a whole is just one slice of the playerbase. Most players don't learn about new patches until they log in when it's released. Which probably seems crazy to everyone here, but that's because you're the type of person that would come to a gaming forum, read the posts, and even less likely actually log in and post(!). Most or all of your friends are probably like you as well, because (unsurprisingly) we tend to make friends with people that are like us. It's that projection bias that makes it difficult to understand that there are people out there that are not like us. We love you for being who you are, passionate people coming and giving us feedback is a big reason for why we exist, but we all have to understand (and account for the fact that) most people just don't do that.

Still, I don't think surveys would be as useful as you may think.

05/09/2013 12:46 PMPosted by Celista
2. Surveys allows players to "vote" once per active account on features they would like to see implemented. No more players jumping on alts to voice their opinion for or against a given feature in forum threads!


Surveys can be great on getting individual's perspectives, but it can also be dangerous in creating expectations. As much as some people say they won't get their hopes up, asking people which of three new features they want to see will absolutely incline them to expect the feature they chose to be implemented. Some more than others. And - not to get into it too much, you can find all sorts of papers and articles and such on it - what people say they want and what they actually want are two different things. Sometimes it's interpreted as game developers telling players what they want, and to some degree that's true. It's important as designers to be able to filter things down to their root causes. Sure someone says they want Dance Studio, but in fact what they're really after is ... I don't know, more character customization options, and in fact the Dance Studio is probably a pretty poor way of achieving that, and the development time investment to do it would compromise our ability to ... have unique raid bosses, item tiers, have to cut an entire zone, or etc. etc. Most people don't care to think about dev resource management, and that's good, because it's not very fun.

05/09/2013 12:46 PMPosted by Celista
3. Surveys tied to accounts allows developers a way to analyze a lot of player opinion data very quickly (who does and doesn't like x feature, their pve/pvp participation, subscription history, etc). This way, Blizzard can see what type of player likes which particular features, and can adjust accordingly.


We do have a lot of data already. We know what PvE/PvP participation is, we know how often people are using their farms, we know often often people take talents or glyphs, we know how often people are using specific abilities, we know subscription history, we know how often groups are clearing content, we know queue times, etc. We can anticipate what the majority of players are looking to do, and develop that content. Not too long ago Heroic dungeons were a big thing and a lot of people were upset and wanted more in this expansion, which isn't possible to pull off before the end of this expansion, but we did hurry to put together some challenging small-group content with decent rewards to see if we could scratch that itch with Heroic Scenarios. It will be interesting to see... well first what people think of them, and second how close it gets for people to filling that role, but also just how many people actually try them, considering how loud of an issue that was.

Ultimately though there is no right single way to gather feedback. We use a variety of methods and venues to see and experience how people are playing the game, what they think, and what they want. We're always looking for more, and certainly surveys could be a part of that, but we get so much feedback and data already I'm not sure the pros for surveys outweigh the cons.
Edited by Bashiok on 5/9/2013 2:12 PM PDT
Reply Quote
90 Human Warlock
12950
05/09/2013 12:46 PMPosted by Celista
1. They are accessible to everyone with an active account. Due to a myriad of reasons, not all players can or choose to post on the official forums to voice their opinions (ahem, players with "forum bans" or players who simply do not utilize the forums).


Lotta people don't know these exist, actually.
Reply Quote
1 Tauren Warrior
0
They need to do something, there's an article on yahoo about subscription loss.
Reply Quote
100 Blood Elf Death Knight
11015
I've taken some cold medicine, and I anticipate some meandering thoughts ahead, so please forgive me. And go easy.

I never realized how much I wanted to see you high on cold medicine until now...

Surveys can be great on getting individual's perspectives, but it can also be dangerous in creating expectations. As much as some people say they won't get their hopes up, asking people which of three new features they want to see will absolutely incline them to expect the feature they chose to be implemented. Some more than others. And - not to get into it too much, you can find all sorts of papers and articles and such on it - what people say they want and what they actually want are two different things. Sometimes it's interpreted as game developers telling players what they want, and to some degree that's true. It's important as designers to be able to filter things down to their root causes. Sure someone says they want Dance Studio, but in fact what they're really after is ... I don't know, more character customization options, and in fact the Dance Studio is probably a pretty poor way of achieving that, and the development time investment to do it would compromise our ability to ... have unique raid bosses, item tiers, have to cut an entire zone, or etc. etc. Most people don't care to think about dev resource management, and that's good, because it's not very fun.

To reemphasize this, surveys are dangerous tools for anyone to use because of the potential ambiguity of language. Even if players know what the intent of the survey is, and what they want (which is already a stretch), if there's any mistake on Blizz's end, they could end up with bad data (which is arguably worse than no data). For an example of how this works, people are far more supportive of "assistance for the poor/disadvantaged" than they are of "welfare," despite it being basically the same thing.
Reply Quote
90 Dwarf Paladin
5595
We do have a lot of data already. We know what PvE/PvP participation is, we know how often people are using their farms, we know often often people take talents or glyphs, we know how often people are using specific abilities, we know subscription history, we know how often groups are clearing content, we know queue times, etc. We can anticipate what the majority of players are looking to do, and develop that content. Not too long ago Heroic dungeons were a big thing and a lot of people were upset and wanted more in this expansion, which isn't possible to pull off before the end of this expansion, but we did hurry to put together some challenging small-group content with decent rewards to see if we could scratch that itch with Heroic Scenarios. It will be interesting to see... well first what people think of them, and second how close it gets for people to filling that role, but also just how many people actually try them, considering how loud of an issue that was.


A bit off topic, but it might illuminate just how much data Blizzard actually does have and collects.

Some other developers have released amusing diagrams showing just how often people use "X feature/weapon/etc." in the game. Releasing something like this might show individuals just what is meant by knowing what skills are used most, etc. ("Did you know 5 skills used the most are...." "Did you know the total time played by all characters is...." etc.)

Just as a fun way to keep the player base informed of "fun facts", and maybe can be used to illustrate designed choices or balance changes.
Reply Quote
90 Goblin Shaman
12965
Add surveys and polls to the forums and to wow in game during loading screens (if possible), but never add in game poll windows that pop up anytime during actual game play.

The forums have become one of the last places ppl go to find info and blizzard talk, and its so irritating to see the main outlet for communication is flopping out of twitter.
Reply Quote
90 Human Warrior
10265
Most players don't learn about new patches until they log in when it's released

This is me :) and I will add that it's a joy.. reading the patch notes in the 10 mins or so it takes to download/install, then jumping in head first.
Reply Quote
100 Night Elf Druid
16630

Ultimately though there is no right single way to gather feedback. We use a variety of methods and venues to see and experience how people are playing the game, what they think, and what they want. We're always looking for more, and certainly surveys could be a part of that, but we get so much feedback and data already I'm not sure the pros for surveys outweigh the cons.

I think that the allure of surveys involves efficiency of data when the data you are looking for is more of a why/why not question--you know players are/are not utilizing x feature, but why is that? Can you find out the answer to that question more accurately using other means or is it most efficient to ask the playerbase directly in as accessible a format as possible? You raise a good point in that there multiple means in which wow data is collected that goes beyond forum usage (how often certain features are utilized, which guilds are clearing current content, etc) and you obviously would be more informed than I am regarding these means of data collection, so it is possible that the current data speaks for itself and a survey would be redundant.

In any case, feel better soon :)
Edited by Celista on 5/9/2013 2:34 PM PDT
Reply Quote
Community Manager

We do have a lot of data already. We know what PvE/PvP participation is, we know how often people are using their farms, we know often often people take talents or glyphs, we know how often people are using specific abilities, we know subscription history, we know how often groups are clearing content, we know queue times, etc. We can anticipate what the majority of players are looking to do, and develop that content. Not too long ago Heroic dungeons were a big thing and a lot of people were upset and wanted more in this expansion, which isn't possible to pull off before the end of this expansion, but we did hurry to put together some challenging small-group content with decent rewards to see if we could scratch that itch with Heroic Scenarios. It will be interesting to see... well first what people think of them, and second how close it gets for people to filling that role, but also just how many people actually try them, considering how loud of an issue that was.


I'm sorry, but know what people do (game data) is not the same thing as knowing what the want to do, or what they think of the stuff they're doing. They could be doing something they hate, knowing that it's the only way to progress in the game or something.

BTW: Heroic Scenarios would be fine if you'd let us queue for them, not require us to have pre-made groups. As long as you can't queue for them, they are not even in the same ballpark as Heroic Dungeons.

Cheers.


I was hoping the last couple sentences would kind of cover that. We don't just look at game data. And, in fact, while it's super helpful, a static list of numbers and percentages isn't nearly as useful as hearing how people feel about something. How something feels is far more important that statistical balance.

We use a variety of methods and venues to see and experience how people are playing the game, what they think, and what they want. We're always looking for more, and certainly surveys could be a part of that, but we get so much feedback and data already I'm not sure the pros for surveys outweigh the cons.
Reply Quote
90 Pandaren Warrior
14185
05/09/2013 01:01 PMPosted by Celista
I have no personal grievances with the game at the moment, and my grievances in the past have been minor.

Now, you say that... but you haven't made a case for why surveys would be good or even needed. You say what surveys are good for, that's about it.

Why I think that you have an agenda in promoting surveys that links back to your own pet grievances has mainly to do with what you say in your second paragraph; that the opinions you see represented on the forums don't feel representative to you of what the people you play with think, and also you suspect, the player-base at large.

Its not a big leap of intuition to say that you believe more accurate data concerning player's opinions could be gained through the survey system you propose, and that the reason you advocate this is because you believe your opinions would be vindicated as a result. That's what I think your agenda is; Ive seen it often on these forums from players with poorly-thought-out or unpopular opinions wanting to poll in order to use the data they believe exists as a platform to pressure developers for change.

It only amounts to rabble-rousing in the end, and never gets far, not least because polling is against forum rules... and that's really all you're doing, just that you're trying to do it in a way that is more insidious and egotistical by asking for it to be incorporated into the game itself.

Anyway... saying more would be pointless because this thread has pretty much come to an early conclusion because /Blue.
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)
Submit Cancel

Reported!

[Close]