Diablo® III

Blizzard, Learn from Yahoo.

I participate in fantasy football, as do most of my other friends. On Sunday afternoon, Yahoo servers went down and I couldn't access anything that had to do with fantasy sports. Not a big deal at all, as I was watching the games anyway. Here is an email I received from Yahoo today. Its quite lengthy, but, if you do read, you will see due to Yahoo's unintended server downtime, they actually took the time to apologize (longer than a paragraph), explain in detail what had happened, and even take the unnecessary extra step in providing some services for free. With the amount of time Blizzard takes servers down, Im sure this strategy would bring back many players and keep the players they currently have.

The email is below:

Dear Yahoo! Fantasy Users,

As I mentioned in my earlier email, Yahoo! works hard to provide great experiences for our users every day. For you, we understand that Fantasy is a core part of your daily routine during this time of year. On Sunday we experienced technical issues which caused the Fantasy experience to be inaccessible. We're sorry. Our entire team worked tirelessly to re-establish the service - however, we had a significant outage. We know this created inconvenience and frustration for millions of our fans, and we feel we owe you an explanation.

So what happened? At Yahoo!, we have giant machines called "filers" that process a lot of the real-time data and stats for us and for you. We do millions of calculations every hour for our games, and normally our machines can handle this with no problem. Recently, we discovered a hardware issue in one of the filers that caused the other one to overload. We replaced some hardware, re-configured the setup, and did some testing. However this Sunday - at approximately 12:15 p.m. Eastern - the new configuration failed. This created an overload on storage capacity and took the Fantasy part of our site down.

We had dozens of engineers from various teams working together to try to determine the cause and fix it. One option was to fall back on another data center, but that would have meant shutting the game down and losing scoring data. We wanted to avoid that at all costs. Ultimately, we were able to move our mobile apps to a back-up data center, free up storage to get the PC version of the game working, and get the mobile apps up in a "read-only" state - meaning you could see scores and data, but you still couldn't set lineups and interact.

We spent Sunday night and most of Monday looking at dozens of potential causes. Monday afternoon, we stress-tested our system. Everything seemed to be in working order, so we turned on all our mobile app functions in time for Monday Night Football. Everything performed as expected and continues to do so. We'll have all hands on deck this coming Sunday to closely monitor performance and ensure we can respond quickly in case of any abnormal activity.

We know many of you have questions about your leagues and games, so we've also set up a FAQ here that will explain how we are going to handle various scenarios.

Finally, we know we failed you this weekend, and there's no way to fix that. As a token of our appreciation for your patience and continued support, we wanted to give something back. First, we're offering a complimentary football weekly Scouting Report for the remainder of the year. Hopefully this will help you down the stretch. The report includes weekly expert analysis, rankings and news to help you make smart decisions. Second, we are offering all our users a 20% discount for the rest the year at the Yahoo! Sports Store.

For any updates, we will keep you informed on @YahooSports and facebook.com/yahoosports.

We won't stop working to provide you with new and improved features that help you enjoy your Sundays.

Thanks again for playing and for your patience.

Best,

Ken Fuchs
Head of Yahoo! Sports
Reply Quote
Posts: 429
I play football or I watch football.

What is this fantasy football?


For real?
Reply Quote
Fantasy football, meh. Regardless, what can they provide for free, that you already pay for, in D3?
Reply Quote
Fantasy Football FTW!

http://funhoo.com/wp-content/uploads/2011/11/Fantasy-Football-Its-nothing-more-than-Dungeons-and-Dragons-for-jocks.jpg
Reply Quote
Since the point was not picked up, I will lay it out as simply as possible.

Yahoo had an unintended server outage. They fixed it promptly, and issued an apology, with details to what they did to fix it, and an explanation of how they are going to attempt to prevent it again. Additionally, even though they didn't need to, they provided services both for free AND discounted other services.

The point is, this company had an accident, that wasn't their fault. To keep their customers, they issued "rewards" or perks at no cost to the company.

Blizzard could simple offer X% bonus exp for a week or something creative. Something that shows they have put an effort into keeping customers through the many random "accidents" they have.
Reply Quote
I fail to see how I am being picky. Would you care to explain?

I am just simply stating that as often as I read about Blizzard servers randomly going down over the years, I have rarely seen anything done about it besides a short, "Servers went down, they will be back up in 2...no 4...no 8 hours. Servers are up." The fact is, Blizzard could easily do more to maintain a firm grasp on their player base, while keeping a good name.
Reply Quote
They are called "Postmortem". Lots of tech companies do this when they fail at something. People like *knowing*. Also prevents you from failing the same way more than once. Because that would just make you look incompetent.

People also like detailed a "changelog". Tinfoil hats aside, Blizz changes lots of things without ever mentioning it to anyone. Which leads to people feeling like things have been "ninja nerfed" every time something like a 12 hour maintenance happens.

It does feel as though Blizzard is entirely dishonest by not providing either of these things. Nor does it feel like they respond to bug reports adequately, nor own up to verified bugs. But I work in open source software, so perhaps my expectations of communications are liberal.
Reply Quote
As was I, and you have still not explained how I am being picky. All Im trying to say is a better effort should be put out by Blizzard. I generally play maybe 2 - 3 hours a week at most, usually Sunday nights. This is just a simple observation I have noticed over the course of a couple years interacting with Blizzard.
Reply Quote
They are called "Postmortem". Lots of tech companies do this when they fail at something. People like *knowing*. Also prevents you from failing the same way more than once. Because that would just make you look incompetent.

People also like detailed a "changelog". Tinfoil hats aside, Blizz changes lots of things without ever mentioning it to anyone. Which leads to people feeling like things have been "ninja nerfed" every time something like a 12 hour maintenance happens.

It does feel as though Blizzard is entirely dishonest by not providing either of these things. Nor does it feel like they respond to bug reports adequately, nor own up to verified bugs. But I work in open source software, so perhaps my expectations of communications are liberal.


This is a perfect example of putting in a better effort. The bonus exp was simple an idea that was similar to what Yahoo did.

Edit - And WoW was released when?
Edited by festavus#1915 on 11/14/2012 12:53 PM PST
Reply Quote
11/14/2012 12:49 PMPosted by ellisD
As was I, and you have still not explained how I am being picky. All Im trying to say is a better effort should be put out by Blizzard. I generally play maybe 2 - 3 hours a week at most, usually Sunday nights. This is just a simple observation I have noticed over the course of a couple years interacting with Blizzard.


And like I said, when warranted, like in the release of wow, they gave me over 2 free months.

Diablo 3 servers run superb and you want something for nothing..... disgusting.


Umm.. where the hell have you been? The servers have not been running superbly since launch, most would agree they don't even run mediocre.
Reply Quote
And like I said, when warranted, like in the release of wow, they gave me over 2 free months.

Diablo 3 servers run superb and you want more..... disgusting.


I disagree that it is 'disgusting'. Using yesterday as an example:

a) first time in a while there was "adequate" notice prior to the "scheduled" maintenance. Some tuesdays no noticed was given at all, less the "system shutting down in N minutes" message while you were playing. Some tuesday there was no maintenance, which voids the "it's always down on tuesday, get over it" argument.

b) In game notification used PST time, the one buried in the forum used PDT. This is a significant difference, or, at the very least, an indication that the two outlets are out of sync and/or not communicating with one another.

c) In game notification listed 11am PST and end. This was "breaking news". The blog/forum said "a few hours". When that time was updated to be 1pm PST, the forum remained "a few hours". The "breaking news" at client login had no timestamp when posted, so disingenuous/dishonest as their initial time had been "ninja bumped". Anyone who read the initial 11am and came back to try then was greeting with "am I insane, I swear it said it'd be ready at 11am". (Why not just add a new paragraph saying "update: We've extended it N hours").

c) The extension came literally minutes before the previous expiration. 10:55am PST -> updated. Surely they can know well in advance that they will need more time, or is the 2 hour/3hour extension just them picking numbers out of the air and hoping for the best? If you think something will take 6 hours, and 3 hours in you aren't half way done ... wouldn't you think you might do better estimation? 5 minutes vs 2/3 hours is a lot.

d) At 1pm PST the servers were still unavailable. The forum notification had not been updated, nor had the in-game client login breaking news. Sure, 1pm PST was an "estimated time". But the twitter account for blizzardcm provided a NEW time. So none of the outlets for "when can I actually come back and play this game" were in sync. The Tweet estimated an additional 3 hours (?) if I recall correctly, forum still said "a few hours (back to 11am PST)", in game locked on 1pm.

e) Somewhere around 5pm EST I was able to login. I started playing, and quit wondering. The forum mentioned things being back and ready. Amazing! Happy. Lets go melt stuff.

f) The system gives a 'shutdown in 15 minutes' for EMERGENCY fix. So someone messed something up during a 'routine maintenance' badly enough to need to shut everything down again? {this is where the postmortem explaination comes in super handy} The forum made a mention, the in-game client I didn't even bother checking. I was able to play again the next time I wandered back to my computer.

This is nothing but a cluster of failed communication and mild dishonesty. They don't _owe_ me any explanation, granted. But the relationship I feel between customer and provider here is very shaky, and why you see such anger. Most of that anger would be pacified with a little open/honest communication on Blizzard's part. At the very least a substantial revamp of their internal communication that is relayed to the community ...
Reply Quote
Another good example of how Blizzard could repair their relationship with their playerbase. Any other ideas/opinions?
Reply Quote
Posts: 811
I play football or I watch football.

What is this fantasy football? What is it, like D&D?


It's much more nerdy than D&D.

D&D players get all the ladies, because women see fantasy football people and D&D players instantly become 100% more attractive.
Edited by Blade#1638 on 11/15/2012 2:41 AM PST
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)

Reported!

[Close]