Why Blizz Needs to Address SimulationCraft

85 Troll Hunter
7360
10/01/2012 12:39 PMPosted by Lhivera
The devs have indeed been very helpful through this dev cycle.


Well, yes and no. I did the first warlock profiles during the Beta Class Analysis Thread (the first one). Zakalwe over at EJ was working on the discrepancy issue (among other things) and he came up with the explanation that Blizz may be using different priority action lists. Several of us asked if Blizz could confirm some of our suspicions in the Class Balance Thread so we could work on the sim. We even posted the action lists for Blizz to take a quick look at. We weren't asking them to debug the sim, just tell us if our suspicions about the priority lists being very different (ours and theirs) were correct. We never got any response. This would STILL be very helpful on their part, because it then says "oh hey, yes we're using the same priority lists, so something else must be wonky" or "nope, we're using a less complicated list for this reason" or something! /insert more frustration.

All this being said, they've done more than they have in the past, which was appreciated. AND . . . if they were serious about taking this issue on, they'd do more (and not with the goal of just getting the sim results to match, but of addressing this pernicious issue that has dogged endgame raiding and the community forever).

Ultimately, though, what we really need is more player eyes on the results, finding things that look wrong. We know that Blizzard's internal numbers put all these specs very close to each other; we can conclude that there's something wrong with the outliers. If you want to improve the sim accuracy, help find the reasons.


I agree with all of the above. And, there's what we need to do, and what Blizzard needs to do. We're doing the stuff on our side as best we can - we're passionate players with real life jobs doing this as a hobby. We need some more support from the guys who get a paycheck and focus on this in a way that we can't.

Ultimately the game environment needs Blizzard to do more to take on this issue, just like they're taking up the 10 v 25 man raiding issue, the ninja looting issue (LFR new loot system) etc, that were originally deemed "community issues" and "outside their scope".
Reply Quote
90 Draenei Shaman
4790
If anyone wanted to go to the trouble, they could create simcraft profiles that roughly approximate each individual raid fight, and post those pages. But nobody stepped up, because raidbots is far better anyway, since it uses actual live parses.

That's why the simcraft overall ranking page should be removed. It serves no purpose. If they want to maintain simcraft outputs online, do individual pages for each class, with each spec and subspec simmed including stat weights. Those would actually be useful.
Edited by Slant on 10/1/2012 1:46 PM PDT
Reply Quote
90 Troll Rogue
21085
This is a repeat of the GS debate from 4.3.

Was gearscore a useful tool in the right hands? Yes, however at the same time the "score" part could be immensely misleading in the wrong hands.

I agree with Slant on this one, not publishing those results would do a lot. Could players still generate the report? Certainly but the hope would be that players with the sophistication to do that would also have the sophistication to not take the class dps rankings as gospel. There would still be people posting the results here from time to time but it would be a lot less often.
Reply Quote
90 Pandaren Shaman
6145
The developers have addressed SimC, quite extensively.

They have repeatedly stated that they appreciate the work that goes into it, they think it's very useful for gauging the effects of changes in rotation and gear, but they think it is pretty inaccurate in terms of absolute DPS numbers. They actually provided extensive feedback and assistance during beta that helped us improve accuracy, but we know for sure that the top-simming specs are still simming very high. We're not sure why.


Because if they admitted to simcraft being correct, they would actually have to balance something?


Shut up.
Reply Quote
85 Troll Hunter
7360
If anyone wanted to go to the trouble, they could create simcraft profiles that roughly approximate each individual raid fight, and post those pages. But nobody stepped up, because raidbots is far better anyway, since it uses actual live parses.

That's why the simcraft overall ranking page should be removed. It serves no purpose. If they want to maintain simcraft outputs online, do individual pages for each class, with each spec and subspec simmed including stat weights. Those would actually be useful.


I agree very much with the above.
Edited by Boomerang on 10/1/2012 2:17 PM PDT
Reply Quote
90 Draenei Shaman
4790
Raidbots can control for player skill. Just select a higher percentile than the median (ie, the 50th percentile). The 80th percentile and above are very strong players, and the 95th percentile and above are elite (or getting rotating tricks from the rogues).

For example, here is the median for Ultraxion on 10H.

http://raidbots.com/dpsbot/Ultraxion/10H/all/14/60/median/#3vvvv

Obviously the numbers are a bit messed up now because nobody is running DS post-MoP, but make believe that they aren't. lets say you're playing a shadow priest, and you did 42.5k DPS on Ultraxion 10H last week. That's well above the median, nice job.

http://raidbots.com/dpsbot/Ultraxion/10H/all/14/60/p80/#3vvvv

And it's above the 80th percentile too! Clearly you're a skilled/geared shadow priest.

http://raidbots.com/dpsbot/Ultraxion/10H/all/14/60/p95/#3vvvv

But it's not above the 95th percentile, so your skill, gear, or support from the rest of your raid isn't world-class.

This allows you to evaluate your performance in a meaningful way.
Edited by Slant on 10/1/2012 2:18 PM PDT
Reply Quote
85 Troll Hunter
7360
Raidbots can control for player skill. Just select a higher percentile than the median (ie, the 50th percentile). The 85th percentile and above are very strong players, and the 95th percentile and above are elite.


I really don't want to go down that tangent (because it's a black hole we've all been down), so I edited my response to remove the references to its accuracy. Sorry about that, I need to get better at not derailing my own threads

:-)
Edited by Boomerang on 10/1/2012 2:18 PM PDT
Reply Quote
MVP - World of Warcraft
90 Human Mage
10015
Boomerang, drop in on the SimC IRC sometime soon.
Edited by Lhivera on 10/1/2012 2:20 PM PDT
Reply Quote
90 Draenei Shaman
4790
Sorry, I will leave it there then. :)
Reply Quote
90 Troll Druid
10805

fwiw, I'm not so concerned about absolute DPS numbers (no one cares) as relative DPS numbers. People won't run a Mage because they do a certain amount of DPS, they will run a Mage because they do more DPS than a Hunter.


Sure, but since the absolute numbers are off by different amounts (and in some cases, may be reasonably accurate), the relative numbers are off as well.

I agree, which is why this discussion will be a lot more interesting tomorrow when we get real world data to look at.

Right now it just isn't possible to add much empiricism to our impressions. I feel that Boomkin is doing fairly low DPS, but I can't compare Boomkin DPS to other classes on raid content because there is no raid content. Give me a couple thousand logs and things get much more interesting.
Reply Quote
85 Troll Hunter
7360
10/01/2012 02:18 PMPosted by Lhivera
Boomerang, drop in on the SimC IRC sometime soon.


I appreciate the invitation. I'm heavily involved in some other extra-curricular activities until a certain first Tuesday in November, but after that I'm planning a drop by!

EDIT: I apologize if some of my early responses seemed cranky. I hadn't eaten anything all day until I just had lunch . . . :-)
Edited by Boomerang on 10/1/2012 2:26 PM PDT
Reply Quote
MVP - World of Warcraft
90 Human Mage
10015
Alternatively, drop me a note on my site if you like. I can fill you in on some of the more recent development stuff.
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)
Submit Cancel

Reported!

[Close]