Balance test map follow-up

Posts: 17,685
You and Azoryen seem to be caught up on the same hitch. The issue isn't whether the MMR/winloss numbers aren't adjusted, it's how.

What's your education on math?


Bachelor's in Applied Math, Bachelors in Physics, currently working on masters/PhD in physics at a state university

(not a statistician, which is really what you would need to discuss specifics, if they posted the model).

And i don't think you'd be factor it out completely, but it would show to some extent, if they actually factor that sort of thing in

edit:

Belated apologies if my posting comes off confrontational, thread is moving fast. ><
Edited by Arianity on 1/9/2014 8:38 PM PST
Reply Quote
Posts: 1
01/09/2014 06:34 PMPosted by Sopraffatto
SC2 is dying, and your posting bogus stats. please continue to ignore the problem and allowing people to leave this game and move to other avenues of gaming. Well done blizzard; well done. I wonder what the "balance team" will be saying when no one is playing SC2.


What is wrong with you? What do you want him to say?

The actual evidence for the 'problem' of protoss at the moment, the EVIDENCE, suggests that the problem is in people's attitude towards the matchup, I've played random since launch of hots and have been continually impressed by how balanced the game is. I'm not going to say sc2 doesn't have it's problems in terms of how easy/fun it is to just jump on the ladder but that's always going to be hard to achieve with such a skill dependant 1v1 focused rts.

The guys on balance do an incredible job and I don't envy the ammount of crap that they have to deal with constantly.

Thank you for this post Mr Kim and keep up the good job
Edited by Squirrel on 1/9/2014 8:39 PM PST
Reply Quote
Posts: 1,243
I would like to see the Vanilla Winrates, Like other posters, I would also like to see what the average MMR is for all 3 races. I think with these 2 sets of data (from the last balance patch to present) we would be able to truly tell if the game is balanced of not.
Reply Quote
Posts: 2,514
They're using a week's stats as balance indicator, yet when Terrans got stopmed in the last 4 premier tournaments not a word was said.

A week's sample pool is ok to infer different then what they thought a week earlier, and yet 700 GM league players is not a large enough sample pool.

They really see nothing wrong with one side having literally 3 openings and the other has 10+

Also, I fail to follow the logic - TvZ was 50/50 before the mine nerf, and they still nerfed it because the mine was badly designed.

The entire PVT MU at the moment is badly designed and they won't change the MSC?

People begging for mech buffs for years, and we keep getting "clues" and "hints" that its going to get buffed, only to be dissappointed time and time again. First the 0.2 "buff" and now all they hype from last week only to learn that they're "thinking about it". They won't buff mech till LOTV, they want your money.
Reply Quote
Posts: 525
I apologize for my typo on that image. It's hard to overstate how frustrating it is when the calendar flips to a new year and my brain completely fails to notice that I've just typed out the previous year even while proofreading. And then that typo is the only thing anyone can see. *hangs head and slinks out*


As the only consistent Blizzard presence on forums, I'm pretty sure you could say you were responsible for aids and cancer and it'd be okay :D
Reply Quote
Posts: 1,301
01/09/2014 08:35 PMPosted by Arianity
Bachelor's in Applied Math, Bachelors in Physics, currently working on masters/PhD in physics at a state university

OK, then you should be able to understand this:

Imagine they introduced a 4th race in SC2, which was super OP against the current 3.

A former gold league switches to this race and quickly raises his MMR to masters level.
At this point, he starts facing players that can compensate the OPness of his race and winrates for this player will stabilize around 50/50 for all matchups.
GET IT?

WINRATES ALWAYS STABILIZE AROUND 50/50 NO MATTER HOW OP A RACE MAY BE BECAUSE THAT'S HOW THE MATCH MAKING SYSTEM WORKS.
YOU WIN MMR POINTS UNTIL YOU REACH A STABLE WINRATE!
Edited by Azoryen on 1/9/2014 8:47 PM PST
Reply Quote
Posts: 15
THANK YOU.

Just knowing that something is happening, that you guys are in fact looking at things, makes all the difference.

Also, how sick is Proleague this season! ^_^
Reply Quote
Posts: 6,100
Alright, I dont play this game very much anymore, but hopefully I can give you my perspective based on how I see this game solely as a spectator.

I used to play a lot of Protoss back in WoL. I was pretty happy to see the MSC and Oracle introduced to fix the two big flaws in Protoss design, namely the lack of defender's advantage in PvP and the lack of detection outside Robo.

But apart from those things, that's all that changed. Protoss is still reliant on Forcefield, a binary crutch that either allows the Protoss player to barely win fights or allowing for crushing victories that the Protoss' frustrated opponent can do little to counter.

The Carrier, Protoss' Stargate counterpart to the Colossus and HT, is still complete garbage. The only time I ever see this used is when the Protoss player is insanely far ahead and feels like building the Golden Armada and rolling over everything with Voids, Tempests, and Carriers in a 200/200 ball of death and pain.

Protoss still lacks units that allow for flashy micro opportunities like Marines. This is one of the huge complaints many people including myself have about the race. A lot of the time it's too easy to forgo strategy altogether and just a-move a ball of Zealots and Archons and do nearly equivalent damage to your opponent as he did to you. This game should be about skill, not about 'Hurr lookit how many Chargelots I made'.

Players complained a lot about Fungal being pretty much a point and click win button, but Storm and Forcefield are still incredibly strong with little available response beyond 'dodge it' or 'EMP the Sentries first'. At least you can juke a Fungal now.

As for Terran, I played a few games as them in unranked and was a bit surprised by how incredibly hard it is to even macro as T compared to P or Z. For Z and P it's pretty much muscle memory to check your Warpgate and Larvae cooldowns, but a Terran needs to swap addons, drop mules, raise and lower Supply Depots, and they cant queue up buildings as easily as Protoss can.

In terms of how games go, I see a lot of PvT games essentially go like this: Terran makes surgical strikes everywhere trying to bring about a situation where the Protoss falls apart due to too much damage being dealt everywhere at once. Sometimes it works. Other times the Protoss holds it off for a few minutes, then reaches max army and then pretty much just walks across the map and wins the game.

Other times the Terran tries a drop or Banshee harass, does a reasonable amount of damage, then leaves. Then a pair of Oracles fly in his base and kill 20+ workers and fly off before his Marines can even make it up the ramp.

It just seems so stupid sometimes. Protoss should feel like the big slow bully that the smaller kids have to outsmart because they know they cant fight him straight up. But a lot of the time it seems like the bully just grabs the small kid from behind and slams his head against the wall before the small guy has a chance to react. Protoss has a lot of unforgiving mechanics to play against like Storm, Forcefield, Dark Templar, and Oracle harass, but other races lack ways to instantly kill their Protoss opponent the way Protoss can kill them.

On the one hand that's a good thing. Mistakes should be costly or fatal. But if a Protoss player carpet bombs a Terran with Storms and he doesnt move fast enough, he can lose like 60 supply instantly. The only way a Terran could do that to the Protoss is through Nukes, and who the heck stands still that long?

I dont know if I'm even making sense, or if my point is even a good one or not. But let me put it to you another way: BL/Infestor was said to be too strong at the end of WoL because it was not only easy to do, but incredibly powerful. Isn't that pretty much the Protoss race in a nutshell?

I'm not even saying Protoss needs to be nerfed, really. I just want there to be a clearer distinction between the average Protoss and the great Protoss. Compare someone like Avilo to MVP and you know instantly who the better player is without even looking at their names. The difference between someone like Minigun and MC is not so easily apparent. Protoss should be hard to play, not hard to play against.
Edited by Ganthor on 1/9/2014 8:59 PM PST
Reply Quote
Posts: 397
Hey David Kim:

Here are the results from this past GSL Code A qualifier stats posted on reddit:

TvP = 30-36 (45.45%)

TvZ = 37-42 (46.84%)

PvZ = 46-41 (52.87%)

Total number of games: 232

To use ladder as a big factor in decisions regarding balance is a mistake, if you are trying to balance it at the highest level. I've always wondered, but why have we never seen the top 200 (GM stats) posted for any of the regions? That's a much bigger indicator, compared to Masters which does see several skilled tiers. Obviously we can't do it for this season right now since GM isn't even open...

Not to mention, ladder does always try to put you towards that ever elusive 50% (somewhat).
Edited by LARGEPHALLUS on 1/9/2014 8:53 PM PST
Reply Quote
Posts: 1,544
I've always found these balance stats from Blizzard to be rather dubious, for a few reasons.

First, the idea that one can filter out such a high-variable concept as skill with a single calculation (http://i.imgur.com/JOoZz.png) is bizarre, especially since Blizzard has officially stated that the calculation does not factor in how a player won their game, which is what skill is all about. It'd make more sense if Blizzard was referring to factoring out (or at least weighting) games won/lost between players with a high MMR disparity as some people have theorized, but without some explanation from Blizzard on this we're left to scratch our heads.

Second, even if we assume "factoring out skill" means weighting or eliminating MMR discrepancies, that does very little to demonstrate overall balance in the current ladder setup, as the numbers will simply gravitate to 50% regardless; there's no real way to see if better players are having their MMR weighed down by racial imbalances with this data alone.

You would need to also look at racial distribution in the leagues, coupled with this current balance data, to see what's going on. If there was truly racial balance, you would expect to see each race populating the different leagues respective of their usage base (give or take some percentage points to accommodate the fact that the best players won't be playing each race at a perfect 33% representation) on top of relatively stable win rates.

e.g. if we were to assume the player base was:

Terran - 75,000 people (25%)
Protoss - 120,000 people (40%)
Zerg - 105,000 people (35%)
Total - 300,000 people (100%)

Then we would expect to see something close to the top 2% of each race in Masters (again, give or take a bit):

Terran - 1,500 people (25%, 0.5% of total player base)
Protoss - 2,400 people (40%, 0.8% of total player base)
Zerg - 2,100 people (35%, 0.7% of total player base)
Total - 6000 (100%, 2% of total player base)

This is, unfortunately, not the data the community is seeing. Terran in particular is under-represented in the higher leagues by more than just a couple percentage points, while bloated in the lower leagues. This leaves the impression that Terran MMR is deflated relative to the other races. Add on top that these players are then still on the losing end in the balance stats for PvT, and it starts to become small wonder that the majority of the Terran player base finds PvT to be unfair.

I say all this without even dipping toes into the concept of quality of design. The idea of a race having a one-button solution to such a wide variety of strategies and tactics, for example, is absurd (though I am glad that the recent balance test map is looking at addressing Photon Overcharge from a balance standpoint, even if it doesn't address the absurdity of its design). Suffice to say, people are frustrated, and this blog provides little relief from players' day-to-day experiences with the game.

Speaking of experiences with the game, gameplay comes first, then balance. It doesn't matter how statistically balanced a game is if people are unable to enjoy it. SC2 isn't in a gutter by any means (let's be honest, it's still the best modern RTS on the market), but there are some pretty obvious gameplay issues that deserve some attention.
Edited by Caustic on 1/9/2014 10:57 PM PST
Reply Quote
Posts: 13
01/09/2014 06:42 PMPosted by Sopraffatto
You are naïve, and a fool. The game is broken, and people are leaving it by the droves. Teams are disbanding, and leagues are dropping. This game is in distress, and Blizzard doesn't give two craps. the same can be said for every other Blizzard made game. LotV and heroes of the storm will be complete flops. mark my words.


I'm betting you are one of those avilo fanboys.
Reply Quote
Posts: 13
01/09/2014 08:45 PMPosted by Ganthor
The difference between someone like Minigun and MC is not so easily apparent. Protoss should be hard to play, not hard to play against.


The difference between MC and Minigun is very apparent and it is sad you can't notice that.
Reply Quote
Posts: 498
Keep up the great work Dayvie.

SC2 is more fun than it has ever been.
Reply Quote
Posts: 2,417
01/09/2014 07:18 PMPosted by Oboeman
b) with the current state of the ladder, do you really think that you have an accurate measure of player skill?


This x1000. I had to slaughter like 40 gold leaguers to get to where I was supposed to be. I sincerely doubt they have a handle on player skill at the moment; so how can they possibly account for it?
Reply Quote
Posts: 6,100
01/09/2014 09:00 PMPosted by Aurora
The difference between someone like Minigun and MC is not so easily apparent. Protoss should be hard to play, not hard to play against.


The difference between MC and Minigun is very apparent and it is sad you can't notice that.


In terms of micro, not really.
Reply Quote
Posts: 5,353
Enjoying the Terran tears here :)
Reply Quote
Posts: 10
01/09/2014 08:57 PMPosted by Aurora
You are naïve, and a fool. The game is broken, and people are leaving it by the droves. Teams are disbanding, and leagues are dropping. This game is in distress, and Blizzard doesn't give two craps. the same can be said for every other Blizzard made game. LotV and heroes of the storm will be complete flops. mark my words.


I'm betting you are one of those avilo fanboys.


Such clever, wow.
Reply Quote
Posts: 5,949
Could we see the metric that you use to calculate these win percentages? How do you define "skill". Is it fair to say that right now the terran "template" subset might be more exclusive than the protoss one?

It just seems a bit odd that these are "skill adjusted" winrates. What does that mean?


One possible method.

Take your list of games. Divide them into a bunch of bins as measured by MMR difference.

From each bin you can then measure the actual winrate, and that predicted by the MMR difference (obviously bin size has to be small for this to be valid).

For a 100 MMR difference you might expect a winrate of 55.6%, but actually see a winrate of 57.8%. Which is a 4% increase over what is expected, which in turn is a 52% expected winrate after accounting for the MMR differences.

Then you do this over all your bins, weight your bins by # of games played, taked the weighted average and you're left with an expected winrate.

EDIT: Actually thinking about it, you don't even need the bins to do that.
Edited by dainbramage on 1/9/2014 9:34 PM PST
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)

Reported!

[Close]