StarCraft® II

Balance test map follow-up

Posts: 266
01/09/2014 08:10 PMPosted by nomufftotuff
Means that they throw out all the matches where the two players MMR is "x" amount different. We don't know what "x" is.


I'm not sure if mmr is the best indicator of what we can qualitatively define as "skill", MMR can be heavily swayed by things like inherent game balance. It could very much be fruit of the poisonous tree, so to speak (usually in the high levels of play).


It's the closest thing we have. Right now, as it stands, two silver league players of any race will have a good chance of beating one another. This means that even if there is some imbalance in the game, the ladder should be matching you with people that will give you a good match, which in some ways is even more important than perceived "imba" at the lower levels.

The thing that's missing from this puzzle is that the MMR distribution of all three races. With that, you might have a better picture at the state of balance as a whole. I'm sure Blizzard has those stats; they said that they're looking at the GM bracket in their post.
Reply Quote
Posts: 15,457
Could we see the metric that you use to calculate these win percentages? How do you define "skill". Is it fair to say that right now the terran "template" subset might be more exclusive than the protoss one?

It just seems a bit odd that these are "skill adjusted" winrates. What does that mean?


Means that they throw out all the matches where the two players MMR is "x" amount different. We don't know what "x" is.

01/09/2014 08:05 PMPosted by Jonathan
I thought the ladder system was designed to pretty much ensure everyone has a 50/50 win rate? If so, why do these ladder stats mean anything at all? It would mean that lesser Protoss players were being matched with better Terrans to even out the winnrate.


No, you can factor those matches out based on MMR, making the statistics "skill-adjusted" as they say.


mmr is not independent from "race strength". he says they are able to separate skill from race strength so that they can evaluate race strength.
it's a mystery to me.
Edited by Oboeman on 1/9/2014 8:19 PM PST
Reply Quote
Posts: 13

Posted by Dayvie

Please keep in mind these are not straight-up win percentages. They’re win percentages with player skill factored out. When we grab win/loss data for balance purposes, we categorize each game with 2 different variables per side: one being player skill and other being race strength. So by factoring the player skill out, we are able to more accurately check how each race is doing at each skill level.


I have a question for you Dayvie, about the variable "player skill". The "player skill" is balanced between the 3 races?, any race has more "player skill" than the others?. Correct me if I´m wrong but I think that this variable alone could indicate some imbalance issues, and it seems to me that you just extract it from those statistical analyses you showed. I think that the "player skill" variable should deserve its own analyses and percentages per race.
Edited by BeNite on 1/9/2014 8:22 PM PST
Reply Quote
Posts: 56
Because...

LADDER WINRATES MEAN NOTHING WHEN THERE IS A MATCHMAKING SYSTEM ENSURING 50/50 WINRATES.

So unless that quote means something special, I'd say:

STOP TRYING TO FOOL US


This is something that David Kim doesn't think about at all.
Reply Quote
Posts: 14
Weird Terran has a slight disadvantage vs Protoss when they only rely on tier 1 (mostly) and tier 2 through the whole game. When Terran start relying on tier 3 and has a disadvantage they are allowed to whine,
Reply Quote
Posts: 17,487
01/09/2014 08:10 PMPosted by nomufftotuff
Means that they throw out all the matches where the two players MMR is "x" amount different. We don't know what "x" is.


I'm not sure if mmr is the best indicator of what we can qualitatively define as "skill", MMR can be heavily swayed by things like inherent game balance. It could very much be fruit of the poisonous tree, so to speak (usually in the high levels of play).


It's the only thing that has any chance of being remotely relevant. It'll be some sort of adjusted MMR. The point is that adjustment (Which in theory takes care of inherent game balance, etc). But without knowing what it is, we can't estimate how accurate it might be.
Reply Quote
Posts: 1,301
Because...

LADDER WINRATES MEAN NOTHING WHEN THERE IS A MATCHMAKING SYSTEM ENSURING 50/50 WINRATES.

So unless that quote means something special, I'd say:

STOP TRYING TO FOOL US


This is something that David Kim doesn't think about at all.


I can't believe that someone balancing a game like SC2 would be so uneducated about the meaning of basic statistics.
Reply Quote
Posts: 14
Sure they use a few ghosts late game but come on
Reply Quote
Posts: 17,487


This is something that David Kim doesn't think about at all.


I can't believe that someone balancing a game like SC2 would be so uneducated about the meaning of basic statistics.


If you bothered to read the original post (which you clearly still haven't), they claim to take that into account. which is indeed basic statistics.
Reply Quote
Posts: 64
David Kim, please allow me to introduce a very basic concept to you. The system works very hard to make everyone's winrates 50-50. So if the entire game is easier for Protoss than it is for Terran, wouldn't Protoss players be promoted until they meet Terrans that are able to beat them 50% of the time?

So while the statistics (from 2013lolololol) will show a 50-50 winrate between the races, LOWER SKILLED PROTOSS PLAYERS ARE HIGHER LEAGUE THAN TERRAN PLAYERS!

WHY OH WHY IS THIS SO COMPLICATED FOR YOU PAL?
Reply Quote
Posts: 5,102
Because...

LADDER WINRATES MEAN NOTHING WHEN THERE IS A MATCHMAKING SYSTEM ENSURING 50/50 WINRATES.

So unless that quote means something special, I'd say:

STOP TRYING TO FOOL US


This is something that David Kim doesn't think about at all.


0/10

The system is not trying to put you at 50% win rate in ALL MATCH-UPS, but at 50% OVERALL, so racial imbalance can indeed show up in stats.

David Kim takes time to write a long update for the community, after all that criticism the blizzard lacked communication with the players, and all Terran trolls have to say is 'QQ Blizzard's official stats doesn't mean anything QQQQQQ'

SERIOUSLY ?? If you guys actually read the whole post he states that this is a good time to discuss small protoss nerfs and small terran buffs, so he is actually working his !@# on balancing this game more than you can think of.

/endrant
Reply Quote
Posts: 291
This kind of response from someone like Dayvie is desperately needed as much as possible. Good work on that front!

Might I ask do you want to see the Swarm Host in Legacy Of The Void with the kind of game-play it produces currently or will it be phased out/ have a role change during the Final beta?
Reply Quote
Posts: 1,301


I can't believe that someone balancing a game like SC2 would be so uneducated about the meaning of basic statistics.


If you bothered to read the original post (which you clearly still haven't), they claim to take that into account. which is indeed basic statistics.

You don't know WTF you are talking about.
I've read it multiple times and I understand it completely.
Reply Quote
Posts: 201
01/09/2014 06:21 PMPosted by Dayvie
We absolutely hear your concerns about GM league representation not being evenly distributed. Obviously, we’d like to see even numbers of players at every skill level, but this one in particular presents two issues: it’s a very small sample size, and it doesn’t represent the pro level very well in all three regions. This is not to say that GM representation balance doesn’t matter – it does matter. We simply believe in putting a bigger emphasis on the pro level as well as all of the other league levels.

It's not just GM though, it's the top percent of your player base. If GM is too small a sample, then expand it and give us the stats. We know what they're going to say. They're going to show that around a quarter of the population is Terran, a small fraction is Random, and the rest is split between zerg and protoss with a heavy protoss bias in the NA and EU regions.

I don't understand how you can account for skill without taking into consideration this shift in racial populations. If one race loses more often to another, their apparent skill will decrease across the board and they will appear slightly stronger in their other non-mirror matchup because on the average the players they face in that matchup will be slightly less skilled than them. This is reflected in your stats. A large imbalance in TvP causing a much smaller imbalance in TvZ.

Accounting for skill will make all matchups tend towards 50%. I have not seen a compelling case from Blizzard that their method of skill removal takes into account the potential for a large imbalance that would cause apparent skill to be reasonably different from where 'actual skill' would be. The relative size of the terran population would be indicative of such a problem. The argument could be made that the other two races simply have better players by chance, but not only would that be improbable, the racial imbalance at the beginning of Wings tells the opposite story. If I remember correctly, 'skill removed' win percentages weren't a huge problem then either.

01/09/2014 06:21 PMPosted by Dayvie
It’s the only major tournament going on right now, and it represents a sample size that is too small to draw any broad conclusions from, but Protoss players have lost at a noticeable clip in that tournament. As of this writing, they’ve recorded only 11 wins in 35 non-mirror matchups.

This supports my idea that you're not taking into consideration population differences. There is a massive difference between the number of Protoss and the number of Terran in SPL. The average skill level of the terrans being fielded is likely higher than the average protoss. Teams are wagering, for example, that their 21st best protoss is a better bet than their 8th best Terran. That's kind of a large gap.
Edited by Winks on 1/9/2014 8:33 PM PST
Reply Quote
Posts: 7,836
01/09/2014 08:16 PMPosted by Starstrider


I'm not sure if mmr is the best indicator of what we can qualitatively define as "skill", MMR can be heavily swayed by things like inherent game balance. It could very much be fruit of the poisonous tree, so to speak (usually in the high levels of play).


It's the closest thing we have. Right now, as it stands, two silver league players of any race will have a good chance of beating one another. This means that even if there is some imbalance in the game, the ladder should be matching you with people that will give you a good match, which in some ways is even more important than perceived "imba" at the lower levels.

The thing that's missing from this puzzle is that the MMR distribution of all three races. With that, you might have a better picture at the state of balance as a whole. I'm sure Blizzard has those stats; they said that they're looking at the GM bracket in their post.


I'm not claiming that it's not some sort of litmus test, it just seems like the lowest picked fruit (love these extended metaphors huh?).

The way mmr works, the system already matches players that have close mmr-but what if a race or regional player set is more highly represented in the higher echelon (by mmr standards) what then? IF some set of mmr ratings corresponds to a player set that has X or Y in common, is it balanced?
Reply Quote
Posts: 17,487


If you bothered to read the original post (which you clearly still haven't), they claim to take that into account. which is indeed basic statistics.

You don't know WTF you are talking about.
I've read it multiple times and I understand it completely.


Clearly not, because you don't seem to understand how racial imbalance would still show up.

Please tell me where I'm wrong, or Dayvie's wrong, without being vague.

ps: You can't.
Reply Quote
Posts: 1,301
01/09/2014 08:26 PMPosted by Velitey
The system is not trying to put you at 50% win rate in ALL MATCH-UPS, but at 50% OVERALL, so racial imbalance can indeed show up in stats.

If one race is OP against all others, how does that show in winrates?????
Players will get promoted to higher leagues and have 50/50 winrates there.
Edited by Azoryen on 1/9/2014 8:33 PM PST
Reply Quote
Posts: 17,487
The way mmr works, the system already matches players that have close mmr-but what if a race or regional player set is more highly represented in the higher echelon (by mmr standards) what then? IF some set of mmr ratings corresponds to a player set that has X or Y in common, is it balanced?


01/09/2014 06:21 PMPosted by Dayvie
Please keep in mind these are not straight-up win percentages. They’re win percentages with player skill factored out. When we grab win/loss data for balance purposes, we categorize each game with 2 different variables per side: one being player skill and other being race strength. So by factoring the player skill o


they state exactly how it works in the main post. (minus the actual workings of the adjustments, which is what we're asking about).

You and Azoryen seem to be caught up on the same hitch. The issue isn't whether the MMR/winloss numbers aren't adjusted, it's how.

01/09/2014 08:31 PMPosted by Azoryen
The system is not trying to put you at 50% win rate in ALL MATCH-UPS, but at 50% OVERALL, so racial imbalance can indeed show up in stats.

If one race is OP against all others, how does that show in stats?????
Players will get promoted to higher leagues and have 50/50 winrates there.


Because you'll see that a person with a 50% overall winrate will be for example, 60% vs P, 50% vs T, 40% vs zerg. (this is one example).

If this shows up for many Terrans, this would tell you TvP and potentially TvZ has an issue.

It can be done, it just matters how.
Edited by Arianity on 1/9/2014 8:34 PM PST
Reply Quote
Posts: 64
01/09/2014 08:29 PMPosted by Winks
We absolutely hear your concerns about GM league representation not being evenly distributed. Obviously, we’d like to see even numbers of players at every skill level, but this one in particular presents two issues: it’s a very small sample size, and it doesn’t represent the pro level very well in all three regions. This is not to say that GM representation balance doesn’t matter – it does matter. We simply believe in putting a bigger emphasis on the pro level as well as all of the other league levels.

It's not just GM though, it's the top percent of your player base. If GM is too small a sample, then expand it and give us the stats. We know what they're going to say. They're going to show that around a quarter of the population is Terran, a small fraction is Random, and the rest is split between zerg and protoss with a heavy protoss bias in the NA and EU regions.

I don't understand how you can account for skill without taking into consideration this shift in racial populations. If one race loses more often to another, their apparent skill will decrease across the board and they will appear slightly stronger in their other non-mirror matchup because on the average the players they face in that matchup will be slightly less skilled than them. This is reflected in your stats. A large imbalance in TvP causing a much smaller imbalance in TvZ.

Accounting for skill will make all matchups tend towards 50%. I have not seen a compelling case from Blizzard that their method of skill removal takes into account the potential for a large imbalance that would cause apparent skill to be reasonably different from where 'actual skill' would be. The relative size of the terran population would be indicative of such a problem. The argument could be made that the other two races simply have better players by chance, but not only would that be improbable, the racial imbalance at the beginning of Wings tells the opposite story. If I remember correctly, 'skill removed' win percentages weren't a huge problem then either.

It’s the only major tournament going on right now, and it represents a sample size that is too small to draw any broad conclusions from, but Protoss players have lost at a noticeable clip in that tournament. As of this writing, they’ve recorded only 11 wins in 35 non-mirror matchups.

This supports my idea that you're not taking into consideration population differences. There is a massive difference between the number of Protoss and the number of Terran in SPL. The average skill level of the terrans being fielded is likely higher than the average protoss. Teams are wagering, for example, that their 21st best protoss is a better bet than their 8th best Terran. That's kind of a large gap.


Finally someone gets it.

@Arianity See post above you fking moron....
Reply Quote
Posts: 1,301
01/09/2014 08:32 PMPosted by Arianity
You and Azoryen seem to be caught up on the same hitch. The issue isn't whether the MMR/winloss numbers aren't adjusted, it's how.

What's your education on math?
Reply Quote

Please report any Code of Conduct violations, including:

Threats of violence. We take these seriously and will alert the proper authorities.

Posts containing personal information about other players. This includes physical addresses, e-mail addresses, phone numbers, and inappropriate photos and/or videos.

Harassing or discriminatory language. This will not be tolerated.

Forums Code of Conduct

Report Post # written by

Reason
Explain (256 characters max)

Reported!

[Close]