Why Handicapping (MMR) is Wrong for Competitive Play

Competitive Discussion
Prev 1 26 27 28 50 Next
Eh you're playing against lesser skilled or higher skill players? Do you not see this?

The matchmaker should create games based on SR not MMR - MMR is for care bare games in quicklay
01/27/2018 10:41 AMPosted by tawT
This has been stated directly by the devs


No, it has not.

https://us.battle.net/forums/en/overwatch/topic/20758686566?page=1#post-6

SR/MMR closely linked unless severe inactivity decay.
nah I watched a few of his videos he seems like a nice guy
Imagine SR as a league in Soccer: Everyone gets to play everyone else, the bottom teams dont get to not play the top ones.

Edit: Or vice versa.
01/27/2018 09:57 AMPosted by tawT
Short of placements and decayed masters and GMs the MM should just fin a group with similar *SR* and match them together.
Instead it tries to be smart about it and creates "Tiers" of people of different MMR at the same SR.


I guess I can never stop posting this. There is NO SUCH THING AS MATCH MAKING PEOPLE AT THE SAME SR WITH DIFFERENT MMRs. SR is not used for match making at all.

Official statements:

1 - https://twitter.com/ww/status/867570441182826499
2 - https://twitter.com/playoverwatch/status/850435344457543680?lang=en

In game proof:

1 - https://youtu.be/yXOwh6l8aBI?t=1m17s
2 - https://www.youtube.com/watch?v=9ZdyrUpEXRs

All they do is get 12 people with as close as MMR as possible and throw them into a game. You asked for them to do the same thing, but with SR instead of MMR, and that would simply be a strictly worse implementation of the same idea. Would you really want a decayed GM stomping platinums? No, of course not, that would be stupid.

You're missing the point. The matchmaking isn't finding 12 players of "equal skill." It is hard to quantify skill (as Blizzard's developers are finding out). Every player in every match has a different level of skill.


Well, you are right about this. It's certainly not easy to measure skill, especially in a complex game with many skill sets.

That said, it's not hard to quantify skill from a system perspective. It's just your MMR. That's it, one number. It's not a perfect representation, but it works well. At least, it works better than any other approach.

01/27/2018 11:17 AMPosted by Cuthbert
The handicapping/Match Making Rating system uses performance metrics, like levels of damage and healing, to figure out roughly how good people are. And then it arranges teams however it wants to give either team a 50% chance of winning.


You seem to think that stats are the primary aspect of MMR, not winning/losing. You know that for 3000+ players, stats aren't used at all, right? It's literally just win/loss for us. For sub 3000 players, performance is just a small bonus on top of the gain/loss from winning/losing.

Those arrangements typically single out the best players, and put them on teams with the worst players. Make sense? If you are the best player in the game, welcome to the worst team. The second, third, and fourth best players in the game have all been placed to oppose you.


This is where the delusion sets in. That you're SO much better than everyone else in the game, that the system has to stack the game to balance you out. Do you really think the match maker can't find players equal or better than you?

Every team has to have a best and worst player (in terms of MMR), by basic logic. Even though usually the difference will be so small it's inconsequential. Some games, you might legitimately by the highest MMR. Just as many games, however, you'll be the lowest MMR.

Cuthbert, your SR is 1769. I hate to break the bad news to you, but the match maker won't have trouble finding players better than you. Most players are better than you. You will actually be the best player in the game sometimes, but no more than 1 of out 12 games.

01/27/2018 11:30 AMPosted by tawT
Imagine SR as a league in Soccer: Everyone gets to play everyone else, the bottom teams dont get to not play the top ones.

Edit: Or vice versa.


Bronze has literal 6 year olds on their first competitive games, while GM contains actual professional players. Do you really think pro soccer players should have to play against kid leagues? Because that's the equivalent.
You missunderstand with gusto dont you. I'm saying that MMR tries to create "fair" games. And that this shouldn't happen. It should just create games of people with similar SR, or average similar SR.

SR is the mesure we care about it's called "Skill Rating" for a reason. So use it for Matchmaking. MMR is redundant - it does nothing but confuse players and fail at creating teams. It also misunderstand skill in general as is evident by peoples rapid rises or thundering losing streaks. Throw it out.

MMR being higher or lower than SR is a thing - how else would the game detect those bastard GM to Bronze people?
It's also possible staying afloat at a given SR while performing "poorly" compared to the average player (ie lower MMR). Same is true with people who are better than their SR.

And this isn't an atempt at avoiding realising you have to get gud to rank up or whatever - quite the oposite. I want a HARDER system that doesn't try to create "fair" matches, just matches that are appropriate for my SR by using SR to matchmake. Neutral, hands-off matches.

Soccer example:
Bronze league players fighting bronzes, gm players competing against GMS. MMR should only be used to weed out the extreeme cases.
01/27/2018 12:05 PMPosted by tawT

MMR being higher or lower than SR is a thing - how else would the game detect those bastard GM to Bronze people?


It's not. You even tried to claim that the devs stated it, but I actually managed to pull a Blizzard quote that says otherwise.
01/27/2018 12:05 PMPosted by tawT
You missunderstand with gusto dont you. I'm saying that MMR tries to create "fair" games. And that this shouldn't happen. It should just create games of people with similar SR, or average similar SR.


That's what they do, create a game with a bunch of people with a similar "number", except that number is MMR, not SR.

Despite the name, SR is explicitly NOT a strict representation of how good you are, and Blizzard doesn't pretend otherwise. Do you think a GM player gets 1000 SR worse if they don't play for a few weeks? SR is a rank, which generally is close to how good you are, but can be impacted by a few other things.

They do what you want, but in a way that isn't [Edited by Blizzard]. Your suggestion would be broken. You could ask for VISIBLE MMR instead, that would make sense. Using SR for match making wouldn't.

01/27/2018 12:05 PMPosted by tawT
MMR being higher or lower than SR is a thing - how else would the game detect those bastard GM to Bronze people?


What do you mean "detect" them? It's not like after 3 games the match maker knows they are a GM player. From the match maker perspective, you have a bronze player suddenly start doing much better. It keeps increasing the rank, putting them into higher ranked games, but it has no way of knowing it's actually a GM player.

01/27/2018 12:05 PMPosted by tawT
Soccer example:
Bronze league players fighting bronzes, gm players competing against GMS. MMR should only be used to weed out the extreeme cases.


I'm not sure what you are saying. Bronzes do play bronzes and GMs player GMs. On the border you play other tiers, like high bronzes play with low silvers. Is that not what you want? Do you want bronze and GMs playing together?

Forum Mod Edit: This post has been edited by a moderator due to language. https://us.battle.net/forums/en/code-of-conduct/
01/26/2018 04:47 PMPosted by pootflute
I've been suspicious of the matchmaking's effectiveness for a while and finding out that there is this totally obscured system really doesn't help.


Honestly, this is just ignorance. I'm not being mean saying your dumb, just the literal meaning of "not knowing". The system really isn't that complicated.


Yes, that's literally what I said. I did not know. I'm not saying that now I read about it, I don't understand it, I'm just saying that just by playing the game, I was never aware it worked like this. The thought never occurred to me that matchmaking was based on something other than SR.

I get it though, MMR is an apparently innocent way to facilitate decay. I'm just saying it's a little weird that it's totally obscured. You have to read dev forum posts to learn it even exists.
Friendly, wth?

MMR/SR differences are apparent from GMs that decay then start playing again - this is a known mechanic.

The devs has also stated that your MMR can decay if you don't play for a while (in some dev blog can't remember wich one, but in the mean time look at this: https://www.youtube.com/watch?v=niyk29hN5kw). Unless your 3000+ this wouldn't affect your SR

Edit: I think it was in a dev update but i can't remember wich - take with a grain of salt.

If SR and MMR is so closely linked then one is redundant.

Soccer example AGAIN, now with more spoonfeeding:
Imagine SR as a division in soccer, for this example lets say 2100SR. Given that MMR can increase or decrease while you over all stay the same rank. Every player at this division has to play the other people in the division. REGARDLESS of their skill. You're in the division and no quarter is given at all - you either play to the level expected or you go down a division (to, say, 2000).

Unless you have people from other divisions (say a plat or a silver in a group) your teammates SR should be at or close to your own SR regardless of all other concerns. This is competitive.

EVIDENCE:
From my own experience: I made a log of my games in season 5 (https://docs.google.com/spreadsheets/d/1O9mprevYPN11aBP9eLP7DOty_Bs9kmKmz4Wmv0hNZRs/edit?usp=sharing), Where my SR was higher than my teammates and the enemy team in the vast majority of games for whatever reason. My bet is a gap between SR and MMR. This was all solo que. If the MMR system was working so close to SR as you white knights are saying then by sheer coincidence surely 1 match would be 150 more than me? No. one match at 25+ and one at +16, the rest very close to me or up to 172 below.

Clearly the MMR didn't believe in me*, or I was getting put into games with some scrubs to make their game better while craping on mine.

Edit: The matchmaker was handicaping me because my bullmanure MMR was lower than it thought i should be. What it should do was just believe I deserved my rank and let not try to do anything about it.
More on the link to my data above:

This was a period of my comp career where I was starting to have more and more horrible games and i started to notice the disparity between my SR and my teammates SR.
And some games you had the type of player who just cant - no grouping up, no syncing of anything usefull, 5dps 1 healer you get the drift. I mean how the hell did they survive to be in mid gold?

Answer: MMR didn't give them a mid gold experience, it matched them with low skill players like themself and thus they were able to stay in the ranks where they had no buisness being.
Stop downvoting posts simply because you disagree with it, that is not how you use the downvote tool. A lot of posts are well written and offer a point in the discussion.
01/27/2018 01:40 PMPosted by tawT
If SR and MMR is so closely linked then one is redundant.


I literally keep telling you the differences between them. They are closely linked, but not identical. SR is a rank, which can change because of factors BESIDES your play and you winning/losing.

Blizzard could definitely get rid of SR. They could do it like CS:GO, where you just have the teir (gold/plat/etc) and a hidden MMR. Or they could do it like Dota and just make MMR visible. SR isn't neccessary but it isn't redundent.

01/27/2018 01:40 PMPosted by tawT
Unless you have people from other divisions (say a plat or a silver in a group) your teammates SR should be at or close to your own SR regardless of all other concerns. This is competitive.


MMR* should be at or close to your own, and that's exactly what happens. In fact, even SR is nearly always the same. Your own data shows this. Speaking of that...

01/27/2018 01:40 PMPosted by tawT
Where my SR was higher than my teammates and the enemy team in the vast majority of games for whatever reason. My bet is a gap between SR and MMR. This was all solo que. If the MMR system was working so close to SR as you white knights are saying then by sheer coincidence surely 1 match would be 150 more than me? No. one match at 25+ and one at +16, the rest very close to me or up to 172 below.


You messed up the average difference calculation. You added the difference of every game, but only divided by 130 (I guess what you call valid games?). You had an average difference between your SR and your teams SR of about 50. Which is basically identical. A difference of even 100 SR is nothing, every player ever fluctuates up and down more than that.

I will admit it does look like your initial MMR is under your SR. For like 10 games there's a consistent and larger difference that then smooths away. Just like if your SR was chasing your MMR. Maybe this was the MMR decay you mentioned? I had never heard about it before, but it does look like it fits.

How close do you expect your team SR to your own SR to be every game?
<Removed by forum moderator for toxicity>
Ignoring the evidence now? Picking some cherry you found on the right there? Ignore the rightside - you only need to look at the SR difference, the "valid" part is games with full statistics where i recorded everything i wanted and verified. My math you can criticise all you want.

The only thing you need to look at is my SR and team SR, I expect it to be a wide spread not almost all under my own. Wouldn't you? Clearly it happened to my teammates because they got a player on their team that was higher SR than them? ofcourse if you cast your dice 100 times it *can* come up all sixes, but you dont expect it.

So my conclusion is that the SR difference in team setup is influenced my something else - MMR, what else could it be?
I'll go one further:
My dataset PROVES that MMR handicaping is happening.

The resons: in a neutral system you'd expect that the distribution of rank in a team to be random, that is you'd get a spread where most teams are at your SR or near and gradually outliers from lower ranks and higher ranks. However my dataset only has players at or close to my SR and then a spread of lower ranked teams.

This is evidence that SR and MMR isn't "linked" tightly (MMR is gained through wins and lowered through losses though), and you can have different skilled people at the same SR.
And it, as stated above, proves that handicapping is happening. And it's happening in the middle of the player base.

A personal note:
Fighting for your rank and then realizing you constantly have to play with the same damned lower ranks you thought you clawed your way away from is demoralising.

Bliz plz, just believe that I and everyone else deserve their current rank and stop it with the MMR allready - stick it in Quickplay where it's welcome.
01/27/2018 01:40 PMPosted by tawT

Soccer example AGAIN, now with more spoonfeeding:
Imagine SR as a division in soccer, for this example lets say 2100SR. Given that MMR can increase or decrease while you over all stay the same rank. Every player at this division has to play the other people in the division. REGARDLESS of their skill. You're in the division and no quarter is given at all - you either play to the level expected or you go down a division (to, say, 2000).

Unless you have people from other divisions (say a plat or a silver in a group) your teammates SR should be at or close to your own SR regardless of all other concerns. This is competitive.

[/quote]

Uh, you're basically advocating that they just... make worse matchups? What's the benefit in not using the more accurate MMR value? The obvious degenerate case is decayed GMs just inexplicably get to stomp for a bunch of matches until they're back in the right division. People already get super frustrated just THINKING that most of their matches aren't fair; could you imagine the outcry if the average match really was heavily favoring one team? Plus you'd get situations where players hover around the division breakpoints as they oscillate between being a liability to most teams and being an advantage.

You're basically advocating that the system go from "make most matches fair" to "make matches randomly", especially since the top and bottom of current divisions are pretty far apart, and dividing things up more and more just results in smaller player pools, making it harder for the system to find people when accounting for other important considerations like group-size (unless you think that should be random too, lol) or geographical location. Small population ranks at the top/bottom would suffer from lack of anyone to play with, and overall queue times would increase. There's just disadvantage after disadvantage from doing it your way. I'd say the only gain is that conspiracy theorists would be happier, but no, they'd just move on to something else; in place of Cuthbert there'd be some guy called Vecna railing against the rigged system stacking matches whenever you ranked up in order to knock you back down.
01/27/2018 04:22 PMPosted by Arzoo

You're basically advocating that the system go from "make most matches fair" to "make matches randomly", especially since the top and bottom of current divisions are pretty far apart, and dividing things up more and more just results in smaller player pools, making it harder for the system to find people when accounting for other important considerations like group-size (unless you think that should be random too, lol) or geographical location. Small population ranks at the top/bottom would suffer from lack of anyone to play with, and overall queue times would increase. There's just disadvantage after disadvantage from doing it your way. I'd say the only gain is that conspiracy theorists would be happier, but no, they'd just move on to something else; in place of Cuthbert there'd be some guy called Vecna railing against the rigged system stacking matches whenever you ranked up in order to knock you back down.


It wont be random it will be based on SR. With the removal of performance based SR gain/loss you'll get a system that reflects the only valuable stats: Winning and teamplay.
Furthermore you will know without any doubt that the games are completely fair (ie neutral). And the matchmaker is allready creating unwinnable matches by teaming you with toxic bastards and people who have NO CLUE about any tactics or gamesense - and they don't go away because the MMR system resists that behaviour from sending them into Bronze where they belong.

And you will know without any doubt that as you rank up you're getting to play who deserve to be there and not people who the MMR system kept from getting stomped on. I mean those who love the MMR so much can play quickplay for days.

The SR based system is a fair system - a competitive system.

Conspiracies:
I have evidence of the handicapping above if you care to look at it. I started loging my games because i noticed my team was nearly allways lower SR than me.
<span class="truncated">...</span>

It shouldn't DO that. that's the entire point. Short of placements and decayed masters and GMs the MM should just fin a group with similar *SR* and match them together.
Instead it tries to be smart about it and creates "Tiers" of people of different MMR at the same SR.

Here's an exsample, three people with the same SR:

A: A player plays realy well over a fair few games but goes 50/50. His skill keeps him floating around and his MMR increases. Still the same SR. Game sees his MMR and matches him with people it thinks are his equal and on average they have higher SR than him.

B: Some guy, average for his SR, goes 50/50. SR stays the same MMR stays the same.

C: Some scrub lucks out and keeps his SR but his MMR decreases because the sytems thinks he sucks. SR is the same but he gets matched with people of his own MMR wich on average have LOWER SR than him.

See? A is handicapped by being put into matches that are harder than they should be, C is getting easier matches than he should have.
MMR is unfair. It has no place in a competitive system worthy of it's name. Let it stay in Quickplay where it belongs.

Let competitive be competitive and just believe that people deserve their current SR and do matchmaking off that.


You're wrongly going with the assumption that MMRs can vary wildly from the same SR.


The PBRS is designed to move smurfs up to where they belong faster.

Take the example of someone who has deranked to Gold intentionally, but is a Masters player, for instance. Said player wishes to climb back to Masters. I'm assuming the amount of SR you get for a win and lose for a loss is affected by hidden MMR, as some people lose more for a loss than they can gain for a win, and some people gain much more SR for good play if the system thinks they don't belong where they are (and everyone can have great games, but not get a lot of SR for that win).

My question, which isn't to take a position on this topic, is this: given that this Gold smurf is in that rank at that time, when they are paired with other Gold players on their way up, what are the teams matched by -- MMR, SR, or both? If it is MMR, then in order to keep the MMRs equal, either each team needs a smurf to provide balance, or each team on which a smurf is placed has to have a lower average MMR to compensate (ergo, the ex Diamond player will be expected to 'carry'). If it is SR, then matchmaking doesn't take into account smurfs.
01/27/2018 04:39 PMPosted by tawT
It wont be random it will be based on SR. With the removal of performance based SR gain/loss you'll get a system that reflects the only valuable stats: Winning and teamplay.


I support the removal of PBSR for all ranks, but it's existence doesn't mean games are rigged.

01/27/2018 04:39 PMPosted by tawT
I have evidence of the handicapping above if you care to look at it. I started loging my games because i noticed my team was nearly allways lower SR than me.


Can you explain to me how this evidence shows handicapping? Let's start by assuming this data is actually legit. It's weird how people who stream on twitch, you know where you can provide evidence of the numbers, don't run into this. But let's say you have legit, non-fabricated, data.

What I see is that your MMR is slightly under your SR, especially at the very beginning of the list. So that you end up on teams that have an average SR about 50 below yours, on average. Note that the enemy team has the same average SR as your teams though. I have two thoughts on this.

  • It is indeed a little odd, I don't know why your MMR would stay just slightly below your SR.
  • The difference is so small that it doesn't matter. 50 is literally the difference between a couple of games. Players 50 SR apart are identical.
  • Yet I'm not making the connection how this tiny, but odd, difference in SR means your games are "handicapped" in any way.

    01/27/2018 04:47 PMPosted by SpiralGalaxy
    My question, which isn't to take a position on this topic, is this: given that this Gold smurf is in that rank at that time, when they are paired with other Gold players on their way up, what are the teams matched by -- MMR, SR, or both? If it is MMR, then in order to keep the MMRs equal, either each team needs a smurf to provide balance, or each team on which a smurf is placed has to have a lower average MMR to compensate (ergo, the ex Diamond player will be expected to 'carry'). If it is SR, then matchmaking doesn't take into account smurfs.


    MMR is used, but that doesn't mean a "smurf" is put on each team. When a masters players smurfs to gold, they lower their MMR to gold level as well. MMR isn't magic, the system can't know the player is actually good despite throwing games to lower their rank. MMR is just a number, like SR, which goes up when you win and down when you lose.

    Join the Conversation

    Return to Forum