Illuminati's titles are based off of achievement in the ladders; with elite generally being top 100 in 1v1 ladder and/or top 30 in RT ladder, Masters' degree #1 in any ladder that is not 2v2, and PhD more than one #1 in any ladders that are not the 2v2 ladder.

Thus, it is possible to move up in title without submitting any written work, which is admittedly unrealistic when compared to true university degrees. Since I feel I am slowly approaching a Masters' title, I would like to challenge myself to submit something written as part of my own post-undergrad work.





My hypothesis is that the ratings in the ladders inflate over time. A 2,000 rating in a ladder is theoretically not as valuable as it used to be: this occurs naturally as more players join the ladders, achieve positive ratings and thus add to the available pool of points. With this in mind, ratings are not useful, alone, to determine skill; a rating's value is determined by comparing it with the value of other ratings, which is why the ladders rank individuals. As more players join the ladders, a higher rank increases in value, because of the increased volume of players, whereas a higher rating may decrease in value.

This is all true in theory, but no one has definitely proved it on Warlight* beyond anecdotal evidence. I have decided, then, to add my own research. I arbitrarily chose the real-time ladder as my focus, since I do not have the time to deal with 300+ players every day for this study, and the 2v2 and seasonal ladders are not the best indicators.

I will be tracking the total number of players and their ratings, once every day. The total amount of points available on the real-time ladder will be added (only counting ranked players, on that day). Since the real-time ladder is relatively small today compared to its high points, I will also average the number of points available on the ladder to adjust for fluctuations in the number of players.

This morning, when I counted there were 46 players ranked, with a sum of 75,861 points available for distribution on the ladder. This gives us an average of 1649.15217.

1. It takes 20 games to get ranked rather than 15 games on the 1v1 ladder. This makes most players with successful runs rated much higher. It makes the people they lose to grow higher which thus spreads to make the whole top 15 balance out some.

2. The 2v2 ladder takes skill. Even if your puppeting or being puppeted. (my team hasn't). I can understand if you choose not to accept 2v2 ladder success due to a very poor template but it has some staunch competition as of late...

3. Top ranked players often lose points from their peak even when they dont lose because their ratings will fluctuate based off the fluctuation of the ratings of previous opponents. This could very well "skew" the type of results you are seeking.

4. Isnt top 100 on the 1v1 a bit too easy to obtain to be classified elite? 30th on the real time ladder is quite biased since not many players are currently joined so everyone will boost their rankings. ---> Use a rating scale for determining entrance to your clan. At least for the real time ladder.

1. It takes 20 games to get ranked rather than 15 games on the 1v1 ladder. This makes most players with successful runs rated much higher. It makes the people they lose to grow higher which thus spreads to make the whole top 15 balance out some.

You have it the other way around. The 1v1 ladder requires 20 ranked games, and the RT ladder's requirement for a rank depends on the player's standard deviation, but usually falls around 15 games.

2. The 2v2 ladder takes skill. Even if your puppeting or being puppeted. (my team hasn't). I can understand if you choose not to accept 2v2 ladder success due to a very poor template but it has some staunch competition as of late...

I'm not disputing that it takes skill. The problem with using it for statistics: teams are not permanent (and also take months to get rated). Therefore a player might switch to a different team, throwing off the statistics, and the data won't change much over a shorter time frame, so I would have to collect data for years as opposed to months.

4. Isnt top 100 on the 1v1 a bit too easy to obtain to be classified elite? 30th on the real time ladder is quite biased since not many players are currently joined so everyone will boost their rankings. ---> Use a rating scale for determining entrance to your clan. At least for the real time ladder.

No. When you're a beginning player or even an intermediate-level student, taking top 100 can be quite the challenge. Consider: we have had over 50 students in the clan; so far, only two have graduated, and one was an alt. The requirements are challenging enough to push a new player to his/her limits, while still being obtainable. Taking 100th on the 1v1 ladder is a nice middle ground; a stepping stone towards taking #1 while not being unreachable for a player willing to push himself.

Use a rating scale for determining entrance to your clan. At least for the real time ladder.

Ratings diminish in value over time, with new players, so I cannot and will not use them to determine anything. Taking the top seats on the ladders will become harder over time, naturally raising the bar for the clan; judging by ratings would lower the bar over time, and I'm not gonna do that. ;)

I think this is a difficult task for Warlight.

Ken Regan studied a similar question for chess, and found that ratings have not inflated over time - top chess players really are much better today than top chess players back in the 80s and 90s, just as their ratings would suggest. However, he had the advantage that chess computers are much better than people, and their time is not valuable - you can use a computer to estimate accurate ratings by looking at the moves played in specific games, and this provides an objective way to compare players from today with players from the past.

Really? Intriguing; I had been under the impression that research had demonstrated chess ratings did inflate over time. I'll have to look into that!

Today we have 51 people ranked on the RT ladder. Sum of available points is 82,603, making an average rating of 1652.06. An 8% increase in the number of points available, but only a marginal increase in average rating.

Edited 6/1/2015 16:55:07

The ratings fluctuate. I had a 2001 rating a few weeks ago, good for 23rd on 1v1 ladder. Today I am 24th but with a 1943 rating. My 2001 would be top 15 today.

Years ago, a 2001 would probably be top 10 on the ladder.

Edit: this is about Chess ratings, in reply to FK's post above

This is the work I was thinking about: http://www.cse.buffalo.edu/~regan/papers/pdf/ReHa11c.pdf You can use Google scholar to find other papers that use a similar method.

The ratings of the top players have increased over time. The question is, why? Are ratings today comparable to those from, say, 30 years ago?

Without a way to directly compare the strength of players today to players from the past, it seems impossible to say. Several authors, including Ken Regan, have devised methods to use computers to derive an inherent rating for players directly from the games they play, based on the quality of their moves. This assumes that the computer program used is much better than the players, and so it can serve effectively as an omniscient judge - which, in the case of chess, is now the case.

Regan's paper linked above concludes that today's ratings are more-or-less comparable to those from 30 years ago - as in many competitive sports, the top players today are simply much better than the top players from 30 years ago.

This is not the final word, just some work I heard mentioned in a talk once. I am not familiar with the literature in the area, but using Google scholar plus the paper I mention above should give you enough to go on.

Edited 6/1/2015 17:23:25

82,603/51 = 1619.67 ?

I'm gonna go ahead and blame the spreadsheet for that error, andy! Thanks for correcting it

Use a rating scale for determining entrance to your clan. At least for the real time ladder.



Ratings diminish in value over time, with new players, so I cannot and will not use them to determine anything. Taking the top seats on the ladders will become harder over time, naturally raising the bar for the clan; judging by ratings would lower the bar over time, and I'm not gonna do that. ;)

I think he meant standardise ratings, but not using the ladders standardisation (i.e. rank) because you may have better or worse people in the ladder at that certain time.

Years ago, a 2001 would probably be top 10 on the ladder.

Back when I was active on the 1v1 ladder I had a best ever rating of 1924 and was 12th, pretty sure 2001 could put you top 7, maybe even top 5 back then.

An 8% increase in the number of points available, but only a marginal increase in average rating.

Surely the number of points available isn't a very good indicator of how the ladder is doing if on the real-time ladder people come of and on it every day depending on when their last game was? I think average rating may be the only relevant statistic, however number of people on the ladder and what day of the week could be interesting, even to see spikes when holidays come around.

Surely the number of points available isn't a very good indicator of how the ladder is doing if on the real-time ladder people come of and on it every day depending on when their last game was? I think average rating may be the only relevant statistic, however number of people on the ladder and what day of the week could be interesting, even to see spikes when holidays come around.

You're right, the number of points available for distribution isn't a great indicator on its own. It is, however, useful when paired with total number of players, average rating etc. :)

Hmm maybe a more effective method would be scraping the rankings and looking at ratings distribution every x-hour period.

@Chess rating

Really? Intriguing; I had been under the impression that research had demonstrated chess ratings did inflate over time. I'll have to look into that!

There is research suggesting that chess rating do inflate, and there is research that suggests it doesn't. As of today no general consensus exist. So no one really knows for sure.

EDIT: As always there is no problem googling for sources, so you can support whatever you want to with "factual" proof.

Edited 6/1/2015 20:42:16

Today, we have 49 players ranked; total points available is 74,635. The average is 1523.163265. The sudden drop in both numbers can be explained by Ollie and CharlueBlue dropping out temporarily, causing a dip in average skill.

I have not been the most active player overall, but I've been around for a while and I've been observing the competetive community quite close, however I've been always studying mostly top players, not average players so I might be biased. When it comes to ladders, I am completely sure average skill of top players increased dramatically. Back when Impaller was #1 pretty much constantly he was facing only HHH on #2 spot and no other viable opposition on 1v1 ladder. Back then 2000+ rating meant always 1st or 2nd place. After that period it got a bit more balanced, but also ratings got a little distorted since many people stalled games. However judging by the quality of moves, I can safely say that people that get real rating higher than 2100 on 1v1 ladder are indeed playing better than Impaller. I have no clue if that is the same for average players. If possible, you could give me 5 sample games of 2 different players that in different periods had an accurate rating of x, where x is a specific rating range. I think I'd be able to tell by moves if I get a suficient amount of games who is a bettter player. When it comes to rt ladder it is a bit harder to determine, I always thouguht over time ratings would deflate because of initial heavy drop in variation being a big boost and for a while it seemed like it was the case for many players. Still, on rt ladder the much more important thing is who is active, considering 50 people is not that much a single person influences what you call "average skill" quite a lot, so a lot of top players dropping out of the ladder in and out seems to be the dominant factor by far when it comes to getting high places. Still, when it comes to skill evaluation it always seemed like rt ladder rating was very close to my estimation of players' skill, so at least I thought it was being quite accurate. If you want to do a serious study on ladder rating inflation I can already tell you it is futile becasue of the sample size and external factors. Pretty uch 99% of the work you'd have to do in order to get any kind of informative results would be isolating specific factors like dropping out when you lose after a streak, vacations, alts, stalling etc. Overall I can safely say that the average skill of "top x" players on any given ladder (except rt ladder which has most rotation) has insreased dramatically over the years. I have not participated in the ladder in a while, but looking through some of the better games from time to time, I see less errors from top players than I did in the past and to me that is enough to make such a statement.

To touch on something lightly discussed, I guess I placed 5th with a rating of 1961 way back in the day. This was probably during the first month or two of the ladder's existence, when only paid members could participate. That was pretty much the only reason I subscribed.

I actually subscribed with my old account, played a few matches, then asked Randy in private to switch my membership over to what I wanted to consider my main account. Which back then confused some people that saw an account that had previously placed high on the ladder, and didn't have a membership.

when i joined WL, i remember you could take #1 in 1v1 ladder with 1925/1950 points.
Basically everyone have a lower rating, and even if you went 15-0, you would ended up with 2000 rating/2050.
Since the last years rating are inflated and it's very easy get 2100+
You can look at ladder points record, and you will see it is increased a lot by time.

About overall skill i agree with szew, there are less strategical mistake/picks in ladder games.

About impaller, when he come back he was barely top 10. looking at his past 1v1 games when he was 1st, it looks like he wasn't that good, but there was no competition at that time.

I think it would be interesting if 1v1 ladder would change in this way:
reset ladder.
put the same rating system of RT ladder. (no expiration = no reason to leave ladder or change account for w8 expiration)
Put a small money prize for top spot. allow to play ladder game with bet.

Illuminati's titles are based off of achievement in the ladders; with elite generally being top 100 in 1v1 ladder and/or top 30 in RT ladder, Masters' degree #1 in any ladder that is not 2v2, and PhD more than one #1 in any ladders that are not the 2v2 ladder.

i think your range is too big for elite players. i think top 30 1v1 ladder would be more accurate, rating lower than 1900 doesn't seems fit with "elite"

Edited 6/5/2015 07:21:03

i suppose back in the days when a 2k was a top 10 it would ne easier to get there for good players as they would be fightign more lower ransk than today.

As there is a big amount of players in the ladder, there is also a bigger amount of good players, so if someone is just fightign against the top 50 he is likely to just fight good players, rather than in the times in which avrage players were in the top 50