But Beren, My point is that currently chess is at the stage were you map out the game before the match. I have myself played games that lasted more then 20 moves, were it wasn't a single move I hadn't seen the day before on my pc. And trust me, every top player (Juniors, Seniors, and adults) prep before the game. And prepping means preparing against the Opponent they are facing in the next round, not studying (general) chess.

Openings like the Berlin wall, is an example of this happening at all levels of chess (but novice and kids), including the world elite. If someone wins that game, they typically found the winning maneuver in advance, with the help of their team.

@FK, Actually more then 1 of my friends have done so. Sorry for misinforming you. I know of plenty of examples outside my circle of friends too, it is not an uncommon thing.

Edited 6/24/2015 23:49:04

Spreadsheet Simulations, or How I Learned to Stop Worrying and Love the RNG

Hmm so I wanted to experimentally (in)validate my own methods of analysis as well as TeddyFSB's suggestion to just check width so I set up a Google Sheet that simulates 1000 games played by 30 players in a "league" and then produces the results according to our methods (visible in the Status sheet if you want to check League Widh and in the Templates sheet if you want to check the bias and p-value-based results).

If you'd like to mess around with it, I suggest making a copy and then enabling/disabling templates as you see fit. Be warned, though, it takes time for the Sheet to simulate those 1000 games, and the simulation runs over from scratch every time you make even the tiniest change. For that reason, I also set up one-template league results sheets so you don't have to wait as long.

The templates used in the simulation were:

Lottery - 50% chance of overdog win
Overdog .7 - 70% chance of overdog win
Underdog .7 - 30% chance of overdog win
Overdog 1 - 100% chance of overdog win
Underdog 1 - 0% chance of overdog win
PerfStrat - % chance of overdog win varies each game based on Elo difference

(PerfStrat is a hypothetical "perfectly strategic" template that behaves exactly as Elo would predict each time- keep in mind that there's still immense room for chance, and it's still theoretically possible, albeit improbable, that this template would yield the exact same results as one of the "extreme" bad templates above- Lottery, Overdog 1, and Underdog 1- in a single run so you're not going to get 0 Bias or a p-value of 0.50/1.00, although the Bias should be very low and the p-values very high)

There are slight inaccuracies in its random selection methods, due to the limitations imposed by Google Sheets. However, their impacts are going to be trivial and cancel themselves out anyway over 1,000 games.

Link to main simulation, with all templates enabled:
https://docs.google.com/spreadsheets/d/1xfvOwCZ-CIxTmOYjgk5ABng5ekDhk9KFZe6izJKVpSo/edit?usp=sharing

Link to simulation with only Lottery enabled:
https://docs.google.com/spreadsheets/d/12dKrE54vHlCQvMYJFvZoCYv6cvEUNg4zViRJQdcmQoI/edit?usp=sharing

Link to simulation with only Overdog .7 enabled:
https://docs.google.com/spreadsheets/d/1XpYrBnkky8YPd1MipgopkoplIvluuAxM2HnQN-PVQb0/edit?usp=sharing

Link to simulation with only Underdog .7 enabled:
https://docs.google.com/spreadsheets/d/1Cwb4DXEPvjnYIKd2H8_mHbeUBCFcs9_UInDmLGhIpWo/edit?usp=sharing

Link to simulation with only Overdog 1 enabled:
https://docs.google.com/spreadsheets/d/1To79EOJEe2kawACItmlOaDWGcQIBd86Zry6hEjIr-Pw/edit?usp=sharing

Link to simulation with only Underdog 1 enabled:
https://docs.google.com/spreadsheets/d/1PtzpMBTyP83WajdzxaSUjRFZSMHrkViS5N9NzW1fLtE/edit?usp=sharing

Link to simulation with only PerfStrat enabled:
https://docs.google.com/spreadsheets/d/1gXfDgfF4-KFg19BTVoiPfFyEH9uRpW-y3QxH37GPx-Q/edit?usp=sharing

Since it takes a while to load, I've also screenshotted some sample results:

MAIN - Width:


MAIN - Templates:


MAIN - Some Results:


LOTTERY - Width:


LOTTERY - Templates:


LOTTERY - Some Results:


OVERDOG .7 - Width:


OVERDOG .7 - Templates:


OVERDOG .7 - Some Results:


UNDERDOG .7 - Width:


UNDERDOG .7 - Templates:


UNDERDOG .7 - Some Results:


OVERDOG 1 - Width:


OVERDOG 1 - Templates:


OVERDOG 1 - Some Results:


UNDERDOG 1 - Width:


UNDERDOG 1 - Templates (value for 2-tail should be 0; a loading error in the sheet):


UNDERDOG 1 - Some Results:


PERFSTRAT - Width:


PERFSTRAT - Templates:


PERFSTRAT - Some Results:


Edited 6/25/2015 10:53:45

Two more sheets:

A "league-free" version of the simulator where no player plays more than one game (i.e., the games within the league don't affect any Elo histories that in turn affect the league), which basically lets you reverse TeddyFSB's method and enter a league width (in the Console sheet) and see how it affects everything else:

https://docs.google.com/spreadsheets/d/1fnMjzZ_vSzoxNVL8nH7DKxeDf1fWZTeFeJPC3ytFwfc/edit?usp=sharing

I made that one to see if I could use the simulation to derive an experimental "expected" overdog win rate since it seems very likely that my assumption that P(average overdog beats average underdog) = average(P(overdog beats underdog)) is flawed.

Also, the distribution here is going to be flat rather than normal when you adjust the "width," so that does create some issues for the analysis.

(The larger the width, as you'd expect, the more likely that Overdog 1 is the "best template" and the smaller, the more likely that Lottery is... but PerfStrat is ultra-flexible and has decent bias ratings and p-values across a very wide range)

An "interactive" version of where you can enable and disable different templates and see how it affects the results:

https://docs.google.com/spreadsheets/d/171Oxr8kfzIxYhVxy7CcdMdOH2b21CjLEwtNaRlXWKe8/edit?usp=sharing

Edited 6/25/2015 10:51:03

I've been trying to understand this thread for a while now. I'm still going to spend some time trying.
Few questions:

I don't understand expected Overdog win %. First off, why it's based on average rating difference? Why not check elo ratings for each game, determine chance that overdog win, and take average of those?
I did that for PERFSTRAT sheet, got expected win % same as simulated.
Elo ratings are based on the games, what you actually check, is whether the way for calculating it is the same as the one you use.
I think there is point bias based on first pick, not sure if it is accounted anywhere in your calculations.

How can a template Underdog .7 and Underdog 1 even exsist? Make no sense. Underdog can never score more than 50% against Overdog, cause then rating would make Underdog an Overdog.

I don't understand expected Overdog win %. First off, why it's based on average rating difference? Why not check elo ratings for each game, determine chance that overdog win, and take average of those?

Adjusting to do that right now.

I think there is point bias based on first pick, not sure if it is accounted anywhere in your calculations.

I couldn't find a good experimental value for that so it's not in the calculations right now.

Elo ratings are based on the games, what you actually check, is whether the way for calculating it is the same as the one you use.

Yeah, I'm not very well-versed in the theory of Elo. Let me adjust the sheets as needed, then. I remember proving somewhere that P(average overdog beats average underdog) is the same as average(P(overdog beats underdog)) and it turned out to be faulty, I guess. Maybe I shouldn't do this stuff at 5 AM.

How can a template Underdog .7 and Underdog 1 even exsist? Make no sense. Underdog can never score more than 50% against Overdog, cause then rating would make Underdog an Overdog.

These are hypothetical. Obviously, there's no template that simply awards the win to the lower-rated player automatically. I'm using "overdog" and "underdog" to refer to the players based on their incoming ratings; the "underdog .7" template is just used to simulate a template that's got a very high chance of upsets, just like how "overdog .7" is a template that's got a high rate of the overdog beating the underdog and "overdog 1" has no chance of upset.

Holy shit Widsisz. That solves a lot. Thanks!

Underdog .7 and Underdog 1 were also designed to be extreme examples of inconsistent templates- if you win a game, as you pointed, you're much more likely to lose the next one.

They were meant to generate extremely bad Elo ratings, to see if tainted Elo ratings like those would still be useful.

Edited 6/25/2015 13:04:15

Holy shit Widsisz. That solves a lot. Thanks!

No problem. To be honest, I don't know much about Elo either, I can be wrong. I love stats, keep up the good work!

Widsisz

>:(

Call me Wid if you can't handle few z and s properly.

^ Sorry! Still not used to names with a lot of "s"'s and "z"'s

Anyhow, I also ran a simulation of 10000 games just to see how (approximated) max league width scales (highest width at any point in the league, that's why you see Underdog .7 and Underdog 1 plateau when the league's instantaneous width is actually decreasing/fluctuating).

PerfStrat only:


All templates:


Lottery only:


Overdog 1 only:


Underdog 1 only:


Overdog .7 only:


Underdog .7 only:


Also, the distribution of player scores after running perfstrat for 10k games:


Edited 6/25/2015 16:20:24

Can you fit rating distribution for each seasonal ladder to a Gaussian and rank them by width of the distribution?

Good idea. It would be the same if I were to just rank them by standard deviation or variance, right? Since that's all I would need to describe the width of a Gaussian. Or is there something else involved?

Yeah, you can just rank them by standard deviation.

Bump.

I've been thinking about why some graphs have linear corelation, and why "Points Distribution" look so weird.
So, I concluded that after more games, the point difference between two players tends to grow - which would mean one of them is getting better (his win % chance grow), and the other is getting worse, is that correct? Or does it mean they are simply ranked more precisely? What I'm saying is, shouldn't the scores stabilize after some time?

For 1vs1 ladder, max score and min score don't differ that much compared to the past, while definitely we have more games now (in the same time interval). Why? Is this because the games expire, and number of games per player is similar to what is used to be?

How would scores for 1vs1 ladder look without game expiration? Or at least with more months? Can someone check that for me pretty please?