With highscore I ment: http://md-ladder.cloudapp.net/allplayers

Do you mean you want to see something different on that page?

Or do you calculate new elo based on the not expired games (which would effect lots of other players)?

Yes. Every 2 hours, I recalculate Elo based on unexpired games from scratch(assuming everyone is rated 1500).

Edited 11/12/2016 17:05:44

With highscore I ment: http://md-ladder.cloudapp.net/allplayers

Do you mean you want to see something different on that page?

Would just be nice to have a column of clan icons and being able to sort players by that. Always nice to know which player belongs to which clan and rate clan activity (e.g. for those who are searching).

Or do you calculate new elo based on the not expired games (which would effect lots of other players)?

Yes. Every 2 hours, I recalculate Elo based on unexpired games from scratch(assuming everyone is rated 1500).

I am no expert of the elo-system, but this may result in the following:

automatism: players tend towards 1500 elo
a) I need to play a lot of games to get a high elo.
b) Players, who do not play a lot of games, may not be rated accurate.

At the one hand a) makes sure, only active players are ranked high. On the other hand activity may substitute skill-level in some way. Not sure how big this effect is, but it may depend on how big the player pool is.

No big problem right now, but a thing one should observe in the future. It also may take some time until the system is really balanced (as all started with 1500 elo).

I see no reason to modify elo with expired games, elo only modifies taking into account the opponent's elo (and yours) in the game you are currently playing, you can forget about old games and your rating will still increase/decrease according to your current skill level. I don't see any benefit for it, maybe me being a chess players makes me like that system ^^U

automatism: players tend towards 1500 elo
a) I need to play a lot of games to get a high elo.
b) Players, who do not play a lot of games, may not be rated accurate.

At the one hand a) makes sure, only active players are ranked high. On the other hand activity may substitute skill-level in some way. Not sure how big this effect is, but it may depend on how big the player pool is.

No big problem right now, but a thing one should observe in the future. It also may take some time until the system is really balanced (as all started with 1500 elo).

Mostly agree with all your points. The player pool is steady at the moment and has around 75-80 active players. If you have few games, your rating maybe inaccurate. But this is true of all rating systems in my opinion.

It is not guaranteed that activity will give you a high rating though, as every additional game you play can drag you down or boost you up. I think active players will be closer to their "true" rating. You can reach a 1600-1700 Elo if you are semi-active though with a good streak of 10 games. But to reach higher ratings, you have to consistently perform.

---

To address the problem of ratings tending towards 1500(assuming the average win rate is 50-50), I'm contemplating the introduction of a new element to the rating system. If we agree that playing more games leads to players reaching their true Elo, I need to provide incentives to stay active.

I was thinking of adding a "rating boost" per game(say +3 or +4) played, up to a certain number of games. This concept is similar to how the seasonal ladder works, where you get bonus points for playing a sufficient number of games.

However, I need to cap this to a certain game count as I don't want someone with 200 active games getting a boost of 200*4 = 800 points! I'm leaning towards capping the game count at about 40(so max rating boost = 40*4=160). This boost is obviously an artificial boost used only for ranking. It is not used by the matchmaking system when looking for possible opponents.

This also discourages players from making "runs" as you see on all other WL ladders. Runs lead to inflated ratings and points "leaving" the system which is obviously a bad thing.

I'd love to hear other opinions on the pros and cons of this approach, before I make sweeping changes to the rating system.

Edited 11/12/2016 17:52:04

As I said, if you don't adjust elo from expired games, it could solve some problems, for example the "runs", even if you leave your elo is saved, you will start with the one you had. If you want to avoid a player having a big initial ranking after the initial 20 games, make a lower K for those games (16?) and change it latter, after you have an official ranking.
Changing K-values also help with activity problems (some of them pointed by Krunx), you can change the K-value depending on the activity. For example, in the regional elo system we use in catalonia, all K-values have that (with K=16 and the activity modification is +-3, so same level players can have from 13 to 19 depending on activity). Not sure if a variable K it's easy or not to implement.

If you don't like not expiring games because you don't one someone who don't play anymore on the ranking, make a "if you don't play for 6 month, your elo is lost" but, to avoid runs and so, if they return they will start with ana ctual elo and no the provisional 1500. The starting elo could be "old ranking - 200" or something like that, that way you will start playing against players not that far from you skill level, and if you still have it, recover it quick. Not sure of how to implement it, maybe it's difficult. But with the initial-K correction, maybe this is not necessary.

Edited 11/12/2016 19:17:21

As I said, if you don't adjust elo from expired games, it could solve some problems, for example the "runs", even if you leave your elo is saved, you will start with the one you had.

While I agree with this, the problem is that runs usually tend to be on a new accounts. So the Elo is never saved for such accounts. Variable K does help with this though.

Another problem with having all games included is that it takes a long time! We currently have about 386 games finished in about 2 weeks. Projecting this out, I don't like the idea of computing Elo after one year with so many games. Expiring games improves running time of the algorithm too.

One potential fix to the long running time, is to update the previously computed Elo using the recent games, but this has other drawbacks. If there is a bug in the system and I need to make a game disappear, it is hard to do as the previous rating will need correction. On the other hand, running a fresh Elo computation every time, does not have this problem.

If you want to avoid a player having a big initial ranking after the initial 20 games, make a lower K for those games (16?) and change it latter, after you have an official ranking.
Changing K-values also help with activity problems (some of them pointed by Krunx), you can change the K-value depending on the activity. For example, in the regional elo system we use in catalonia, all K-values have that (with K=16 and the activity modification is +-3, so same level players can have from 13 to 19 depending on activity). Not sure if a variable K it's easy or not to implement.

This sounds like a good addition to the system. Could you elaborate on how the K varies by activity? Would it be something like if

0  < gamecount < 10, K= 32 - 15 = 17
10 < gamecount < 20, K= 32 - 10 = 22
20 < gamecount < 30, K= 32 - 5 = 27
30 < gamecount < 40, K= 32

Some updates:

• Added clan icons
  Icons for clans and players' clan affiliations are updated once a day. It may take up to 24 hours for changes to reflect on the CLOT. If a player joins a new clan, leaving and rejoining the CLOT will update their clan tag as well.
  
  
  
  
• Updates to player page.
  The veto menu is hidden by default and can be viewed by clicking the "Change Templates" button. There is also a gaping hole on the right of the page. Wonder why ;)
  
  
  

Edited 11/12/2016 20:46:30

This sounds like a good addition to the system. Could you elaborate on how the K varies by activity?

In fact it's the opposite, the more activity the lower the K.
For the initial games I would use:
- If you can limite the max games to 5 --> k=16
- If you can't change the limit only for initial games --> if nº of games <6 then k=16 else K=10

For normal games, after you have a rating:
A = (finished games in last 30 days)/30 = average finished games per day during 30 days
B = (finished games in last 90 days)/90 = average finished games per day during 90 days
C = (2A+B)/3 <-- Activity
If the player has less than 90 active days, C = total games / total days

I though about A for 60 days and B for 180 but...maybe too long?

And then modify the K with C into account:
K = 32 - 5C

This is an example, not sure what's the normal activity...for example, MotD has 54 games in half a month (roughtly), so 100 a month, but he has a lot of activity, I don't think that's normal :P That would be A=3.33. If he doesn't lower the activity, K=15. Unlikely someone will have lower K's than that.

I have finished 11 games in 12 days, let's say A = 1. Playing 3 games at a time but playing quick, not sure if this is "normal". If I keep that activity my K is 27, not much variation.
If we consider my activity "normal" then it's ok, if we consider it low or high the formula should be reviewed. It's a very simple one, obviously it can be improved, but I like the activity formula ;)



Another problem with having all games included is that it takes a long time! We currently have about 386 games finished in about 2 weeks. Projecting this out, I don't like the idea of computing Elo after one year with so many games. Expiring games improves running time of the algorithm too.

You use all the games to compute elo? You only need the last ones. Maybe the computer check all of them to see which ones finished since last update? In that case, if you have a "expire variable" to avoid expired games, change it to "checked games" and the computer only look for "not checked ones" and:
Elo = current elo +"not checked games elo"

We need to think of something to improve this because the expiring games will end in people losing elo even winning if they or the others lower activity (and not all people will be super active continuosly).

P.D. Thanks for your hard work!

Edited 11/12/2016 21:38:17

Ah. A lower K for active players ensures lesser fluctuation, so I see why you are recommending it for the active players. But doesn't this encourage runs? Once you have 100 games, no matter what you do, your rating is very stable and players feel like they're not making progress even though they are winning.

On the other hand, you can get an inflated rating by playing just 20 games. If we adopt such a system, I feel like we need to encourage activity too(which is why I was suggesting adopting the "bonus points" used by the seasonal ladder.)

We can tweak the constants in the formula after observing the ladder for a month or two.

You use all the games to compute elo? You only need the last ones. Maybe the computer check all of them to see which ones finished since last update? In that case, if you have a "expire variable" to avoid expired games, change it to "checked games" and the computer only look for "not checked ones" and:
Elo = current elo +"not checked games elo"

Yes, I use all games. The reason for this is

One potential fix to the long running time, is to update the previously computed Elo using the recent games, but this has other drawbacks. If there is a bug in the system and I need to make a game disappear, it is hard to do as the previous rating will need correction. On the other hand, running a fresh Elo computation every time, does not have this problem.

Yes it encourages runs. But it is also more accurate, since a rating system know less about a player who plays less games. :(

Online chess has had this problem too, that players makes new account to get a better rating. Some site solved this by being very strict on alts (particularly on top/leaderbords players) The goal is to get like RL chess, where you...only have one human/account, thus only one rating. That means banning anyone who uses more then 1 account from the ladder :/

Activity/elo-system:

Personally I would recommend to adjust the K-value dependent on activity like Memele mentioned. But I would not do it linear, as then there would be no appeal to play 9 games in parallel instead of 1. In addition to that set a minimum of games per month to show up in rankings.
In theory the elo-value of a player is more accurate.

In praxis you are right, inflation by activity drifts players towards playing more games. This is a decision one has to make: having the most exact rating system showing skill or having more games. This is a general decision the creator of the ranking system faces. And it is a decision that will influence the behave of the players of the multi-day ladder fundamentally. Through my eyes the ranking system is one of the bigest set screws of the developer of a game/ladder.

I do not have a deep knowledge of the history of the warlight ladder, but through my eyes the current rating system of the ladder is one major reason why people do "ladder runs" and then leave the ladder.

start-value/ladder-runs:
Thorugh my eyes not starting with a fixed value has advantages, but one has to ensure that noone uses alts. I do not know how complex this problem (api data) is in this case.

Once you have 100 games, no matter what you do, your rating is very stable and players feel like they're not making progress even though they are winning.

If you use a good correction, this should no happen. For example, with the one I said earlier a VERY high activity means k=15, that's still high enough. For comparison, FIDE uses 20 for almost all people...
The only problem would be having much more elo than your rivals, but that's not something that can't be solved easily it happens in chess (Carlsen has problems to reach 2900 because of lacking of +2800 rivals).

On the other hand, you can get an inflated rating by playing just 20 games.

If the initial K is low, you could inflate it a bit, but nothing too exagerated.


One potential fix to the long running time, is to update the previously computed Elo using the recent games, but this has other drawbacks. If there is a bug in the system and I need to make a game disappear, it is hard to do as the previous rating will need correction. On the other hand, running a fresh Elo computation every time, does not have this problem.

This can be solved modifying it manually if an error appears but that's most work for you ^^U
The erros are mostly people not joining, righ? If that's the case, there is some way to make elo variation being 0 in those games?

@krunx

Personally I would recommend to adjust the K-value dependent on activity like Memele mentioned. But I would not do it linear, as then there would be no appeal to play 9 games in parallel instead of 1.

Yes. This is my main worry. The proposals so far encourage players to play 1 at a time as opposed to 9.

In addition to that set a minimum of games per month to show up in rankings.

This is a bit tricky. While I agree with your sentiment, I need to be careful that I don't disqualify too many players from attaining a rank as that discourages participation too.

This is a decision one has to make: having the most exact rating system showing skill or having more games. This is a general decision the creator of the ranking system faces. And it is a decision that will influence the behave of the players of the multi-day ladder fundamentally.

Completely agree. While we strive for accurate ratings, if it comes at the cost of reduced activity, that is not a good thing. An active player pool ensures variety in match-ups and makes the CLOT much more attractive. Activity also allows us to estimate "true Elo" more accurately. But in the worst case, I'd rather sacrifice some accuracy if it is at the cost of activity.


@Memele

If you use a good correction, this should no happen. For example, with the one I said earlier a VERY high activity means k=15, that's still high enough. For comparison, FIDE uses 20 for almost all people...

I haven't worked out the math yet. But my gut tells me that in the system proposed, it is better to go 18-2 than 90-10. Would you agree with this? On the same note, what is the objection to an artificial rating boost(just for ranking and display)? This boost will not apply to the matchmaking system or the Elo calculations. It is just an incentive to play more. It seems to work really well for the seasonal ladder where everyone ensures that they play the minimum amount of games necessary. If they do not play N games, they fall significantly behind. However, if everyone plays this number of games, there is no relative gain and we fall back to relying on true Elo to judge players.

This can be solved modifying it manually if an error appears but that's most work for you ^^U
The erros are mostly people not joining, righ? If that's the case, there is some way to make elo variation being 0 in those games?

Yeah. It can be solved manually but is a big pain to do on a large scale. The errors can be due to bugs in the system, which happens every now and then. This design is based solely on the ability to run the CLOT with minimal human intervention. However, if it leads to a broken system due to poor scalability, I will consider the alternative.


p.s - Thanks for the continued discussion. It is very helpful and will lead to better solutions.

For calculating the elo, I understand that using all games it's done to avoid bugs and so. Maybe a solution to avoid this without lots of time in calculations and the need to expire games could be:
We set a period of time, let's say 3 months (but it could be different). You have a variable, initial elo (=1500) and create an auxiliar variable, elo-1. After 1 months we save the players elo to elo-1. We do the same after 2-3 months (elo-2 and elo-3). For now 3 month passed (the time period we decided before).
One month after that we do:
initial elo = elo-1
elo-1 = elo-2
elo 2 = elo-3
elo-3 = new elo this month
After this doesn't matter if games expire, the elo value those old games had it's taken into account because of the change in the initial elo and you don't need more than 3-months-old games for the calculations.

The only problem to this is if a bug affect games more than a month old. I guessed that bugs happen to more recent games, but maybe I'm wrong. Depending on this the system cold be changed, but I hope that my idea it's understandable.

the downside of letting games expire is that it will encourage people making runs. On the other hand it will also be a motivation for people who have improved their skill. The reason why the RT ladder has so many problems with runs on alts is that it is nearly impossible to get a good ranking when you started with a bad start. So people who are new to the ladders/strategy games could get discouraged by having a bad start and never being able to compete for the top spot ever

@Ollie
But that's because WL ladders use another elo system, with normal elo a bad beggining it's not that big a deal, if you improve you will go up.

@Ollie, Yeah I agree with Memele. The concern you raised is only due to bayesian Elo and will not occur on this CLOT. An early loss is not a big deal imo.

@Memele,
Yep. I understood what you said :) I will implement it soon.

What about my other point? Do you dislike this? And if so, why?

But my gut tells me that in the system proposed, it is better to go 18-2 than 90-10. Would you agree with this? On the same note, what is the objection to an artificial rating boost(just for ranking and display)? This boost will not apply to the matchmaking system or the Elo calculations. It is just an incentive to play more. It seems to work really well for the seasonal ladder where everyone ensures that they play the minimum amount of games necessary. If they do not play N games, they fall significantly behind. However, if everyone plays this number of games, there is no relative gain and we fall back to relying on true Elo to judge players.

ahh i didn't know that. I never worked with the elo rating systems. Forget about what i said :)

What about my other point? Do you dislike this? And if so, why?
The more the games the more accurate the elo, so 18-2 would be better than 90-10 depending of how reliable the initial calculation is. That's why I propose a lower K for the beggining, to avoid this impact. This shouldn't discourage players because when they take the normal K, they should be able to adjust their elo (if not close enough to their true level) relatively fast. In fact, there is no really that big of a deal with the artificial initial boost but the "runs". For a normal players who doesn't care about that, after the initial elo, they would reach their true elo eventually, being it lower or higher to the initial one after the 20-games. The only problem is that if it's a system than let you take a high elo "easy" it will encourage runs.
For the variable K, there is no need to change it a lot, or maybe to change it at all (after the initial games). In the chess example I used the elo is calculated per torunaments and not games so the impact it's bigger, maybe, going game by game it's good enough like it is, like in online chess. But, as I said before, it could still be good for the initial ranking.

What about my other point? Do you dislike this? And if so, why?

The more the games the more accurate the elo, so 18-2 would be better than 90-10 depending of how reliable the initial calculation is.

Through my eyes the K-factor only depends on the number of games one is able to make in a period of time. So I would compare the number of games per player per month in warlight with the number of games in chess and intialize the K-factor to that value.

There are also some psychological factors:
1st: How much luck is involved in a warlight game? It is frustrating to loose a game and loose a lot of points only because you had bad luck. Just imagine the following situation:
A, elo 2000; B, elo 1900 => A wins 2 out of 3 games (if I am right); But lets imagine in the game there will appear (with a very high probability) a situation where winning chances are 50%/50%; Than A will loose most probably loose elo against B.
2nd: Someone who plays activly, usally wants to see his elo increasing. But sometimes there are some barriers and he wont do it that fast. This is sometimes very demotivating.

I am not sure how, but I would definetly try to push activity activly and keep players motivated. Maybe a small inflation of the system is wanted as it will push activity.

One also has to think about the time it takes to balance the elo-system in the ladder. Wouldnt a higher K-factor lead to a faster balancing (I know it wouldnt be that accurate at the start, but at least one would see different elo-numbers)?

Edited 11/15/2016 23:52:59