Jump to content


Photo

Proposal Log Discussion 4 - Use of Kleier Rankings


  • This topic is locked This topic is locked
36 replies to this topic

#21 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 23 March 2018 - 03:52 PM

My point is that the Kleier ranking has been accepted for some time by the ISF.
The motivation is a fairer tournament due to more evenly matched groups by eliminating a small number of discrepancies where for example - a player is much more talented than their ELO suggests. (eg playa1)
There is also a basis to seed newcomers (eg Garulfo).

If playa1 or garufalo are concerned about their ratings not reflecting their ability then they should play more games.

The ISF website is hosted by Kleier, so it would be damning if it did not use his rankings.

Other problems:

1. Tiebreak games are not counted in kleier ratings.

2. It assigns a 50 percent weight to tournaments from 2015, this is far too high.

3. The sample size is too small for a number of players for ratings to be accurate.

I am fine with using it if the methodology is published so that it can be reviewed.

Edited by malcom.jansen, 23 March 2018 - 03:57 PM.

  • Master Mind likes this
I'm in love with Stacy's mom.

#22 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 24 March 2018 - 10:00 AM

If playa1 or garufalo are concerned about their ratings not reflecting their ability then they should play more games.

The ISF website is hosted by Kleier, so it would be damning if it did not use his rankings.

Other problems:

1. Tiebreak games are not counted in kleier ratings.

2. It assigns a 50 percent weight to tournaments from 2015, this is far too high.

3. The sample size is too small for a number of players for ratings to be accurate.

I am fine with using it if the methodology is published so that it can be reviewed.

 

I agree with all your points 1-3. Without the methodology being published it is actually difficult to argue the pros and cons of the Kleier system in detail. ELO is also not without its issues. For example, there are no tournament games whatsoever that contribute to a player's ELO or Max ELO - yet we are using it for seeding because it is what we have. So if you are arguing that Kleier shouldn't be used (because it is not verifiable), perhaps you should also be arguing that ELO should not be used because it has no basis in tournament results.

 

A verifiable TC ranking would be ideal, but would also pose a formidable challenge to create. 



#23 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 24 March 2018 - 03:00 PM

I agree with all your points 1-3. Without the methodology being published it is actually difficult to argue the pros and cons of the Kleier system in detail. ELO is also not without its issues. For example, there are no tournament games whatsoever that contribute to a player's ELO or Max ELO - yet we are using it for seeding because it is what we have. So if you are arguing that Kleier shouldn't be used (because it is not verifiable), perhaps you should also be arguing that ELO should not be used because it has no basis in tournament results.

A verifiable TC ranking would be ideal, but would also pose a formidable challenge to create.

ELO does not measure tournament results, but this is not a problem. Ranked games do not differ from tournaments other than double chasing being allowed. Double chasing is an issue in very few games, so site ELO is an appropriate proxy.

ELO does not model Stratego results perfectly and there is a significant amount of cheating. However, for nearly every player you are going to have a decent idea of relative ability after 10 wins and losses. You could probably construct a 95 percent confidence interval of plus or minus 75 points as being someone's true relative ability.

Max ELO is not the best metric since you are measuring a player's peak. However, it is probably within the upper range of most player's confidence interval and you do not want to encourage rating sitting, so it is fine.

Edited by malcom.jansen, 24 March 2018 - 03:34 PM.

I'm in love with Stacy's mom.

#24 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 27 March 2018 - 10:37 PM

I have discussed this further and would like to offer an additional proposal:

 

Any player who has won multiple tournaments will be assumed to have a minimum Max ELO of 1000.

 

With this rule, the TC will not have to rely on a ranking system without documentation and the TC will be able to properly seed someone like playa1.


I'm in love with Stacy's mom.

#25 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 06 April 2018 - 04:11 PM

TC has now voted on this proposal.

 

Result: 3-1 (Against)

 

Kleier-converted ELO will be used in the upcoming Champions League to better seed the small number of players whose Kleier Ranking is significantly better than their MAX ELO. TC will contact Mr. Kleier to attain information about the ranking calculations to be able to make a more informed decision in the future.


  • KARAISKAKIS likes this

#26 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 06 April 2018 - 04:18 PM

 

TC has now voted on this proposal.

 

Result: 3-1 (Against)

 

Kleier-converted ELO will be used in the upcoming Champions League to better seed the small number of players whose Kleier Ranking is significantly better than their MAX ELO. TC will contact Mr. Kleier to attain information about the ranking calculations to be able to make a more informed decision in the future.

 

I expect my Kleier converted ELO of 9000 to be used or for my calculation to be refuted before the tournament starts.


Edited by astros, 06 April 2018 - 04:19 PM.

I'm in love with Stacy's mom.

#27 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 06 April 2018 - 04:22 PM

I expect my Kleier converted ELO of 9000 to be used or for my calculation to be refuted before the tournament starts.

 

Sure.

 

Your Kleier ranking is 1453, therefore your Kleier converted-ELO is   

 

(1453/2)+140 = 866.5



#28 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 06 April 2018 - 04:24 PM

Sure.

 

Your Kleier ranking is 1453, therefore your Kleier converted-ELO is   

 

(1453/2)+140 = 866.5

My Kleier rating is actually 9000 (his value is wrong), you do not need to verify my calculations, just take my word for it. This is what you guys are effectively doing by using an undocumented system.


I'm in love with Stacy's mom.

#29 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 06 April 2018 - 05:34 PM

This website does not provide game logs, so the same can be said of ELO. We cannot verify those ratings either. In fact, many players often say that their ELO is wrongly calculated after a game.

The Kleier ranking - from an observational perspective at least - doesn't appear to be throwing out spurious results. Players are more or less ranked in the same approximate order. There is a big difference between not using something because you cannot verify it, and not using something because it isn't fit for purpose.

In terms of seeding a very small number of players more fairly (observationally speaking) KLEIER is in my opinion fit for this purpose.

#30 Master Mind

Master Mind

    Major

  • Tournament Manager
  • 1,174 posts
  • Coat of arms
  • Platinum General

Posted 06 April 2018 - 05:36 PM

Also, various players already pointed out you can ask Hermann Kleier for the documentation if you want to see if it is really based on stratego results instead of turnips.



#31 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 06 April 2018 - 05:41 PM

Also, various players already pointed out you can ask Hermann Kleier for the documentation if you want to see if it is really based on stratego results instead of turnips.

I have and never got a response...


I'm in love with Stacy's mom.

#32 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 06 April 2018 - 05:42 PM

This website does not provide game logs, so the same can be said of ELO. We cannot verify those ratings either. In fact, many players often say that their ELO is wrongly calculated after a game.

The Kleier ranking - from an observational perspective at least - doesn't appear to be throwing out spurious results. Players are more or less ranked in the same approximate order. There is a big difference between not using something because you cannot verify it, and not using something because it isn't fit for purpose.

In terms of seeding a very small number of players more fairly (observationally speaking) KLEIER is in my opinion fit for this purpose.

The underlying methodology behind ELO is understood, the same cannot be said for Kleier rankings. It is not necessarily a terrible ranking, but you cannot backup its validity. Additionally, as I pointed out and The Prof agreed, the sample size for tournament games is too small.

 

I am not going to continue to argue this, but this is a silly decision.


Edited by astros, 06 April 2018 - 05:44 PM.

I'm in love with Stacy's mom.

#33 astros

astros

    Stratego TM

  • WC Online Team
  • 768 posts
  • Coat of arms
  • Platinum Colonel

Posted 11 April 2018 - 04:06 PM

TC has now voted on this proposal.
 
Result: 3-1 (Against)
 
Kleier-converted ELO will be used in the upcoming Champions League to better seed the small number of players whose Kleier Ranking is significantly better than their MAX ELO. TC will contact Mr. Kleier to attain information about the ranking calculations to be able to make a more informed decision in the future.

  

Also, various players already pointed out you can ask Hermann Kleier for the documentation if you want to see if it is really based on stratego results instead of turnips.


Can the TC please share their findings about the methodology behind the Kleier system?
I'm in love with Stacy's mom.

#34 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 11 April 2018 - 04:48 PM

  
Can the TC please share their findings about the methodology behind the Kleier system?

 

 

Mr Kleier has provided the following methodology for the Kleier Ranking:

 

-----

In short: It’s a probabilistic rating based on Zermelo’s 88 year old article [Zermelo, E. Math Z (1929) 29:436.  https://doi.org/10.1007/BF01180541]. Elo’s merit was to make a  very simplified recipe out of it. The bonus was that it could be calculated on a pocket calculator and yielded numbers that will approach reasonable ratings “under good conditions”. In general, however, his results are meaningless. The rating I use is a measure of the playing strength.

Currently my rating can be characterised as follows:

1 The probability function used, is for the “last error” game. An example of these kind of games is the Nim game. Elo uses a ”mean error” game (like coin flipping).

2 I use a physical time scale with a 2.5 years window. Elo uses individual time scales. The clock for a busy Elo-player runs faster. I contrast to Elo I consider all games between rated players.

3 It requires no more than three players to prove that a rating number cannot be calculated in general. The remedy is to eliminate “overloaded bridgers”. Unfortunately, this is anything but trivial and will still take further investigation. During January / February I wasted about 200 hours and 500 € in trying to improve the algorithm to no avail. This issue is still work under progress.

I started a documentation about the algorithm but it hasn’t been finished yet.

 

------

 

TC will discuss the implications of this response and whether to use KLEIER going forwards.



#35 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 12 April 2018 - 08:24 PM

TC have voted unanimously to discontinue using KLEIER going forward for seedings purposes. 



#36 Don_Homer

Don_Homer

    Lieutenant

  • Members
  • PipPipPipPipPipPipPip
  • 616 posts
  • Coat of arms
  • Gold Marshal

Posted 14 April 2018 - 05:26 PM

TC have voted unanimously to discontinue using KLEIER going forward for seedings purposes. 

What are the reasons? 


Molto Bene, Thats a nica Donut !


#37 TheOptician

TheOptician

    General

  • Tournament Manager
  • 2,960 posts
  • Coat of arms
  • Platinum Marshal

Posted 15 April 2018 - 11:24 AM

What are the reasons? 

 

Aside from some of the issues with the KLEIER ranking that have been discussed in this thread (and also: http://forum.strateg...?hl=alternative), the main reason to discontinue using KLEIER for seeding purposes is that there is no available methodology for the Kleier ranking (despite it being in use for 20 years) and even the creator of the ranking concedes that the algorithm requires improvement.


  • Don_Homer likes this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users