Re: Rankig of Vtes Debate


<jtduffin@xxxxxxxxx> wrote in message
news:1122325675.685913.300070@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>
> Frederick Scott wrote:
>> <jtduffin@xxxxxxxxx> wrote in message
>> news:1122150180.877691.158760@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > I don't think the ratings are totally bogus. Such as they are. Yeah,
>> > they're not perfect,but having seen them in operation for a while, I
>> > think they are pretty good. They reflect, to a large extent, who the
>> > good players are, because good players win tournaments (and that's most
>> > of what the points get awarded for).
>>
>> But how many tournaments do 'good players' win? I think the more
>> accurate statement is, "Good players tend to win the tournaments they
>> play in." Because the rating system factors in level of participation,
>> it is, indeed, totally bogus (for rating skill).
>
> If you don't play in tournaments, there's no way to evaluate whether
> you're a good player or not. Your argument here is like saying that we
> might never know who the best chess player in the world is, because he
> or she may never have actually played chess.
>
> Every rating system factors in participation. It is simply not
> possible to rate players who don't participate in the events you're
> trying to base their ratings on.


Oh, bull. That's not what this argument is about. Of course, every
rating system needs information to function. What's at issue is whether
the existance of information should _by its very existance_ count as a
positive factor in the calculation of its output.

> The *amount* of participation required by the current VTES rating
> system is hardly outlandish.

"Outlandish" is a subjective description. It clearly affects the
ratings significantly, even amongst the most highly skilled players.
And that makes them totally invalid as a description of skill.

> You can achieve the highest rating levels
> by playing in at least eight tournaments every eighteen months.

I don't believe that's true. I don't know what the calculation is for
winning one single four-player, one table, one game, final-only
tournament (I believe that is a valid tournament) but I'll bet if you
calculated it and multiplied by eight, you wouldn't be very near the
top. If so, your statement is strictly wrong - for what it's worth.

More to the point, the number and type of high-level tournaments even
the best players can get to probably differs significantly, making the
number useless as a way of understanding their skill levels. In short,
there's no point in taking this number and comparing it with other
players - either highyl-skilled-player to highly-skilled-player or
highly-skilled-player to any unknown quality player.

>> Now, it may be true that every single player who is, in fact, one of the
>> best 10 players in the world participates in tournaments on a level
>> sufficient that the participation factors are greatly reduced. I don't
>> know. I would be leary of even this, however, given that some great
>> players may participate far more in continental qualifiers and continental
>> championships than other great players. If so, I believe the former would
>> have a pretty good leg up on the latter for ratings purposes.
>
> So what? I'm not saying that the system is going to be accurate to a
> high degree of precision in distinguishing the best player in the world
> from the second-best from the third-best.

Well, what the hell is it supposed to be accurate for?!? You really have
very little information. You know it's probably hard to score as many points
as the top players have but you know nothing about:
1) If other players could score that many points given the same opportunity
levels;
2) If the top players have had the same opportunities to score points as
each other;
or 3) If it's easy or hard to even get as many opporturnities to score points
as the top players have gotten.

There's too many unknowns here to say much about what this number truly
means in terms of skill.

> In many games and sports,
> and I would say VTES is one, it's not even possible to make such
> rankings definitively, because players' results aren't consistent
> enough that you can say player X is always better than player Y who is
> always better than player Z.

Sure. But you can do a much better job than this.

> I don't think the qualifier/championship bonus is making that much
> difference - based on who attends those events - in player rankings
> either. The best you can do is get 90 extra points (for winning a
> Continental Championship) and there are only four of those to play in
> per year. You'd have to make the final table of more than one of them
> to get a really significant leg up on your compatriots in the "top
> Championship players" category.

Yea, but when you're limiting people to eight tournament results this
can make a big difference. And the tournaments differ by number of
players as well, which may turn out to be a much greater factor in
terms of the chance to put up high numbers. And I also suspect that
over eight tournaments, at least for the next dozen tournamennts or
so, additional opportunities to play again and wipe out poor chance
results is big.

I think this argument amounts to trying to divide and conquer by
individual detail. That is, the difference in the finalist points of
a singler tounament may not sound like much but when added to how the
number of participants affects ratings and how many tournaments you
can attend affects ratings, in the end, you get ratings that don't
say much about skill. You know guys who have numbers are probably
good but you don't know how good, how they compare to other good
players, or who's missing from the list.

>> I think you may be getting fooled by seeing a bunch of name players on the
>> top of the list and assuming the sytem must, therefore, by 'working' in
>> some form. But it isn't working if better players are actually getting
>> left off the list because of participation factors. An example would be
>> Brian Moritz, a past North American Continental Champion. He told me there
>> there was no NW Regional Championship this year. It had been scheduled but
>> its planning was fumbled and too few people showed up to have an official
>> tournament. He blew the competition away at the 18-player SW Regional
>> Qualifier, possibly one of the smallest ones in North America. (I don't
>> know, but from the numbers I've seen, it appears to be a small one.) I
>> don't know if he's attended any other regional qualifiers or any non-
>> qualifier tournaments of any note. His rating is now 479, not high enough
>> for him to be ranked in the top 100. I REALLY doubt that's anything like
>> an accurate ranking, myself.
>
> Why? You're sure that there are fewer than 125 players as good as Mr.
> Moritz in the world? (He's ranked #126 in the world in constructed at
> the moment.)

I'm not. I don't know anything at all exactly "for sure" about anyone's
skill level any more than you do. But I'm just very suspicious there aren't
and I'm very suspicious that the reason is the numbers are bogus for exactly
the reasons I'm telling you they're bogus: the system doesn't correct for
unequal opportunity to attend tournaments. And Brian Moritz seems likely
to be a case-study of a player who's likely to be very good but doesn't have
anywhere NEAR the same opportunity to score points under this system as,
for instance, players on the U.S. east coast or in heavily populated areas of
Europe.

> Now, who those top 10 players are, and what their
> order is, definitely varies over time because of the sliding 18-month
> window - these ratings are indeed based on how well you've done at
> tournaments relatively recently. But I don't think there's anything
> significantly wrong with that.

Yea, but you're undervaluing how far off the system is likely to be.
I mean, it would be fair to say that any rocket capable of taking me
to the moon doesn't need to be capable of taking me to Alpha Centauri.
But by that standard, both the Saturn rocket and a bottle rocket fail.
Does that make them both "good enough" to get me to the moon? You
don't define acceptable by what a thing doesn't need to do that it
fails to do.

This system is a bottle rock as far as measuring skill level goes.

> Who only plays in one tournament a year? How great could such a player
> realistically be? This isn't a game you can achieve mastery of by
> living as a hermit and meditating for ten hours a day, you know. :-)

Sure. But this isn't a valid reason to connect participation in
official VEKN tournaments directly to skill level. A player might
have developed great skill playing with a few other really good
players in constant pick-up games. Why they don't go out and tear
up the VEKN tournament scene isn't important. In fact, I suspect
there are a number of chronically underrated players in Los Angeles
for just that reason. Even though there are tournaments in Los
Angeles, I suspect a player's opportunity to attend them there isn't
equal to an east coast player's opportunities. And some players
may not desire to attend many official tournaments in 18 months
yet be quite good. That's wrong if all you're measuring is skill
level. And it's wrong because it's not designed to be right (in
that sense) in the first place.

Fred


.

Quantcast