Tuesday, March 20, 2007

Rating and Fleecing GMs

One of the most discussed posts in the baseball blogosphere yesterday was from U.S.S. Mariner on the proper evaluation of trades. Two models were put forward:

1) Each trade should be evaluated on what’s known at the time. If a trade turns out much better than expected, or much worse, that shouldn’t affect our opinion of the trade.

2) Each trade should be evaluated on the results of the trade. If a trade looks like it’s an amazing rip-off, even if at the time everyone acknowledges it as such, but the victim turns out the winner due to unforseen circumstances, the victim’s still the victor.

This might even be a more profound post than you might believe. The actual issue raised by the question can be argued in circles for hours on end. GM's have to make their decisions based on the information at hand when the trade is made. We, the fans, have the opportunity to Tuesday morning quarterback (sorry for mixing my metaphors) the trades.

Two interesting questions:

1) Clearly, GMs should be held responsible for what information that they have available when completing the trade, but what information can they be expected to know, assuming due diligence?

2) Why do GMs make trades that are clearly a rip-off, everyone else seems to know are a rip-off, and actually turn out to be a rip-off?

On the first question, further advancements in quantitative analyses mean that accurately modeling future performance is becoming more and more accurate. But, the nature of the game is such that they'll never be perfect, and even the best systems leave a large amount of error in their estimations. Obviously, GMs can't be held responsible for players who suffer freak career-altering/ending injuries (meteorites falling from the sky and hitting Albert Pujols in the left knee, etc.) But, I suppose we know a thing or two about which players are more likely to suffer a game-related injury. We have an idea of which players will break out based on their statistics and which are likely to suffer a quick decline. The biggest problem is that even the best predictions (indeed, anything in statistics) are just a matter of probabilities. Perhaps GMs might be considered little more than glorified professional gamblers who, instead of playing with poker chips or stocks, trade players? It's an intriguing question: What can we properly critique GMs for knowing/not knowing?

Consider: the average GM gets to make how many trades of relevance over the course of a year? Or even over five years? Two or three per year? Suppose that the GM has a true ability of getting 60% of his trades "right" and 40% "wrong." Over five years, he makes twelve trades. According to the olde binomial distribution, he's got a 33% chance of getting half or more of them wrong! The problem here, like a lot of problems in analyzing baseball, has to do with a too-small sample.

On the second question: May I introduce to you the concept of Prospect Theory by Amos Tversky and Daniel Kahneman. Consider the following scenarios:
You are arrive at the site of a massive natural disaster and are charged with evacuating 100 people from a remote village. You have a plane capable of bearing the weight of about 65 of the people safely, but if you put more people into the plane, the chances of it making the flight to safety drop. You could probably physically fit another 20 people onto the plane, but it means that the chances of making it back to safety are about 75%, with a 25% chances of crashing and everyone dying. There isn't enough time to do two runs with the plane and no backup is available.

Do you take a) the 100% chance of saving 65 people or b) the 75% chance of saving 80 people, but the 25% chance of saving no one?

Made your decision?
Another disaster strikes. You and that same plane go to another village with 100 people, and find a similar set of circumstances.

Do you allow a) a 100% chance that 35 people will die or b) take a 25% chance of having everyone die and a 75% chance of having 20 people die.

An interesting finding. First off, you might have figured out that the two scenarios are mathematically equal to each other (although the two conditions in each are not.) Oddly enough, the same people, when presented with similar scenarios often pick opposite choices. Kahneman and Tversky's explanation: humans are reward-seeking and risk-averse. This isn't a revelation. What makes it interesting is that you can get the effects simply by framing the information in a different way. In the first incarnation, the material is presented as saving lives (a reward). Many people pick the riskier option to try to save more people. However, when the material is presented in terms of people dying (a risk), people suddenly become conservative.

Your favorite GM, therefore, might pull the trigger on a bad trade thinking only of the potential rewards (perhaps egged on by his GM trading partner). This one happens more than you might think. To go over to the NFL Draft, how often do you see a team draft a player who doesn't really fill a need based on his "upside?" and how they'd hate to have passed on what could be a "very special player." (How many of you have seen friends do the same in fantasy drafts?) How many GMs have been seduced by the promise of a "young, live arm" or a "five-tool" player? How many of them would make the same decision if they were told, "He has an 85% chance of becoming a complete failure and a 15% chance of not failing?"

The other point to be made here is that the proper course of action in both disaster scenarios is the more conservative one. Consistently picking the safe bet of 65 lives saved over and over again will save more lives in the long-run. (Admit it, you were tempted to take you chances and save extra lives, even at the risk of peril to everyone). Human beings take plenty of stupid chances that rationally make no sense. If you've ever bought a lottery ticket, you are one of them. So, a GM who is thinking only of the "upside" and does not understand the basics of probability will, on average, be taken advantage of. It's as easy as exploiting simple weaknesses in human behavior.


While I'm here:

Tango Tiger's got his dollar values for roto leagues.

No comments: