A Replacement Level Baseball Blog

Tuesday, January 22, 2008

Snake Oil Salesmen







Last year, the Arizona Diamondbacks won 90 games and the NL West with a mix of young talent (Conor Jackson, Mark Reynolds, Chris Young) and peak-aged regulars having career years (Eric Byrnes, Orlando Hudson), plus another solid season from Brandon Webb (good for an ERA+ of 156) and innings ably ate by Doug Davis and Livan Hernandez. In the postseason, the D-Backs looked pretty good running over the Cubs in the divisional series, and not terrible having the favor returned by the pick-of-destiny Rockies in the NLCS.


But to the statheads, none of this mattered as much as how definitively they beat their Pythagorean Projection. By +11 games, to be precise.


As the Snakes put up a 46-35 record to open the season (while being badly outscored) baseball analysts were falling all over themselves to explain, or explain away, this beating of the odds. Sal Baxamusa (whose real name used to be Sal Baxamusamusabaxa) at Hardball Times took this as an opportunity to rant.


When reading Sally's beef, I couldn't help but think about Dustin Hoffman's tantrums in Rain Man, especially the scene in which he mumbles "397 toothpicks, defnly defnly 397 toothpicks" over and over while gently rocking himself. Nothing upsets a left-brainer more than an outlier.




Baxamusa picks on guys like Jim Rosenthal, who made the seemingly coherent observation that the Diamondbacks tend to win their close games and lose in blowouts. Others were so audacious as to suggest that the relevant factor might be conscientious leveraging of a quality bullpen.


Mere superstition, cried the grad student. Cast it off like Biblical Creationism, the Female Orgasm, and so many other untenable myths. You see, for the sabermetrically devout, there are only numbers and noise. Any variance in independent variable X not explained by the best predictor variable(s) Y must be random. Never mind that Pizza Cutter's study at MVN (which the guy cites!) shows that the Pythagorean residuals--the difference between the winning percentage predicted by the model and the actual winning percentage, basically a measure of over/under performance--is correlated rather strongly with several of the factors Baxamusa calls "rationalizations": winning% in one-run games, offensive consistency and inconsistenty in run prevention.

Now, maybe Sal's point is just that these factors aren't representative of particular athletic or managerial "skills". But it seems that the first and last (performance in close games and inconsistency in run prevention) are rather closely related to bullpen leverage. An optimally levered bullpen is one in which the best arms pitch in the situations that have the highest impact on the outcome of the game. This is fairly obvious. What maybe isn't so obvious is that the opposite holds as well: an optimally levered bullpen is one in which the worst arms pitch in the situations with the least effect on the outcome of the game. In other words, mopup duty in blowouts.


And then there's offensive consistency. All offensive consistency means in this case is that variance in runs scored per game is clustered tightly around the mean. There are infinitely many ways to for this to happen, most of which are either random or at least unintentional and thus don't speak to any discernible "skill". Some ways of producing consistency--like managers using consistent lineups and batting orders, or GMs putting a premium on non-streaky hitters, are intentional but still don't necessarily reflect a skill (the idea of lineup/batting order optimization is to score as many runs as possible given a certain group of hitters, not to score the same number of runs as often as possible).

In the present case, it doesn't matter to me whether the D-backs offensive consistency was reflective of a skill, I'm only interested in whether it was reflective of something besides randomness. That is, whether the outcomes that account for the error between the D-Backs projected and actual wins record were more the result of choices or chance. Even the best play-by-play data can't successfully quantify all the choices made within an organization over 162 games. But statistical analysts often make the mistaking randonmness with unmeasurability.

That's all for now.

No comments: