clock menu more-arrow no yes mobile

Filed under:

Evaluating the Orioles bullpen usage using Leverage Index

Has Buck put in the best pitchers at the most crucial situations?

Matthew Emmons-USA TODAY Sports

If there is one way a manager can change the odds of his team winning significantly, it's bullpen usage. Lineup construction has proven to be a matter of a few runs at most over a season. Bullpen management, however, has an amplified effect on a team's winning percentage due to the increased leverage at the end of games. Buck Showalter has a reputation as a skilled manager of the bullpen, at least among non-Orioles fans. Let's see how he fares so far this season.

Leverage index is a measure of how important a particular situation is in a baseball game depending on the inning, score, outs, and number of players on base, created by Tom Tango. An average (or neutral) LI is 1. High leverage is 1.5 and above, and low leverage is below 1. 10% of all real game situations have a LI greater than 2, while 60% have a LI less than 1.

First, the leverage index as the relievers enter the game plotted against their ERA. I remove Miguel Gonzalez from the data for obvious reasons.


You see a modest negative trend, which is what we want to see. We want the pitchers with the lowest ERA to pitch in situations with the highest leverage. However, you still see some pitchers with ERA over 6 pitching in important situations. One is Tommy Hunter, the O's demoted and now injured closer, as you would expect. The other is Evan Meek. I am not even attempting to understand what Buck had in mind with Meek, who had been DFA'd since. Of course, early season ERA is subject to a large amount of random variation, especially in the 20 odd innings for relievers. A more precise exercise would be to plot the leverage index versus the pitcher's true talent, which we can best estimate using projection systems. Below is the same graph except with ERA replaced by average projected ERA by Steamer and ZiPS at the season's start.


A stronger relationship is detected, though most of it stems from Josh Stinson's projected ERA of 5.6 and leverage index of 0.18. Once Stinson is removed from the sample, there is essentially no correlation between projected ERA and leverage index. One thing to note is that Zach Britton is projected as a starter, and T.J. McFarland as a part-time starter. Adjusting Britton's and McFarland's projected ERA would make the relationship stronger, but still not anywhere close to the relationship I expected before doing the study.

Yet somehow despite Buck's management, the O's bullpen has collectively pitched better in the clutch. They have accrued 2.02 WPA, or Win Probability Added, sixth in the league, while having a middle-of-the-pack ERA and the third worst FIP. Simply, the O's bullpen has been clutch, though the pitchers pitching in those clutch situations haven't been the O's best relievers.