Evaluating the Orioles bullpen usage using Leverage Index

Matthew Emmons-USA TODAY Sports

Has Buck put in the best pitchers at the most crucial situations?

If there is one way a manager can change the odds of his team winning significantly, it's bullpen usage. Lineup construction has proven to be a matter of a few runs at most over a season. Bullpen management, however, has an amplified effect on a team's winning percentage due to the increased leverage at the end of games. Buck Showalter has a reputation as a skilled manager of the bullpen, at least among non-Orioles fans. Let's see how he fares so far this season.

Leverage index is a measure of how important a particular situation is in a baseball game depending on the inning, score, outs, and number of players on base, created by Tom Tango. An average (or neutral) LI is 1. High leverage is 1.5 and above, and low leverage is below 1. 10% of all real game situations have a LI greater than 2, while 60% have a LI less than 1.

First, the leverage index as the relievers enter the game plotted against their ERA. I remove Miguel Gonzalez from the data for obvious reasons.


You see a modest negative trend, which is what we want to see. We want the pitchers with the lowest ERA to pitch in situations with the highest leverage. However, you still see some pitchers with ERA over 6 pitching in important situations. One is Tommy Hunter, the O's demoted and now injured closer, as you would expect. The other is Evan Meek. I am not even attempting to understand what Buck had in mind with Meek, who had been DFA'd since. Of course, early season ERA is subject to a large amount of random variation, especially in the 20 odd innings for relievers. A more precise exercise would be to plot the leverage index versus the pitcher's true talent, which we can best estimate using projection systems. Below is the same graph except with ERA replaced by average projected ERA by Steamer and ZiPS at the season's start.


A stronger relationship is detected, though most of it stems from Josh Stinson's projected ERA of 5.6 and leverage index of 0.18. Once Stinson is removed from the sample, there is essentially no correlation between projected ERA and leverage index. One thing to note is that Zach Britton is projected as a starter, and T.J. McFarland as a part-time starter. Adjusting Britton's and McFarland's projected ERA would make the relationship stronger, but still not anywhere close to the relationship I expected before doing the study.

Yet somehow despite Buck's management, the O's bullpen has collectively pitched better in the clutch. They have accrued 2.02 WPA, or Win Probability Added, sixth in the league, while having a middle-of-the-pack ERA and the third worst FIP. Simply, the O's bullpen has been clutch, though the pitchers pitching in those clutch situations haven't been the O's best relievers.

Log In Sign Up

Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Camden Chat

You must be a member of Camden Chat to participate.

We have our own Community Guidelines at Camden Chat. You should read them.

Join Camden Chat

You must be a member of Camden Chat to participate.

We have our own Community Guidelines at Camden Chat. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.