The Orioles bullpen from 2014-2015: Still a strength

Welcome to the third and final installment of: What Happened to the 2014-2015 Orioles?" If you're just joining us, Part 1 looked at the team's offense and defense and Part 2 looked at the team's starting pitching (and by necessity, defense). In Part 3 today I'll examine the team's bullpen.

You can't talk about a bullpen without talking about context; that is, the score of the game, the number of runners on base, and the number of outs remaining for your team. Relievers are often called upon on a moment's notice to get out of a jam into which the starting pitcher, or another reliever, has placed the team. They have no control over when they are inserted but must focus on stranding inherited runners, protecting a lead, and so on. Occasionally they are called upon to preserve a close game and sometimes they're pulled into mop-up duty. It's part of a manager's job to pick the right reliever for the right situation, such as a sinkerballer when a double play is needed or a flamethrower when a strikeout is needed.

We'll get to context later. For now it's enough to say that the 2015 bullpen has improved on the 2014 version by nearly every context-independent metric there is. Two pitchers in particular, Zach Britton and Darren O'Day, have stepped up their game, and newcomer Mychal Givens has flashed some impressive fielding-independent numbers. Let's start with my favorite metric, FIP-:

• 2014: 91 (ranked 10th)
• 2015: 82 (ranked 1st)

That's right. When you factor in the park the Orioles play in, the 2015 bullpen is the best in the game when it comes to striking out batters while limiting walks and home runs. This, despite the fact that whenever Brian Matusz gives up a long ball or Britton blows a save, the fanbase gets a queasy feeling in its collective gut. This is why it's important to look at the long term.

Let's delve into the components of FIP to see what's going on in detail:

K%:

• 2014: 21.2% (ranked 21st)
• 2015: 24.3% (ranked 5th)

BB%:

• 2014: 7.6% (ranked 6th)
• 8.7% (ranked 14th)

K-BB%:

• 2014: 13.6% (ranked 17th)
• 2015: 15.7% (ranked 6th)

As suggested by the FIP- improvements, by the K-BB% metric the bullpen is far more effective this year than last year. Given what I showed about the team's defensive decline in Part 1 of this series, an increased strikeout rate is very important for the Orioles, especially considering that the bullpen is walking more batters overall. Here are the most improved relievers (with at least 20 IP) by K-BB% year-over-year:

• Britton: +12.1 percentage points
• O'Day: +6.7pp
• Brad Brach: +5.5pp

Britton has really elevated his game. His strike out rate is up nearly 10 pp while his walk rate is down by about 2.5 pp. This, while his groundball rate has increased by more than 2 pp. Simply amazing. This is not to take away from O'Day and Brach who have made improvements as well. I should note that Matusz has also improved his K-BB rate by 1.8pp to a well-above-average 17.7%.

The club has also gained an effective reliever in Mychal Givens. His 26.7 K-BB% gives the team an effective late-inning weapon and is ranked 9th overall among relievers with at least 20 IP. He's ranked just behind super-reliever Dellin Betances and just ahead of the aforementioned Mr. O'Day. His inclusion helps compensate for the fact that both Tommy Hunter and T.J. McFarland showed decreased K-BB rates this year and that Jason Garcia has a paltry 5.7% rate. Givens also replaces the semi-talented 2014 arms of Evan Meek and Ryan Webb. Finally, Chaz Roe has brought an okay-ish 11.9% K-BB rate to the team this year.

The final component of FIP is home runs, and here the team is somewhat weird. Their HR/9 rate has gotten a tad worse but their rate relative to the league has also increased:

• 2014: 0.74 (ranked 13th)
• 2015: 0.79 (ranked 7th)

The increase in home run rate makes sense given the increase in fly ball rate:

• 2014: 30.9%
• 2015: 32.9%

It seems the Orioles are slightly fortunate to be giving up more home runs but not nearly as many as the rest of the league's bullpens. But don't conclude that they are just lucky. The metric xFIP- shows that the team's improvements are to be believed:

• 2014: 93 (ranked 9th)
• 2015: 89 (ranked 5th)

All those strikeouts are offsetting the slight increase in walks and home runs, when you factor in the park the Orioles are playing in.

So the Orioles' bullpen has, overall, improved and been a real source of strength for the team. Or have they? The metrics above are context-independent, but when examining how relievers have performed, you need to factor in context. The most useful metric here is RE24, or "run expectancy given the 24 base-out states". It this situation, it measures how many runs the bullpen has saved given the base-out states they work in. This is particularly important because relievers are often brought into situations with runners on bas. If the baserunners score on their watch, the starter's ERA goes up. But with RE24, the reliever is penalized by subtracting RE24.

Since RE24 is cumulative and we are not comparing equal time periods, I've pro-rated the stat to 162 games. The values are as follows:

• 2014: 0.228 runs saved above average per 162 games
• 2015: 0.133 runs saved above average per 162 games

By looking at RE24 we find that the 2015 bullpen has been slightly worse than in 2014. I suspect that more runners, inherited from starters (or even other relievers), are crossing the plate than last year. This would be yet another reason why the 2015 Orioles' starters ERAs are higher than they are last year. But RE24 shows that the relievers share some of the blame. I would also hypothesize that since the starting pitching and defense have declined, there are overall more situations in 2015 in which relievers enter with runners on base than in 2014.

The difference overall equals 15.39 runs or about 1.6 wins over the course of a season. That's a literal amount of wins, but relative to the league the drop is not that bad. 2014's RE24 value ranked 5th; this year's ranked 7th. This makes sense given the absolute, context-independent performance as measured by FIP- described above.

Another usual context-dependent metric is Win Probability Added (WPA). If you look at this metric, the bullpen appears to be doing much worse than last year's:

• 2014: 6.24 (ranked 3rd)
• 2015: 3.54 (ranked 7th)

Yes, this metric is cumulative and the periods of time comparison aren't equal. But there are only a handful of games left in the season. What gives? Why has the bullpen's WPA dropped so much?

The key is not the bullpen's performance, but the leverage of the situations in which they're being asked to pitch. The former has remained steady but the latter has dropped a lot. To see this, let's remove leverage from the equation by comparing WPA/LI:

• 2014: 2.54 (ranked 9th)
• 2015: 2.00 (ranked 8th)

This shows that, absent any differences in leverage, the bullpen has been about the same, if not a bit better than last year.

Now let's focus solely on leverage by looking at gmLI, which is the game's leverage in which the reliever enters the game:

• 2014: 1.27 (ranked 4th-highest)
• 2015: 1.04 (ranked 24th-highest)

That's quite a disparity and shows, on average, the 2015 bullpen is being asked to work more in the following kinds of situations:

• The Orioles are already well ahead due to a lot of runs scored
• The game is still in the early innings, which also means the Orioles are likely well behind, since a starting pitcher would only get pulled if they and the defense allowed a lot of runs early on

The only point I can prove is the second one. In 2014 relievers entered after 5.8 innings of work by a starter. In 2015 they are entering after just 5.3 innings. Since they are entering the game earlier, there is more opportunity for the team to score runs, thus the leverage is lower. The leverage is also lower because a reliever entering the game indicates the starting pitcher wasn't being effective for as long, meaning the team is likely behind by more runs than when a reliever entered in 2014. You could even argue that since the bullpen is working for longer each game, they should be given more credit than the 2014 bullpen.

So, despite being put into fewer important situations than last year, the 2015 Orioles bullpen has been quite effective.

All data from FanGraphs prior to the 9/26 game.