PECOTA vs Vegas - Who's been closer to projecting O's wins?

Ah, spring. Flowers come to life, allergy season starts in earnest, and the baseball world is full of projections about the season to come.

Let's take a look at two very different visions of projecting the fate of Orioles and the other four teams in the American League East division during the past five years. One will be the PECOTA projection system used by Baseball Prospectus. According to BP,

"There are three elements to PECOTA:
1)Major-league equivalencies, to allow us to use minor-league stats to project how a player will perform in the majors;
2) Baseline forecasts, which use weighted averages and regression to the mean to produce an estimate of a player's true talent level;
3) A career-path adjustment, which incorporates information about how comparable players' stats changed over time."

The second comes to us from Las Vegas - the over/under wins total lines for each team. Those are set to attract a relatively even number of bettors on each side. As Wikipedia explains in its primer on basic over/under betting,

"The goal of a sportsbook is to have an equal number of bets on both sides of the over-under. In theory, this means that the manager could set the value at zero and then readjust based on either the number of incoming bets and/or events that influence the potential outcome. In practice, the initial value is based on both quantitative (e.g. win-loss record, average points per game, etc.) and anecdotal information (e.g. media reports, injury status of players, etc.)."

So, in the case of PECOTA, lots of historical data on every player is used to help formulate a projection of what each player will do in the coming season, and a propriety formula is used to project a win total for each team. In the case of Vegas, they're trying to find a number the betting public will find intriguing enough to lay money on both sides in relatively even amounts.

So, how have the two very different methods compared over the past 5 seasons in the American League East? Let's review...

PECOTA's projections were closer in two of five cases in 2010, with Vegas was closer with the Orioles and the Yankees, and Red Sox. There will be a trend developing in that regard. The Vegas odds and PECOTA were virtually identical Boston and Toronto. That will occur less often than you might think over the past half-decade. One interesting note - PECOTA vastly over-projected the Orioles' win total, to the tune of 13 games, while Vegas bettors came in with a line 8.5 games too high. It won't be the last time.

Almost a clean sweep for Sin City, but the bettors in Vegas were just a little too optimistic about the Yankees' chances. And once again, PECOTA over-projects the Orioles' win total by 13 games, while Vegas set a total that was 7.5 games too high. So, yeah, whenever I said PECTOA hates the O's, I obviously meant from 2012 on.

PECOTA adjusted its win total for the Orioles down from the previous year, and Vegas even more so. Both were spectacularly far from the actual result, but then again, everyone but a certain Camden Chatter was, and even Dan O'Hare was under by three games. And spoiler alert - this is the only year we review where PECOTA was closer on Baltimore than the Vegas line. Vegas was closer on three of the five AL East teams

It wasn't just PECOTA that didn't believe in the O's repeating the magic of 2012 - Vegas bettors had a win total line that was just 4.5 games above the math projection. And again, Vegas was closer on three of the five AL East teams.

PECOTA gave the O's a four-game increase in expected wins, while Vegas did the same. Both were wrong, again, about how much the O's would improve from 2012. Both models expected better from Boston, and there was no more than a 2.5 game variance in expected wins in the other three AL East teams. The two systems ended in a 2-2 tie, with both predicting the same 88 wins for the Red Sox. Yeah, about that...

Both Vegas and PECOTA see the Red Sox as the team to beat in 2015. PECOTA likes their roster, Vegas bettors have their own reasons, I suppose. And yet again, the O's are projected by PECOTA to be under .500, and Vegas has them barely above water. After 162 games, we'll see who was closer.

So what can we learn from this historical comparison? Honestly, probably not much. It's only five years' worth of data, and 15 total team seasons. That's a fairly small sample size. And both systems have been really, really wrong a bunch of times.

But it is worth noting that the Vegas win total over/under line has been closer to the Orioles' actual results (and the Yankees, too) than PECOTA for four out of five seasons since 2010. The best math could do was tie, once. You can take that to the bank ... or the betting window.