A Historical Review of ESPN Power Rankings

by Ben Ulene

 

It’s September again, and with the major wintertime sports starting their 2016-2017 regular seasons, sports fans across the U.S. get to participate in the annual tradition of poring through expert predictions – including various outlets’ preseason power rankings.

While Vegas odds and other betting markets offer a general take on how various teams may stack up, there is something satisfying about reading power rankings reports. With written blurbs for teams that function as justifications for their rankings, sites like ESPN and Bleacher Report impart a qualitative aspect to the numbers that may mirror, or perhaps even spark, debates among fans across the country. And like professional odds-makers, many sporting news outlets update rankings as the season goes along – bookending the year with a final set of rankings leading into the playoffs – which provides a mechanism for determining how “predictable” any given regular season is.

For ease of use, I analyze ESPN’s Power Rankings in this article. With rankings from the first to last week of the regular season across all four major U.S. sports, ESPN’s rankings let us perform cross-sport comparisons to test the accuracy of predictions by different sets of “experts,” as well as dive deeper into individual sports to determine the predictability of different teams and seasons.

 

Cross-Sport Analysis:

Upon analyzing the past six years of rankings across MLB, NBA, NFL, and NHL, one thing is apparent: The NBA is consistently the most predictable league, and by far. As a few scatterplots show, in the NBA, Week 1 Power Rankings are much more predictive of regular season success than in any other sport:

 

Here, a straight diagonal line would represent perfect prediction accuracy for every team; while the NBA plot is far from perfect, it still seem to be noticeably less random than the other plots.

In fact, as a two-sided t-test shows[1], the inter-league difference in predictability (measured by the average absolute value difference for every team’s beginning ranking and their final ranking) is statistically significant between the NBA and every other league, but insignificant among the other three. In other words, the predictabilities of MLB, NFL, and NHL regular seasons cannot be proven to be different from one another – but the NBA is more predictable than all three:

League 1 Mean Difference League 2 Mean Difference p-value
NBA 4.33 MLB 6.94 1.151e-07
NBA 4.33 NHL 6.54 6.178e-06
NBA 4.33 NFL 7.52 3.242e-09
MLB 6.94 NFL 7.52 0.3327
MLB 6.94 NHL 6.54 0.4735
NFL 7.52 NHL 6.54 0.101

 

Why then – besides the unlikely explanation that ESPN’s NBA analysts are that much better than their analysts for other sports – is there such a drastic difference in ranking accuracy? Season length is what first comes to mind, but upon closer inspection cannot be the primary cause; not only is the NBA season easier to predict than the equally-long NHL season, but there is a noticeable lack of statistical difference between predicting the MLB (162 games) and NFL (16 games) seasons.

There are, however, a few possible explanations:

1) Fewer, more impactful players: The NBA mandates that teams carry 14 players at any given time, as opposed to the NHL’s 20, MLB’s 25, and the NFL’s 53, giving star players – who are usually apparent at the beginning of the season – more of an impact on results.

This is magnified by the nature of the game: Only in the NBA do the most impactful players consistently play for more than three quarters of the game, making it easier for talented teams to rise to the top. Pitchers in baseball can play only every five games; hockey superstars may see only 20 minutes on the ice in a game. Even in the NFL, elite quarterbacks are only on the field for offensive snaps, and the comparatively high injury rate makes it difficult to predict even the best teams.

2) Higherscoring games: The high number of possessions in NBA games, compared to the other three sports, may mean that there is less of a risk that any given NBA game is determined by chance. NBA teams possess the ball over 90 times on average in any given game; even in baseball, teams will rarely send up more than 40 batters in a game. Therefore, skill differences have more opportunities to manifest themselves in basketball, while mistakes can have a larger impact on the other big sports with fewer possessions.

3) Injuries: Since hockey and football are high-contact sports, teams in the NHL and NFL are much more susceptible to being gutted by injuries midseason than NBA teams. Even in baseball, a non-contact sport, pitchers – perhaps the most important players on their teams – are highly susceptible to season-ending arm injuries that can tank a team’s season.

 

Seasonal Analysis:

Taking the league-based differences into account, it is no surprise that six of the seven most predictable seasons in our sample are in the NBA – with the 2013 NBA season clocking in with a tiny 3.47 average difference between beginning and end rankings.

seasons

Additionally, the NBA is unique in its consistency – while other sports like the NHL vary wildly from season to season in predictability, the NBA has stayed relatively constant. The NBA has been the most predictable for each of the past six years, while no other league has repeated as least predictable – the NFL led in 2010 and 2013, MLB in 2011 and 2015, and NHL in 2012 and 2014.

seasons2.png

 

Team Analysis:

Just as interesting are the numbers for different teams across sports. Not surprisingly, the Miami Heat was the most predictable team in the country over the past six years – buoyed by four years of their LeBron-powered “Big Three.” But the top of the list is not just dominated by consistently good teams – bottom-dwellers like the Philadelphia 76ers, the Edmonton Oilers, and the Houston Astros (bad until 2015) also lead in predictability. Perennially unpredictable teams like the Minnesota Vikings and Boston Red Sox dominate the bottom of the list, as well:

teams.png

To conclude, what can this data tell us about predictability in American sports as a whole? For a large majority of teams across the big four U.S. sports, not a ton – moving four spots up or down in rankings can be the difference between making the playoffs and missing out. And assuming the randomness in professional sports doesn’t change anytime soon, it is probably safe to say that expert accuracy will continue along this trend for years to come.

 

[1] I compared the absolute value of first week–last week differences, with n=180 for every league except the NFL, which had n=192 (since the NFL has 32 teams).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s