MIT Sloan Sports Analytics Discussion


Princeton Sports Analytics writers Max Kaplan and Philip Chang are stat nerds. They have just returned from the so-called Geekapalooza, MIT Sloan Sports Analytics Conference in Boston. The conference included many faces of the sports analytics world, including Daryl Morey, Nate Silver, Mark Cuban, and many others. However, there were also ex-coaches and GM’s who were also behind the learning curve. Here is a discussion between Max and Philip about the most interesting aspects of the conference.

Max Kaplan: Sup, Phil. 2 days. 2700 people. 1000 Students. Dozens of panels. Who did you find most interesting among the panels?

revengeofnerdPhilip Chang: Howdy, Max. And before I say anything, let me give a big thank you to the Princeton ORFE department for allowing us to go.

Anyway, while the opening panel Revenge of the Nerds (this year, featuring Mark Cuban, Nate Silver, Daryl Morey, Paraag Marathe, and Michael Lewis) in past years has been the conference’s highlight, it really featured a lot more fluff than I had expected. There was almost no observation of the analytical aspect itself; rather, it seemed to be more of a discussion of different applications of those statistics, and how the structure of player/team evaluation has changed throughout the years. Not that that’s a bad thing, but that steered me away in response to your question, and thus, *upset pick* I really gotta say that the “Predictive Sports Betting Analytics” panel seemed the most contentious and informative of the bunch.

MK: Upset pick? What was the spread? Or the least you could do is explain to me why you were interested in the gambling panel.

PC: Haha, good one. Gambling is typically seen as a “dirty” part of sports, but through the eyes of professional NBA bettor Haralabos Voulgaris, and his interaction with “21” star Jeff Ma, bettor blogger Chad Millman, and director of bookkeeping organization Matthew Holt, we were able to examine how practically sports games, and seasons, could be predicted based on a) the models one chooses, and b) how closely one follows that sports. Haralabos (Bob) described how he closely followed the NBA, which allowed him to place bets with winning strategies on particular games. For the gaming commission, however, it was much more difficult because Holt and his compadres have to place lines on nearly every sporting competition on the planet, with not nearly as much research on a particular competition as Bob has had. To me, it seemed to be a really interesting, practical, and eye-opening discussion that captured exactly what is possible to predict in sports with purely a model, and how those models sometimes don’t take into effect things like lockout seasons, personal issues, etc. Thoughts?

MK: Yes, I agree that the opening panel did not have the “raw” data so to speak that many of the attendees (we included) may have been yearning for. This may be the result of the structure of sports hiring. Out of the panelists of Revenge of the Nerds, Nate Silver and Paraag Marathe (COO of the 49ers) seemed like the only two who did their own analyses. And even Nate Silver, as he has become well-known through his blog fivethirtyeight on the New York Times website, has started to delegate some pieces to other writers while he gets face time at the conference.

It seems as though a sports analyst (much to the chagrin of the 1000+ students – mostly MBA’s) rarely ascended to the ranks of upper management of a sports franchise. Through all of the panels, you could instantly tell who was a TV/management presence (hello, Tony Reali and Herm Edwards) and who combed through the numbers. Paraag, perhaps the highest a pure stats nerd like us has ever gotten in a franchise, advised that the communication of sports analysis is more important than the actual analysis itself. In essence, being good at math and being correct is enough to surmount the ideological differences between the old and the new.3aac6557a6d0eb2059f46db84c8692d8

PC: Great point. Indeed, it seems that sports analytics has been more about who you know than what you know. However, as we see with Paraag, things appear to be changing. Would you argue, though, that gamblers like Voulgaris could be as, if not more, successful than people like Paraag?

MK: Bob Voulgaris was recently featured in an article in ESPN the Magazine. It was about his shift from the subjective to the objective: he created a huge database to forecast possessions, games, and seasons. However, in my eyes, the panel duped the audience. The world of sports betting is not to correctly predict the outcome of a game. It is to outpick other sports betters. The existence of a betting line is to make sure that the sports book always wins regardless of the outcome. They choose the line based on who they think will bet for who. The job of Matthew Holt and the casinos is to measure behavior, not to predict the outcome of the game. It is in this way that successful gamblers like Bob are not actually at odds with the sports books. It is not a zero sum game between the two, as there are the other gamblers who take the losses. Maybe this is why Holt was daring the audience to try it themselves.

On a different note, Bob said that he specialized in basketball because that was his favorite sport. But it seemed like two decades of basketball analysis has pushed him away from all enjoyment of watching the sport. It is a business for him. In short, it did not seem fun to overanalyze the game that we all enjoy so much.

PC: I agree with your assessment that Bob’s goal, at the end, is not to always correctly predict the outcome of every game, but rather to beat the house/predict a little better than everyone else. However, I don’t think his tools and expertise should be understated, and even though he says he enjoys watching the games, you’re accurate in saying that there’s no way he could possibly enjoy watching basketball as much as he did before.

MK: Here’s a question. There was a whole panel on the Science of Randomness. There are many, many people who try to make a living by betting on sports. Most fail. The law of large numbers state that outliers are not only possible but likely, how do we know that Voulgaris is not just a product of luck?

PC: In a large sample size as large as Bob’s track record, it’s tough to attribute his continued success to luck, per se. Perhaps he lucked into the correct algorithm or model, and that has paid dividends, but in examining trends as objectively as he says they are, it’s much more probable that he really has figured out the system as well as he could. After all, that’s what his staff is for.

MK: Hiring top math students sounds like Wall Street. But Wall Street is often *cough cough* not correct. I think Wall Street has made the complete transition from the subjective to the objective. It takes time though. How far along are we to quantifying sports?

PC: We are simultaneously far and near, and I’ll explain why. You brought up another one of the more interesting panels of the conference, “True Performance and the Science of Randomness.” Here, the panel of Alec Scheiner, Silver, Morey, Ma, Phil Birnbaum, and Ben Alamar dissected what it meant to separate the “signal” from the “noise;” put another way, it seems that we as observers are much to quick to base decision on their results rather than the process. In doing so, we often do not give a theory or method enough time to really thoroughly test its accuracy. The Greg Oden/Kevin Durant draft decision was one of the most prominent examples. In agreeing to eliminate aspects such as emotion, overactive imagination in creating hypotheses, and avoiding stupid decision (Ma drew a direct comparison with MIT-Sloan-Sports-Analytics-Conferenceblackjack here, saying that one dumb decision could undermine years of work and credibility), the panel agreed that short-term success could easily be attributed to luck, while long-term success could only really be achieved with skill. Silver hammered the nail, in my eyes, by saying that you can’t understand a theory until it’s failed or succeeded, and that was really a shame in our world, when methodology could be undermined by luck in the short-term.

My question to you, then, is whether or not our tendency to evaluate results with 20/20 hindsight, emotion, and am almost unwillingness to let the data and methodology speak could ever really be changed. After all, that was the theme of the majority of this conference: how can we, as stat dorks, change and communicate, more than anything, what our numbers say to complement “gut” decisions? Should “gut” decisions really ever be taken into account?

Max: That’s a great point to bring up. At the end of that panel, a person in the crowd asked that very question: “How do you build a culture that appreciates the process?” Then there was silence from the panel. And then a panelist said, “It’s impossible in sports.” This was one of the more depressing moments of the conference. In a sense, the system inhibits innovation.

Should gut decisions ever be taken into consideration? Well, Bob Voulgaris had a great answer to that one. He said that a person’s sensory perceptions are a type of information or data. They must be considered as well as the numbers. On the other hand, Nate Silver told the crowd that playing backgammon is a great way to learn probability and use pure numbers to make the right decisions. He did concede that some things are inherently unpredictable (like perfect weather predictions due to chaos theory). The question boils down to: is sports predictable? It must be to a certain extent; it is a game. But there are far more rules and externalities than in a game of backgammon.

PC: Oftentimes, though, it seems that those externalities revolve around how “things have always been done,” such as with personnel or game decisions.

MK: Exactly. Should we follow the numbers or experience? The best algorithms take experience into account but there are natural limitations. This can best be seen in the panel “Monday Morning Quarterback: In-Game Decision Analysis.” Brian Burke, founder of Advanced NFL Stats, faced two ex-head coaches (Herm Edwards and Jack Del Rio) in an analysis of in-game choices. Kick or go for it? Go for 1 or 2? Go for the fake? Let the other team score? These were some of the situations brought to the table. In general, the crowd through a polling system favored the numbers rather than “the book.” Herm Edwards said numbers only matter when you have all the information and KYP (know your personnel). However, he seemed a bit antiquated sitting next to the much more stats-oriented Del Rio.
How do we balance the impulse decisions of playcalling (you only have a couple seconds) with making the correct decision? I truly do not know the answer. In Revenge of the Nerds, the consensus was when the answer is “I don’t know” even after analysis, go with gut and experience. Therein lies the problem.
How can we fix the playcalling problem or the bigger problem of convincing current coaches/managers to change the status quo?

PC: Indeed, your latter question is the one that this conference was really all about. Sports analytics is really meant to be a tool, not an end-all; Burke, I believe, had one of the conference’s most memorable moments when he quipped that “statistics are like a lamppost to a drunk; useful for support but not for illuminations” during the Break-ups in Sports panel.
This idea of “convincing” the coaches and general managers to obey the numbers, though, I believe is still inherently flawed as well. No coach, manager, or owner is the same, clearly; as such, they’ll all have different strategies for each situation. It’s completely different for Mike Woodson to say “Push the ball in transition!” as opposed to Mike D’Antoni, for example; that’s simply the latter’s style, regardless of what the statistics say about its effectiveness. Granted, we could argue about different methods’ relative success for eons. However, the point of sports analytics in a practical sense is more of a study guide than an answer sheet; of course, in this day and age you have to take the sheer numbers and models into account, but in the end, D’Antoni’s decision to become the fastest-paced team in the NBA is his own and NOT the numbers’.

MK: How then can a team balance numbers and “the book?”

PC: This fear of an over-reliance on numbers is perhaps what keeps some coaches from adopting those techniques wholly; Doug Collins of the Philadelphia 76ers, for example, was quoted as “hating that number stuff” in a panel. However, a bigger reason that coaches sometimes work against numbers is because given a smaller time span, some unsustainable behaviors could be successful. A coach, after all, is concerned about his or her employment, and wants to guarantee that job security. Stan van Gundy mentioned exactly this in the “Break-ups” panel; too often, coaches are concerned with their own selfish desires than for the good of the team. Van Gundy, on the other hand, seemed utterly convinced that his disagreements with the Orlando Magic association were entirely based on his desire to win, and get the whole saga with Dwight Howard over with. He legitimately did not seem to care what the Magic did with him, so long as they continued to give the organization the best ability to win in the future. Too often, that seems to be a problem, Max; teams will forfeit a longer-term solution for a win-now one, perhaps because of employment problems. Look at the San Francisco 49ers versus the Philadelphia Eagles, for example; the 49ers, through a few years of patience with Harbaugh and their own players’ development, managed to create one of the most dangerous teams in the NFL, while the Eagles went out and tried to sign a “dream team” defensively. Obviously, the latter’s strategy didn’t work out, while the 49ers were yards away from winning the Super Bowl. This idea of looking out for your own safety, rather than slowly building a long-term contender, seems very prevalent in today’s sports world, Max.

MK: When asked how he would run a team, ESPN President John Skipper answered that he would trade for Lebron the 1st day, trade for Kobe the 2nd day, and Harlem shuffle the 3rd day. Of course, Daryl Morey’s Rockets traded everyone away in the post-Tracy, post-Yao era. Big names are not always the answer. Even owners, in addition to coaches, sometimes have different incentives than success on the field. In the sports ownership panel, each had his own metric of success. Robert Kraft based his success on the impact on the community. Stan Kasten (Dodgers CEO) saw success through anything that made his “customers” happy.

You’re right in that conferences like these are a forum for debate. Sports analytics has gone mainstream (well, almost) where the coaches can no longer ignore it. I know that you did not attend the soccer analytics panel. It was interesting for the very fact that it was not interesting. Soccer analytics is almost non-existent. Half of the time was spent debating whether Messi or Ronaldo was better. I’m not even sure data exists publicly. And even if it did, very few managers would heed its advice. Add this to the fact that the average EPL football manager lasts fewer than 2 years, and there is very little incentive for long term success. Soccer is the fastest growing sport in America and my guess is that it is the one with the most to gain through analytics.

Baseball, on the other hand, may be reaching a point of diminishing returns, according to Nate Silver. It is perhaps the only “crowdsourced” revolution in sports statistics. Do you think Moneyball has completely changed how we see all sports or have we yet to see a Moneyball-like breakthrough in the others? Secondly, off-topic, but I was surprised at the lack of talk about fantasy at an obviously fantasy-obsessed crowd. What did you think?
Indeed, throughout the conference, it seemed as though the “truth” may lie between the scouts of old and the quants of new. However, I don’t think anyone can argue (other than maybe some intransigent coaches) that we have over statistics-fied things yet. Can we?

PC: Matthew Berry had a fantasy panel early on in the conference; unfortunately, no one attended because it was at the same time Mark Cuban was hamming it up with Nate Silver. I talked to some people about it, though, and it seemed to be pretty resigned. The fact is that fantasy games are niche and very un-fun if one does lots of research on it (empirically), and maybe the people attending Sloan were interested in the bigger picture anyway.
Back to Moneyball, while it revolutionized (or accelerated the trend, at least) the baseball scene, I’m loathe to say it really affected every sport. As we obtain better tools and more thorough technology, these breakthroughs were likely going to happen regardless. Michael Lewis was excited to just to be there and geek out.

In 100 words or less, what did you dislike most about Sloan?

MK: As sports analytics has gone mainstream-ish, it is becoming an ever more popular field for graduates. The vast majority of students there would not find jobs in the industry. Also, further innovation in the field is now expected, losing the this-time-its-different feel of Moneyball.

My attempt to get on the big screen behind the panel that had the running #ssac twitter feed. I never saw my post though.
My attempt to get on the big screen behind the panel that had the running #ssac twitter feed. I never saw my post though.

In 100 words or less, what did you like most about Sloan?

Sloan candidly brought to light the challenges and rewards facing sports analytics down the road. It was really humbling and meaningful to chat with Daryl Morey in the hallway and hear how excited he is for the field to grow and develop. It’s clear that we have only cracked the surface of analytics’ potential in sports, and this panel has convened the sports worlds’ greatest minds to provide updates and progress in that realm from many different viewpoints.

[Late Edit]: The conference also featured a few research papers, almost all by PhD students and professors. The only exception was Princeton University senior Kevin Whitaker. You can read his paper here.

Does a ‘coattail effect’ influence the valuation of players in the Major League Baseball draft?

One thought on “MIT Sloan Sports Analytics Discussion

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s