Thanks to everyone who submitted a question to BCS stats guru Ken Massey. Ken actually went above and beyond the call of duty and answered pretty much all the questions that people submitted. A big, big thanks to Ken to fielding all these questions.
Q: "What is the most unintuitive thing you've learned by studying college football statistics?"
- Anonymous
A: I am always amazed at how thin the line is between a winner and a loser. The better team loses 30% of the time, and the media never seem to realize that head-to-head may not always give you an accurate indicator of which team is really better. It takes an entire season of results to get any sort of accurate assessment.
Q: "Several years ago Bill James wrote an article urging statisticians to avoid participating in the BCS because he felt the rules surrounding what the computers could and couldn't do was fundamentally flawed. Apparently you disagree. Why?"
- Anonymous
A: I take it as a challenge to create a model that gives reasonable results using censored data. Bill James is a very good "hacker" statistician, but I don't think that gives him the right to be critical of others that take a more abstract approach.
Q: "We do this as a hobby, but you do it for the BCS. How much time do you spend working on your algorithm in a given week? Do changes require external approval?"
- Eddie P.
A: The algorithm itself doesn't change much. I did have to modify it to remove all margin of victory from the BCS version several years ago. I spend more time maintaining the ranking comparison. Yes, if I were to change my BCS compliant model, I would have to get approval from the BCS.
Q:"How often do you tweak the inner workings of your system? What situations/scenarios in the past have caused you to take another look at how your rankings work?"
- TempoFreeGridiron
A: I have always started with the mathematical model, and have not been tempted to tweak because of results that I don't personally agree with. Sometimes I don't like how the rankings turn out, but I trust the mathematics. I have tinkered with adding more data to the system (scoring, statistics, etc), but the BCS version doesn't allow any of that.
Q: "Last year there was an issue when the Colley rankings used an incomplete data set for the pre-bowl rankings. It was found and fixed, but how did this happen? Doesn't the BCS give you all the data or spring for a STATS, Inc. subscription?"
- Anonymous
A: No, the BCS does not provide the data. Each of us collects the scores independently. In previous seasons, we informally compared results, and so the Colley mistake was caught, but not until after the official release. Starting this year, we have a protocol to compare checksums of the data before the BCS standings are released.
Q: "Understanding that your formula is proprietary, what factors are considered by your rankings? What factors are not considered?"
- ral315
A: BCS version - who you played, whether you won or lost, and where the game was played. Your results vs teams of your own caliber are most important. So a good team is primarily judged on how it does against other good teams.
Q: "How did you get the BCS gig, anyway? That's a huge accomplishment for a 26 year old statistician."
- Eddie P.
A: I did an undergraduate math project on rating models. After that, I created a web page to continue the project as a hobby. Roy Kramer, commissioner of the SEC, found my site as his group did research on the computer component of the BCS formula. He just called me out of the blue.
Q: "Even though margin of victory is not technically used or allowed in BCS rankings, isnt GOF [Ed. note: Game Outcome Function] affected to some degree by the margin of victory ?"
- mika
A: [Y]es, in my main ratings margin of victory goes into it, but for the BCS I cannot use margin of victory.
Q1: "What are the reasons for dropping Wisconsin 9 spots after losing to a ranked MSU team, in East Lansing, on a last second Hail Mary pass, and then only dropping Oklahoma 6 spots, after losing at home, to an unranked Texas Tech team on the same day?"
- pico
Q2: "So Wisconsin, statistically overall a better team than Oklahoma, loses to a ranked Michigan State and falls more spots than a Oklahoma losing to an unranked Texas Tech, how can that be said to make any sense?"
- WhyteRaven74
A: Another Wisconsin loss answers this question.
Q: "Why do you hate college football? And why isn't a playoff good enough for Division 1-Fbs football when it's good enough for every other football league in the United States?"
Aaron
A: College football may be the only league where the regular season really matters. I'd hate to ruin that with a large playoff. We currently have a 2 team playoff. I'd support a 4 team playoff; 8 at the very most.
