Irish Boy wrote:
Dave In Champaign wrote:
Irish Boy wrote:
(One minor exception: I'm increasingly uncomfortable with football outsiders in general. I still look at and use the numbers but I worry about them. His general points still stand.)
I'd be interested in hearing more about this.
Problems I have, from most general to most specific:
1.) The stats are proprietary. I can't deconstruct them. I worry about that. I'm on the opposite side of JORR on the trusting-advanced-statistics scale, but I agree with him that you should be able to explain any statistic that you use. I can explain the gist of their statistics. I'm not sure that's good enough for me.
2.) I've become increasingly distrustful of omni-stats that try to distill the entirety of performance to a single number (in all sports, but especially in football). I think the components that make up the omni-stats are often more interesting than the amalgamated whole.
3.) They've admitted to futzing with certain numbers when they think look wrong. I think this is only true of projections but the point still stands. Rely on the numbers your computer spits out, then rely on them. If they look stupid, let me know that.
4.) In general, I think their worldview is wrong. I've increasingly come to see football like basketball, where the unit of measurement should be by possession and not by play. I understand the other side but have gradually come to disagree with it. Your mileage may very. But I fear that the commitment to per-play statistics is driven by their desire to do fantasy-football stuff.
5.) I've never understood their rationale for not stabilizing their numbers by using a cross-reference across seasons. All of their numbers use the current season averages as the baseline. That seems weird to me.
6.) Opponent adjustments seem off. I don't know that what I'm going to say is true (see point one), but I think it is:
FO's opponent adjustments assume linear adjustments. The only way I can explain this is through a hypothetical. Let's say the average defense gives up 400 yards a game, but the Bears give up 300 yards a game. If the Packers gain 300 yards, they'll treat it is having gained 400 yards (I'm simplifying quite a bit.)
That's fine as far as it goes. The problem is in the extremes. Let's say we have the Chiefs, who give up 500 yards a game. And let's say the Bronco's gain 525 yards. FO looks at that and says "decent, but not that much better than average." I see that and say, holy crap, how many more yards can they realistically hoped to have gained? The assumption of linear adjustments breaks down after a while.
7.) They refuse to drop metrics that have been proven to be absolute garbage, or even directly acknowledge that they've been garbage-y (I'm looking at you, Lewin Forecast).
I tend to agree with all of this. FO is at their best in measuring things that are tough to quantify with the naked eye (catch rate, defense and special teams, offensive line play), but Serious Football Talk has so saturated the discourse that I think their stats just don't offer as much value-added as they did even five years ago. So, the numbers say the Falcons are overrated, huh? So does Skip fucking Bayless. After the week 1 Sunday nighter, Schatz did an interview in which his primary observation about the game was, "I think Peyton Manning looked like Peyton Manning, and I think the Steelers looked like the Steelers." I'm paying $50 a year for this?
The efficiency stuff puts them in a box at the player level, too. Because their stats are composite, and contextual, and tied to single-year baselines as opposed to larger sample sizes, everything has to be hedged to a Bernsteinian degree. I really hate to dip into the "WAR sucks because lol Ben Zobrist" argument, but Brandon Stokley is 2nd in the league in DVOA right now. If you brought this up to Schatz or Tanier, I know they'd hem and haw about how they're not claiming Stokley is the 2nd
best wide receiver in the league, and that it's all about
catch rate and
offensive efficiency and
high-leverage plays. I also know that my knowledge of stats is pretty basic and that this is probably a dumb criticism and that I'm coming off as a philistine. But sweet merciful crap, Brandon Stokley is a billion years old and slow as molasses and has been targeted less than fifty times and has five TDs this season. Absolutely no one would consider him anything more than a 3rd receiver, at best. So any stat considers him to be 2nd in the league
at anything isn't really giving us a lot of actionable information, you know? It's interesting to think about, but it doesn't really get us anywhere. Makes me think you're on to something in re: per-possession vs. per-play stats.
Smaller nits to pick: with the exception of Speed Score (sort of), they get way out over their skis on college scouting. The point of SackSEER, for example, was to cut through the noise and identify undervalued collegiate pass-rushers, and yet they showed their ass by dumping on JPP--exactly the sort of raw, inexperienced athletic specimen the formula was designed to evaluate correctly. Even better, a year later, the new-and-improved version did
the exact same thing with Bruce Irvin. Lewin had the opposite problem. It's not exactly a breakthrough to claim that quarterbacks with lots of collegiate experience and a high completion percentage would be more likely to succeed than quarterbacks who are inexperienced and inaccurate (again with the hedging). With the exception of outliers like Mark Sanchez or Cam Newton (or Timmy Chang or Case Keenum), those are exactly the most sought-after quarterbacks in any given draft. So to refine it, they add...BMI? Come on.
I also wish they'd track and grade individual defensive players. It's one of the reasons that I find myself visiting Pro Football Focus more and more often.
Login
Register
FAQ
Search
The only way I can see that happening is the team gets sold and someone decides to clean house. Otherwise, it's going to take a massive bottoming out.
So true. People from Minnesota are nicer about it though I assume.
