Many of you likely read this when I posted on MWSFMB but just in case you missed it:
Quote: Inspired by Jon Bois' brilliant dissertation on the history of successes and failures by NFL franchises -- and the subsequent hilarious experiment of throwing darts at a board to engineer a more successful draft than Matt Millen could muster in 2006 -- I set out to better understand the formula of NFL front offices. Because, these guys have to actually be trying, right?
Despite Bois' dart-board randomness success, in reality preparing for the draft is a complex, laborious and painstakingly detail-oriented and continuous journey. It all starts in earnest about eight months before Roger Goodell calls out the first overall pick.
THE GROUNDWORK
Each team's scouting begins as college football season starts. The GM will assign scouts to certain parts of the country; his minions travel city to city, state to state, attending games and visiting schools to begin scouting.
"The area scout would grade, go out to games of course, and grade guys," former Packers and Chiefs scout (and now agent) Marc Lillibridge told me. "Then you'd have your over-the-top guy, maybe your college director, he'd do the top-100 seniors. Along the way, probably September or October, you get a sense of what juniors are coming out. The coaches of teams you're scouting will even say, 'this guy's coming out, you probably want to look at him,' and we'd understand that."
"Each scout is assigned an area of the country to cover."
"Each scout is assigned an area of the country to cover," former Ravens and Browns scout (and now NFL Network analyst) Daniel Jeremiah explained back in 2010. "We had cross-checker scouts (national scouts or the college director) that also made a fall visit to study every draftable prospect. In the last week of November, we would send a third scout to visit the players that we had major interest in."
That's just the beginning, though. Once all the raw data is collected during the college season, the difficult task of coming to something like a consensus as a group commences. "After the college season was over," Jeremiah continued, "each scout was assigned a position to evaluate. During meetings, we would spend time going over each player. All of the fall reports would be read, all-star game evaluations and Combine results would then be discussed. Then the scout that interviewed the player would discuss what he had gleaned from the conversation.
"After all of the information was discussed, our GM would then ask the scouts to compare the player we just went over to other players at his position. Once we found a landing spot for him at his position, we compared him to players across the board that were given a similar grade."
This is how players with similar grades are given hierarchy on a team's board.
Jeremiah went more in-depth on the process with fellow former Seahawks and Panthers scout Bucky Brooks on their CFB 24/7 podcast recently, expounding on the method to the madness of trying to land on a grade. "A lot of times when we'd get a discussion on a player," he said, "we would go up to the whiteboard, go around the room and say, okay, 'Which games did you watch? Which games did you watch?' Then literally, the director in the room would be like, ‘Okay, Bucky, why don't you go watch these four late-season games, D.J., you need to go watch these three games you missed early in the year. Let's come back together and see if we can't figure this guy out. We got to get him right, we got to get him where he belongs."
That's a classic methodology developed by Ozzie Newsome, the Ravens' highly-respected and preeminent scout-GM. However, as Bucky Brooks relates, that's certainly not universal from team to team.
"It's funny," Brooks says, "when I worked in Seattle -- and the Green Bay Packers did this with Ted Thompson and Ron Wolf -- when we had these pre-Combine meetings, we would go in at the end of January, we would sit in this room, we'd pop on the tape, Kony Ealy (for example), and we would watch, as a group, three games. At the end of watching three games, as a unit, we would then decide where that player would go on the board."
Completely opposing schools of thought, and the Seattle/Green Bay method of working as a group is almost repugnant to those scouts that come from the Newsome tree. Why? It comes down to avoiding dreaded groupthink.
"That is how San Francisco does it," Jeremiah says, "they bring all their scouts in there for a long time and watch all the tape. That's how Seattle does it, and those teams have been very successful, Green Bay still does that, now Kansas City, with John Dorsey, they're doing it in Kansas City. I come from Ozzie Newsome in Baltimore, and we were completely against that, and obviously, Baltimore just won a Super Bowl, Seattle just won a Super Bowl, so there are different ways of doing it."
There's evidently no hard-line guide to scouting -- it's as much an art as it is a science -- and the recent success by the two schools of thought reinforces this.
Still, aware of its trappings, Seahawks GM John Schneider is very careful to foster a culture that looks to avoid groupthink and encourages differing opinions among scouts. "We take a lot of pride in giving our scouts a lot of leeway in terms of their opinions on players," Schneider says. "So there is a concern about [groupthink], but in giving our guys a lot of leeway and confidence in the job they do, they know they're going to be heard and at the end of the day we're going to take all the opinions and put them together. I don't feel we do anything necessarily different than other clubs. We try to work it where we feel like we don't have all the answers all the time. We're looking for more and more questions, and answers to be questioned."
"We're looking for more and more questions, and answers to be questioned."
It's something that permeates the Seahawks' facility, and you hear both Pete Carroll and Russell Wilson talk of the constant quest for knowledge. It's an integrated, evolving process.
"Part of the duty of our system is that we don't -- and this isn't something I ‘developed' or anything -- it's a scouting philosophy," Schneider says, "where we don't hold our scouts true to their grades throughout the fall."
"[Scouts are] not hiding behind their numbers and their grades and things," Seahawks head coach and vice president Pete Carroll adds. "We've made it so they will speak out and we will hear them and let them feel comfortable about it. Then if we agree, we agree. If we don't, we all deal with that and we have to understand how that works and even more so [this year] with both sides, with the coaches being able to be so involved with it. We want the input and we feel like we can figure it out and make sense of it at the end of it. So we are getting everything that everybody has to offer, we hope. That's kind of the background goal is to draw out the best that everybody has to offer."
Getting coaches involved with the evaluation of a prospect is a common way to break stalemates within the scouting team.
"However you end up grading a player, you put your name and initials on the magnet on the draft board," former Packers/Chiefs scout Lillibridge says. "If there’s a huge discrepancy between scouts on the same player, then the whole room would watch, and give him a grade, but if everyone’s pretty much in the same area -- let me just give you an example. Let’s say you have, in a cluster, Sammy Watkins, Marqise Lee, and Mike Evans, and realistically they’re all even. You just feel like, they’re all the best."
What do you do?
"Well, that’s when you’d have the coaches come in. You’ll have your coaches in that category, probably your receivers coach, your offensive coordinator, then head-to-head, the coaches will do their own grades. So, that will then sometimes help to break the tie for who you’d want."
In fact, getting coaches involved before the prospects are even evaluated is key to a robust scouting methodology.
"From the scouts' perspective, they must know exactly what to look for at each position, and that comes with guidance from each position coach," Greg Gabriel, former scout and director of college scouting for the Giants and Bears, said. "The position coach must articulate what he needs and wants at his position because the scouts are the eyes and ears for the coaches when they are on the road evaluating college players. The scouts are the ones the club entrusts to present the right players that will allow the organization to compete for a championship."
It's an ongoing and necessary relationship that that requires all parties acting in concert.
"When we're putting our board together and we're choosing players," Seahawks GM John Schneider emphasized, "we're selecting players for the coaches that we know will fit the coaches' philosophy at each position and have a legitimate chance to compete. That's all you can ask for from a coaching staff -- guys that are willing to teach and let guys compete."
EVALUATING THE INTANGIBLES
There are countless factors that go into scouting NFL prospects, and the Combine displayed a few of them. Height, weight, speed, and power are obviously important, and it's been famously noted that guys like John Schneider, Scot McCloughan and San Francisco's Trent Baalke emphasize arm length and hand size. More important than that, though, is how well a prospect actually plays his position. This is the nuance of tape analysis, learning how a player moves, how he recognizes defensive/offensive schemes, and how skilled he is on the field. This is huge part of the scouting process and the inordinate amount of time that goes into it. The ironic part about scouting though, is that while game-tape analysis is one of the most important components of a report, it's ultimately the easy part.
"The hardest thing we do is try to figure out what's in a man's heart."
"The hardest thing we do is try to figure out what's in a man's heart," John Schneider says. "In the realm of scouting, the easiest things to do are the evaluations of the guys -- how he plays, what you think his future holds, how high his ceiling is, what his basement is. You can do all the work in the world, you can do every psychological test you possibly can, but at the end of the day, you don't truly know what's in a man's heart or how he's gonna react in a certain situation. You hope you have a really good feel for that. And hopefully, nine times out of 10, your psychological assessment is correct in how they're gonna handle certain situations. But you don't know."
Patriots head coach Bill Belichick echoed this sentiment at the Combine. "When we evaluate players, it's a long, thorough process we go through; obviously it's very inexact. We do the best that we can and that's a long process that's involved. Visiting the school, interviewing the player, talking to the people who have had the most involvement with him -- like his college coaches, even high school coaches, even beyond that. Other people that have had associations with him -- former teammates, so forth, so on. It's a mosaic composed of a lot of different pieces and you try to fit them all together and put some type of valuation on the player. And you do that for all the players. Each one's different. Each one's unique."
"A great science of this draft business is trying to figure out what is the makeup of the athlete and what kind of a competitor you get when you draft him," Pete Carroll said at the Combine. "There is a long process that goes into that with a tremendous exchange of information to try to figure the guys out. We can measure this stuff --- this stuff is not the hard part. The hard part is taking the measurements and then connecting that with the mentality of the player and figuring out what that's really going to turn out. It's a tremendous science there. It's a challenge and for us, it's that competitiveness that we are trying to find in the guys, that chip on the shoulder, that mentality that they have that will take them beyond where normal people go.
"The most important characteristic is grit," he continued. "That's the most crucial characteristic that helps somebody be successful. If they continue to hang, continue to fight -- and that's the competitiveness we're looking for -- there's always a chance they can pull it together. If you do a little homework, read up on what grit is all about, it's about persistence and resiliency, and an inner strength and belief you can get it done. That's what's most important. If guys don't have that, you can only take them so far."
THE PLAYER GRADING SCALE
Most teams use a grading scale from 1.0-to-9.0 to evaluate players (some teams only go up to 8.0). The only 9.0 grade that Ron Wolf has ever given, Lillibridge told me, was for Bo Jackson. A 9.0 is a generational talent, and that's so rare that it's not even worth mentioning. So, in practice, an 8.0 scale is used. Greg Gabriel breaks it down thusly:
8.0 grade: Special player, will impact a game and dominate at his position
7.0 grade: A potential pro bowler, a player you win because of
6.5 grade: A solid rank and file starter you could win with
6.0 grade: A solid backup who could start, but limited
5.5 grade: A role player but not a starter. A specialist
5.0 grade: A talented player, but not draftable. Developmental
"If you’re a 7.0 and above, you expect that person to be a starter in year one and then eventually, a Pro Bowl player, "Lillibridge said. "That’s what you’re looking at when you’re looking at the first two rounds."
BUILDING YOUR DRAFT BOARD
Legendary scout-GM Ron Wolf, whose scouting tree sprouted limbs in Ted Thompson, John Dorsey, Reggie McKenzie, Scot McCloughan, and John Schneider (and Schneider has sprouted his own branch in John Idzik), wrote a book called The Packer Way, and in it, he detailed the methodology he developed to build Green Bay's draft board. Here's an abridged snippet. Note the fact I said abridged, because he goes into insane detail of their system:
Putting together our draft board takes weeks. We consider players who have received a grade of 5.0 or higher, that's the minimum rating for a draft-able prospect. Anyone below 5.0 is considered a free-agent-level player.
We take each position and go through each player at those spots one-by-one. I then ask the scout who has seen him and has graded him how he arrived at his decision. I ask him if he's changed his mind because we're constantly reviewing our information. I ask him if there's anything he wants to add to what we know. At this point, he'll tell us whether we need to look at tapes of the prospect.
If we do, we'll pull out five games against his best opponents, I run the tape machine - and we start talking about what we're seeing. If the player isn't showing well, maybe I'll ask the scout if he'd rather put another game on the screen. By now, the scout is melting in his seat because his evaluation isn't holding up. After two or three tapes, you get a feel for the prospect. You give him a final grade, which could differ from the scout's. The player winds up in one of two categories. If he's bad enough, he becomes a reject and doesn't go on the board. If he has what we call a makable grade - he has enough ability to perhaps play in the NFL - he goes on the live board.
If we're lucky, we review 20 players a day in these draft meetings. They continue for five or six weeks, athlete by athlete, with each player allotted enough time to be evaluated properly. If you rush, you could make mistakes, which is what this process is trying to avoid.
The pace changes according to position. If we like a receiver, we'll take a look at every pass that's been thrown to him during his career, whether it was a completion or a miss. Same with running back.
Every run.
On quarterbacks, we just watch them play as much as we can. This is a long and arduous process. How many times can you review people blocking each other? But you have to maintain and edge. You have to be correct.
Easy.
DRAFT DAY: VERTICAL VS. HORIZONTAL BOARDS
Teams most commonly use two types of draft boards for use in the "war room" -- the vertical board and the horizontal board.
The vertical board is just a numbered list of players -- 1 to 150, for instance -- that represents your hierarchy for prospects that the months and months of debate and evaluation has produced. In theory, a GM can just look at his board and pick the highest-graded remaining player left out there. "The whole goal of the draft process was for our general manager to have a top-150 list," explains former Ravens scout Daniel Jeremiah. "All of our meetings before had led to this whole thing vertically, so we have meetings before we get to that point in time. That's all been discussed. So we have it by position up on the draft board, but on his sheet of paper, he has his 150. And, it's really paint by numbers. 'He went?' Check him off. 'He went?' Check him off. 'He went?' Check him off. 'It's our pick? Who's our pick? Who's our highest rated guy?' Boom, turn in the card."
"The whole goal of the draft process was for our general manager to have a top-150 list."
In other words, a true best player available (BPA) approach. Theoretically, once all the pre-draft work is done, draft day should be easy.
On the other hand, there is the horizontal board. Teams in the Ron Wolf tree of scouting more typically use this, where draft prospects are graded and compared to players on that current roster. "We grade for our team," John Schneider explains, "we don't grade for the league. Our board basically represents that. We grade a guy based on whether we think he can compete with Bruce Irvin or Malcolm Smith or Bobby Wagner, and that's the way our board falls."
They want to select players that can compete with and hopefully beat out players at different positions on their roster. This makes draft day a little more hectic. It's a process that is grounded somewhere near BPA, but more flexible based on need and depth.
Former Packers scout Marc Lillibridge, who spent time working side-by-side with John Schneider under Ron Wolf, knows just how this goes. "There were times where we’d get into debates on whether we were going to take, say, a linebacker or a defensive end," he told me. "Or if we were going to take a quarterback or a defensive back. So I think it just comes down to, in those cases, nine out of 10 times, from people I’ve talked to and been around and had conversations with, if it’s a dead heat between two players, it comes down to need. You go with need.
"So, you’re saying you’re taking the best available player, but if you’re loaded at, say quarterback -- you have two great quarterbacks and your board is sitting there tied with Derek Carr and, say, Phillip Gaines, the corner from Rice. They’re both the exact same [score], but at corner you have two legit starters but then your nickel guy is coming up for a contract after next year and in two years, your other corner is up, then that’s really all you have. Then, in that case, you’re probably going to end up taking the corner."
MUST READS
First round mock draft
Draft big board
This is where moving up and down the board becomes a strategy. And this is where things can get really complicated and stressful.
"In those kinds of situations, it’s a moving target. You want value, "Lillibridge said. "You ask: 'do we think there are any teams behind us that really want Carr? Can we trade out and get Gaines two spots lower? Or maybe four spots lower?'"
He plays out a scenario:
"Let’s say that you have Gaines as a 7.2 and let’s say you have Antone Exum from Virginia Tech as a 7.1. You say, if we trade back four spots, Carr goes, the team after that takes Gaines, and then we know the third team probably doesn’t need a corner. Would we then be okay with taking Exum with that next pick? Or, do we feel that Gaines is worth that 0.1 in score differential? Does it make that much of a difference? Those kinds of conversations are going throughout the whole draft, and you’re doing that with every position."
This is where the pro personnel department comes in. The GM, working closely with his team, tirelessly researches other teams' needs, follows up on rumors, voraciously reads respective teams' local media, and makes calls non-stop to better try and gauge where teams are going to be targeting certain players or positions. There is legit football espionage going on here.
"So, it comes down to: you really have to have your ducks in a row." Lillibridge said. "It can get kind of hairy a little bit."
VOLUME VS. TARGETED DRAFTING
The size of a given team's draft board can vary fairly greatly. The "Ravens Way" is to build a vertical board of 150 to use as what's called their "front board." Most teams have an extensive "back board" -- or "coffin," because they're "not going digging in there" -- which holds 150-200 players they've deemed to be free agent level, but for obvious reasons, great focus goes to draftable players. The number of players that teams focus in on can vary.
"In Carolina, at one point, I wanna say we had 250 people on the draft board," Bucky Brooks said.
Considering the process that goes into making each grade, sorting the order, and coming to a consensus on the hierarchy of players, that seems extreme compared to most team's standard of 150 or so. "In New England, it's even less. It's 75. It's 75 guys, so when you think about how many guys are picked in the Draft, they only have 75 guys that they want to take," Jeremiah said. "That's why a lot of time you'll see them, they get in the fifth, sixth, seventh round, they'll start punting picks. They'll say, oh, just give me something for next year, we don't have anyone we want. They won't make our team, why are we going to waste a pick on a guy that's not even going to make our team."
New England has picked seven times in each of the last two NFL Drafts. Compare to Seattle, which drafted 11 times in 2013 and ten times in 2012. It's a different approach, and Schneider (and all of the Wolf tree of GMs) believes in building in some insurance for players that they miss on by drafting in volume.
THE BOTTOM LINE
While certain picks may seem capricious or uninspired to fans and the media, hundreds of man-hours and highly developed methodologies go in to each and every player grade and each and every selection. Each decision, at least in most cases, is backed up by layers of quality control and safe-guards in order to somehow try to mitigate risk, but ultimately it all comes down to one thing: it's exceedingly difficult to evaluate the future performance of a human being.
|
