Next Gen Stats: Intro to new Route Recognition model

Written by: Gambyl Nation Editorial Team
on January 28, 2021
at 4:12 pm
filed in NFL

< wpimage aligncenterid4693sizeSluglargelinkDestinationnone > <div class=wp block image><figure class=aligncenter size large><img src=httpswwwgambylcomwp contentuploads202101nflll 1024x576jpg alt= class=wp image 4693><figure><div> < wpimage > < wpparagraph > Conventional counting stats like receptions and receiving yards provide a way to measure an individual players ability to catch and move the football but they only tell part of the story Advanced stats like depth of target separation window and completion probability provide greater insight but they still leave out an important factor Namely which route did the pass catcher run to get open before catching the ball < wpparagraph > < wpparagraph > With the help of player tracking technology the Next Gen Stats Analytics team set out to answer that exact question decoding one of the key elements of an offensive play call by using player tracking data to measure which routes pass catchers are running on any given pass play < wpparagraph > < wpparagraph > Last month we revealed a new set of rushing metrics derived from the ability to calculate <a href=httpswwwnflcomnewsnext gen stats intro to expected rushing yards>Expected Rushing Yards<a> Today we are introducing another new machine learning tool the Route Recognition model which classifies routes by type in real time with the help of player tracking data < wpparagraph > < wpparagraph > Lets dig into the methodology behind the Route Recognition model < wpparagraph > < wpheading level3 > <h3>How the model works<h3> < wpheading > < wpparagraph > The Next Gen Stats player tracking system records the x y location speed acceleration direction and orientation of all 22 players on the field in real time Our new Route Recognition model leverages this data as inputs into a model that assigns a route type to every eligible receiver on every pass play including tight ends and running backs Our architectural approach uses a combination of convolutional neural networks CNNs and long short term memory LSTM networks trained on <a href=httpsawsamazoncomsagemaker>Amazons SageMaker platform<a> CNNs allow us to engage with the spatial nature of our dataset that is where each player is on the field in a given play while LSTM networks allow us to engage with the temporal nature of our dataset what happens as the play develops over time < wpparagraph > < wpparagraph > We approached routes run by players aligned in the backfield separately from routes run by players aligned out wide in the slot or tight because of clear differences in route archetypes Below are the 15 unique route types assigned to all route runners based on their location when the ball is snapped Note that while NFL playbooks have hundreds of variations of routes weve narrowed it down to these high level categories including 10 routes for those in typical wideout alignments and five for those aligned in the backfield < wpparagraph > < wpparagraph > Wideout Routes 10 Screen flat slant crossing out in hitch corner post go Backfield Routes 5 Screen flat angle out wheel < wpparagraph > < wpparagraph > The model was trained and validated against all routes from every passing play from 2018 and 19 including both the regular season and the postseason All route runners were included regardless of whether or not they were targeted given that we could not find a difference between the shapes of targeted routes and non targeted routes we saw no reason to train on only targeted routes In total the wideout model was trained on over 100000 routes while the backfield model was trained on over 15000 routes < wpparagraph > < wpparagraph > To avoid noise in the data from broken plays during which pass catchers often stop running their assigned routes and player movement after the catch which would not tell us much about the efficacy of any given route all routes were capped at either the moment the ball was passed forward or at a given time 46 seconds after the snap for wideout routes and 4 seconds after the snap for backfield routes whichever came first The optimal time limits for these route types were determined by analyzing how pass attempts played out over the past two seasons for reference 44 seconds accounted for the 75th percentile of all pass attempts by time to throw in that span < wpparagraph > < wpparagraph > Heres a visualization of route paths sorted by our predicted route type < wpparagraph > < wpimage aligncenter > <div class=wp block image><figure class=aligncenter><img src=httpsstaticwwwnflcomimageprivatet editorial landscape 8 desktop mobilef autoleaguezzzjndwgvhbgxzdob2ayjpg alt=Wideout Route Paths 1><figure><div> < wpimage > < wpparagraph > For training purposes the tracking data for the wideout model has been normalized such that all the pass catchers are to the left of the quarterback with the rationale being that the route paths are symmetrical We found the shape of the routes aligned with our expectations of route paths no egregious model predictions seem apparent < wpparagraph > < wpparagraph > Future iterations of the model will look to delve deeper into the route tree to account for the nuance of route running at the pro levelhttps91c9f45c7ceaa574696f04786ac1c7e4safeframegooglesyndicationcomsafeframe1 0 37htmlcontainerhtml < wpparagraph > < wpheading level3 > <h3>What we can learn<h3> < wpheading > < wpparagraph > Real time route classification enables us to contextualize the passing game in new ways We can study league wide trends to gain a new understanding of offensive strategy and tendencies and we can break down and rank individual players by advanced performance metrics < wpparagraph > < wpparagraph > The table below combines our more descriptive NGS receiving metrics with the results of our Route Recognition model The values corresponding to each route represent league averages over the last two seasons Only wideout routes are included ie players aligned wide in the slot or tight < wpparagraph > < wpheading level4 > <h4>Wideout Tendencies by Route NFL Average 2018 19 Seasons<h4> < wpheading > < wptable > <figure class=wp block table><table><thead><tr><th scope=col>Route type<th><th scope=col>Route pct<th><th scope=col>Rank<th><th scope=col>Target rate<th><th scope=col>Rank<th><th scope=col>Air ydstarget<th><th scope=col>Rank<th><tr><thead><tbody><tr><td>Go<td><td>223<td><td>1<td><td>108<td><td>10<td><td>237<td><td>1<td><tr><tr><td>Hitch<td><td>183<td><td>2<td><td>201<td><td>5<td><td>77<td><td>6<td><tr><tr><td>Crossing<td><td>116<td><td>3<td><td>248<td><td>4<td><td>73<td><td>7<td><tr><tr><td>Out<td><td>101<td><td>4<td><td>278<td><td>2<td><td>83<td><td>5<td><tr><tr><td>In<td><td>89<td><td>5<td><td>169<td><td>7<td><td>104<td><td>4<td><tr><tr><td>Post<td><td>78<td><td>6<td><td>15<td><td>8<td><td>213<td><td>2<td><tr><tr><td>Flat<td><td>68<td><td>7<td><td>179<td><td>6<td><td>17<td><td>9<td><tr><tr><td>Slant<td><td>62<td><td>8<td><td>252<td><td>3<td><td>60<td><td>8<td><tr><tr><td>Corner<td><td>45<td><td>9<td><td>146<td><td>9<td><td>210<td><td>3<td><tr><tr><td>WR screen<td><td>34<td><td>10<td><td>407<td><td>1<td><td> 23<td><td>10<td><tr><tbody><table><figure> < wptable > < wplist > <ul><li>The average pass catcher runs a go route on nearly a quarter of all routes 223 the highest percentage of any route type in our data However those routes are targeted roughly 1 out of 10 times 108 percent the lowest target rate of any route<li><li>The WR screen is the least run route 34 and its the only route where the average target is behind the line of scrimmage But its also targeted at the highest rate 407 and early in the play 16 seconds average time to throw<li><li>The most targeted routes outside of the WR Screen The out 278 and slant 252 routes are the next most popular across the league<li><ul> < wplist > < wpparagraph > How often a pass catcher runs a route can give us insight into strategy and tendencies at the league wide team and individual levels of the game Evaluating route types by advanced performance metrics can tell us which routes are the most valuable on a per target basis as you can see in the chart below < wpparagraph > < wpparagraph > NOTE EPAtarget is expected points added per target this measures the value of individual plays in terms of points comparing the down distance and field position situation at the start of the play relative to the end of the play CROE is catch rate over expectation which measures performance relative to completion probabilitieshttps91c9f45c7ceaa574696f04786ac1c7e4safeframegooglesyndicationcomsafeframe1 0 37htmlcontainerhtml < wpparagraph > < wpheading level4 > <h4>Production by Non backfield Routes 2018 19 Seasons NFL Average<h4> < wpheading > < wptable > <figure class=wp block table><table><thead><tr><th scope=col>Route type<th><th scope=col>EPAtarget<th><th scope=col>Rank<th><th scope=col>Catch rate<th><th scope=col>Rank<th><th scope=col>CROE<th><th scope=col>Rank<th><tr><thead><tbody><tr><td>Post<td><td>+048<td><td>1<td><td>512<td><td>8<td><td>+19<td><td>2<td><tr><tr><td>Corner<td><td>+043<td><td>2<td><td>45<td><td>9<td><td>+14<td><td>4<td><tr><tr><td>In<td><td>+031<td><td>3<td><td>62<td><td>7<td><td> 08<td><td>8<td><tr><tr><td>Crossing<td><td>+027<td><td>4<td><td>69<td><td>4<td><td> 07<td><td>7<td><tr><tr><td>Slant<td><td>+026<td><td>5<td><td>674<td><td>5<td><td> 24<td><td>10<td><tr><tr><td>Out<td><td>+025<td><td>6<td><td>674<td><td>6<td><td>+21<td><td>1<td><tr><tr><td>Go<td><td>+019<td><td>7<td><td>341<td><td>10<td><td> 21<td><td>9<td><tr><tr><td>Hitch<td><td>+015<td><td>8<td><td>693<td><td>3<td><td>+14<td><td>5<td><tr><tr><td>Flat<td><td>+007<td><td>9<td><td>797<td><td>2<td><td> 02<td><td>6<td><tr><tr><td>WR screen<td><td> 008<td><td>10<td><td>90<td><td>1<td><td>+15<td><td>3<td><tr><tbody><table><figure> < wptable > < wplist > <ul><li>The most valuable routes by expected points added per target were the post +048 and corner +043 routes The go route +019 ranked seventh on the list of 10 route types One possible reason for this Its harder to separate on go routes which put the player on a straight path than on posts or corners which ask the player to make a cut Targeted pass catchers on posts and corners average 24 yards and 23 yards of separation from the nearest defender respectively while pass catchers targeted on go routes average just 18 yards of separation<li><li>Target location on go routes has a dramatic impact on the average net value of the play Go routes targeting a pass catcher outside the numbers average +013 EPA per target while targeted pass catchers inside the numbers average +042 EPA per target Over the last two seasons pass catchers running go routes were targeted outside the numbers four times as much as they were targeted inside the numbers<li><li>The top three routes by catch rate screen flat and hitch were the least valuable routes by EPA per target As we found in our analysis of our <a href=httpswwwnflcomnewsnext gen stats introduction to completion probability 0ap3000000964655>completion probability model<a> there is a strong negative correlation between catch rate and air yards which indicates targets closer to the line of scrimmage are not as valuable on a per target basis as deeper targets<li><li>Its worth noting four of the five most valuable routes by EPA per target are in breaking routes post +048 in +031 cross +027 and slant +026<li><ul> < wplist > < wpparagraph > Contextualizing routes at the league wide level gives a macro look into the value of route classification At the individual player level we can learn more about the micro nuances of route running < wpparagraph > < wpheading level3 > <h3>Route versatility<h3> < wpheading > < wpparagraph > Which wide receivers run the most diverse route tree relative to the average player With our new route classification model we can evaluate which receivers are essentially the most predictable or different from the average We calculate route versatility by averaging the sum of the absolute difference between a players route percentage and the average NFL receivers for each of the 10 route types see the last paragraph of this article for further explanation < wpparagraph > < wpparagraph > The top five and bottom five most versatile route runners from the 2019 season among 72 wide receivers with at least 300 routes < wpparagraph > < wpheading level4 > <h4>Wide Receiver Route Versatility Rankings 2019 Season min 300 routes<h4> < wpheading > < wptable > <figure class=wp block table><table><thead><tr><th scope=col>Top 5<th><th scope=col>Bottom 5<th><tr><thead><tbody><tr><td>1 Christian Kirk Cardinals<td><td>68 Robert Woods Rams<td><tr><tr><td>2 DJ Chark Jaguars<td><td>69 Jamison Crowder Jets<td><tr><tr><td>3 Auden Tate Bengals<td><td>70 Allen Robinson Bears<td><tr><tr><td>4 Stefon Diggs Vikings now with Bills<td><td>71 Mike Williams Chargers<td><tr><tr><td>5 Golden Tate Giants<td><td>72 Ted Ginn Jr Saints now with Bears<td><tr><tbody><table><figure> < wptable > < wplist > <ul><li>In his second season as a pro and first in head coach Kliff Kingsburys offense the Cardinals Christian Kirk ranked as the most versatile route runner of the 2019 season Kirk was targeted at least 13 times on five different routes hitch 28 targets crossing 17 WR screen 16 go 13 and out 13 While Kirk ran every route in the tree its important to note 77 percent of his routes came on the right side of the formation<li><li>Ted Ginn Jr now a member of the Chicago Bears ranked as our least versatile route runner of 2019 among qualifying wide receivers Ginn ran a go route more often than any receiver in the sample 42 of routes Ginns production on those routes in 2019 9 targets 1 reception for 25 yards targeted only 6 of the time<li><li>Not listed in the bottom five but relevant based on his reputation Seahawks receiver DK Metcalf ranked 66th out of 72 receivers by our measure of route versatility Metcalf ran a go route at the second highest rate of qualified receivers on this list 38 of routes behind only Ginn 42<li><ul> < wplist > < wpparagraph > This only scratches the surface of the analysis possible with our route recognition Who were the best performing wide receivers by route type NFLcoms Nick Shook takes a look at <a href=httpswwwnflcomnewstop 3 nfl wide receivers by route michael thomas reigns>the top receivers of 2019 by route type<a> < wpparagraph > < wpparagraph > Mike Band Next Gen Stats Analyst Follow Mike on Twitter <a href=httpstwittercomMBandNFL target= blank rel=noreferrer noopener>MBandNFL<a> < wpparagraph > < wpparagraph > Route versatility calculation explanation If Player A runs a go on 25 percent of routes a hitch on 19 percent and an out on 12 percent and the NFL averages are 22 percent 18 percent and 10 percent respectively the absolute difference from the average across those three routes would be 3 percent 1 percent and 2 percent < wpparagraph >