International Simulation Football League
*Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback a - Printable Version

+- International Simulation Football League (https://forums.sim-football.com)
+-- Forum: Community (https://forums.sim-football.com/forumdisplay.php?fid=5)
+--- Forum: Media (https://forums.sim-football.com/forumdisplay.php?fid=37)
+---- Forum: Graded Articles (https://forums.sim-football.com/forumdisplay.php?fid=38)
+---- Thread: *Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback a (/showthread.php?tid=34521)

*Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback a - slate - 08-27-2021

2,275 words, ready for grading. Please pay 5% each to @mithrandir, @Kotasa, @Pat, and @Modern_Duke for help with development and analysis.

With the end of the ISFL season tonight comes the beginning of awards season in the league. As many people around the league argue which players are most deserving of postseason honors, I myself am lucky enough to be able to contribute to the selection of the first and second team ISFL All-Pros as part of the Simulation Football Writers' Guild (always looking for new members, if you're interested please DM Mithrandir on Discord).

This season in particular, there were several difficult positions to sort through this year with plenty of standout performances, and when faced with difficult decisions like these I'm always interested in developing some new way to analyze and break down the various players' performances to arrive at a quantitative answer for who performed the best. To that end, I developed two new metrics (I bet you'll never guess what they're called!) and am writing this media piece to share them with you all. Maybe you will find them useful either for helping you decide on awards votes if you happen to have them, or inspire you with new ideas for your own metrics or analysis. Or maybe this will just make me a bit of money after I already spent the time making these metrics, who knows. Let's get into it!


Adjusted Pass Defense Rate

In the ISFL, we have only a handful of metrics to use when evaluating CB play. Tackles, PDs, INTs, FF/FRs, and defensive TDs have long been the only tools available to us. With the transfer to DDSPF21 we have added to that targets and catches allowed, which has unlocked a new angle to evaluate CBs on plays where they didn't record any of the other stats. However, it still leaves us with a tricky question: how to compare PDs and catch rate?

PDs, or passes defensed, are presumably plays where the CB got his hand on a pass and broke up a would-be completion. However, there may be many incompletions that aren't PDs that are still caused by the cornerback, such as tight coverage leading to a difficult throw, or pressing at the line leading to the receiver not being at the spot in the route they're expected to be. I know these examples break down with the simplistic gameplay of the sim,  but the same principle should apply with whatever esoteric logic the sim is using instead. These non-PD incompletions generated by the corner's coverage are why we care about catch % allowed as a metric to evaluate CB play.

Still, there are some incompletions that aren't due to the CB at all either. A bad overthrow or a WR drop don't reflect at all on how well the CB covered. So the question remains, how can we holistically evaluate a CB's performance taking into account the impact of their coverage on the catch rate they surrendered to offensive players?

To answer this, I looked to distill the possible outcomes of a pass targeted at a CB into three separate categories. First is an interception, the best category. What happens after the interception (return yards / TDs) don't matter to me as much as the fact that the play resulted in a turnover, since I'm trying only to evaluate the coverage quality of the CB. Second is passes defensed. For these plays the CB's coverage clearly had an impact on the offensive play and led to an incompletion that probably wouldn't have happened otherwise. And the final category is a non-PD incompletion. This type of incompletion might or might not have been caused by the player in coverage, but if a CB is able to generate more of them that's generally a good sign that they're playing well and shouldn't be dismissed out of hand just because they don't translate to more PDs.

I then create an "Adjusted Pass Defense Rate" (APDR) by taking a weighted sum of these three types of outcomes, divided by the number of times the CB was targeted in coverage. This attempts to describe what % of passes were influenced by a CB in coverage, with more disruptive coverage (INTs) and that attributable directly to the CB (PDs) are given more weight than other incompletions. The base formula is as follows:

[Image: unknown.png]

Here, "INTs" is the number of interceptions, "PDs" is the number of passes defensed, "Targets" is the number of times the player was targeted in coverage (as determined by the sim), and "CtA" is the number of catches allowed on plays where the player was targeted in coverage. Thus the final term of the numerator (Targets - CtA - PDs)  is the number of incompletions in coverage (Targets - CtA) that weren't PDs. Because each Interception also counts as a pass defensed in DDSPF21, INTs are subtracted from PDs so as not to double count their contribution.

From here, we need to determine the weights, x and y, which translate INTs and non-PD incompletions into "PD equivalents". That is, how much value is provided by getting an INT as opposed to a PD (greater than 1), and how much value is provided by a non-PD incompletion as opposed to a PD (less than 1). After some discussion with fellow SFWG members @Kotasa and @Pat we decided on x=5 and y=1/3 . This means that a statline of X tackles 1 PD 1 INT would be treated equivalently to a line of X tackles 5 PD 0 INT (because we count only non-INT PDs), and that having 3 incompletions in coverage is as good as recording 1 PD and allowing 2 completions. Another way to think of y is that it represents the proportion of incompletions that a CB is responsible for in coverage. To simplify it a bit, there are 3 parties at play in any passing play (QB, WR, and CB) and so it makes intuitive sense to say that 1/3 of incompletions are the CB's fault. If you dislike the values of either x or y that we decided on, please let me know in the comments what you think more reasonable values would be and why, or I would be happy to talk with anyone about potential data-driven ways to assign values that reflect the real in-game impacts.

With the formula described, here is a ranking of all ISFL CBs this past season according to their Adjusted Pass Defense Rate (ADPR) (minimum 100 targets):

[Image: unknown.png?width=631&height=590]

I'm pretty happy with how this formula ranks players at first glance. It seems to do a solid job of combining stats like Interceptions and PDs with general coverage ability. Rickie Vaughne had an outstanding season with 8 interceptions and a good catch rate allowed with a solid number of PDs, but we see a few players with fewer interceptions beat out Maverick Bowie, who had the 2nd most interceptions in the league but forced only 50 incompletions on 170 targets. Similar types of comparisons can be found throughout the full list. That said, I definitely think there is continued room for improvement on this and I don't believe this is close to portraying the entire story of CB play, just coverage ability.


Return Expected Points Added

Leaving behind the murkiness of CB play, we move on to the relative clear cut nature of kick and punt returning. Who cares about all the nuances of running routes, attributing responsibility for incompletions, and comparing the value of interceptions vs. PDs? Just try and run the ball as far downfield as you can.

Expected points added (EPA) are one of the building blocks of advanced stats in football. Although there are a lot more details than I can explain quickly, the fundamental gist is to convert field position into the likelihood of scoring points for your team (and therefore the expected points that your current situation is worse), and then compare the expected points after a play to the expected points before it. The change in expected points from the play is therefore the expected points added by that play.

For returning, we aren't looking at a before/after scenario as much. Instead, I'm comparing the expected points from a drive that started after a return of X yards to a baseline. To do this with complete rigor, we would look at each individual return, work out the expected points of a 1st & 10 at that field position, and compare it to the expected points of a 1st & 10 at the baseline field position. However, the sim comes nowhere close to tracking things in that close of a manner in a convenient way, and I'm not about to scrape all of that data, so instead we take a slight simplification and try to convert the raw return yards into EPA directly.

The SFWG founder @mithrandir provided a very helpful Football Perspective article that listed the value of 1 yard of field position as equivalent to 0.064 expected points. So the expected points added by a return is equal to 0.064 * return yards, minus the expected points of the baseline scenario. This applies for both kick and punt returns, but the base scenario for both will differ.

For a kick return, the baseline is a touchback starting at the 25 yard line. If no one were there to return the ball, the offense would start at the 25 yard line (basically), which using our estimate of 0.064 EP/yard gives us an expected points for a drive starting at the 25 yard line of 1.6 points. However, there is a special case: a kick return touchdown. In this case, the offense's drive doesn't have an expected value of 0.064*return yard points, it has an expected value of 7 (well.. 6.988, because of the possibility of a missed point after attempt. At least this is what it is in the NFL, to be completely rigorous we should use the missed XP rate of the ISFL, but I didn't.) This means that we also need to take away the return yards of the TD return so as not to double count. So the EPA from kick returns is given as:

[Image: unknown.png]

KRTDYds is the number of yards gained on kickoff return TDs, subtracted from total kickoff return yards so as not to double count them. Everything else should hopefully be self explanatory.

For a punt return, the formula is basically the same with two major differences. First, the baseline is no longer a touchback, but instead the ISFL average punt return yardage. I think it's worth having some debate over this, but in my mind a punt return adds expected points only if it outgains your expected field position, and the ISFL average punt return distance effectively gives this expected field position. For punt returns, this kind of transforms it into a "points above replacement" metric rather than a pure EPA as with kickoff returns, but if I didn't make this adjustment the punt return yardage totally dominated the total return EPA calculation so I decided to make this adjustment.

The second difference is that a punt return TD is not a fixed distance, and so we need to include the punt return TD yardage in the calculation as well to find the baseline expected points of a drive had the returner not scored a TD. For an intuitive example, a 90 yard punt return TD is worth more than a 40 yard punt return TD, because an offense starting at their own 10 yard line was less likely to score on that drive than one starting at the opponent's 40 yard line. The expected points added for a punt return TD of X yards is thus the expected points of a TD (6.988) minus the expected points of a drive starting with X-8 yards left to go (8 because we are still assuming an average ISFL return as the baseline). This drive would be expected to score 0.064*(108-X) = 6.912 - 0.064*X points. The complete formula is thus:

[Image: unknown.png]

Which simplifies to:

[Image: unknown.png]

(This second point is not really relevant seeing as no one in either league had a punt return TD this season, but hey, word count.)

Total Return EPA (RetEPA) is the sum of EPA from both kick and punt returns, KREPA + PREPA.

With all of that said, let's look at the total expected points added by returners in the ISFL this season:

[Image: unknown.png?width=1037&height=583]

Sakura incredibly led the league in kickoff return EPA while recording 0 kickoff return TDs (take that TD truthers!), while Witten had an unbelievable 3 EPA off of only 11 punt returns while the next highest player had less than 2 off of 28 returns (oh yeah, and it was also Sakura). One of these two should be your ISFL returner of the year and it's anyone's guess as to which it is.

I also pulled the stats from the DSFL to calculate the same metrics. For the DSFL I used a baseline 20 yard kickoff return and 6.5 yard punt return for the base rather than a 25 yard touchback and 8 yard punt return, as returns in the DSFL aren't as good and deserve lower benchmarks.

[Image: unknown.png?width=1037&height=404]

While a couple players had more EPA off of solely kickoff returns, Toastovich's ISFL-quality 8.72 yard punt return average as well as a top tier 4 EPA off of kickoff returns should land him DSFL returner of the year without too much question.


Thank you very much for reading! If this type of stuff is interesting to you and you'd like to help incorporate this type of analysis more into postseason honors, strongly considering helping out the Simulation Football Writers' Guild next season. We'll be back soon™ with the actual all-pro teams for S30.

RE: Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback an - Kotasa - 08-27-2021

lmfao @j00 sucks

RE: Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback an - Pat - 08-27-2021

I swear to God if you only posted this so you could dunk on Doy.

RE: Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback an - slate - 08-27-2021

(08-27-2021, 07:47 PM)Pat Wrote: I swear to God if you only posted this so you could dunk on Doy.

Nah Kotasa had it right it was to dunk on @j00

RE: *Adjusted Pass Defense Rate and Return EPA: A Deep Statistical Dive into Cornerback a - j00 - 08-29-2021

FeelsBadMan