Task 4
I very much appreciate the opportunity to write about something that has tortured me, as a GM, throughout this season - the sim luck of my Arizona Outlaws.
I hope this would be known, but the Arizona Outlaws finished in a 5-way tiebreaker atop the ASFC, coming in fifth due to the cascading head-to-head rules (though we were confident we were eliminated a week before due to conference record). Five teams in the conference had nine wins. I will be using this UW task to explain clearly and statistically why Arizona should have had 10 wins, and dropping some sim knowledge that is soon to be out of date.
Let’s start with personnel - Arizona were stronger than the team that finished the playoffs last year.
Jay Cue regressed from around 1400 TPE last season to approximately 1050 this season, but he got better. Most people would quickly draw the conclusion that Cue’s MVP-level play this season was the result of luck, and there is always an element of that, but the truth is that a 1050 TPE Game Manager QB with a 79 Speed build is worth around 3-4% more to a team’s win rate than a maxed Mobile QB (which Cue was previously). I tested this extensively before he made the switch. Interestingly, my testing also showed that Mobile QB was actually the best archetype besides the 79 Speed cheese when maxed out. Cue was robbed of being the GOAT by Wolverine and their programming.
Another important area of improvement was at cornerback. Zamir Kehla retired and was replaced by safety Wesley Eriksen. A safety playing cornerback - how could that be an upgrade? Well, their respective levels of TPE through Seasons 25 (Kehla) and 26 (Eriksen) were roughly the same, but the real answer lies in archetypes. The difference between Eriksen’s Center Fielder archetype and Kehla’s Man-to-Man archetype are as follows:
Strength +5
Tackling +5
Hands +5
Agility -5
The safety archetype sacrifices 5 Agility for 5 points in three other stats that are highly valuable to a cornerback. It pretty much dominates it. Given they both had these stats maxed all season, this was a second upgrade.
The losses of Julio Jones and Mathias Hanyadi were mitigated by Mattress Cadaire and Darren Pama (who was very productive at lower TPE) respectively, and Heath Evans was replaced by creative depth chart work. In total, these “downgrades” summed to far less of a reduction in win rate than the improvement from the aforementioned upgrades. Mason Gillion and L’Gazzy Burfict coming in for Atlas Quin were more or less a scratch, with Gillion and Burfict certain to be the superior options going forward.
Also, Arizona only had a few players in regression. Everyone else improved. In short - Arizona were an objectively better team than the one that finished 9-7 and second in the ASFC last season.
Was that improvement worth a crucial tenth win, though? Let’s have a look at our sim testing results, with our win percentage and the actual result of the game:
1 OCO 66% L
2 NYS 48% L
3 NOLA 76% L
4 @ SJS 55% W
5 HON 79% W
6 SAR 70% L
7 @AUS 51% L
8 @PHI 64% W
9 BAL 79% W
10 @HON 54% W
11 @OCO 36% L
12 @YKW 43% W
13 SJS 76% W
14 NYS 75% L
15 AUS 76% W
16 @NOLA 46% W
Now, using these to draw conclusions in isolation is problematic. These were our sim test results from prior to each game, not the actual win probabilities. Of course the opposition teams are sim testing too, which will usually shift these probabilities downwards. From experience, I know that the average shift is around 3% downwards, so all the analysis from this point forward will be based on the above, reduced by 3%. I could load up all the game files and find out the real win rates after the fact but I don’t have the inclination. 3% will do as an assumption. (Incidentally, after the fact simming was about 95% of the hidden workings of my old CRUNC Power Rankings system, which proved disappointingly controversial in certain low-win-rate circles.)
Let’s start with Expected Wins. Basically, we take a win probability and give exactly that number of wins, and add it up through the season. So that 63% against Orange County in game one would be worth 0.63 wins, and so on. If we add all those up for the Outlaws’ season, we get 9.46 wins.
Should that number round up to 10? Well, yes, for two reasons. Firstly, Arizona outperformed 9 wins, and 10 is the next option going upwards, but that’s a little bit flawed. The second and real reason is that better teams tend to outperform their expected win total, and worse teams tend to underperform. Allow me to explain.
Take two games for a team with a 80% win rate. They’ll have an expected win total of 1.6, but they’ll get two wins from those games 64% of the time, outperforming their expected win rate.
The above is just an illustrative example, not conclusive proof (don’t try it with a 70% win rate) that is intended to show the characteristics of skewed data. Without going too deep into means, medians and the like, the skew of a teams expected win total means that 9.46 wins over a 16 game season will usually be outperformed. To put it another way: Arizona should have had 10 wins.
But enough about hypothetical win percentages, let’s look at what happened.
Arizona had a +91 point differential on the season
Arizona had an average margin of victory of 15.8 (over two scores)
Arizona had an average margin of defeat of 5.6 (less than one score)
Of Arizona’s 7 losses, 5 were by 7 points or less (a one score game being widely held as an unreliable measure of skill difference, a “coin flip” if you like)
In the 7 Arizona games this season that had a winning margin of 7 points or less, they won only 2.
That’s right, we went 2 out of 7 on coin flips.
Sim luck robbed a top tier team of a mere shot at the playoffs, and robbed the iconic Jay Cue of his last shot at playoff success.
But hey, we go again. Sim gonna sim.
I very much appreciate the opportunity to write about something that has tortured me, as a GM, throughout this season - the sim luck of my Arizona Outlaws.
I hope this would be known, but the Arizona Outlaws finished in a 5-way tiebreaker atop the ASFC, coming in fifth due to the cascading head-to-head rules (though we were confident we were eliminated a week before due to conference record). Five teams in the conference had nine wins. I will be using this UW task to explain clearly and statistically why Arizona should have had 10 wins, and dropping some sim knowledge that is soon to be out of date.
Let’s start with personnel - Arizona were stronger than the team that finished the playoffs last year.
Jay Cue regressed from around 1400 TPE last season to approximately 1050 this season, but he got better. Most people would quickly draw the conclusion that Cue’s MVP-level play this season was the result of luck, and there is always an element of that, but the truth is that a 1050 TPE Game Manager QB with a 79 Speed build is worth around 3-4% more to a team’s win rate than a maxed Mobile QB (which Cue was previously). I tested this extensively before he made the switch. Interestingly, my testing also showed that Mobile QB was actually the best archetype besides the 79 Speed cheese when maxed out. Cue was robbed of being the GOAT by Wolverine and their programming.
Another important area of improvement was at cornerback. Zamir Kehla retired and was replaced by safety Wesley Eriksen. A safety playing cornerback - how could that be an upgrade? Well, their respective levels of TPE through Seasons 25 (Kehla) and 26 (Eriksen) were roughly the same, but the real answer lies in archetypes. The difference between Eriksen’s Center Fielder archetype and Kehla’s Man-to-Man archetype are as follows:
Strength +5
Tackling +5
Hands +5
Agility -5
The safety archetype sacrifices 5 Agility for 5 points in three other stats that are highly valuable to a cornerback. It pretty much dominates it. Given they both had these stats maxed all season, this was a second upgrade.
The losses of Julio Jones and Mathias Hanyadi were mitigated by Mattress Cadaire and Darren Pama (who was very productive at lower TPE) respectively, and Heath Evans was replaced by creative depth chart work. In total, these “downgrades” summed to far less of a reduction in win rate than the improvement from the aforementioned upgrades. Mason Gillion and L’Gazzy Burfict coming in for Atlas Quin were more or less a scratch, with Gillion and Burfict certain to be the superior options going forward.
Also, Arizona only had a few players in regression. Everyone else improved. In short - Arizona were an objectively better team than the one that finished 9-7 and second in the ASFC last season.
Was that improvement worth a crucial tenth win, though? Let’s have a look at our sim testing results, with our win percentage and the actual result of the game:
1 OCO 66% L
2 NYS 48% L
3 NOLA 76% L
4 @ SJS 55% W
5 HON 79% W
6 SAR 70% L
7 @AUS 51% L
8 @PHI 64% W
9 BAL 79% W
10 @HON 54% W
11 @OCO 36% L
12 @YKW 43% W
13 SJS 76% W
14 NYS 75% L
15 AUS 76% W
16 @NOLA 46% W
Now, using these to draw conclusions in isolation is problematic. These were our sim test results from prior to each game, not the actual win probabilities. Of course the opposition teams are sim testing too, which will usually shift these probabilities downwards. From experience, I know that the average shift is around 3% downwards, so all the analysis from this point forward will be based on the above, reduced by 3%. I could load up all the game files and find out the real win rates after the fact but I don’t have the inclination. 3% will do as an assumption. (Incidentally, after the fact simming was about 95% of the hidden workings of my old CRUNC Power Rankings system, which proved disappointingly controversial in certain low-win-rate circles.)
Let’s start with Expected Wins. Basically, we take a win probability and give exactly that number of wins, and add it up through the season. So that 63% against Orange County in game one would be worth 0.63 wins, and so on. If we add all those up for the Outlaws’ season, we get 9.46 wins.
Should that number round up to 10? Well, yes, for two reasons. Firstly, Arizona outperformed 9 wins, and 10 is the next option going upwards, but that’s a little bit flawed. The second and real reason is that better teams tend to outperform their expected win total, and worse teams tend to underperform. Allow me to explain.
Take two games for a team with a 80% win rate. They’ll have an expected win total of 1.6, but they’ll get two wins from those games 64% of the time, outperforming their expected win rate.
The above is just an illustrative example, not conclusive proof (don’t try it with a 70% win rate) that is intended to show the characteristics of skewed data. Without going too deep into means, medians and the like, the skew of a teams expected win total means that 9.46 wins over a 16 game season will usually be outperformed. To put it another way: Arizona should have had 10 wins.
But enough about hypothetical win percentages, let’s look at what happened.
Arizona had a +91 point differential on the season
Arizona had an average margin of victory of 15.8 (over two scores)
Arizona had an average margin of defeat of 5.6 (less than one score)
Of Arizona’s 7 losses, 5 were by 7 points or less (a one score game being widely held as an unreliable measure of skill difference, a “coin flip” if you like)
In the 7 Arizona games this season that had a winning margin of 7 points or less, they won only 2.
That’s right, we went 2 out of 7 on coin flips.
Sim luck robbed a top tier team of a mere shot at the playoffs, and robbed the iconic Jay Cue of his last shot at playoff success.
But hey, we go again. Sim gonna sim.