International Simulation Football League
*Who over/under performed this season? An Amkalytics S50 Season Review - Printable Version

+- International Simulation Football League (https://forums.sim-football.com)
+-- Forum: Community (https://forums.sim-football.com/forumdisplay.php?fid=5)
+--- Forum: Media (https://forums.sim-football.com/forumdisplay.php?fid=37)
+---- Forum: Graded Articles (https://forums.sim-football.com/forumdisplay.php?fid=38)
+---- Thread: *Who over/under performed this season? An Amkalytics S50 Season Review (/showthread.php?tid=53555)

*Who over/under performed this season? An Amkalytics S50 Season Review - Amkamkamk - 10-14-2024

It’s been a few seasons since I revved up the Amkalytics machine. If you missed previous installments, this series is where I pretend to know what I’m doing with looking at ISFL data. You should treat this like your nephew showing you something they made in Minecraft. It’s not analytics, an actual application of math; it’s amkalytics:  one grown man’s fever dream in Excel.

Shout out to @AustinP0027 for helping me think differently about tracking this. His insights influenced and inspired this process, but none of this is his fault. He's getting a cut of this article for his effort. 
Amkalytics Values

I’ve made some edits to the Amkalytics engine, but the heart remains the same.

Amkalytics has a few defining first principles:
  • I refuse to open the Sim Engine: I know that what we see on any given day is just one data point from the sim, not a true reflection of how good a team is. I don’t care; I’m treating this like real sports. The only thing that’s real is what’s recorded.
  • I refuse to learn how to scrape box scores—not doing it.
  • I refuse to learn how to do statistics -- Not doing it.

The conceptual premise for Amkalytics is that football games are smashing two rocks into each other and seeing what rocks break.  Not all rocks are created equal; some are harder than others. Some may have extra pointy edges, some forged in the bellies of volcanos. So, how can we recognize the expected damage compared to the actual damage? We can predict that “rock that is actually a Faberge Egg” would be turned to dust when hit with a chunk of obsidian. However, does that Faberge egg only break into 3 or 4 large chunks? Not too bad, Faberge Egg!

In the inverse, we expect Obsidian to smash smaller or weaker rocks. So, only breaking a Faberge Egg into three or four pieces could be a good job for those atoms in the Faberge Egg not completely evaporating. However, if we see that week after Week, Obsidian only dissembles weaker completion and does not obliterate them, it's interesting. It's worth noting.

Amkalytics looks at each game and tries to suss how strong of a performance each game was for each opponent. Who underperformed, who overperformed, and what that tells us about the standings/Margin of Victory that is in the index.

SICKO WARNING - SICKO WARNING -SICKO WARNING -SICKO WARNING -SICKO WARNING

The section below details all the steps that go into the Final Amkalytics ranking. Reading this back it has real "cabin fever" vibes, and I am not responsible for the psychic damage you might encounter trying to parse through it.

If you just want the charts, scroll down!

SICKO WARNING - SICKO WARNING -SICKO WARNING -SICKO WARNING -SICKO WARNING

Prelude: Points For - The Amkalytics Lingua Franca 

The metric of concern for amkalytics is Points For. Why? It’s the easiest to pull. See above. Starting after Week 2, when teams have a PF/PA average based on their first two games, we look at how each team performs for PF/game and PA/game based on the average points scored per game.

Let’s look at the Colorado Yeti. In week 1, COL completely dismantled OCO, 57-6. In week 2, however, COL ran into a YKW team that smacked them 21-47.

[Image: ohxlj46.png]

Through W2 of the ISFL season, the average team scored 24 points per game. If a team scores 24 points per game, its PF value would be “1,” showing that its offense is on par with the league average. COL has a PF value of 1.6 (they are 1.6x better than the average offense), and its defense is 1.09 (it allows slightly more than average points per game)

So COL beat the brakes off OCO, and got reality checked by YKW. OCO lost to HON by 5 in Week 2, and before they played COL, YKW beat a good BAL team by 10. With the context available, amkalytics sees that OCO might be bad, and YKW might be really good. So, how can we judge those games? And what does that mean for the COL/YKW game?

Finding Amkalytics Game Score
Step 1: Adjust the opponent's PF/PA average
Only scoring 25 points isn’t very impressive when the league average is 24.3, but it is impressive when the teams you play only allow 13 points on average. We adjust how good your offense is by good the opposing defenses you play, and same is true for your defense vs. opposing offenses. We'll call this "Opp Adj PF" or "Opp Adj PA" for Opponent Adjusted PF/PA. For Opp Adj PF, you want a higher value (scoring more points on average) for Opp Adj PA, you want a smaller value (allowing less points than average).

Code:
Opp Adj PF = Team A's PF Value / Average PA Value of their Opponents
Opp Adj PA = Team A's PA Value / Average PF Value of their Opponents

Considering COL played a really bad (and it would turn out historically bad) OCO defense and a red-hot YKW offense, COL’s Opp Adj PF dips down from 1.6 to 1.17. Their PA gets stingier, moving from 1.09 to .94. 

Step 2: Weigh a team's Opp Adj PF against their opponents Opp Adj PA.
 After the COL/YKW game, COL’s Adj PF was 1.09, and YKW’s adjusted PA was .61. Colorado’s offense is pretty good, and YKW’s defense is playing incredibly. So, our quick and dirty way to establish how many points we’d expect a team to score is

Code:
Team A’s Adj PF score * Team B’s Adj PA score * League Avg.Points Scored = Projected Score.


1.17 * .61 * 24.3 = 17.34 projected COL points 
1.53 * .94 * 24.3 = 34.9 projected YKW point

Amkalytics will view how well COL did as: Did you score more or less than 17.3 points? Did you allow more or less than 34.9 points?

Step 3: How did a team perform relative to the expected point value?
Game scores are zero-sum in amkalytics. If one team looks better for scoring more than expected, the other team looks worse for allowing points.

In week 2, Colorado scored 21 points, 3.6 points more than expected. Colorado’s offense gets a grade of +3.6, and YKW’s defense gets a -3.6 for that game. Colorado’s defense gets graded at -12.1 for allowing two scores more than expected. We then add these two scores together.

Code:
Amkalytics Game Score = (Actual Offensive Points - Expected Offensive Points) + (Expected Defensive Points - Allowed Defensive Points)

 COL gets graded at -8.5 and YKW at 8.5.

Cool, but Yellowknife was at home! And what about home-field advantage? As of Week 2, the home team scored 1.64 points more than expected, so we subtract 1.6 from the home team and give 1.6 to the away team.

Code:
Homefield Adjusted Game Score= Game Score - (League Avg Home Game Score)
Away Adjusted Game Score = Game Score  - (League Avg Away Game Score)

So this game's score of +/- 8.5 gets moved to +/- 6.9.

Step 4: …But what about TPE?
This part is my nightmare. I’m not smart enough for this, so I have a bootleg process that gives me an (incorrect, not scientific, shoddy) estimation of the relationship between the difference in Average TPE between the two teams and the Margin of Victory and how predictive that is.

Code:
TPE Adjusted Game Score = Home/Away Adjusted Game Score - ((Team A Average TPE - Team B Average TPE) * Amkalytics TPE Secret Sauce**)

** I just plot the Average TPE difference and Margin of Victory on a scatter plot and find the slope and the R^2 value. I have a small data set, but I will build this out as I go. I'm not going backward to past seasons to build it out; it's not that deal. I use the Slope as a rough estimate for how many TPE leads to how many points for the MOV and the R^2 to moderate how predictive it is. So it'd (TPE Game * Slope * R^2 value). My PhD work is not quantitative, let me live.

At the end of the season, YKW had about 200 more Average TPE than COL, which, as of week 2, shaved about 1.4 points off their game score. If you’re a better team, you should beat a worse team. Inversely, COL saw their game score increase by 1.4

So, at the end of week 2, COL received a game score of -5.5, and YKW received a game score of +5.5 for that game.

Adjusting Game Scores Over the Season

Amkalytics needs data points to understand what's going on. It’s a resume tool; it can only judge what teams do when the sim is run and recorded. It does, however, reevaluate games based on how the season goes. If a team gets better, your near loss to them looks better and better. If you beat a team by 20, and it turns out everyone beats them by 20, that win starts to mean less.

For example, at the end of the season, the early-season COL/YKW game looks very different. Yellowknife was a good team but not a world beater, and we knew that COL had a pretty good offense and a suspect defense. Throughout the season, that +/- 5.5 game was adjusted to a +/- 17, with the Colorado offense getting dinged for scoring fewer points than its season average and allowing way more points than we’d expect Yellowknife to score.

Amkalytics obtained more data about COL, YKW, their opponents, the importance of TPE in each game, and the importance of home-field advantage.

Interlude: S50 Superlatives

Biggest Overperformance AND Offensive Overperformance in a game: Week 5, CTC (65) CTC hosting COL (17) COL . (TPE Adjusted Game Score: +/- 39.1 CTC Off Game Score: 33.5, CTC Defensive Game Score: 6.6)
Hoo boy, this one wasn't even close. While it isn't the most significant Margin of Victory of the season (which belongs to the aforementioned COL/OCO game), CTC earned the biggest game score of the season with 39.1 (with COL, unfortunately then, getting a league lowest -39.1)

Based on the CTC Offense and COL defense at the end of the season, amkalytics says this game would normally be closer to a 31 - 24 game in CTC's favor in Cape Town. Then Thor went hammers! (Pun intended). CTC's +33.5 Offensive Game Score was also the best offensive overperformance (and/or COL's 33.5 defensive underperformance) was the season leader.

Quote:Quick note about a limitation of amkalytics: Special Teams/Defensive scores. I just include them as part of the offense. It's not great, it overinflates the offense and penalizes the opposed defense for something that's not their fault -- but, y'know. It is what it is. Sorry COL Defense that you got a raw end of the deal on a pick six.


Biggest Defensive Overperformance/Offensive Underperformance: Week 1, COL (57) COL  vs. OCO (6) OCO . (COL Defensive Game Score: 20.5)

When I started this article, I didn't mean to talk so much about Colorado, but here we are! I noted that Colorado's defense wasn't very good this season (they ended up with the 2nd worse defense -- getting slightly beat out by BER -- but they started off the season strong! 

A team who found themselves blowing people out and also getting blown out, they put the clamps on a OCO team that amkalytics, at season end, would have predicted OCO to score closer to 26-27 points in that game.

Quote:Another amkalytics quirk? It loves dapping up bad teams when they play just "kinda bad". To their credit though, COL also has the 2nd highest Defensive game score when they held Yellowknife to just 10 points in their week 12 loss to the Wraiths. 

So amkalytics noted that they shut down a really bad team who should have scored some garbage time points and losing small (and low scoring) to a team that should have put them in a locker.

"Looks about right!" The most amkalytics expected outcome: Week 1 - SAR (22) SAR vs OSK (21) OSK  (TPE Adjusted Game Score: .1)
Yeah! These teams are good on both sides of the ball, have very similar TPE, and when the season-long dust settled, this game had a TPE-adjusted game score of .1. Amkalytics saw the game and said: "Yeah, that makes sense!

Best Loss aka Worst Win: Week 2 - OCO (33) OCO  vs.HON (38) HON  (TPE Adjusted game score: 18.5 for OCO)

Honolulu's story this season is messing around and letting bad teams make them sweat. This is real masochist behavior by our friends on the Island. They get the dub, but it's not pretty, according to the Excel spreadsheet!

Quote:Both OCO/HON match-ups are in the top two spots of "Best Losses aka Worst Wins" and OCO has the top 5 values for this for being in one score games with better teams with a generationally, all time terrible defense but a feistier offense. Their TPE game adds an extra 2 points or so to the game score, but it's their consistent rate of getting blown out that makes their competency in these games stand out. Love you, Cord and AJ.

Season-Long Stats

Cool. We have now established how the games are scored individually. Let’s look at team-level data.

The Amkalytics “rank” score is based off the following:

Code:
Amkalytics Rank = (Opp Adjusted PF/game - Opp Adjusted PA/game)*League Average Points + Average TPE Adjusted Game Score.


Season Long Opp Adj Off/Deff Stats

To calculate the opponent-adjusted PF/PA, we must examine each team’s schedule. Below is a chart of each team's opponent's average PF/PA per game.

[Image: RgtcEtj.png]

If big numbers are good for PA and small numbers are good for PA, we can see which teams have better or worse opponents.

HON’s HON opponents (across all games, not just games vs. HON) scored slightly under the average point, and their defenses allowed more points than the average. HON’s offense and defense would get slightly nerfed since their opponents were better.

We also can see, according to amkalytics, which teams had easier/harder schedules (based on the PF/PA metric)


Quote:An  aside: The OCO problem, again

OCO’s defense being historically bad is a real problem for my whole deal. Just having them on your schedule instantly makes your offense downgraded because the model expects you to score 45-50 points on this team. The ASFC, across the board, gets slammed because they played OCO twice and didn’t beat them by 50 every game. And OCO’s “difficulty” in schedule is mainly due to the fact that everyone beats the breaks off of them, so the PF/PA of their opponents are inflated by how bad they are. So, just add that context in there when looking at the rest of these stats


This allows us to map what teams are like regarding opponent-adjusted offense and defense.

[Image: mIN9swE.png]

Most of our playoff teams are in the top right quadrant (good offense, good defense), with NYS NYS  and HON HON sneaking in due to their solid defenses and offense, respectively. The Outlaws ARI  are balled OUT, due in part to their hyper-efficient running game, and The lower left quadrant includes the teams that filled up the bottom of the standings. 

The whole damn NSFC is up there, man. What a tough season to be in that side of the league.

Average (Non-TPE Adjusted) Game Scores

Now, we can see the average of the teams' “Game Scores” on offense and defense. How many average points are they exceeding their expectations on offense and defense for each game, looking just a PF/PA, not TPE.

[Image: RqKnrpA.png]

CTC CTC and SAR SAR are the only two teams to that, on average, outperformed what amkalytics expected from them over the course of the season on both sides of the ball. CTC held the top spot as adding about half a point in their favor each game, but a large part of that was their MASSIVE game against the Yeti.

Quote:Amkalytics wrinkle: This doesn't mean these were the best two teams, but it does mean that they were cooking. AZ has a 0 value for their non-TPE adjusted game score. That doesn't mean that they were average, they had a very good season. It just means that, on average, theri games went how the model expected. 

No team, without factoring TPE, had higher or lower than a point (in either direction) for their game score - but CTC, SAR, and BER each clocked about >= .3 points better per game than amkalytis would have thought.

[Image: YWuzQOp.png]

TPE Adjusted Game Score

But! That’s not the whole story; we gotta adjust for TPE. I’m not smart enough to figure out the impact that TPE has on offense v. defense, so I just adjust the overall score.

[Image: Gsb57cj.png]


OCO’s OCO rock-bottom TPE level makes their league-low game score now a league-best game score! They got their teeth kicked in every game but didn’t always get their nose broken! COL COL takes the top spot by etched almost an extra point per game, while NOLA NOLA  is banished to the shadow realm.

We need to acknowledge the ASFC tax once again. Almost the entire conference, except for AZ ARI , is at the bottom. This was weaker than their sibling conference, but OCO OCO dragged it down. We know the model's beef with HON HON . 

Quote:The Bayou Bottom Out
NOLA almost made the playoffs this year and is coming off their rebuild yielding two titles. Their league-leading TPE though makes amkalytics think they should be MUCH better than they are. Amkalytics isn't mad @Tmoney6996, just disappointed. 

It's not slander if it's from a formula!

Okay, where are we at? 

Adjusted PF/PA? Check.
Adjusted TPE Game Score? Check.

Final ranking time!
IF YOU WANTED TO SKIP ALL THE NERD SHIT AND SEE THE CHARTS, STOP SCROLLING HERE

Amkalytics Team Rank

Below you'll see the teams in order, by conference, and their "Better/Worse" value. Which is the difference between their amkalytics Margin of Victory and their Margin of Victory on the index. Let's add some color,  shall we?


[Image: UgWAZcr.png]

Tier 8: Okay, they get it. I'm not gonna pile on.
Orange County: -16.36 (+1.96) OCO

Since amkalytics is a zero-sum game, OCO successfully stole some rank points from the whole of the ASFC for all the one-score games they were in. They weren't as bad as their splits show, but I think we're all glad this season is over. On to the next one!

Tier 7: The Competent, but underperforming teams
Berlin: -6.15 (+1.1) BER 
San Jose: -3.81 (-.31) SJS 
Colorado: -3.27 (+2.04) COL 

For context, the gap between OCO and BER is close the gap between BER and the 3rd best team. The spreadsheet does what it wants! Colorado got a big bump due to their TPE gap, and Berlin (with more wins) got dinged for a smaller adj PF/PA margin. San Jose had a fine season, but got leapfrogged by Colorado due to their more built out roster not putting together a stronger MoV. 

Tier 6: NOLA
New Orleans: -1.13 (-.94) NOLA 

They just outside the playoff race, but NOLA just got dunked on for being in this position with the TPE lead it has. Their Opp Adj PF/PA split gave them a bit of a bump compared to their MoV in the index, but they just got buried due to their veteran roster—Scoreboard, ring check, etc. etc. 

Tier 6: Someone's gotta go to the playoffs
New York: .41 (-.96) NYS 
Austin: .70 (-.86) AUS 

The Copperheads boasted a STOUT defense, tied for the 3rd best Opp Adj Defense in the league. New York, opponent-adjusted, was just a hair better than the league-average defense. Both teams had their offense dinged by amkalytics, but the Silverbacks were more so. Their better-than-average offense got knocked back down to earth at the end of the season, falling behind the league benchmark, and they slipped behind Austin in the amkalytics ratings.

Didn't slip behind them last week of the season, though! #NewYorkPosting #BigAppleAgenda

Tier 4: TheyCantKeepGettingAwayWithThis.gif
Honolulu: 2.3 (-2.2) HON 

I thought they were cooked like 6 times this season. Part of it is the OCO tax, but ... both games were closer than they deserved to be. I think if you're Honolulu you are over the moon to be this early in your rebuild and have 12 wins on the board. Amkalytics is skeptical, but that doesn't matter in the sim! 

Tier 3: Fight! Fight! Fight!
Osaka: 3.73 (1.10) OSK 
Yellowknife: 3.76 (.89) YKW 
Cape Town: 3.94 ( -.19) CTC 

Amkalytics has been high on Osaka all season, due in part to the bump they get for playing such a difficult schedule. Their race to the playoffs was exciting, and amkalytics was an early adopter. Yellowknife started out red-hot but slowly rode the wave down over the season and had a great season that just fizzled out before the finish line. Cape Town won big and had a strong season, but the MoV now stands for Moral-Only Victory. 

EXPAND THE PLAYOFFS!!

Tier 2: The Rebel Alliance
Sarasota: 4.66 (+03) SAR 
Baltimore: 5.9 (+.31) BAL 

Sarasota is an amkalytics darling. Their opp adjusted PF/PA stayed the same, but they are the only playoff team with a positive TPE-adjusted game score. Baltimore is Baltimore, steady behind their high-octane offense. They saw a bump in their Opp Adjusted PF, but their defense is where they made their money with a league-leading defensive rate AND a positive defensive game score. Their defense was great and still beat expectations.

They have one job to do.

Tier 1: I'm sick.
Arizona: 5.98 (-1.71) ARI 

C'MON! Amkalytics knocks them down 1.71 points (2nd most behind Honolulu), and they'd still be number one by almost a point. I thought we were past this! I thought they were supposed to be cycling out! We gotta do this again?! Even with nerfing their offense and defense a bit, even when you consider TPE, they average a negative game score. Doesn't matter. As you'll see in the graph below, the Outlaws slowly but surely took the top spot and never let it go. Ugh.

Week-by-Week Ranking Movement

Here are the ranking scores for each week. As a reminder, a ranking changes not only when a team wins or loses, but based on how other teams perform as well.  Here's the league as a whole. Also, I didn't change the colors to actual team colors -- I'm almost 4k words in. We're letting it rock.

[Image: cg1P5KW.png]

ASFC

[Image: WRGoXVd.png]

NSFC

[Image: EQOFN01.png]

Considering all this, we can determine who over/underachieved this season based on their end-of-season average TPE.
TPE Ranking and The People's Champ Award

When I first did Amkalytics in S45, I did it for the DSFL and made "The People's Champ Award", which went to the team that most overperformed in the amkalytics ranking based on their TPE. This went to the Portland Pythons in S45, and this is my first version for the ISFL. So it'll be the first ever ISFL People's Champ.

Stealing a play from @caleb.grim, I standardized the values for Average TPE (based on updated TPE values)  and the values for the final Amkalytics rank. Here's what it looks like. Avg TPE is on the X axis and Amk rank on the Y axis.

[Image: fwkApKt.png]

The chart looks like how you'd expect, but it's hard to get a feel for the distances:
[Image: lyZtnSI.png]

Orange County OCO  actually outperformed their massive TPE gap, Austin AUS didn't make the playoffs but have a lot to feel good about as they outpaced their TPE, and the Yeti COL had a great year given expectations.

Ironically, the People's Champ Award this year goes to the Arizona Outlaws ARI . Their performance was almost a total standard deviation better than their TPE performance. You simply hate to see it.

Quote:The most upsetting part is that, with this win, @Assistant to the POR GM is now a 2-time People's Champ GM. They were GM when Portland won it in S45 with the last edition, and they won it this time. Amkalytics can't stop giving Bamford awards.


Ultimus prediction:
Silverbacks by a million. (Don’t look at my ballot)

RE: Who over/under performed this season? An Amkalytics S50 Season Review - IR15H0000000000 - 10-14-2024

Wow, seeing that OCO actually over performed with only 2 wins is... interesting.

RE: Who over/under performed this season? An Amkalytics S50 Season Review - AustinP0027 - 10-14-2024

Took me a minute to figure out how I influenced this at all. We had that convo back in June  Big Grin

Great work!

Really interesting to see where all the teams stack up when you compare expectations and especially when you adjust for TPE differences!

RE: Who over/under performed this season? An Amkalytics S50 Season Review - Baron1898 - 10-14-2024

On the one hand, I'm so happy that this is back in all its glory; on the other hand, fuck Arizona

RE: Who over/under performed this season? An Amkalytics S50 Season Review - .HoshinoIchika - 10-14-2024

(10-14-2024, 01:56 PM)IR15H0000000000 Wrote: Wow, seeing that OCO actually over performed with only 2 wins is... interesting.

1 win*

RE: Who over/under performed this season? An Amkalytics S50 Season Review - BOOM™ - 10-14-2024

This was well researched and presented some interesting data/results.

[Image: headache-ba%C5%9Fa%C4%9Fr%C4%B1s%C4%B1.gif]

RE: Who over/under performed this season? An Amkalytics S50 Season Review - UptownCord - 10-14-2024

Bring back the Gemini Awards just for this