International Simulation Football League
S36 Sim Balance Team Newsletter - Printable Version

+- International Simulation Football League (https://forums.sim-football.com)
+-- Forum: Announcements (https://forums.sim-football.com/forumdisplay.php?fid=495)
+--- Forum: Announcements (https://forums.sim-football.com/forumdisplay.php?fid=32)
+---- Forum: Department Announcements (https://forums.sim-football.com/forumdisplay.php?fid=493)
+---- Thread: S36 Sim Balance Team Newsletter (/showthread.php?tid=40031)



S36 Sim Balance Team Newsletter - slate - 06-23-2022

Introduction

Hello all! While the Sim Balance Team has no concrete plans to release any balance patches for player archetypes, playbooks, or anything like that in the near future, I thought it would be helpful to provide some communication regarding our thoughts on the state of the game currently and the impacts of our most recent balance patch prior to S34. This will also serve as an opportunity for members of the general userbase to yell at me about how the sim sucks voice any comments or concerns regarding current sim balance and future actions you would like to see us take.

For those who don’t know, the Sim Balance Team is a league department tasked with monitoring gameplay balance and making periodic changes as necessary to the sim “infrastructure” like archetypes, playbooks, sim-specific rules, etc. Our main goals include ensuring that the simmed games are fun and engaging to watch, and that an adequate variety of well-balanced options are available both to players choosing position and archetype as well as GMs in constructing rosters and creating gameplans.

Going forward, I hope to continue to keep more open lines of communication regarding the state of sim balance to ensure that the community better understands the team’s thought processes and has a clear way to make their opinions heard. Unfortunately that is something that fell by the wayside over the past year as I was in the final push for completing my Ph.D., but with my newfound freedom it is something I plan to prioritize going forward for the team.

With all of that said, let’s get into it!



Executive Summary (aka tl;dr)

Areas where the S34 patch achieved our goals:
- Reduced disparity between rush and pass effectiveness
- Improved DE stats across the board, especially relative to LB

Areas where the S34 patch has made progress, but that require continued monitoring to understand its effectiveness and what more needs to be done:
- Increasing ability for TEs to impact gameplay
- Better stats for DT players
- Reducing LB stats to make DT and DE positions more competitive for leading league stats and receiving consideration for end-of-season awards
- Increased diversity of archetype use at some positions where a single archetype had been deemed dominant (Speed RB / Speed WR / Pass Rusher LB)

We are currently working on tools and methods that help us to use league data to analyze and monitor league balance and identify issues for the next balance patch, expected to come around roughly January next year.



S34 Balance Retrospective

The team’s most recent large effort was releasing the Season 34 Sim Balance Patch, which overhauled several player archetypes as well as making adjustments to the playbooks and introducing some new ones. With two post-patch seasons under our belt, this is a good time to take a look at some of the trends coming out of that and how they compare to before the patch.

The main goals of the balance patch to address a few problematic trends in the league that affected both overall gameplay and individual player experiences. These trends included:

Running plays produced too little value compared to passing plays, leading to lack of diversity in game planning.
Defensive line positions received too few individual stats, while linebackers received an outsized share. This was leading to a noticeable dropoff in the willingness of users to create at DE and DT.
Some positions, especially LB and RB, had low diversity in archetypes among high TPE users. This reduced player agency and also contributed to a lack of diverse gameplay.
It was too difficult for players lined up at TE to accumulate stats - successful TEs were by and large those who lined up out wide more often.

With these in mind, we will look at trends in league stats compared between the post-patch seasons of S34 and S35 and the 4 seasons pre-patch (S30-S33). These four seasons were chosen as the comparison because they were a couple of seasons removed from the sim transfer and subsequent balance patch 1 season afterwards. Teams therefore had a decent amount of time to understand and respond to the prior set of changes, so this stretch represents a bit more stability in terms of league trends.

League-Level Trends
We first look at broad league trends, averaging across all teams to understand the broad characteristics of what gameplay is like. The following table shows key indicators of offensive and defensive performance trends from S30 to S35, as well as grouping them by “pre-patch” (<=33) and “post-patch” (>=34). There is a lot of data there, so we will work through it piece by piece.

[Image: unknown.png]

In terms of offense, scoring and yardage stayed roughly stable pre- and post-patch, as can be seen under offensive points per game (“Off PPG”) and offensive yards per game (“Off YPG”). While S35 and S34 had the highest and 3rd highest scoring seasons in the sample, the gradual increase from S30 to S33 indicates that this is more a continuation or potential plateauing of an existing shift favoring offense over the period. This is likely a product of the defense-heavy classes of S22 and S25 regressing and retiring, leading to their replacement by more offense-heavy recreate classes since then.

The “Pass %” column indicates the frequency with which teams attempt to pass the ball compared to run it (computed by taking pass attempts + sacks divided by pass attempts + sacks + rush attempts). Teams’ passing rates are driven by many things besides just what is the most effective strategy, but we felt that the high rates of passing pre-patch were indicative of broad belief that passing was generally better than running among teams. While post-patch results are mixed as S34 saw the 2nd highest passing rates within the sample, the aggregate numbers across both seasons are more in line with what the balance team hopes to see going forward in terms of league wide run/pass splits.

We also looked individually across teams to see how pass frequency correlated with offensive scoring, and found little correlation either before or after the patch. The highest scoring team of all time, the S30 Arizona Outlaws, passed the ball just over 55% of the time, over 10 percentage points below league average at the time. Each individual team in the sample is plotted in terms of their passing play frequency and offensive points per game scored in the plot below.

[Image: unknown.png]

In terms of the effectiveness of passing vs. rushing, we see that although completion % has risen, yards per passing attempt and yards per completion are both down post-patch. These changes are especially noticeable in comparison to the two seasons immediately prior to the patch, when the league had further shifted towards offense. Meanwhile, rushing yards per attempt are up considerably post-patch as a result of changes to RB archetypes as well as creating new playbooks to allow teams to make more use of their running QBs. These changes combine to bring the value of running and passing closer to alignment and decrease incentives for teams to skew heavily towards passing, creating more diverse and interesting gameplay. Given this reduction in passing efficiency, I am especially excited by the increasing completion rate as it mitigates the effect on the stats of individual pass-catchers.

While the changes discussed so far have focused mainly on the offensive side of the ball, we also are interested in how defensive stats are affected. Across three of the most impactful defensive stat categories - sacks, interceptions, and TFLs - we see that the two seasons post-patch have resulted in greater frequencies of defensive players having these exciting high-impact plays. In particular, the large increase in TFLs has been especially noticeable, resulting from our efforts to give more stats to the previously lacking DTs and DEs in the sim. Achieving this increase in TFL rates while still increasing the value of running plays is very satisfying in accomplishing several of our competing goals simultaneously.

Player-Level Trends

Aside from overall league trends, we are also keenly interested in how equitable different positions and archetypes are across the league. If overall league numbers appear to be balanced and interesting gameplay, but every player at each position is the same archetype and some positions get vastly more stats than others, that is not ideal in terms of giving users interesting choices to make for their players.

In this section I’ll break down three overall areas we tried in particular to improve balance with the S34 patch: tight ends, defensive tackles, and a comparison between DEs and LBs. While these analyses present numbers based only on the overall position, we are also continuing to break this data down further into archetypes to understand balance within positions as well.

Tight Ends
Tight Ends were a large area of complaint for many users pre-patch, as it appeared difficult to make meaningful contributions to team performance at the position and even the best performers often had middling stats when compared to their nearest comparison in terms of WR. Through tweaking the Vertical Threat archetype and altering playbooks to feature more TE use, the S34 balance patch sought to address some of these concerns.

[Image: unknown.png]

Looking at the average performance of tight ends, both in terms of top performers at the position as well as all players at the position in the league, we can get a sense of how TE stats have varied over the past 6 seasons. Before the balance patch from S30 to S33, TEs overall averaged 104 targets - in season 34, this increased greatly to an average of 123 targets before dropping to 98 average targets in S35, the second lowest in the sample. There is a similar story in regards to receptions and yards - these initially showed a large increase in S34 followed by returning to numbers in line with S30-S33 or worse afterwards.

Top performers at the TE position have had a more clear increase in receiving yards in particular than the position overall, with S35 receiving yards among the top 5 TEs higher than all but one season in the pre-patch sample. We note that at a shade higher than 800 yards, this is still not quite the average level of performance we would hope to see from the league’s top TEs. The one clear area of improvement for TEs, both among top players as well as the position overall, is TDs, with numbers in both S34 and S35 significantly greater than pre patch.

Overall, these early returns show some mixed indications as to the effectiveness of the patch in accomplishing our goals for TEs. There are many factors convoluting this analysis, with the number of players at the position dwindling over time as well as season-to-season variation in terms of which TEs happen to line up more at WR, which given the small number of users can have an outsized effect on overall aggregate statistics. This is an area we will continue to monitor and potentially adjust further in the next round of rebalancing if we find that TEs are still unable to perform at the highest level without lining up at WR.

Defensive Tackles
Like tight ends, defensive tackle numbers across the league are low, giving a small sample size to draw conclusions from. In addition, especially in recent seasons, teams have increasingly been using lower TPE players at the position which makes it difficult to discern what the position stats would look like if DT TPE was more in line with say LB or CB TPE.

[Image: unknown.png]

With that said, we find that the season 34 balance patch generally had a positive impact in terms of stats accumulation among top players at the position. Sacks were much higher than all but 1 pre-patch season in both S34 and S35, and TFL numbers were much higher in S35 as well (although not in S34). Looking beyond the very top players, the trends are less clear across the position as a whole, but we feel this is likely due to the aforementioned trend in terms of decreasing DT TPE. We also observe that tackles are down both among top players as well as the general DT playerbase - potentially this could be due to increased competition from DEs thanks to their changes as well?

While we feel that the position’s situation has improved it’s been difficult to tell given overall trends at the position taking time to change in response to rebalancing efforts. We do think there could be more ways to improve the position’s on-field performance and will continue to look at DT as a particular area for improvement in the league.

DEs and LBs

Related to the goal of DT improvement, we also wanted to improve their fellow defensive linemens’ impact on the field, especially relative to LBs. Pre-patch, DEs were falling out of favor as LBs served as the primary pass rushers for many teams, with many teams having multiple 10+ sack LBs and no 10+ sack DEs. Additionally, DE TFL numbers had reduced significantly when compared to the pre-sim transfer era which had been a defining characteristic of the position, and we were interested in finding methods to improve their performance in that aspect of the game.

[Image: unknown.png]

Compared to the two positions previously, the data clearly shows a significant improvement across the board for DE impact. Both top users (either elite or just starter-level) and the position overall saw significant increases in total tackles, TFLs, and sacks after the patch. Overall, we feel that DE numbers themselves reflect the achievement of many of our goals for that position when taken in isolation.

However, we find that top DEs are still a notch below top LBs and have had difficulty competing for defensive awards, though it is still early after the change to draw too many conclusions thus far. Contrary to our expectations, LBs also captured most of the added TFLs post-patch where we expected them to be more concentrated among DL positions, creating another statistical disparity between the two groups. It has only been two seasons so we plan to continue analyzing LB vs. DL balance, but in terms of DE specifically we feel that their numbers are close to where we want them to be, with LB stats needing to be brought down more than DE numbers need to be brought up..

Archetype Balance
While we are still working on methods to analyze how different archetypes within the same position perform, the team has some general ideas based on our observations of the league as to how successful our goals in this space have been. The S34 balance patch saw us make changes with regards to several archetypes which had been dominant in terms of large majorities of the userbase at the position choosing them, in particular Speed RB, Speed WR, and Pass Rusher LB.

From our observations, it appears that existing users largely stayed at the same archetype as before rather than changing archetypes as a result of the balance patch. I think this response is perfectly understandable - an archetype swap requires effort, and our goal is to bring the archetype in line with others, not make it worse, so there’s no inherent reason to change. However, we have also found that new creates have increased variety; for instance, none of the LBs in the S36 ISFL Draft were Pass Rusher archetype!

While the populations of less favored archetypes at high TPE is still low enough that many of the best performances still come from the previously dominant archetypes, we have also seen some positive signs over the past few seasons with strong performances from previously neglected archetypes. For instance, Possession WR Saleem Spence posted a great season in S34 en route to 1st team All-Pro, and Slot WRs Vincent Sharpei Jr. and Nick Kaepercolin Jr. were two of the best receivers in S35, with NKJ winning WRotY. Another example is Bender Rodriguez posting great stats and being nominated for LBotY as a Versatile LB, and one that lines up at the traditionally less stat-accumulating inside LB positions.

These positive signs indicate that we have improved the situation compared to pre-patch in terms of archetype diversity and viability of various player strategies, but we will continue to monitor and gather harder data to use to make further decisions in this space.



Current Sim Balance & Moving Forward

Overall, while we plan to continue to monitor and improve balance across positions and within position archetypes, we feel that early returns on the balance patch indicate that it has addressed many of the issues we had identified. As shown above in some of the tables of data we are able to present, much of our current work before we ramp up for the next round of rebalancing changes is to develop methods that allow us to use league data to monitor the health of the league so that we have concrete information to go off of when proposing new changes or evaluating existing ones.

Harnessing some of the data presented above also provided us with a chance to begin to formulate ideas and priorities for the next rebalance effort, which is likely to arrive sometime around January of next year (we aim to do about a large pass roughly once every IRL year). The following is a brief discussion of just two of the topics that we have already started discussing, but is by no means meant to be a concrete list of planned changes or anything of the sort:

Quarterback Archetype Changes - As pointed out in @Twenty6’s recent media piece, quarterbacks currently take some of the lowest TPE to max every attribute out of the current archetypes. Given the priority that teams place on having max earners at the position, this leads to many seasons for QBs that amount solely to banking TPE and the lack of differentiation of high level QBs after about 2-3 seasons in the ISFL. Additionally, the individual passing and rushing archetypes are not as well distinguished in terms of gameplay as we had originally hoped when designing them during the sim transfer. We feel that taking a new pass at QB archetypes and increasing the TPE to max them will create a more fun experience for QB players as well as more diversity across teams and players within the league.

Further Defensive Tackle Rebalancing - While we believe the S34 balance patch has improved the situation, DTs appear to still be below our targets in terms of their performance relative to other front 7 positions in DE and LB. We are working to explore further areas to improve DT performance beyond the previous set of changes, including the impact that specific playbooks and formations have on them.



Conclusion

Thank you very much for reading! Please let us know your thoughts regarding the newsletter format of communicating our thoughts with you, current balance issues with the sim or player archetypes, or any more general suggestions for us!

If you think that this all sounds really interesting and you might be interested in helping out, the team is currently seeking 1-2 interns to join us and help discuss some of these coming changes and use the sim to evaluate the impact they might have on the league! Please take a look at our job application post for more information about the position and its requirements.

Slate, Sim Balance Team Head


RE: S36 Sim Balance Team Newsletter - Mooty99 - 06-23-2022

The sim team in this league does such a great and important job and it is really appreciated, just wanted to say that


RE: S36 Sim Balance Team Newsletter - zeagle1 - 06-23-2022

Whatever y'all did to DTs was awesome


RE: S36 Sim Balance Team Newsletter - Starboy - 06-23-2022

(06-23-2022, 03:35 PM)zeagle1 Wrote: Whatever y'all did to DTs was awesome

And the DE edits were not… Sad


RE: S36 Sim Balance Team Newsletter - RussDrivesTheBus - 06-23-2022

(06-23-2022, 03:35 PM)zeagle1 Wrote: Whatever y'all did to DTs was awesome

Just made'em fatter. My favorite tweak of all time.


RE: S36 Sim Balance Team Newsletter - Nathan - 06-23-2022

Is there any way to improve the sim engine? Multiple times we saw teams punt the ball away, not play with urgency, or have incredibly poor playcalling within the last five minutes. It was insane the amount of times I saw a team down multiple scores or one score within 90 seconds and would just dink and dunk the ball until the time ran out. How often I'd see teams just checkdown on a 4th and long.


RE: S36 Sim Balance Team Newsletter - MattBake12 - 06-23-2022

as a former DT I appreciate the work you are doing to make them more rewarding to build- keep up the good work team. look forward to seeing the QB mods too


RE: S36 Sim Balance Team Newsletter - LB3737 - 06-23-2022

(06-23-2022, 04:00 PM)Starboy Wrote:
(06-23-2022, 03:35 PM)zeagle1 Wrote: Whatever y'all did to DTs was awesome

And the DE edits were not… Sad
Speak for yourself, david moyes goated


RE: S36 Sim Balance Team Newsletter - Pat - 06-23-2022

(06-23-2022, 04:35 PM)Nathan Wrote: Is there any way to improve the sim engine?  Multiple times we saw teams punt the ball away, not play with urgency, or have incredibly poor playcalling within the last five minutes.  It was insane the amount of times I saw a team down multiple scores or one score within 90 seconds and would just dink and dunk the ball until the time ran out.  How often I'd see teams just checkdown on a 4th and long.

Unfortunately there is nothing we can do in that regard other than hope that we fine a suitable engine that fits our needs. Anything beyond modifying our player builds and tweaking playbooks is really beyond our reach.