*My attempt at creating an objective scoring system. I introduce the WADI! - Printable Version +- International Simulation Football League (https://forums.sim-football.com) +-- Forum: Community (https://forums.sim-football.com/forumdisplay.php?fid=5) +--- Forum: Media (https://forums.sim-football.com/forumdisplay.php?fid=37) +---- Forum: Graded Articles (https://forums.sim-football.com/forumdisplay.php?fid=38) +---- Thread: *My attempt at creating an objective scoring system. I introduce the WADI! (/showthread.php?tid=28177)

*My attempt at creating an objective scoring system. I introduce the WADI! - Arvot - 12-08-2020 I have tried to create an objective rating system for players. I’m a safety playing in the DSFL so I looked at free safeties from the DSFL as my test study. I thought it would be interesting if I could come up with some way of giving people an overall rating. So using my limited knowledge of maths and stats I jumped right in and started crunching numbers to see what happened. I took all the data for free safeties from the DSFL seasons on the index, so from season 3 to season 25, stuck them in a spreadsheet and tried to find a way to get a meaningful rating. I knew I would have to create some sort of weighting system for each stat, and a way to collate all these different stats into an overall stat, but I wasn’t sure how to go about it. How do you decide if a sack is more important than an interception? Does a defended pass have the same value as a tackle for loss? To start I got rid of any players who had not played 14 games in their season or the records where it’s a gm who has a safety and has 14 games with 3 tackles. It was a shame as some people had great per game stats but just hadn’t played the full 14 games. I felt this was better than extrapolating a 10- game season into a 14, sorry to our fallen brothers and sisters. I then found the means for each stat. This seemed a good starting point. If I could figure out what an average season looked like I could know whether someone should have a higher or lower overall score. I had to find a way of then measuring each stat against the mean to generate a score for that stat. My plan was I could grade them out of 1 on each stat then add that together and create an overall score. I then found out about percentiles. I found a way to measure percentiles for scores. So as an example, the mean tackles were 61.9. Anyone with 62 tackles in a season was in the 50th percentile, as they scored more than 50% of the rest of the free safeties and would get .5 added to their score. This worked well for tackles as there is a lot of different numbers and the scores are spread evenly. However, for things like Interceptions, Sacks, Forced Fumbles etc. most player only got 1 or 2 a season. This created issues. Sacks is a good example. The mean was 1.6, so anyone with 1 sack got 0.5 and anyone with 0 got a 0. I felt like this was too harsh on those without sacks, so I added a 0.9 to all percentile scores. This meant you got 0.299 for 0 sacks and 0.562 for 1 sack. This isn’t ideal but I didn’t want to limit those safeties that played mainly in coverage. Another big issue was it didn’t reward people really excelling in one stat. Someone with 3 sacks got 0.8, someone with 4 sacks got 0.94 but after that the difference was minuscule. Someone with 6 sacks got 0.98, which is so tiny in their overall rating. I realised this way of rating players probably wasn’t going to work. I tried to give things different weights by multiplying their scores and to be honest the people who had good seasons got good scores, but it felt a bit off. I then found out about z-scores. A z-score is a way of rating how far away from the mean a score is. It is pretty much another name for a standard deviation. Standard deviation is how spread out the data is, so how much variance is there. Excuse my simplistic explanation if it’s a bit off, I’m no expert, but this seemed like a better way of ranking players. If someone got the mean they would have a z-score of 0. If they were one standard deviation above the mean they got a 1, if they were 1 under they got a -1. Let’s look at sacks again as an example. The mean sacks were 1.6 and the standard deviation is 1.7. If you had 0 sacks you would score -0.9 (0.9 standard deviations below the mean) and if you have 1 sack you scored -0.4 (0.4 standard deviations below the mean) as these were both below average this seemed fair. For 2 sacks you score 0.2 and for 3 you would get 0.7. Someone with 6 sacks would get a score of 2.5 which reflected how much better they performed than the average. The strength of this is that people get rewarded for doing well in a metric compared to the historical averages. Another strength of this is because you convert all the metrics into the one type of score you can combine them easier to create an overall score. If someone is really good at sacks but not so good at interceptions, they are being scored on each of those relative to the historical means. So maybe they get +2 for sacks -.05 for ints. These scores mean the same thing, so it feels better to use as part of an overall grading without having the top end squeeze I found in percentiles. To get an overall score I had to use some mathematical jiggery-pokery. I took a players average of all their z-scores. This was their base overall score. I think this is good as this is how much better or worse they performed compared to the historical average, which is what I’m trying to create a rating for. To convert a z-score into an actual score I used a bit of algebra with the formula for determining z-scores. This ends up with you multiplying the z score by the average of the standard deviations then adding on the average of all the means. My maths could be way off but this produced scores that seemed to fit the stats I was looking at so that’ll do! The scores were a bit big, so I multiplied them all by 0.75 purely for aesthetics so most of them were under 10. I also multiplied the Safety (the stat, not the position) by 0.25. they are so rare that anyone who got one was 4 s.d. over the mean. This skewed their scores massively. I wanted them to get some reward for getting a safety so multiplied their score by a quarter to reduce its impact on the overall score.   This does favour all-round players better. If someone doesn’t have an Interception or no forced fumbles they will struggle to compete with others who have even had 1. Players who are highly specialised get undervalued which is annoying. I suppose that comes down to an argument of is it better to be really good at one thing, or good at many things. I could try and amend this by using weights for stats but I’m not sure it would change too much and I’m not sure if those specialist players should be rated higher than someone who does a bit of everything. In season 25 Maple has 84 tackles and 6 sacks which are impressive numbers, however, they have nothing else. This system rates them higher than more all-round players who didn’t pop as much in some categories, but not as good as all-round players who excelled in a few categories. Maybe this is the right spot for them to be. Here is last season’s scores. I’ll include the different types of scores I came up with so you can see the difference. The Score column is the rating using z-scores and multiplying that by 0.75 to squish the numbers a bit to make them look better. The next column along is without squishing the numbers. The first percentiles column is scores using the percentiles but I’ve added multipliers to the different stats to try and give them better weighting and the last column percentile flat is without those multipliers. You can see that people tend to get similar scores, with people in the middle of the pack being favoured by different methods. For fun here is the top 20 of all time using the official WADI scoring system. It’s still a work in progress but I think there is some promise, and it’s been a lot of fun delving into the stats. RE: My attempt at creating an objective scoring system. I introduce the WADI! - Nokazoa - 12-08-2020 This article doesn't talk about Mccormick. Commenting for algorithm though /s For real nice article. It's an interesting question on all around players vs specialized ones RE: My attempt at creating an objective scoring system. I introduce the WADI! - GuitarMaster116 - 12-08-2020 Dang that’s some big maths RE: My attempt at creating an objective scoring system. I introduce the WADI! - Jay_Doctor - 12-08-2020 Needs more Keleven