Adapt and Move On: a New Rating System for 2021

I tracked Elo ratings across the league during the 2020 season.  Elo ratings are intended to go on in perpetuity, so I simply started with the 2019 season and worked my way forward.  Using personal liberties, I adjusted each team’s starting point in the 2020 season in hopes of finding a balance between recognizing the higher performing teams from the previous season while also accounting for the volatility that exists in lower league American soccer.  The resulting ratings for 2020 weren’t as accurate as I had hoped they’d be.  Perhaps this was because of the shorter season, but the ratings seemed to fluctuate more than the IMDB ratings for the last two seasons of Game of Thrones.  And just like the show’s last season, things just didn’t feel right in the end.

So for the 2021 season, I will be maintaining my own personal rating system.  It borrows some of the coefficients used last season, but will include methods that those who are familiar with RPI ratings will likely recognize.  Please reference my previous guest article from last season to familiarize yourself with the Elo formula I used if you’d like a refresher, and to compare it with what I will use for this upcoming season as you read.  Fair warning, I really enjoy talking and writing about numbers and formulas.  I almost enjoy it as much as the hosts of this podcast, a Union Omaha podcast, like talking about teams that aren’t Union Omaha. (Editor’s note: we all know the Ajax and Bayern fans are to blame.)  You know, on a Union Omaha podcast.  Anyway, let’s get started!

So, the main reason I maintained an Elo rating system last season was because I had never done one before.  I was intrigued with the facets of the formula that remain static over time, because the rating systems I’ve designed in other sports for the last ten or so years were fluid.  And by fluid, I mean the value of an individual game would change over the course of the season.  For example, in the 2019 college football season LSU played an early non-conference game at Texas.  LSU won this game by a touchdown, and it was considered a substantial result for them because Texas started that season strong.  The significance of this result had a pretty long shelf life, as Texas won their next three games and continued to impress.  However, Texas had a very mediocre finish to their season.  While LSU obviously ended the season as one of my highest rated teams, the win against Texas slowly became less valuable as the season progressed.  Those that follow college football as much as I do are well aware of a popular point made by talking heads across the country of how “Team A is a much different team now than they were last month”, so I like to develop systems that can account for that.  I had a couple bullets in this chamber to mock supporters of certain schools in this part of the country, but I want to remain friends with them so we’ll just move on with me hoping you understand the logic.

To calculate the value of a game, we will consider the following:

• Location of the match
• Final goal difference
• Each team’s current winning percentage
• The current cumulative winning percentage of each team’s previous schedule

Let’s go back to August 23^rd, 2020.  On this day, Union Omaha came from behind away to North Texas in a match that ended in a 2-2 draw (it feels good to remember that, doesn’t it?).  After this match, North Texas’ performance garnered a rating valued at 0.950, and Union Omaha a rating of 1.390.  These values will have more meaning later, once I reveal the full ratings from 2020, but all you need to know for now is that this was the value of the game on that night; a night that ended with North Texas having a record of 1-3-2, and Union Omaha a record of 2-3-0.  By the end of the season, North Texas’ performance rated slightly higher, changing from 0.950 to 1.045, while Union Omaha’s was considerably higher, changing from 1.390 to 2.035.  What happened here is the formula recognized North Texas’ improvement over the course of the season, and as more data was collected the value of the individual match adjusted accordingly, giving Union Omaha more credit for securing a draw away to a team that was better than its record indicated at the time.

So what goes into the calculation?  Do you really want to know?  If not, that’s ok.  Feel free to skip this section, I’ll let you know when it’s safe to return.  If you want to read at the same pace as the others, for some reason or another, may I suggest putting the kettle on, or perhaps asking Alexa to play a song of moderate length?  Why don’t you ask Alexa to play “Keep me Hangin’ On”?  No, not the original version by The Supremes, but the cover version by Vanilla Fudge.  Did you know that version existed?  It does, and it’s vastly different from the original.  Go on, give it a spin. 

Ok, only nerds here now.  So the formula itself isn’t complicated, it’s just that there are several calculations going on simultaneously, and different variables based on location and results.  Essentially, four numbers are always multiplied together:

• A goal difference coefficient similar to the one used in the Elo calculation (I increased it by a factor of 10)
• A number based on the opponent’s winning percentage:
  • If the match is a win or a draw, it’s the winning percentage itself
  • If the match is a loss, it’s (1 – winning percentage)
• A number based on the cumulative winning percentage of the opponent’s previous opponents.  Cumulative is used to account for an unbalanced schedule.  For example, if you played three games against two opponents, twice against a team with a 3-0-1 record and once against a team with a 0-0-4 record, that works out to a 6-0-6 record and a .500 winning percentage.  Same rules above apply based on the match result (actual percentage for a win or draw, (1 – percentage) for a loss
• A coefficient based on the match result and location:
  • Home wins: 0.667
  • Away wins: 1.333
  • Home losses: -1.333
  • Away losses: -0.667
  • Home draws: 0.333
  • Away draws: 0.667

What I’m hoping I’ve done with this formula is minimalize the difference of a 5-0 result vs. a 4-0 result, which is something I might not need to worry about anymore (Bye OCB!), give more credit to results by away teams, even if they are draws, and adjust results over the course of the season as we learn more about all of the teams in order to avoid those pesky narratives I talked about above.

To calculate the overall rating of a team, I simply average these individual results.  But, I’m not always averaging the same numbers.  As the season goes on, I will remove matches that are considered outliers from the calculation.  This process will remove those cases where a team does something that kinda doesn’t seem normal for them.  Like that time in Old School where Frank the Tank outdebated James Carville but failed to remember it, things sometimes happen for no explainable reason.  Therefore, I will remove matches from each team’s overall averages throughout the season in the following manner:

• After 8 matches, each team’s worst and best performance aren’t included in their average
• After 16 matches, each team’s 2 worst and best performances aren’t included in their average
• After 24 matches, each team’s 3 worst and best performances aren’t included in their average

Thank you so much for being willing to read all of this crap above without seeing what the 2020 ratings looked like.  So now we’ll get straight to it, but I first need to remind the normal people that stopped reading the previous portion to come back.

NON-NUMBER NERDS: IT’S SAFE TO RETURN

Welcome back, normal people!  Let’s go over the final ratings for the 2020 USL League One Season:

1. Greenville Triumph: 1.568
2. Union Omaha: 1.316
3. North Texas SC: 1.270
4. Chattanooga Red Wolves: 0.919
5. Forward Madison: 0.685
6. Richmond Kickers: 0.483
7. FC Tucson: 0.446
8. South Georgia Tormenta: 0.024
9. New England II: -0.475
10. Fort Lauderdale CF: -0.752
11. Orlando City B: -1.873

I will of course be keeping up with the 2021 season as we go, but don’t expect to see an update for a while.  This model will not produce worthwhile results for a few weeks because each team needs to have a few games under their belt before ratings settle down and don’t bounce around so erratically. I usually don’t start looking at my college football ratings until most schools have played four or five games, so I’m expecting to publish the first 2021 ratings sometime in late May or early June.  Until then enjoy yourselves, and thank your lucky stars that you don’t have as much spare time as I do.