A CHANCE FOR REDEMPTION

Then, one fatal error. One sneaky interception, one dropped fly ball, one untimely foul, and the other team gains the advantage. Watching your team lose is agonizing, but you can’t turn away. You watch as their chances of going to the playoffs drop to nothing.

As you dejectedly turn off the TV, you think about how tragic it is that your team seemingly did everything right and that one fluke stood between them and the postseason. Your favorite player deserves to be going head-to-head with the best, not going home with his teammates. Instead, it’s almost like nothing your team did mattered. All they have to show for it is a measly score, one moment in time where they succumbed to weakness.

You start to wonder, “Is there a better way? How can my favorite player make those great plays count, even if my team lost? What if he got a chance to keep playing because he proved himself as a good athlete?

“What if it could be different?”

Athletes Unlimited

Sporting events require a thoughtfully designed scoring system that can translate how athletes perform into something quantifiable. The best scoring systems strike a fine balance between simplicity (where even the newest fans can grasp the game) and complexity (where even the most nuanced plays can be quantified). 

Many of the conventional scoring systems fans are familiar with have remained unchanged for years. While many metrics have been developed – both officially and unofficially – to quantify individual athletic performance, these systems have less to do with contributing to league standings and more to do with the likelihood of a player extending his contract or getting drafted. At the end of the day, wins and losses are what determine whether all of the members of a team get to progress, or just get to go home.

Athletes Unlimited is taking a new approach to scoring. Rather than giving the trophy to the softball team with the most wins (or the win where it most counts), players will earn points on an individual basis based on their performance within their teams. Here’s how it’ll work:

The Scoring System:

Players will accumulate points based on their offensive performance over the course of a softball game:

• +10 points for singles
• +20 points for doubles
• +30 points for triples
• +40 points for home runs
• +10 points for:
  • Walks (BB, IBB, and HBP)
  • Sacrifice hits
  • Sacrifice flies
  • Successful stolen bases
• -10 points for getting caught stealing

Pitchers also partake, with the system giving +4 points for each out recorded while subtracting -10 points for each earned run allowed.

How did Athletes Unlimited choose these point values? Remember that a successful scoring system prioritizes simplicity just as much as it values complexity. Round numbers are easy to calculate, and the degree to which they scale depending on how impressive a play is should be intuitive to fans.

“Sure, I can tally these up on a napkin if I have to,” you might say. “But how do you know if these points actually reflect how good a player is?”

To find the answer, we need to compare how a player would fare in the Athletes Unlimited system to how they are measured by some other widely accepted metric. If individual standings remain consistent across the two systems, we can reasonably conclude that the new scoring system reflects athletic performance accurately.

Analysis

The quickest way to perform a simulation and make this comparison is to draw upon historical data of individual athletic performance. Unfortunately, there just isn’t a lot of high quality historical data for professional softball players that is available to the general public. So, we decided to use the next best data set: Major League Baseball historical data.

Now that we have our data set, we need to identify a widely accepted metric that baseball fans use to evaluate individual players. For this, we turned to a statistic called Wins Above Replacement, or WAR for short.

Why WAR?

WAR is a baseball statistic that measures a player’s total contributions to their team by placing a value on the number of additional wins that the player’s team earned because the player was on their team, versus if the player were not on the team. The biggest benefit of WAR is that it’s one of the few metrics that encompasses all aspects of player performance. WAR is context, league, and park neutral. This means you can use WAR to compare players across the board, no matter if they played decades apart.

This is the breakdown of how Cody Bellinger, first baseman and right fielder for the LA Dodgers, would’ve earn points based on his contributions to his team during the 2019 MLB season if he were being scored by Athletes Unlimited’s scoring system:

Now that we know what this looks like, we can repeat the same process for all of the players in the 2019 MLB season, and we end up with the following top 10:

If we go back to our original question, which is whether or not the Athletes Unlimited scoring system accurately reflects players’ performance, we need to do a side by side comparison with the Athletes Unlimited rankings based on WAR:

So, are the top players according to the Athletes Unlimited scoring system actually the top players in the league? 

Eight of the MLB’s top 10 offensive players – according to their WAR scores – make an appearance in the top 10 offensive players according to the Athletes Unlimited scoring methodology. The two exceptions, Ketel Marte (No. 21) and Rafael Devers (No. 11), are in the top 25 in the Athletes Unlimited rankings. These results let us reasonably conclude that the Athletes Unlimited individual scoring system accurately reflects on how well these players did over the course of the season.

Another interesting thing to note is the percent difference in measured ability between the Athletes Unlimited scoring system and WAR. The difference between the No. 1 player (Cody Bellinger) and No. 10 player (Freddie Freeman) according to the Athletes Unlimited scoring system was 10%. Meanwhile, the difference between the No. 1 player (Mike Trout) and No. 10 player (Rafael Devers) according to MLB’s Offensive WAR is a whopping 31%. Sure, there are some differences in how WAR is calculated that may lend itself to this larger percentage gap, but when you factor in that there are roughly 750 active players in the league, the top 10 players shouldn’t experience as much of a range of performance as WAR suggests.

If you read last week’s blog post, we discussed how parity is an important mechanism for making sports competitive and compelling. If the top 10 players are competing at a similar high level, the stakes are raised when they compete within a scoring model that reflects their performance and increases the parity between them. A 10% range of difference in competitiveness creates more unpredictability and tension that makes for an interesting game; a 31% range of difference, not so much.

“Okay, this new scoring system works for the top players. What about everyone else?” you might be asking. We can continue this exercise for the top 200 players in the league, but putting the rankings in tables like we just did would be mind-numbing and not all that useful. For larger groups of data like this, we can use scatter plots to understand the amount of correlation between the two systems:

Correlation is measured as a value between zero (completely uncorrelated) to one (100% correlation); the closer to one the correlation, the more certainty we have that the two systems we are observing similarly rank player performance. The correlation we found between the Athletes Unlimited rankings and the WAR rankings is 0.70 which implies that the two values are highly correlated. This bodes well for the scoring system and  proves its validity on an individual athlete basis.

Conclusion

Scoring systems play a huge role in manifesting the aspects that make for a good athletic competition: close matches, suspense, twists, stakes, you name it. Measuring how good players are by comparing team scores and tracking wins and losses over the course of a season works, but only to a certain extent. With fans increasingly moving towards engagement with individual athletes versus entire teams, it can be extremely frustrating to watch a good player do everything right, but be punished by playing on a bad team or being swept by the other team because someone made a fatal mistake. 

Athletes Unlimited believes we can do this better, and our scoring system is the answer. We know, based on our research, that our evaluation of players reflects other widely accepted standards of performance, which is important to get right when players influence their prize money by how well they play. It also means that if your favorite softball player gets drafted on a weaker team, it doesn’t impact her opportunity to perform. In fact, it’ll motivate her to play harder to improve her league standings and get herself on a better team next week, which means you’re in for a better watching experience.