The foundation of the Athletes Unlimited Volleyball scoring system

However, volleyball is a highly team-oriented sport, with athletes who build entire careers out of specializing in certain positions that function differently. This means that some positions are more likely to perform certain skills than others, creating an imbalance in opportunity to score individual points. In developing the individual scoring system for the Athletes Unlimited volleyball league, we needed to answer two questions:

1. What measurements provide an accurate assessment of a player’s performance in their position, as well as their contributions to the team’s success?
2. How can the chosen measurements allow each individual, regardless of position, to have a relatively equal opportunity to be ranked highly on the leaderboard?   

In part one of this two-part series, we’ll explore the statistical analyses designed throughout the history of volleyball that lay the foundation for the scoring system we have today.

TRADITIONAL STATISTICS

Volleyball has benefited from the work of scientifically oriented coaches in its rise to prominence as an international sport. In the 1960’s, revered head coach Jim Coleman started looking at how to evaluate player performance, building the framework for the traditional statistical methods that are still in use today. At its most basic level, individual performance can be quantified by identifying the type of touch, then assigning a quality grade to the touch. A grade can be as simple as a binary result (e.g. whether a serve landed in or out) or something much more complex (e.g. the serve at what velocity, to what area of the court, what trajectory, with or without float, passed at what level, and if the opponents killed the first attack after the serve).

The benefit of analyzing players by traditional statistical methods is two-fold: Any skill can be recorded in as much detail as someone could want; and, given a large enough sample, the numbers can give us a very good idea of how adept a player is at performing a skill relative to others that perform that same skill.

However, it becomes trickier to assign point values to these statistical numbers. How many points should it be worth for an outside hitter with a hitting efficiency* of .250 for a match? How many points for .300? The scale would become logarithmic in point assignment and would vary by position, quickly becoming complex and confusing. The other issue is that these statistics are helpful in comparing athletes who perform the exact same skills, but they become less helpful when trying to evaluate across the entire spectrum of a position (e.g. comparing an outside hitter that passes to one that doesn’t), let alone across different positions, where the skills performed are much different.

PLUS / MINUS

Basketball statisticians often track something called Plus/Minus, which is a numeric representation of how well a team does when a specific athlete is on the court. During his time at Brigham Young University in the 90’s, coach Carl McGown adapted this statistic for volleyball: “When you are on the court, how many rallies does your team win, and how many rallies do the opponents win?” This is an appealing way of looking at player performance because it’s simple, easy to track, and takes into account the entirety of a player’s contributions, including intangible contributions such as energy or team dynamic.

However, this system raises a few issues within the context of the Athletes Unlimited model. The first is that you need an extremely large sample size to approach valid numbers. The NBA reports that a minimum of 45 games are needed to achieve statistical significance, and the most reliable numbers tend to emerge after five seasons of play, which could mean 350+ games. A five-week season makes achieving this data set size impossible, and we wouldn’t be able to draw out data from a single match this way.

The other thing is that the Plus/Minus method tracks too closely with a reward system that’s already in place. Because players receive points for winning a set, measuring Plus/Minus yields an almost 1:1 proxy for points scored. In essence, we’d be rewarding them on an individual level practically in parallel to what they were already being rewarded on a team level.

PERFORMANCE AGAINST A STANDARD

Another possibility was to create some sort of statistical standard based on world-class players and measure an individual’s performance against that standard. Imagine the best players in the world having their best days attacking — that would be a 10/10 on the standard — and their worst days attacking would represent 0/10. We set out trying to create a distribution curve for international players and initially arrived at what we thought were good models to compare our athletes against.

However, the differences in positional play rear their head again. An outside might have a great night passing and digging, but an average night attacking. Their composite score would be hindered by their attacks and end up lower than the composite score of a libero, who meanwhile spent the match passing and digging worse than the outside, but not attacking at all.

While each of the strategies discussed had meritable aspects to consider, none of them could serve as the foundation for the individual scoring system we needed. However, this exploration led us to work that had been conducted by my colleague, Joe Trinsey, during his time with the United States Women’s National Team. I’ll let him take it from here.

THE ORIGIN STORY

Written by Joe Trinsey

When I first started working with the USA National Team, I was assisting then USA Assistant Jamie Morrison. As part of the U.S. Men’s National Team staff from 2005-2008, Morrison helped launch USA Volleyball into the modern era of volleyball analytics. 

A constant subject of debate was the answer to the age-old volleyball question, “Who’s more valuable: a big hitting outside who struggles to pass, or a ball-control outside who struggles to terminate?” Even setting aside the complex factors of how these two players fit into a team, this is a surprisingly difficult question to answer, and almost no published work in volleyball research addressed this topic to our satisfaction.

I had been researching baseball performance metrics like Wins Above Replacement (WAR) and basketball metrics like Player Efficiency Rating (PER). I knew there had to be a way to create an equivalent metric for volleyball. When Morrison showed me the amount of data he had been compiling on International Women’s Volleyball, my stat-nerd heart jumped for joy! He had the data I needed to create these metrics, and he encouraged me to dive into it to my heart’s content.

I felt that a Markov Chain was the best mathematical tool to solve this problem. Imagine a Markov Chain like one of those cheap carnival games: you drop a ball in at the top and it rattles down and bounces off a whole bunch of pegs and ramps until it reaches the bottom where it inevitably just barely misses dropping in the big prize bucket.

via GIPHY

Fortunately for me, Markov Chains aren’t rigged like carnival games. What they are good at, however, is analyzing the value of processes where events move from state to state, dependent upon the state before it. That sounds a lot like volleyball to me!

A DIFFERENT SCOREBOARD

Imagine you’re looking at a volleyball scoreboard at a match, only this scoreboard is special. Instead of showing the score of the match, it shows you, in real time, the odds of each team winning the point. As each player contacts — or doesn’t contact — the ball, this scoreboard changes to reflect the teams becoming more or less likely to win the point, until the point is finally scored and it shows 1.0 for one team and 0.0 for the other team.

In women’s Fédération Internationale de Volleyball (FIVB) competition, the receiving team wins the rally about 60% of the time. So when the ref blows the whistle, but before the serve has been contacted, the scoreboard would show 0.60 for the receiving team. When the ball is served and crosses the net, but just before the ball contacts the passer’s arms, the receiving team’s probability drops closer to 0.56 because the prospect of a missed serve giving them a free point has now been eliminated.

Let’s say the ball is passed well. The receiving team’s odds of winning are now closer to 66%, so the scoreboard shows 0.66. Now, the setter delivers a perfect set to the hitter with only one blocker in front of her. The odds increase as the scoreboard now shows 0.74 for the receiving team. The hitter hits the ball hard and at the moment the ball clears the net and is headed into the defender’s court, untouched by the block, the odds have increased yet again and the scoreboard now shows 0.97.

The ball glances off the defender’s arms and into the stands, and now the scoreboard finally shows 1.0 for the receiving team and 0.0 for the serving team.

RUNNING THE CHAIN

The hitter gets credit for the kill, but we know that every player on the scoring team contributed to earning the point. In our example, the passer gets some credit, the setter gets some credit, and so does the hitter. On the other team, the server gets some negative credit for allowing the ball to be passed well, and the blocker gets dinged for allowing the ball to be hit by her block. The defender who shanks (aka mishits) the final dig is actually assigned a small negative value, because by the time her action was called upon, the odds were stacked heavily against her.

Markov Chains allowed me to analyze every unique game state to see a team’s odds of winning the rally at any given point. Some of these states are common: every rally begins with a serve, so it’s easy to define the odds of winning the rally after a given quality of serve reception. Other states happen less frequently and require much more data to analyze.

Ultimately, this analysis puts a number to the old volleyball adage of, “better the ball”. Did a player’s action (or lack of action) move her team closer to winning the rally? Can we take all those fractions of points (or small deductions) and add them all up? If so, we can get a numerical value for the player’s overall contribution to winning or losing a match.

When asked to contribute to the Athletes Unlimited scoring system, this analysis immediately came to mind. I knew, however, it would have to be modified in order to be easily referenced and understood by anybody watching the match in real-time. Back to Chris!

CONCLUSION

Written by Chris McGown

Thanks, Joe. By examining previous work done in the field of volleyball, we could start to put the pieces together for our own individual scoring system. Ultimately, they all had upsides, but also substantial downsides. We found that none of the proposed methods answered both of the questions we had set out.

Joe’s work with Markov Chains, on the other hand, provides a method that could successfully be adapted to the system that Athletes Unlimited required. The background calculations would be complex, but they could yield a simple point system to measure individual player performance, one that was both accurate and afforded relative parity across positions. In our next post, we’ll get into the details on how we used this statistical approach to arrive at the system we have now.

*Hitting efficiency is a common statistic that measures how well an athlete executes attacks and is akin to a batting average in baseball. It is calculated using the equation: (kills – errors)/attacks. For more information, visit https://www.theartofcoachingvolleyball.com/calculating-hitting-efficiency/

Have questions? Athletes Unlimited’s team of data analysts and volleyball experts will be answering questions from readers in early February. Send us your questions on Instagram and Twitter @AUProSports, and they may get included in our Q&A!