When the Numbers Get It Wrong: What Sports Analytics Cannot Tell Us

I’ve spent much of the past two years trying to understand how sport puts a price on talent. Whether it’s a footballer’s weekly wages or the amount an IPL franchise is willing to pay for a batsman, I’ve always found it interesting that two players who appear equally good can have completely different valuations. That curiosity led me to build statistical models, study performance data and eventually write a research paper examining inefficiencies in football wages. I started with a fairly simple question: could I use numbers to identify players who were worth more than they were being paid? What I didn’t expect was that some of the most interesting findings would come from questioning the numbers themselves.

For my football research, I developed a linear regression model using UEFA Champions League performance data and applied it across six leagues, including the Premier League, La Liga, Bundesliga, Serie A, Ligue 1 and MLS. The idea was to compare what players earned with what their performances suggested they should earn. If the model consistently valued someone above their actual wages, perhaps that player was being undervalued by the market.

The results were quite interesting. The relationship between predicted and actual wages varied across leagues, with some showing much larger discrepancies than others. At first, these differences seemed to support my original theory about market inefficiencies. Then I began thinking about what I was actually comparing. A player in MLS operates in a very different financial environment from someone in the Premier League. Salary restrictions, league revenues and the way clubs develop or recruit players can all influence wages, regardless of what happens on the pitch. If two players produce similar statistics but earn very different salaries, is one necessarily undervalued? Or am I expecting two very different markets to behave in the same way?

That distinction changed how I thought about my results. A model can identify a gap between predicted and actual wages, but it cannot automatically explain why that gap exists. Some discrepancies might represent genuine opportunities for clubs to recruit overlooked talent. Others might reflect circumstances the model hasn’t accounted for. I realised that identifying an apparent inefficiency was only the beginning. Understanding whether it was actually an inefficiency was a much harder problem.

I encountered a similar challenge while developing my IPL salary predictor. Unlike football, where wages are negotiated through contracts, IPL player prices can be determined through competitive auctions. A player’s value depends not only on individual performance but also on what a particular franchise needs at that moment. Two batsmen might have almost identical averages and strike rates, yet one could command a much higher price because a team desperately needs a finisher rather than another opening batsman.

Consider a franchise with only one overseas slot remaining and a limited auction budget. It might be willing to spend considerably more on a specialist bowler than on a statistically stronger all rounder, simply because the bowler fills a gap in its squad. From the perspective of a performance based valuation model, that decision might appear irrational. From the franchise’s perspective, it could make perfect sense. The auction isn’t just pricing individual talent. It’s pricing talent in the context of scarcity, competition, perceived value and team requirements.

This made me realise that predicting a player’s salary and determining whether that salary is justified are two different tasks. A model might become very good at estimating what teams will pay without necessarily telling us what they should pay. Equally, when a prediction is wrong, it doesn’t automatically mean the market has made a mistake. Sometimes the model simply hasn’t understood what the market is valuing.

Of course, this goes beyond salaries. Anyone who watches football knows that statistics don’t always capture what is happening in a match. A team can dominate possession, create more chances and still lose. A player might contribute very little to the score sheet but completely change the way the opposition defends. Even something as simple as confidence can alter a game, yet it’s difficult to assign it a reliable number.

I find this particularly interesting because I’m naturally drawn to numbers. I like being able to test an idea, compare results and see whether a prediction holds up. But the more models I’ve built, the more aware I’ve become of how much depends on the assumptions behind them. Which variables should I include? How should I weigh them? And perhaps most importantly, what happens when something that matters can’t easily be measured?

Data can reveal patterns that would otherwise be difficult to spot, challenge assumptions about players and help clubs make more informed decisions. But I think there is a danger in treating a model’s output as a definitive answer simply because it comes with a number attached.

What I’m learning from getting it wrong

As I continue developing Armaanalytics, I’m becoming as interested in the differences between predictions and actual outcomes as I am in the predictions themselves. When a model produces an unexpected valuation, I want to know why. Is the player genuinely overlooked. Is there something unusual about the league or auction? Have I missed an important variable? Or is the model telling me something I hadn’t thought to ask?

These questions also extend beyond sport. Employers assess candidates through qualifications and experience, investors value companies using financial projections, and organisations make decisions based on whatever information they can measure. In each case, the numbers are useful, but they rarely tell the whole story. The difficult part is deciding what weight to give them and recognising when something important might be missing.

When I first started researching football wages, I thought the most exciting outcome would be finding a player whose performance suggested he deserved to earn twice as much. I still find that possibility exciting. But now I’m equally interested in understanding why a player might earn half of what my model predicts and whether the model or the market has the better explanation.

I’m not trying to build models that replace human judgement. I’d rather build ones that make us question it, including my own. And sometimes, the most useful thing a model can tell me is that I am wrong!