Rating our scouts
Player recruitment is a process that should ideally combine objectivity and subjectivity. Both aspects, however, should regularly come under suitable scrutiny and review: we should question whether we are looking at the right data, and we should question the predictions of our scouts.
While the former can require detailed statistical checks, the latter needn’t be an arduous process. As key decision makers, we should have a clear view on what constitutes success from new signings. Our scouts can then provide predictions as to whether a player is likely to meet these success criteria. In due course, the predictions can be assessed against the actual outcome to give an overall score, illustrated below:
In this case, the scout was confident on the player’s ability to become a first-choice player (row 1) and provide at least 8 assists in his first season (row 2). While correct on the player’s appearance record, he is penalised for overconfidence on the player’s assist output, which failed to meet the success criteria standard. He is penalised less for being less confident on the player’s ability to provide a return on investment (row 5), but in overall terms provided a good prediction on the player, denoted by the ‘+4’ score.
Whether the player signs for us, or even a different club (where the scout may make a different set of predictions), this approach allows us to see who are our ‘star’ scouts – the people who consistently make accurate predictions about successful or unsuccessful signings.
Of course, each club will have its own unique criteria, different to the ones above. Some clubs may detail expectations around a player’s ability to his certain on-field performance outputs, while others might value off-field and commercial factors.
Whatever our priorities though, there’s no reason why we shouldn’t better understand the strengths and weaknesses of our scouts. Accumulating predictions over the course of a few windows should provide a useful perspective on key skills and any biases, and allow us to make the best possible decisions in the transfer market.