Performance appraisals, football-style
It’s important to have the best possible people in talent ID and recruitment roles. Each decision to sign or not sign a player can cost or earn a club millions. But how can we best evaluate the judgement of our scouts, when our clubs make just a few transfers each year?
Individual scouts often write reports on hundreds of players in a year, but the vast majority if not all of the players observed will never sign for the club. Therefore as well as evaluating scouts on their judgements of the players we did sign, it makes sense to evaluate their reports on players we didn’t sign. If a scout recommended a player whose career nosedived – or vice versa – that’s useful to know.
This could be done on an anecdotal, case-by-case level if evaluating just a few scouting reports. But such an approach is impossible to scale to evaluating hundreds or thousands of player judgements.
This is where our Player Contribution model can help. Our model tracks the level at which players are playing, and places a value on the minutes they play. Take an imagined scenario in June 2017, where two scouts filed two reports on Freiburg’s Vincenzo Grifo and Sevilla’s Pablo Sarabia. After strong seasons, both had similar market values at the time, and our model suggested their performance levels were roughly equal. Let’s say Scout A recommended Grifo over Sarabia, while Scout B recommended the opposite. Let’s also say our club decided to sign another player, who both Scout A and B recommended. Normally we might just assess the judgements of our scout on this third player, but we can look at the career arcs of Grifo and Sarabia to see who made the better recommendations at the time.
Grifo’s decline in performance in spells at Gladbach and Hoffenheim, and Sarabia’s continued progression at Sevilla means that Scout B probably provided the better assessment of the players. Repeating this exercise for the hundreds of other players that our scouts have reported on gives us a more complete view on who is more consistently providing the better reports on players.
Any organisation – whether in football or otherwise – benefits from evaluating the performance of staff, and feeding that back in an appropriate manner. We’re lucky to be in an industry where that process can be easier than in others; smart clubs use approaches like the one above to improve their talent ID.