An average cost Markov decision process model to decide when to challenge a call in a tennis match

In a standard tennis match each player has an unlimited opportunity to challenge an umpire’s call, but if three incorrect challenges are made in a set he is not allowed to challenge anymore in that set. If the set goes into a tie break the limit on incorrect challenges increases by one. These limited incorrect challenges are not carried over from one set to another. So this is kind of a limited resource available to the player and if he knows how to use this resource in a best possible way, there is a scope for increasing his overall chances of winning a match. With the motive of gaining insight on when to challenge a call, we have modeled a single game in a tennis match as a Markov decision process. We have also studied the impact of variables like player’s probability of winning a point, the player’s perception of the challengability of a call and proportion of challengable calls on the decision making process.