Who’s Getting Their Money’s Worth From the ABS Challenge System?
Over the weekend, a friend of mine asked if ABS challenge data gets incorporated into WAR. (I have extremely cool friends.) It doesn’t, and I explained that while an individual challenge can absolutely change the course of an inning (or a game, or even a season), a single ball-strike call is of negligible run value out of context. If anything, we should be adding challenges to WPA, not WAR, except WPA only gets calculated on a per-plate-appearance basis, rather than a per-pitch basis.
At any rate, challenges aren’t like catcher framing in the early days. In 2008, the first year for which we have framing data, the difference between the best framer in baseball (Brian McCann) and the worst (Ryan Doumit, the poor guy) was 97.7 runs. At 10 runs per win, well, you can do the math on your own.
Doumit was an outlier, some 36 runs adrift of the back of the pack. In 2007, he was a 2.1-WAR player, with a 109 wRC+ in 279 plate appearances. Add in framing data, and in 2008 Doumit increased his offensive output to a 123 wRC+ in 265 plate appearances… and lost more than five wins, finishing at -3.1 WAR.
McCann, who was one of three catchers with 30 or more framing runs, posted 8.3 WAR, thanks to a .301/.373/.523 batting line from behind the plate. That made him the second-most valuable position player in the league that year, trailing only Albert Pujols.
Once the implications of framing became clear, the gap closed quickly. If the Pirates had known then what we know now, Doumit would’ve been a first baseman or an outfielder. Those who stayed behind the plate quickly learned pitch framing as an art. In 2008, the difference between the best and worst framers in the league (taking Doumit out of the equation for a moment) was about 60 runs. In 2025, the gap was half that size. If you disregard another outlier (a good one this time, Patrick Bailey), every catcher in the league was within 10 runs either side of average, give or take a run or two.
Challenges aren’t like that. Everyone has known from the start that they’re being watched, so this is a skill everyone has known they had to strategize for. And insofar as challenging ball-strike calls is a skill, it’s one that only gets deployed a couple times per game, rather than dozens.
Through this past weekend, Sal Stewart leads the league in run value from challenges won, with 3.7. This metric is a counting stat, unlike framing, which is pegged to league average. There’s no direct penalty for losing a challenge, and while it’s probably possible to quantify the opportunity cost of losing a challenge, I know I’m nowhere near good enough at math to take that problem on. It seems like more trouble than it’s worth.
That means that Stewart, who’s won 23 challenges, gets his 3.7 runs. Troy Johnston, who’s challenged a league-leading six times without a win, gets zero. That’s a rounding error, in the world of WAR.
But batter challenges are a small piece of the puzzle. The opposing pitcher and catcher can challenge as well, and catchers get credit for challenges they initiate while on defense. (Whether it’s fair to debit a hitter for challenges initiated against him is a reasonable methodological question, but I’m going to do it anyway.)
The catcher who’s profited most from challenging this season has been Shea Langeliers, who’s 52-for-86 behind the plate, racking up 7.19 runs. Langeliers, somewhat puzzlingly, has only challenged once as a hitter, but even so, he leads all position players with 6.55 total challenge runs.
At the other end of the spectrum is James Wood, who is a pretty shocking 6-for-19 on challenges. Opposing catchers, by contrast, have won 23 of 35 challenges on pitches thrown to Wood, costing him 4.55 runs. That’s the largest figure in the league by a huge margin: three quarters of a run over second-place Ian Happ, and a full run over third-place Brett Baty. On the whole, Wood is 3.65 runs in the red on challenges. Sucks to be tall in the world of ABS, I guess.
| Name | Total Rev. | Batter Rev. | Overturned | Pct. | Runs | Opp. Rev. | Overturned | Runs | Total |
|---|---|---|---|---|---|---|---|---|---|
| Cal Raleigh | 22 | 15 | 10 | 66.7% | 3.36 | 7 | 4 | -0.66 | 2.70 |
| Sal Stewart | 57 | 41 | 23 | 56.1% | 3.71 | 16 | 9 | -1.27 | 2.43 |
| José Caballero | 33 | 24 | 14 | 58.3% | 2.25 | 9 | 3 | -0.31 | 1.93 |
| Kyle Karros | 28 | 15 | 9 | 60.0% | 2.73 | 13 | 4 | -0.83 | 1.91 |
| Bryan Reynolds | 38 | 19 | 10 | 52.6% | 3.59 | 19 | 13 | -1.70 | 1.89 |
| Nolan Schanuel | 51 | 33 | 19 | 57.6% | 3.40 | 18 | 13 | -1.68 | 1.72 |
| Ronald Acuña Jr. | 26 | 17 | 7 | 41.2% | 1.71 | 9 | 0 | 0.00 | 1.71 |
| Coby Mayo | 18 | 12 | 7 | 58.3% | 1.71 | 6 | 1 | -0.18 | 1.53 |
| Juan Soto | 30 | 16 | 10 | 62.5% | 2.28 | 14 | 7 | -0.76 | 1.52 |
| Heliot Ramos | 17 | 14 | 8 | 57.1% | 1.52 | 3 | 1 | -0.08 | 1.44 |
Good to see that something’s going right for Raleigh this year.
So there’s about a win’s worth of difference between the best challenger this year (Langeliers) and the greatest victim of ABS (Wood). Take out catcher defense and that gap gets narrowed down to about six runs.
Looking at the leaderboard got me curious. Stewart leads the league in challenge runs and in successful challenges. But that’s the result of a spray-and-pray approach. Stewart has challenged 41 calls this year, which is the most in the league. He’s also tied for the league lead in unsuccessful hitter challenges. With no penalty for missing a challenge (at least not by this metric), quantity has a quality all its own.
The second-most successful catcher on the challenge leaderboard is Tyler Stephenson. And in contrast to Langeliers, he’s made five challenges from the batter’s box and won all of them. The only other hitter with as many challenges and a perfect record is Matt McLain, who — like Stephenson and Stewart — is a member of the Cincinnati Reds.
Obviously, some teams will have benefited from ABS more than others.
| Position | Batter | Pitcher+Catcher | Total | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Team | Rev. | OvrTrn | Pct. | Runs | Rev. | OvrTrn | Pct. | Runs | Rev. | OvrTrn | Pct. | Runs |
| MIN | 141 | 67 | 47.5% | 12.0 | 187 | 114 | 61.0% | 21.5 | 328 | 181 | 55.2% | 33.6 |
| ATH | 107 | 53 | 49.5% | 9.0 | 168 | 104 | 61.9% | 21.9 | 275 | 157 | 57.1% | 31.0 |
| CIN | 125 | 67 | 53.6% | 13.5 | 138 | 99 | 71.7% | 16.4 | 263 | 166 | 63.1% | 29.9 |
| COL | 131 | 57 | 43.5% | 10.4 | 154 | 97 | 63.0% | 17.3 | 285 | 154 | 54.0% | 27.8 |
| LAA | 133 | 59 | 44.4% | 13.5 | 138 | 84 | 60.9% | 14.0 | 271 | 143 | 52.8% | 27.4 |
| MIA | 105 | 52 | 49.5% | 10.7 | 174 | 104 | 59.8% | 16.3 | 279 | 156 | 55.9% | 27.1 |
| NYY | 140 | 70 | 50.0% | 12.0 | 158 | 92 | 58.2% | 14.8 | 298 | 162 | 54.4% | 26.8 |
| NYM | 148 | 74 | 50.0% | 12.0 | 135 | 73 | 54.1% | 13.5 | 283 | 147 | 51.9% | 25.5 |
| KCR | 114 | 64 | 56.1% | 12.0 | 140 | 80 | 57.1% | 13.0 | 254 | 144 | 56.7% | 25.0 |
| BOS | 114 | 56 | 49.1% | 11.9 | 109 | 62 | 56.9% | 12.8 | 223 | 118 | 52.9% | 24.7 |
| BAL | 125 | 61 | 48.8% | 13.3 | 110 | 56 | 50.9% | 10.8 | 235 | 117 | 49.8% | 24.1 |
| DET | 119 | 64 | 53.8% | 11.8 | 113 | 76 | 67.3% | 12.2 | 232 | 140 | 60.3% | 24.1 |
| MIL | 86 | 38 | 44.2% | 7.0 | 184 | 100 | 54.3% | 16.8 | 270 | 138 | 51.1% | 23.7 |
| HOU | 132 | 75 | 56.8% | 12.9 | 141 | 72 | 51.1% | 10.6 | 273 | 147 | 53.8% | 23.5 |
| CHC | 115 | 56 | 48.7% | 11.1 | 118 | 79 | 66.9% | 12.3 | 233 | 135 | 57.9% | 23.4 |
| LAD | 121 | 65 | 53.7% | 13.4 | 101 | 61 | 60.4% | 9.9 | 222 | 126 | 56.8% | 23.3 |
| SEA | 119 | 52 | 43.7% | 11.1 | 112 | 66 | 58.9% | 11.9 | 231 | 118 | 51.1% | 23.1 |
| CHW | 108 | 49 | 45.4% | 8.3 | 189 | 87 | 46.0% | 14.5 | 297 | 136 | 45.8% | 22.7 |
| ATL | 101 | 38 | 37.6% | 8.8 | 124 | 75 | 60.5% | 13.8 | 225 | 113 | 50.2% | 22.6 |
| TEX | 102 | 51 | 50.0% | 10.7 | 113 | 72 | 63.7% | 11.6 | 215 | 123 | 57.2% | 22.3 |
| TOR | 112 | 56 | 50.0% | 9.3 | 134 | 76 | 56.7% | 12.7 | 246 | 132 | 53.7% | 22.0 |
| PHI | 133 | 66 | 49.6% | 10.0 | 104 | 67 | 64.4% | 11.9 | 237 | 133 | 56.1% | 21.9 |
| SFG | 117 | 57 | 48.7% | 11.4 | 118 | 62 | 52.5% | 9.9 | 235 | 119 | 50.6% | 21.3 |
| CLE | 116 | 46 | 39.7% | 8.8 | 131 | 67 | 51.1% | 12.1 | 247 | 113 | 45.7% | 20.9 |
| ARI | 100 | 55 | 55.0% | 9.0 | 109 | 73 | 67.0% | 11.5 | 209 | 128 | 61.2% | 20.5 |
| STL | 99 | 47 | 47.5% | 7.8 | 113 | 64 | 56.6% | 12.5 | 212 | 111 | 52.4% | 20.3 |
| WSN | 95 | 41 | 43.2% | 7.5 | 152 | 83 | 54.6% | 12.3 | 247 | 124 | 50.2% | 19.8 |
| PIT | 129 | 58 | 45.0% | 11.3 | 117 | 53 | 45.3% | 8.3 | 246 | 111 | 45.1% | 19.6 |
| SDP | 89 | 38 | 42.7% | 6.3 | 139 | 79 | 56.8% | 12.8 | 228 | 117 | 51.3% | 19.1 |
| TBR | 103 | 45 | 43.7% | 6.7 | 128 | 75 | 58.6% | 11.4 | 231 | 120 | 51.9% | 18.1 |
Again, the volume approach pays off. Across all positions, the Twins have initiated 30 more challenges than the second-place team. But the Reds not only have the highest challenge success rate in the league, they’re getting the most value per challenge out of any team.
| Team | Challenges | Win% | Runs | Runs/Challenge |
|---|---|---|---|---|
| CIN | 263 | 63.1% | 29.9 | 0.114 |
| ATH | 275 | 57.1% | 31.0 | 0.113 |
| BOS | 223 | 52.9% | 24.7 | 0.111 |
| LAD | 222 | 56.8% | 23.3 | 0.105 |
| TEX | 215 | 57.2% | 22.3 | 0.104 |
| DET | 232 | 60.3% | 24.1 | 0.104 |
| BAL | 235 | 49.8% | 24.1 | 0.102 |
| MIN | 328 | 55.2% | 33.6 | 0.102 |
| LAA | 271 | 52.8% | 27.4 | 0.101 |
| ATL | 225 | 50.2% | 22.6 | 0.100 |
| CHC | 233 | 57.9% | 23.4 | 0.100 |
| SEA | 231 | 51.1% | 23.1 | 0.100 |
| KCR | 254 | 56.7% | 25.0 | 0.098 |
| ARI | 209 | 61.2% | 20.5 | 0.098 |
| COL | 285 | 54.0% | 27.8 | 0.097 |
| MIA | 279 | 55.9% | 27.1 | 0.097 |
| STL | 212 | 52.4% | 20.3 | 0.096 |
| PHI | 237 | 56.1% | 21.9 | 0.092 |
| SFG | 235 | 50.6% | 21.3 | 0.091 |
| NYM | 283 | 51.9% | 25.5 | 0.090 |
| NYY | 298 | 54.4% | 26.8 | 0.090 |
| TOR | 246 | 53.7% | 22.0 | 0.090 |
| MIL | 270 | 51.1% | 23.7 | 0.088 |
| HOU | 273 | 53.8% | 23.5 | 0.086 |
| CLE | 247 | 45.7% | 20.9 | 0.085 |
| SDP | 228 | 51.3% | 19.1 | 0.084 |
| WSN | 247 | 50.2% | 19.8 | 0.080 |
| PIT | 246 | 45.1% | 19.6 | 0.080 |
| TBR | 231 | 51.9% | 18.1 | 0.078 |
| CHW | 297 | 45.8% | 22.7 | 0.077 |
The margins, as you can see, are small. The Reds are one one-thousandth of a run per challenge better than the A’s, and just 0.037 runs per challenge better than the last-place White Sox. That comes to one additional win every 270 challenges; at the current league-average challenge rate, that’s a difference of an extra win and a quarter over a 162-game season.
Here’s a graph of the relationship between total challenges and runs earned from challenges, by team. It’s called “Total Challenges vs. Runs, by Team.”
There’s a strong correlation between number of challenges and runs won from challenges.
The correlation between challenge success rate and challenge runs is still strong, but not as much as the raw challenge figure. Which makes sense, considering the lack of penalties for an unsuccessful challenge. But you can see the teams that have benefited the most from their challenges are in the middle of the pack in success rate.
The Reds are way over there on the right side of the chart with their 63.1% challenge win rate. Of the top 11 teams in challenge runs, everyone but Cincinnati is within one standard deviation of the league-average success rate.
Given the data we have here, an extra 2.88% of challenge success rate leads to one additional run from challenges. At the same time, every 11.6 challenges — successful or otherwise — correlates to an additional run.
It looks like teams, on the aggregate, could stand to be more aggressive. We know that’s true across plenty of situations within the game: Baserunning, manager challenges, using the backup catcher. An optimal strategy would occasionally leave a manager facing a serious downside; you don’t want to have rallies stifled by losing baserunners, or be forced to use the emergency catcher.
But while prioritizing avoidance of disaster minimizes the number of awkward postgame press conference questions, it might not be the best guiding principle for winning baseball games.
Ultimately, the effects of ABS challenges are minuscule on the whole; the best and worst teams, let alone players, are separated by little more than a run. But every edge counts. It’s time to loosen the reins.