Starting Pitcher Ratings: Adjusted Game Score

So far in this series, we’ve discussed why an alternative to pitching WAR might be beneficial, and introduced Game Score as the basis on which an alternative might be built. Now, let’s put Game Score through a few basic adjustments so we can make it more viable for historical comparisons.

We’ve seen already how Game Score takes a variety of results (innings, hits, runs, strikeouts, etc.) and combines them into a single number reflecting the effectiveness of a pitcher’s performance in a particular start. But there are still a few obvious factors in the pitcher’s results that have yet to be accounted for: what team was he facing, and when and where did the game take place?

The approach here has two steps. First, find the expected runs per game by the opponent in the park and year in question. To do this, start with the opponent’s raw scoring rate in runs per game, then adjust for their overall park factor (including both home and road games) to get their normalized scoring rate, then multiply that by the park factor of the stadium that hosted the game in question.

If the gritty details of park adjustments don’t interest you, skip past the formulas that are a couple paragraphs down. For anyone still reading, I'm using a regressed three-year park factor, depending on how many of the surrounding seasons the team spent in the same park. (Yes, it’s probably more rigorous to use more seasons of park data, but that also adds more complications when teams move to a new park and would delay finalization of seasonal results even further. As it is, I already had to wait for the 2025 regular season to end so I could run preliminary rankings for 2024.)

The formulas, to be shown below, apply to both overall park factor and home park factor. APF is the adjusted park factor, PF(0) is the raw park factor for the season under examination, and PF (-1) and PF (1) are for the previous and next season, respectively. The formulas are:

One year in park: APF = 0.67*PF(0) + 0.33

Two years: APF = 0.5*PF(0) + 0.25*PF(±1) + 0.25

Three years: APF = 0.4*PF(0) + 0.21*(PF(1) + PF(-1)) + 0.18

Park adjusted opponent scoring can have a pretty wide range even within a single season. If you compare the figures across history, the gap is often prodigious. The 1999 Giants have an enormous expected output of 8.36 runs per game in Coors Field, while the 1968 Mets in Dodger Stadium would anticipate an anemic 2.47. (Just for fun – the ’99 Giants actually scored 58 runs in their 6 games in Coors, averaging 9.67; the ’68 Mets put up 21 in 9 games in Dodger Stadium, or 2.33 per.)

None of the above is original; park and league context adjustments are one of the first topics of sabermetric study. The question for our purposes is, how do you apply this to Game Score, which is not laid out in run-based units?

This leads us to step 2: develop a Game Score adjustment based on scoring environment. Early in my pitching-based work, I decided to use a single formula across history for this purpose rather than modifying it year-to-year. This approach has benefits and drawbacks. One of the primary benefits at the time was making the numbers easier to work with, but it’s also nice to have a fixed formula that can capture changes in average starting pitcher production over time. The downside is a fairly marginal loss in accuracy in the adjustment, which would primarily show up in extremes that most pitchers won’t see very often (even the 1968 Dodgers don’t face the Mets at home in every start, after all).

To generate this adjustment, I pulled the GS2 numbers for three seasons: 1965, 1978, and 1994. This gave a wide range of scoring environments, but kept the league context reasonably modern without being influenced by the wonkier tendencies of current-day pitching (such as the opener). I ran a regression of GS2 against total runs scored in the game for each year, which gave the following results:

1965: GS2 = 75.7 – 2.79*R

1978: GS2 = 74.6 – 2.68*R

1994: GS2 = 72.6 – 2.46*R

Note that for any relatively normal scoring context (say, total runs per game between 7 and 11), there is less than one point of difference between the outcome of any of these formulas; for R=9 and R=10, the differences are 0.2 or less. For simplicity’s sake, I ended up using a rounded compromise formula:

Expected GS2 = 75 – 2.7*R

If you want the average to end up as 50, you adjust the pitcher’s actual Game Score by adding the difference between 50 and this value. Also, since R was expressed in total runs per game (rather than runs per game for the individual team), we double its coefficient to account for park-adjusted opponent scoring:

GS2 Adjustment = 5.4*(PAOS) – 25

For a pitcher facing the aforementioned ’99 Giants in Colorado, this adjustment is +20.2 points of Game Score; for the ’68 Mets visiting LA, it’s -11.7.

What do these adjustments look like over a full season? We’ll stick with ’68 and ’99 as extremes in each direction. In 1968, the pitcher with the most hitter-friendly set of opponents and environments (with at least 25 starts) was Joe Niekro of the Cubs, whose Game Scores are adjusted an average of -4.7 points; on the pitcher-friendly side, it’s LA’s Don Drysdale with a -7.9. In 1999, the friendliest conditions were given to San Francisco’s Shawn Estes with a -0.1, while the most hostile were unsurprisingly suffered by the poor souls condemned to Colorado, with Brian Bohanon taking the crown at +8.8. Note the complete lack of overlap between the two seasons; in fact, there’s less distance between the extremes in 1968 than there is between the highest ’68 adjustment and the lowest ’99 adjustment.

How does the formula fare over time? Average opponent-park adjusted GS2 (OPAGS2) by decade:

Years	OPAGS2
1901-09	56.1
1910-19	54.0
1920-29	52.1
1930-39	51.6
1940-49	50.8
1950-59	49.6
1960-69	50.2
1970-79	50.0
1980-89	49.4
1990-99	49.6
2000-09	49.0
2010-19	49.4
2020-24	48.9

That stabilizes reasonably quickly; once the early modern game sets in (integration and expansion), the average only varies by a point or so. There’s still an obvious difference in the earliest part of our dataset, but that’s understandable – and ultimately we’ll be adjusting for league context on a yearly basis, so the final numbers won’t be polluted by this change.

Speaking of polluting the numbers, one additional note I should make before talking results: I am including postseason starts in the consideration set. Yes, this is different from my approach in the weighted WAR system, but in that case I was constrained by what Baseball Reference factors into its WAR calculations. Here, I’m building my own system and can use whatever data I choose. And I choose to include playoff starts for two reasons. First, the postseason is considered MORE important than the regular season by the teams, players, and fans. For pitchers in particular, good or bad postseason performance can have an enormous impact on how the player is regarded. (Ask Madison Bumgarner... or Clayton Kershaw.) Second, postseason participation often costs pitchers regular season starts, either in the same year (as the team manages workload and sets the playoff rotation) or in the future (due to wear and tear, and sometimes due to immediate injury). Counting postseason stats does raise some issues of fairness to pitchers on bad teams, but it strikes me as being a more reasonable option than ignoring them.

All right, you’ve stuck with me through three posts (and a few dozen repetitions of the word “adjustment”), so it’s time for our first actual fun table of numbers. Here are the top 50 single seasons in average adjusted Game Score (25-start minimum):

Rank	Year	Pitcher	Starts	OPAGS2
1	2000	Pedro Martinez	29	79.1
2	1999	Pedro Martinez	31	76.2
3	1913	Walter Johnson	36	74.9
4	1910	Ed Walsh	36	74.3
5	1912	Walter Johnson	37	74.0
6	1994	Greg Maddux	25	73.5
7	1918	Walter Johnson	29	73.4
8	1901	Cy Young	41	73.0
9	1997	Roger Clemens	34	72.9
10	1968	Bob Gibson	37	72.6
11	1905	Ed Reulbach	29	72.3
12	1997	Pedro Martinez	31	72.2
13	1931	Lefty Grove	33	72.1
14	1995	Greg Maddux	33	71.9
15	1924	Dazzy Vance	34	71.8
16	1915	Pete Alexander	44	71.6
17	1910	Russ Ford	33	71.5
18	1909	Mordecai Brown	34	71.4
19	1919	Walter Johnson	29	71.3
20	1905	Christy Mathewson	40	71.3
21	2001	Randy Johnson	39	71.3
22	1902	Rube Waddell	27	71.0
23	1995	Randy Johnson	33	70.7
24	1912	Smoky Joe Wood	41	70.6
25	1946	Hal Newhouser	34	70.1
26	1928	Dazzy Vance	32	69.8
27	1908	Mordecai Brown	32	69.8
28	1985	Dwight Gooden	35	69.7
29	1910	Walter Johnson	42	69.7
30	1911	Ed Walsh	37	69.6
31	1965	Sandy Koufax	44	69.5
32	1908	Cy Young	33	69.5
33	1940	Bob Feller	37	69.4
34	1936	Lefty Grove	30	69.3
35	1915	Walter Johnson	39	69.3
36	1902	Cy Young	43	69.3
37	1932	Lefty Grove	30	69.2
38	1999	Randy Johnson	36	69.2
39	1972	Steve Carlton	41	69.2
40	1936	Carl Hubbell	36	69.1
41	1969	Bob Gibson	35	69.1
42	1914	Russ Ford	26	69.0
43	1971	Tom Seaver	35	68.9
44	1914	Walter Johnson	40	68.9
45	1986	Mike Scott	39	68.9
46	1911	Vean Gregg	26	68.8
47	1914	Claude Hendrix	37	68.8
48	1902	Bill Bernhard	25	68.7
49	1937	Lefty Gomez	36	68.6
50	1935	Cy Blanton	30	68.6

If I may say so, that’s a pretty fun list. You can definitely see the effects of the change in pitcher usage –Russ Ford makes as many appearances as Roger Clemens and Sandy Koufax combined. But I don’t mind letting the deadball pitchers have a moment in the sun; you can already infer from the table of league averages that their numbers are going to have some serious air let out moving forward.

Also, even with the deadball advantage on the table, the difference between Pedro Martinez’s highest average and anyone else’s highest average is the same as the gap between #3 and #23 on the list. Pedro, it turns out, was pretty good.

Pedro was not, however, especially durable; note that his exceptional 2000 season included only 29 starts, while several of the seasons behind him had totals in the high 30s and a few even cleared 40. Next time, we’ll take our context-adjusted Game Score and work on turning it into a more robust measure of seasonal performance, one that balances excellence and availability.

Starting Pitcher Ratings: Game Score

To introduce this series, I discussed the three major WAR systems for pitchers and the issues I have with each of them. Having pointed out the problem, let's get to work on a solution.

Whatever issues I may have with it, one of WAR’s benefits is very compelling: it reduces the player’s complex contributions to a single number, which can be compared to a similarly derived single number for other players. How does a pitcher with a 2.60 ERA and 220 strikeouts in 280 innings in 1968 compare to a pitcher with a 3.40 ERA and 250 strikeouts in 210 innings in 1999? After sorting through all the complications under the surface, WAR can give you a simple answer. If I’m suggesting an alternative to WAR, it should ideally come in a similarly straightforward package. Fortunately, just as in many other statistical problems in baseball, we can start by borrowing from the work Bill James. In this case, we’ll use the Game Score.

(Before anyone else says it: Yes, Game Score is only set up for starting pitchers; it is not generally applied to relief outings, even relief outings made by pitchers who are usually starters. We’ll look more at the effects of this on the rankings once we get far enough to actually discuss rankings, but there definitely is an effect. It is also worth pointing out that we can only pull Game Scores for pitchers who have box score records available. Since I’m pulling my data from Baseball Reference, that means I have nothing before 1901, and nothing for the Negro Leagues. So we’re working with half of Cy Young’s career, under 20% of Kid Nichols, and none of Smokey Joe Williams.)

James introduced Game Score in the ‘80s as a fun way to evaluate the effectiveness of a starting pitcher’s effort in a particular game. His original formulation is:

GS = 50 + Outs + Strikeouts – Walks – 2*Hits – 4*(Earned Runs) – 2*(Unearned Runs) + 2*(Innings completed after the fourth)

The description is a little clunky (and so is implementing it in a spreadsheet of game logs), but the results are pretty reasonable. League average will usually be around 50. 60 is a solid start, 70 is a very good one, 80 is excellent, and 90 will usually be among the 25-50 best overall starts of the season. The same is true in the other direction; 40 is shaky, 30 is pretty bad, 20 is lousy, etc. My initial foray into starting pitcher evaluation used the original formulation of Game Score, and it generally performed well… until it didn’t.

The major issue I ran into deals with the earned/unearned run distinction. Whether this distinction has value in general is a bit of a rabbit hole (most of the research I’ve seen suggests it does not), but even if you think it’s worth using earned runs in some circumstances, it becomes a huge problem trying to use them for game-by-game evaluation of baseball seasons from over a century ago.

Here is a sample of the Baseball Reference game log page for Christy Mathewson’s outstanding 1905 season. See if you can spot the issue:

Yeah, that’s not going to work. We know Matty’s overall total of earned runs allowed for the year (48, out of 85 runs total), but game-to-game we have no idea of his earned/unearned run split.

Having encountered this problem, I went looking for an alternative Game Score formulation. Fortunately, Fangraphs already had one, courtesy of the inimitable Tangotiger. Game Score 2 is laid out here, and the formula is as follows:

GS2 = 40 + 2*Outs + Strikeouts – 2*Hits – 2*Walks – 3*Runs – 6*(Home Runs)

To clarify, the home run is also a hit (and will result in at least one run scoring); the 6 points subtracted are in addition to the other penalties, just as the point for a strikeout is added on top of the points already given for outs in general. If you’re curious about where the specific coefficients come from, Tango explains it further here (link is to part 3, which in turn links to parts 1 and 2). The short version is that it’s a combination of the run value of each event type and the pitcher’s estimated share of responsibility for those events. To take the obvious example, a hit is more damaging than a walk, but the hit is partly the responsibility of the fielders whereas the walk is completely the pitcher’s fault, so the pitcher is penalized the same amount for each of them.

As noted earlier, the GS2 formulation resolves the issue of missing data by removing the earned/unearned run distinction. Conveniently, it also addresses another problem with Original Recipe Game Score, which is the handling of short starts. Usually if your pitcher gets pulled after one inning, you’re going to consider that a below average result. As such, rather than starting each game at the expected average of 50, it makes more sense to start from a below-average 40 and build up from there. (Sorry, openers! But not all that sorry. We’ll come back to them later in the series.) For additional detail, here is a table of the baseline score earned by a pitcher who completes each whole number of innings 1-9 in both Game Score versions:

IP	GS1	GS2
1.0	53	46
2.0	56	52
3.0	59	58
4.0	62	64
5.0	67	70
6.0	72	76
7.0	77	82
8.0	82	88
9.0	87	94

The modified formula places a higher premium on lasting longer in the game, but also includes larger penalties for negative outcomes; the 5-point baseline difference for a 7-inning start can be erased by allowing one solo home run (worth -6 points in GS1, 2 for the hit and 4 for the earned run; -11 in GS2, 2 for the hit, 3 for the run, 6 extra for the homer).

The wider variance in GS2 can be more fully demonstrated by looking at a bit of seasonal data. In 2016, the original Game Score formula produced no scores over 100, and five negative scores; GS2 (with my modifications, to be revealed shortly) saw six starts clear 100, and a whopping 31 dip below 0. The standard deviations differ accordingly (16.8 for GS1, 19.5 for GS2). These results are pretty typical.

The improvements in handling unearned runs and truncated outings combine to make GS2 my clear choice of metric moving forward in this project. Being who I am, though, I couldn’t resist making a couple of small modifications. You may have noticed in the screenshot above that Baseball Reference has hit batters and wild pitches recorded on a game-by-game basis even as far back as 1905, and I can think of no reason not to account for those when I already have the numbers handy. (Other data, such as stolen bases allowed, pickoffs, passed balls, or intentional walks, might also provide value, but aren’t available for the oldest games in the sample. Balks are available, but are so rare that it’s impossible to imagine them making a difference in an overall evaluation. To pick a season at random, MLB in 1955 saw 506 total hit batters, 477 wild pitches, and only 36 balks.)

How do we adjust for hit batters and wild pitches? HBP are easy; they are exactly equivalent to walks and are therefore penalized equally (-2 points). WP are more complicated. They’re similar to stolen bases, which run estimators generally consider to be worth less than half as much as a walk. But in comparison to stolen bases, wild pitches are more likely to score a run from third, sometimes advance multiple runners at once, and when coming at the end of a strikeout, can even occasionally put a runner on base, which should make them proportionately more damaging. We’ll value them at -1 point. The final formula is:

GS2 = 40 + 2*Outs + Strikeouts – 2*(Hits + Walks + Hit Batters) – (Wild Pitches) – 3*Runs – 6*(Home Runs)

To get a sense of Game Score’s scale, here are examples of various Game Scores, each of them an actual start from the 2016 season:

Pitcher	Date	IP	H	R	BB	K	Other	GS2
Danny Duffy	8/1/16	8	1	0	1	16		100
Matt Shoemaker	7/16/16	7.1	3	0	0	12		90
Jake Arrieta	5/14/16	8	3	2	2	11	1 HBP, 1 WP	80
Hector Santiago	6/15/16	6	2	1	2	5		70
Chris Archer	8/1/16	7.1	6	3	1	6	1 HR, 1 WP	60
Jharel Cotton	9/13/16	5.2	7	3	1	2	1 WP	50
Robert Gsellman	9/9/16	5	7	4	2	6	1 HR	40
Joel De La Cruz	8/10/16	4	7	4	2	2	1 HR	30
Jordan Lyles	5/23/16	2.1	5	6	3	3	1 HBP, 1 WP	20
Jacob deGrom	8/24/16	4.2	12	5	2	3	3 HR	10
Chris Young	6/25/16	2.1	7	7	4	2	2 HR, 1 WP	0

However you feel about the details, the broad strokes are reasonable; those starts generally get worse as you go down the list.

Let’s consider the characteristics of this formula. It includes aspects of pitching performance beyond simply counting innings and runs allowed, but also reflects actual hits and runs rather than completely disclaiming the pitcher’s responsibility for events to which other players contribute. It indirectly accounts for fielding by putting extra weight on things that are clearly the pitcher’s responsibility (strikeouts, walks, and homers) compared to things that have shared responsibility (hits and runs), but doesn’t assume a fixed level of fielding contribution based on a team’s overall performance. Which is to say that, to varying extents, it addresses the issues I have with all three versions of pitcher WAR. That doesn’t make it a perfect metric – but it does make it an intriguing option to build around.

Next time, we’ll continue the building process by going through a central question of any comprehensive baseball measure: how do you adjust for context?

Starting Pitcher Ratings: Motivation

Around this time last year, I wrote a series of posts introducing the Weighted WAR framework for evaluating position players. At the time, I specifically excluded pitching value from consideration. At the risk of being a bit self-indulgent, I’ll quote my own post from last year on this topic:

“The question still remains: which form of WAR to use? This is a thorny question for pitchers, because unlike the batting and fielding components of different WAR systems (which may differ in outcome but are generally trying to accomplish the same thing), pitching WAR varies philosophically from system to system. To hugely simplify a complicated topic, Baseball Reference WAR starts from pitcher runs allowed and makes a blanket adjustment for the general quality of the team’s fielders. Fangraphs WAR uses fielding independent numbers (strikeouts, walks and home runs) to measure a pitcher’s entire contribution and assumes any variance outside of that is attributable to the team’s fielders. And RA9 WAR (also presented by Fangraphs these days, and used in other sources as well, including Seamheads for NeL stats) takes the pitcher’s park adjusted runs allowed and ignores any fielding effects entirely.

I don’t particularly like any of the freely available WAR systems for pitchers; I think they all simplify the pitcher-fielder dynamic too much and produce extreme outcomes as a result. As such, both because of the extra work that would be involved and a lack of faith in a reasonable outcome, I’m not considering pitching value as part of this project. Again, this is by no means intended to imply that pitching value shouldn’t count; I may circle back to this topic in the future.”

Welcome to “in the future.” In my next several posts, I’ll be introducing a new method of evaluating starting pitchers, one which I think provides a compelling alternative to any of the readily available WAR systems. But before we get into that, it’s probably a good idea to further explore those WAR systems and explain why I think an alternative is worth introducing to begin with.

As mentioned above, there are three versions of pitching WAR that can be easily found on major baseball websites. I’ll take them on in reverse order of my own preference.

My least-favorite option is also the simplest: RA9 WAR. RA9 stands for runs allowed per 9 innings; it’s similar to the more-familiar ERA, but with unearned runs included (a choice that we’ll discuss later in the series, but one that I agree with). RA9 WAR simply park adjusts the pitcher’s rate of runs allowed (different sources may use different styles of park adjustment, either general to the team or specific to the pitcher), then compares it to the league’s replacement level for the year. Outside of the park adjustment, no other modifications to the pitcher’s performance are made. In particular, there is no attempt to separate the pitcher’s performance from the contributions of the fielders behind him (whether helpful or harmful), or from his bullpen’s success or failure in stranding runners who were on base when the pitcher was removed.

To take an extreme example: Say the starting pitcher records two outs in the seventh inning, then strikes out the next hitter, but the ball gets away from the catcher and the hitter reaches base. The starter is pulled from the game, and the new pitcher immediately allows a line drive into the gap. The center fielder has a chance at the ball, but can’t quite reach it; the right fielder picks it up and tries to throw out the runner at home, but his throw is slightly up the line and the runner scores. At minimum, five players contributed to allowing that run, and one could argue that the starter bears the smallest share of that responsibility. But the official stats charge him with the run, and therefore so does RA9 WAR.

While most runs allowed are not this heavily skewed away from the pitcher’s own contributions, it’s still obvious that the efforts of the fielders and relievers are crucial pieces of context for the starter’s performance. If you compare Jim Palmer and Rick Reuschel’s 1974 seasons, but fail to account for the fact that Palmer was pitching in front of brilliant fielders such as Mark Belanger and Brooks Robinson and Paul Blair while Reuschel suffered through the butchery of Bill Madlock and Rick Monday, you won’t get the full picture of their relative effectiveness. Ignoring fielding and relief work entirely strikes me as eminently counterproductive.

Next up, we’ll go to the other extreme with FIP WAR, primarily presented on Fangraphs (and therefore usually abbreviated as fWAR). Per Fangraphs’ own explanation of their system, fWAR uses five components on a per-inning basis: home runs, walks, batters hit by pitch, strikeouts, and infield popups. (Walks and HBP are treated as equivalent, as are strikeouts and popouts.) The theory behind this is that these components are the best representation of the pitcher’s actual ability, as they are the events for which the pitcher is most directly responsible. As such, they also tend to have better year-to-year correlations than rates of hits or runs allowed.

As a predictive measure of pitching ability, FIP is almost certainly preferable to RA or ERA; it does in fact correlate better year-to-year, both with itself and with future ERA. But WAR is not generally advertised as a forward-looking projection system; it’s presented as a measure of backward-looking value. A pitcher who posts a solid FIP but also allows a high number of line drives to remote areas of the park might still have a bright future, but the actual hits and runs he allows still hurt the team.

FIP is also overly reductive in that it ignores less-measurable but still existent pitcher abilities, including things like wild pitch/passed ball tendencies, ground ball rate, and performance with runners on base (some pitchers work better from the stretch than others). Just to pick one, let’s focus on the pitcher’s own fielding, specifically with regard to four-time Gold Glove winner Mark Buehrle. Buehrle was one of the best-fielding pitchers in the game on batted balls, and also a master of controlling baserunners. He allowed 59 total stolen bases over his 16-year career (less than four per season); by comparison, he picked off an even100 runners (just over six per year). Given that FIP by definition ignores the pitcher’s own fielding, it is not a coincidence that Buehrle’s career RA9 WAR (59.9) exceeds his fWAR (52.3) by a substantial margin, and this is clearly a case in which fWAR gets it wrong. There are enough similar errors (in both directions) to reduce the utility of fWAR for purposes of historical evaluation, which is my primary interest.

That brings us to Baseball Reference WAR (generally called bWAR), which is my preference among these options. Like RA9 WAR, bWAR starts with the pitcher’s park adjusted runs allowed. However, it then adjusts for the quality of the fielders behind the pitcher based on his team’s performance in the fielding metric used in position player WAR calculations (which changes depending on the level of data available for the year in question). This effect is calibrated for the number of balls in play the pitcher allows; it will typically be worth between 0.5 and -0.5 runs per 9 innings on all but the most extreme teams.

Sounds great, right? Unlike fWAR, it doesn’t ignore actual runs allowed, but unlike RA9 WAR, it accounts for the team’s fielders. And in practice, this usually works reasonably well; there’s a reason it’s my preferred option among the three WAR systems. But just as with the other two, there are edge cases that cause trouble. The primary problem here arises from the assumption that all of the pitchers on a team get the same level of support from the team’s fielders – an assumption that is not ridiculous on its face, but can be cast into doubt by a few examples.

In this case, let’s look at a quartet of Texan aces in 2019: Justin Verlander and Gerrit Cole of the Astros, and Mike Minor and Lance Lynn of the Rangers. All four put up very good seasons, and all four received Cy Young votes. By conventional numbers, Cole (20-5, 2.50 ERA) and Verlander (21-6, 2.58) were solidly better than Lynn (16-11, 3.67) or Minor (14-10, 3.59). Unsurprisingly, given the difference in their ERAs, RA9 WAR prefers the Astros pair as well; the gap narrows somewhat thanks to Texas’s status as a hitter’s park, but the AL’s top four still goes Verlander 9.0, Cole 8.0, Minor 6.5, Lynn 6.3.

Houston, however, was a much better team than Texas (or almost anyone else; the Astros won 107 games), so it’s not a surprise to learn that they had better fielders. You might think, then, that FIP would close the gap for the Rangers. And it does… for one of them. Lynn’s FIP of 3.13 is a marked improvement on his 3.67 ERA, while Verlander’s 2.58 ERA balloons into a 3.27 FIP. Cole, on the other hand, posted a 2.64 FIP (thanks to an absurd 13.8 strikeouts per 9 innings), very comparable to his ERA. And Minor? His 3.59 ERA climbs to a 4.25 FIP. As such, fWAR has a top 3 of Cole-Lynn-Verlander, and Minor doesn’t show up until the #8 position.

And what does bWAR make of this group of pitchers? Well, as noted, bWAR evaluates Houton’s fielders as very good (bWAR’s fielding adjustment for Cole is +0.46 runs per 9) and Texas’s as very bad (Lynn’s adjustment is -0.30). So, starting from the baseline of RA9 WAR and accounting for quality of fielding, bWAR gives a result of Minor 8.0, Lynn 7.7, Verlander 7.4, Cole 6.7.

This is a bit surprising, given the other results, so what’s going on? As noted, B-R gives Minor substantial credit for pitching effectively in front of a bad defense. As a sanity check for this, let’s look at batting average on balls in play (BABIP for short). The AL average BABIP in 2019 was .299; the Rangers allowed a mark of .314, 15 points higher than average and consistent with a poor fielding team. (By contrast, Houston allowed a .272 BABIP.) Lance Lynn’s BABIP allowed was slightly worse than his team’s, at .322; adjusting for this seems totally fair in his case. As for Minor? His BABIP allowed was .288 – lower than the AL average. There are three possible interpretations of this. First, hitters had a tougher time hitting Minor and his balls in play were easier for the Rangers’ subpar fielders to handle. Second, the Rangers’ fielders performed better with Minor on the mound than they did otherwise, for whatever reason. Or third, both factors had an effect, and the relative extent of those effects is unknowable.

Any time a discrepancy of this type arises, bWAR uses the first interpretation. As such, it is also intermittently prone to startling swings toward or against certain pitchers. And that is how you get 2019 Mike Minor (3.59 ERA, 4.25 FIP, .288 BABIP) rated substantially ahead of Gerrit Cole (2.50 ERA, 2.64 FIP, .276 BABIP). For an even more extreme example, Aaron Nola in 2018 allowed a .254 BABIP for a team that bWAR thinks was below average in the field by 0.41 runs per 9 innings; as a result, bWAR rates Nola’s admittedly very good campaign as the single best season any pitcher has had in the last 15 years, narrowly outpacing Jacob deGrom’s 1.70 ERA/1.98 FIP effort the same year.

Where to go from here? We have three options for pitching WAR, but all three seem to have the same failure point: the pitcher-fielder interaction. In my opinion, all three systems handle this interaction too rigidly by either by ignoring it entirely, assuming the pitcher can only affect a few specific outcomes,  or assuming that all pitchers on the same team have the same fielding performance behind them. It would probably be possible to amalgamate all three systems into some sort of WAR hydra, allowing each system’s extremes to be smoothed out by the other two.

Or we could abandon the WAR framework entirely and do something completely different. Next time, we’ll start to explore the basis for a new option.