Exploring Top Prospect Lists: Repeaters

So far in the examination of top prospect lists, we’ve introduced our Baseball America dataset and examined prospect performance by ranking, role, and age. This time, we’ll be looking at a trait that is less inherent to the player than either of the last two breakdowns. The question at hand: has the player appeared in a previous top 100 list?

A couple of notes to start off. First and most obviously, this question inherently limits our data even more than usual; we can’t use the 1990 list to find the answer, since there was no previous top prospect list that could have included those players. That leaves us with 2200 player-entries rather than 2300. Second, I’m not technically looking at whether the player has appeared on any prior list, I’m looking at whether he was on last year’s list specifically. This should make a minimal difference; there are some dropouts who later return, but likely less than 100 in the sample. In exchange for this small loss of information, we get two benefits: this approach is easier to implement, and it opens up an extra check we can run.

For starters, let’s do the simple yes-or-no question: How many players in our sample were repeaters? Of the 2200 entries we’re considering, 909 had been on last year’s list (about 41%); 1291 had not. By ranking bucket, here’s the breakdown:

Rank	Repeater	Newbie
1-10	163	57
11-25	172	158
26-50	253	297
51-75	179	371
76-100	142	408

Unsurprisingly, it’s far more common for a top-10 player to be a returner than for someone toward the bottom of the list. The other 26% of the top 10 will be predominantly players who weren’t eligible for the list in prior seasons; it would be a rare prospect indeed who made the leap from afterthought to future star in one year.

Usually, we would jump into percentile tables here – but this time, I’m going to precede the tables with the introduction of our second question of the day. We can easily check whether the player was on last year’s list, but why stop there? If he was on last year’s list, where did he appear, and how does that compare to this year?

This is a trickier question than one might think. The immediate instinct is to examine this by looking at change in ranking, but that is easily dismissed as a viable option; going from #93 to #73 is a very different outcome than going from #23 to #3, even if both are 20-position improvements. I settled on an option that made me wish I’d been using different ranking buckets in all the prior posts in this series: square roots. If the square root of a player’s ranking decreases by at least 2 between seasons (say, if the ranking goes from 100 to 64, or 25 to 9), we’ll call that a large jump; if it changes by the same amount in the other direction, we’ll call it a large drop. Everything else, we’ll consider reasonably similar to the prior year. Here are the player totals in each category:

Rank	Jump	Similar	Drop
1-10	77	86	0
11-25	70	94	8
26-50	85	137	31
51-75	5	122	52
76-100	0	78	64

The 0’s are there for obvious reasons; it’s impossible to have your square root improve by 2 and stay between 76-100, and nearly impossible to do the opposite while staying in the top 10. The 8 and 5 totals in the next categories toward the middle are small enough both to demonstrate the unlikelihood of those criteria being met, and to be discarded as usable samples. But the other groups are big enough to be functional.

(Just for the sake of completeness: The five jumpers in the 51-75 range include four unimpressive results, but the fifth player was Jim Thome. Three of the eight droppers from 11-25 posted at least 10 WAR: Brad Penny, Josh Hamilton, and Desmond Jennings. Hamilton is more or less the essence of what you hope to eventually get out of a declining prospect, as he eventually recovered to have a few seasons consistent with his initial hype.)

On to the percentiles! We’ll start with top-10 players, broken down between new, similar, and jump:

Percentile	Jump	Similar	New
90	57.6	59.1	56.6
80	41.6	41.8	39.3
70	34.5	27.9	30.5
60	23.6	17.2	21.2
50	18.3	11.9	15.9
40	11.9	7.3	12.0
30	9.1	5.0	7.5
20	7.3	0.5	3.2
10	2.5	0.0	0.0

Not overpowering differences, especially at the top end. But for anything between 20th and 70th percentile, you’d rather have someone new to the top 10 (one way or another) than someone who’s hung out there for a while. There may be something to this, as the lion’s share of top-10 prospects will usually graduate to the majors by the end of the season if they perform well and stay healthy. However, our method of selection may also be contributing to this effect, as jumpers are more heavily weighted toward the top of the rankings than players who stayed in similar ranges (that is, a player ranked #2 is more likely to be a jumper than a player ranked #8).

Next group, rankings 11-25:

Percentile	Jump	Similar	New
90	49.3	35.0	43.4
80	31.4	26.8	26.4
70	27.8	19.2	18.3
60	16.2	14.4	12.3
50	9.5	8.7	7.2
40	6.0	6.6	4.1
30	2.8	2.5	0.6
20	0.4	-0.1	0.0
10	-0.2	-1.6	-0.8

The jumpers stand out again, this time more on the high end than toward the middle. Again, there’s likely some quality leakage going on here; ranks 11-13 include 23 jumpers over our sample, while ranks 23-25 include only eight. By comparison, I’m surprised how competitive the new player category is, as this group skews lower in the rankings.

With that, let’s get to the middle group (26-50), the only one in which we have both Jump and Drop categories in a reasonable sample size:

Percentile	Jump	Similar	Drop	New
90	37.1	34.3	38.0	31.9
80	20.4	17.7	21.2	18.6
70	12.9	11.5	13.5	12.3
60	8.2	6.5	8.9	8.7
50	6.0	5.0	5.6	4.2
40	2.6	1.7	2.8	1.5
30	1.4	0.1	0.0	0.1
20	0.0	-0.4	-0.3	-0.2
10	-0.8	-1.0	-1.2	-0.6

Unfortunately, what might in theory have been the most interesting group (since it covers the most categories) produces results that honestly look like a lot of nothing. The Similar group is maybe a bit on the low end (it doesn’t lead a single category), but not significantly so; meanwhile, Drop scores highest in the upper percentiles, but by small margins each time. Let’s move on to the 51-75 and 76-100 tables, in order.

51-75:

Percentile	Similar	Drop	New
90	27.5	20.3	28.2
80	19.6	13.2	15.2
70	14.3	8.7	8.5
60	8.4	3.5	4.6
50	2.0	1.6	1.4
40	0.3	0.7	0.1
30	0.0	0.0	0.0
20	-0.5	-0.1	-0.3
10	-1.3	-1.2	-0.9

Similar stands out nicely in the upper middle here, which is a nice change of pace compared to its performance in every other ranking split so far.

76-100:

Percentile	Similar	Drop	New
90	15.3	15.1	22.4
80	6.5	11.9	12.9
70	2.1	5.8	8.4
60	0.4	1.6	4.7
50	0.1	0.8	1.4
40	0.0	0.0	0.1
30	-0.6	0.0	0.0
20	-0.9	-0.3	-0.3
10	-1.3	-1.1	-1.0

This one makes sense to me – your 76-100 prospects are guys that you don’t have enormous expectations for, but you hope they’ll progress. If a guy is in that range twice in a row, he failed to progress the prior year. If he dropped into that range, then you actually had higher hopes for him previously, and if he can put it together then the original expectations might still be realized. And if he’s new, then the initial hope of progress hasn’t yet been denied. (None of the medians are especially impressive, but that’s to be expected from the bottom group.

Conclusions appear to be a little thin on the ground this time, but I’ll throw a couple tentative ones out there. Even accounting for quality leakage, I suspect you’d still slightly prefer a jumper in the top 25 over someone who’s stayed steady. And at the bottom of the rankings, it seems pretty clear that you’d rather have a new guy over a repeater. Beyond that? It looks like BA’s evaluators have again done a pretty solid job.

Up next, we’ll move on from year-to-year top 100 standing, and instead look at what we can glean from the most basic information about the teams’ own prospect evaluations: draft position.