Evidence Why the RPI is Garbage for Determining March Madness Seeds

The Rating Percentage Index, or RPI, is the biggest aid for the NCAA March Madness Selection Committee to determine which teams make it in March Madness, where they rank, and which teams are left out (ESPN). The RPI is also the simplest of the many ranking indices out there (ESPN), like:

  1. Basketball Power Index or BPI by ESPN
  2. Sagarin Index by Jeff Sagarin
  3. KenPom by Ken Pomeroy
  4. Massey Ratings

This would be fine if the RPI had the most meaningful indicators of success, for example. Just because you have more data doesn’t mean you can do a better job at predicting outcomes. In fact, it would be a beautiful thing if the RPI had the simplest model and the best model. However, from its performance in this year’s ESPN’s tournament challenge that I simulated in a set of brackets as described below, the RPI was the worst of the indices… by a long shot! It’s so bad, in fact, that more than half of the 18.8 million brackets will beat it! That meant the “average” Joe or Jane entry from a sample of 18.8 million entries will beat it! The RPI entry has 0 potential points remaining, with two rounds to go, sitting in the 46th percentile (100 is best)!

In short, the RPI is GARBAGE! Rip it up and let it R.I.P. !!!

It’s time the Selection Committee stop relying on it, along with their “human” input to try and improve it for determining March Madness selections! Schools are being robbed of being in the tournament, or having a chance to show how good they are by being put up against another really strong team early on, i.e. Wichita State. So which index should the Selection Committee use? I’ll let you decide from the experiment I did and the results below, but the short answer is pretty much any other one.

The Theory

In using something like an Index to predict outcomes for a March Madness bracket, it’s pretty simple. The bracket set up doesn’t matter. You look at the two teams playing in any match up. The higher ranked team in the index you are using, not in the bracket, should win.

Filling out the brackets basically means you start from the top of the rankings and push that team all the way to the championship. Take the second team and push it as far as it can go before it hits the first team. Then take every subsequent ranked team and push it as far as you can until it hits a game already determined to be won by another team, a team higher ranked in it by the index.

If the brackets were set up well, all the highest ranking teams should go pretty far. None should have to hit another high ranking team until at least the Sweet 16, meaning to get to the Elite 8. If not, like poor Wichita State hitting Kentucky in the second round, then someone got robbed! The RPI had Wichita State as 31st ranked, which means it should only go as far as the second round even if not up against a powerhouse like Kentucky. The other indices had Wichita State anywhere from 8th to 15th, meaning it should be at least the Sweet 16, with Kentucky ranked 3rd to 9th. Wichita State only lost by 3, by the way, 65-62.

The better an index is at predicting games outcome due to its correct ranking of teams, the more points it should get. Now, March Madness sees lots of upsets, but there are 18.8 million other brackets against which to compare how much of a fluke any given index’s performance might be if it does poorly due to the upsets, so the one year sample is quite valid with a gauge like millions of human minds giving their best opinion.

The Contenders

I pitted four indices against each other in the Index Me group on ESPN’s Tournament Challenge.

  1. BPI (ESPN)
  2. Sagarin (Jeff Sagarin)
  3. KenPom (Ken Pomeroy)
  4. RPI (Selection Committee)

I left out the Massey Index in having ran out of time, and it didn’t do well anyway from a quick check lately. I had other entries which I will cover later, that had equal averages and weighted averages of these indices ratings based on what I knew of them.

The Results After 4 Rounds

I will update the results after the tournament is over next week, but the results are clear enough now to make a statement. You can click on the links above to see the outcomes that aren’t easily copied to a blog, but here’s how it stands:

Sagarin 810 640 94.4 Gonzaga
KenPom 760 640 89.1 Gonzaga
BPI 760 160 89.1 Villanova
RPI 600 0 46.4 Villanova

As you can see, Jeff Sagain’s Index is doing the best on points and potential points remaining (PPR). It has actually already won the battle of the indices due to common choices remaining with KenPom and RPI, the only indices that could catch it on PPR.

But look at the percentile (PCT) where the indices placed. Sagarin is beating out 94.3% of the 18.8 million brackets entered. KenPom and RPI beat out 89.0 percent. But the RPI? 53.5% of those brackets are beating it! That’s more than half the entries, and presumably people entering! The average person could have drawn up a more accurate index based on how they picked the match ups! Reverse engineer what I did, basically, to create the indices.

It’ll only get worse for the RPI as the RPI has no potential points remaining. You could argue the last few games are way skewed on points, but it makes no difference. Even the results right now show the RPI being decimated!

The Selection Committee is making its decisions on that, and whatever other “human” input they have to do things like putting Wichita State where it were??? C’mon man!!! You’re supposed to be smart. At least act that way, eh?

Here’s how the Indices compared for ranking. Look at the difference between the RPI and the rest of the indices, with “big” differences of 7 or more (cost of a fair Sweet 16 round match up) to BPI, SAG and POM indices, shown in red. I included Massey but didn’t use them as they were also wacky enough I didn’t put the time into it. I didn’t have access to all 64 teams in all the indices but after 32 teams for each index, just about every other team was going to lose in the first round so the 47 teams I had were more than sufficient to do the experiment.

Villanova 1 1 2 2 2
Gonzaga 2 8 1 1 1
North Carolina 3 5 3 3 9
Virginia 4 18 9 7 23
Louisville 5 7 7 6 17
Duke 6 6 8 13 6
West Virginia 7 24 4 5 12
Florida 8 10 10 9 19
Kentucky 9 4 5 4 3
Kansas 10 3 6 10 5
Purdue 11 20 12 15 21
Saint Mary’s 12 17 25 14 13
Oregon 13 9 13 16 14
UCLA 14 16 15 18 10
Wichita State 15 31 11 8 11
SMU 16 15 18 11 7
Baylor 17 11 14 12 15
Michigan 18 30 23 20 18
Cincinnati 19 12 21 22 20
Florida State 20 13 20 19 22
Wisconsin 21 32 17 22 24
Notre Dame 22 23 22 25 16
Iowa State 23 22 16 17 8
Arizona 24 2 19 21 4
Butler 25 14 26 26 29
Oklahoma State 26 39 24 23 25
Creighton 27 26 27 27 26
Miami 28 42 28 30 32
Marquette 29 61 29 29 50
Rhode Island 31 37 47 36 31
Xavier 34 36 32 40 42
Dayton 36 28 39 35 44
VCU 38 19 49 47 40
Vanderbilt 39 38 43 34 37
Minnesota 40 21 34 33 30
South Carolina 43 43 30 32 48
Michigan State 44 51 35 41 49
Middle Tennessee 45 35 58 46
Nevada 46 29 54 54 33
Arkansas 47 25 37 38 27
Northwestern 48 49 40 37 283
Maryland 49 34 41 43 36
Virginia Tech 50 47 45 44 34
Seton Hall 53 44 48 52 35
Princeton 54 50 74 56 56
UNC Wilmington 59 27 69 58 45

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s