The Rating Percentage Index, or RPI, is the biggest aid for the NCAA March Madness Selection Committee to determine which teams make it in March Madness, where they rank, and which teams are left out (ESPN). The RPI is also the simplest of the many ranking indices out there (ESPN), like:
- Basketball Power Index or BPI by ESPN
- Sagarin Index by Jeff Sagarin
- KenPom by Ken Pomeroy
- Massey Ratings
This would be fine if the RPI had the most meaningful indicators of success, for example. Just because you have more data doesn’t mean you can do a better job at predicting outcomes. In fact, it would be a beautiful thing if the RPI had the simplest model and the best model. However, from its performance in this year’s ESPN’s tournament challenge that I simulated in a set of brackets as described below, the RPI was the worst of the indices… by a long shot! It’s so bad, in fact, that more than half of the 18.8 million brackets will beat it! That meant the “average” Joe or Jane entry from a sample of 18.8 million entries will beat it! The RPI entry has 0 potential points remaining, with two rounds to go, sitting in the 46th percentile (100 is best)!
In short, the RPI is GARBAGE! Rip it up and let it R.I.P. !!!
It’s time the Selection Committee stop relying on it, along with their “human” input to try and improve it for determining March Madness selections! Schools are being robbed of being in the tournament, or having a chance to show how good they are by being put up against another really strong team early on, i.e. Wichita State. So which index should the Selection Committee use? I’ll let you decide from the experiment I did and the results below, but the short answer is pretty much any other one.
In using something like an Index to predict outcomes for a March Madness bracket, it’s pretty simple. The bracket set up doesn’t matter. You look at the two teams playing in any match up. The higher ranked team in the index you are using, not in the bracket, should win.
Filling out the brackets basically means you start from the top of the rankings and push that team all the way to the championship. Take the second team and push it as far as it can go before it hits the first team. Then take every subsequent ranked team and push it as far as you can until it hits a game already determined to be won by another team, a team higher ranked in it by the index.
If the brackets were set up well, all the highest ranking teams should go pretty far. None should have to hit another high ranking team until at least the Sweet 16, meaning to get to the Elite 8. If not, like poor Wichita State hitting Kentucky in the second round, then someone got robbed! The RPI had Wichita State as 31st ranked, which means it should only go as far as the second round even if not up against a powerhouse like Kentucky. The other indices had Wichita State anywhere from 8th to 15th, meaning it should be at least the Sweet 16, with Kentucky ranked 3rd to 9th. Wichita State only lost by 3, by the way, 65-62.
The better an index is at predicting games outcome due to its correct ranking of teams, the more points it should get. Now, March Madness sees lots of upsets, but there are 18.8 million other brackets against which to compare how much of a fluke any given index’s performance might be if it does poorly due to the upsets, so the one year sample is quite valid with a gauge like millions of human minds giving their best opinion.
I pitted four indices against each other in the Index Me group on ESPN’s Tournament Challenge.
I left out the Massey Index in having ran out of time, and it didn’t do well anyway from a quick check lately. I had other entries which I will cover later, that had equal averages and weighted averages of these indices ratings based on what I knew of them.
The Results After 4 Rounds
I will update the results after the tournament is over next week, but the results are clear enough now to make a statement. You can click on the links above to see the outcomes that aren’t easily copied to a blog, but here’s how it stands:
As you can see, Jeff Sagain’s Index is doing the best on points and potential points remaining (PPR). It has actually already won the battle of the indices due to common choices remaining with KenPom and RPI, the only indices that could catch it on PPR.
But look at the percentile (PCT) where the indices placed. Sagarin is beating out 94.3% of the 18.8 million brackets entered. KenPom and RPI beat out 89.0 percent. But the RPI? 53.5% of those brackets are beating it! That’s more than half the entries, and presumably people entering! The average person could have drawn up a more accurate index based on how they picked the match ups! Reverse engineer what I did, basically, to create the indices.
It’ll only get worse for the RPI as the RPI has no potential points remaining. You could argue the last few games are way skewed on points, but it makes no difference. Even the results right now show the RPI being decimated!
The Selection Committee is making its decisions on that, and whatever other “human” input they have to do things like putting Wichita State where it were??? C’mon man!!! You’re supposed to be smart. At least act that way, eh?
Here’s how the Indices compared for ranking. Look at the difference between the RPI and the rest of the indices, with “big” differences of 7 or more (cost of a fair Sweet 16 round match up) to BPI, SAG and POM indices, shown in red. I included Massey but didn’t use them as they were also wacky enough I didn’t put the time into it. I didn’t have access to all 64 teams in all the indices but after 32 teams for each index, just about every other team was going to lose in the first round so the 47 teams I had were more than sufficient to do the experiment.