What Is “Ranking The Rankers”?

Is Anybody Actually “Good” at Fantasy Football?
Ranking the Rankers is an ongoing experiment to try to answer this question.

How It Works
I’ve selected 12 fantasy football experts from three major sites (ESPN, NFL, CBS), and each week I have them play matchups against each other based on their weekly player rankings. To make things more interesting they also play a naive computer ranker named DAMO (Dumb-Ass Machine Orderer). If fantasy football is a legit skill then we should see the very best experts pull away from the others over the course of a long season.  At the very least, they should all handily beat DAMO.

DAMO
DAMO does nothing more than rank players according to how many total points they’ve scored over the course of the season up to the current week.  It does not consider opposing defenses, game plans, field conditions, previous games missed, etc. — nothing but total points on the year.  In Week 1, DAMO uses the total points of the players from the previous season.  If a player is not ranked by at least one other ranker, DAMO does not rank this player and moves everybody else up a spot.  This is to account for injuries and byes.

Matchups
Each week each ranker plays every other ranker in a head-to-head matchup. Matchups are scored according to ranker “battles”. A battle occurs when Ranker 1 has Player A ranked ahead of Player B, and Ranker 2 has Player B ranked ahead of Player A. If Player A outscores player Player B on the week then Ranker 1 gets the point differential added to his total. If Player B outscores Player A then Ranker 2 gets the points.*

For example, in Week 1 of the 2013 season, Dave Dameshek of NFL.com ranked Drew Brees (1) ahead of Peyton Manning (3), and Nathan Zegura of CBS.sportsline.com had Manning (5) ahead of Brees (6). In the actual football games, Manning scored 50.9 points, and Brees scored 24.1 points.  Therefore, in his matchup against Dameshek, Zegura got 26.8 (50.9 – 24.1) added to his score for the Brees-Manning battle.

In a matchup, we look at all possible battles across all possible positions and add up the totals. Whoever has the most points wins the matchup. Basically. There is some slight scaling and adjusting to make things more in line with an actual fantasy game.  Specifically, “starter battles” count four times as much as “non-starter battles”. A starter battle is any battle in which at least one of the rankers has at least one of the players in “starting range” for a standard 10-team league (top-10 QB, top-20 RB, top-30 WR, top-10 TE, top-10 K, top-10 DST). I did this because I don’t want a bunch of low-ranked players affecting the overall score too much. Correctly predicting Davone Bess will outscore Brandon Stokley is not as useful as correctly predicting DeSean Jackson will outscore Hakeem Nicks, because not many fantasy players are agonizing over a Bess-or-Stokley start. Nicks or Jackson is a legit decision.

To put matchup scores on the same scale as the scores of actual fantasy games, I divide the total score for each ranker across a position by the total number of battles at that position** and multiply the result by the number of starters on a typical fantasy roster at that position (1 QB, 2 RB, 3 WR, 1 TE, 1 KR, 1 DF).  A ranker’s total score for a matchup is the sum of his scores across all positions. Whoever has highest total score wins. That’s it.

PDPM
PDPM stands for point differential per matchup.  This is the average difference between a ranker’s score and his opponent’s score.  For example, in Week1 of 2013, Michael Fabiano of NFL.com had the best week with a 13.8 PDPM, meaning he beat his opponents by an average of nearly 14 points.  Another way to think about this: if you followed a ranker’s rankings to a tee, his PDPM is how many points you’d win your fantasy game by against an average expert.  If you want to go sabermetric on everybody, PDPM can be thought of as a value above average stat.  A positive PDPM means the ranker was better than average, a negative PDPM means he was worse than average, zero means exactly average.

Final Thoughts
DAMO is designed to implement minimal fantasy football strategy.  It’s meant to be a baseline against which to judge the experts.  If DAMO can do as well as the experts then it shows there really is no such thing as an expert, at all.  If all you need to do to rank players effectively is look at their total points for the season, then nobody is actually “good” at fantasy football.

Note that Ranking the Rankers applies only to setting a lineup. It doesn’t expressly test other key aspects of running a fantasy team like drafting, in-season transactions, and keepers.*** Maybe I’ll try to come up with an experiment for these next year.

*Standard scoring for players is assumed.  An unranked player is considered to be ranked below all ranked players.  I made sure to cap ranking lists, so that they are all the same size, i.e., every ranker ranks x QBs, y RBs, etc.

**Technically I divide by (non-starter battles + 4 x starter battles) to account for the extra starting battle weight.

***Although, the fundamental decision behind these other aspects is often the same one behind setting a lineup — Will Player A score more points than Player B?