Analyzing Reader and Expert Evaluations of NFL Quarterbacks

The Athletic’s release of Mike Sando’s Quarterback Tiers provides an annual survey assessing NFL quarterbacks. This year introduced a variation by inviting readers to rank quarterbacks into tiers, from Tier 1 (elite) to Tier 5 (not an NFL starter). More than 45,000 readers participated, casting over half a million votes.

Reader Insights and Expert Consensus

Overall, readers and experts largely agreed on quarterback rankings, with slight variances. Analysis revealed a noticeable home-team bias in reader data. While The Athletic cannot determine the specific teams readers support, it utilized app data to gauge team biases. This data showed team followers rated their quarterbacks higher, often significantly, than other readers did.

An example is how Patriots and Ravens fans rated Drake Maye and Lamar Jackson as Tier 1, contrasting with most readers who saw them as Tier 2. This home-team bias was approximately a half-tier difference on average, akin to how experts rated Jackson (1.6) versus Jared Goff (2.1).

Impact of Home-Team Bias

Analysis revealed that Tyler Shough and Drake Maye topped the list for receiving the most significant home-team bump. Conversely, Browns’ Deshaun Watson did not benefit from this effect. His followers, alongside other readers, rated him as Tier 5. Despite being named the Browns’ starter, expert panelists placed him mostly in Tier 4.

The readers’ top quarterbacks ranked similarly to experts. Josh Allen, Patrick Mahomes, and Matthew Stafford received minimal differential in fan versus broader evaluations, highlighting widespread regard transcending team loyalty.

Consistency Between Readers and Experts

Reader opinions closely aligned with expert panel evaluations, with only minor shifts in rankings for the 35 quarterbacks. Reader ratings were generally about 0.3 tiers lower than those of the experts. This pattern illustrates that casual readers may approach ratings less seriously than NFL professionals.

However, noticeable disparities existed, particularly for players like Deshaun Watson, Aaron Rodgers, and Tua Tagovailoa. Watson, despite missing significant game time due to injury and off-field issues, showed the largest gap in ratings.

Disparities in Evaluations

“Watson’s average reader rating was 4.8, while panelists rated him at 3.8, highlighting the reader’s sense of him being overrated.”

Rodgers, returning for a final season with the Steelers, and Tagovailoa, starting for the Falcons, were perceived by readers as less capable, placing them both in Tier 4 versus the experts’ Tier 3 placements. This highlights areas where public perception diverges significantly from expert evaluation.