RateBeer.com Proves (Once Again) That No Best Beer Poll is Perfect

by Mike Pomranz, Posted Feb 17th 2009 @ 1:00PM

Since I took a few constructive swipes at BeerAdvocate over their "The People Have Spoken!" opinion polls, I thought it only fair that when rival RateBeer.com released their annual best of lists for 2009, I'd reflect on these lists as well.

One of my primary complaints with the BeerAdvocate list was its bias towards Stone Brewing, who landed 5 beers in the top 25 and grabbed the coveted top brewery spot. RateBeer.com reviewers seemed slightly less blown away by Stone. The California brewer still took a respectable 8th place in RateBeer's brewery list, but did not score a beer in the top 25. Instead, though, RateBeer readers found a bias of their own: Munster, Indiana's Three Floyds Brewing Company. Three Floyds not only earned the top brewery of 2009 spot, but also placed an impressive four beers in the top 25. Luckily, this feat once again gives me the opportunity to reiterate that almost any user-based poll will have its leanings. With thousands of beers to choose from, any brewery having four beers in the top 25 seems a bit over-the-top. [Writer's note: For the record, I have biases of my own, such as greatly preferring having my points reinforced rather than refuted. So thank you, RateBeer users!]

My other chief criticism of the BA list: Not much international influence. RateBeer brags that it is "truly worldwide" saying "90% of RateBeer's top 10 raters are from outside the United States." Apparently that reached them all the way into Scandinavia, landing a Swedish beer and a Danish brew in their top 10. Though RateBeer included 5 international brewers (if you're willing to include Canada, which I am) on their top 25 breweries list -- over double the number on the BeerAdvocate list -- I'm still skeptical that these user generated polls are "truly" an international affair. Do I agree that the most interesting craft brewing is happening right here in the United States? Yes, emphatically. Still, having two beers from the same brewery in Sweden find their way onto your list doesn't entirely convince me that every beer in the world is getting its fair shake.

RateBeer.com (in a similar manner to BeerAdvocate) calls these lists "The Best Beers in the World" and "The Best Brewers in the World" so I would be irresponsible to not render the same criticism: It's a bold assessment from an English speaking website. Are all these beers awesome? Yes. These brewers certainly represent some of the world's best and these lists make great guides for finding new brews. Still, I don't think RateBeer.com has gotten us any closer to deciding what the top beers in all of the world are. User generated feedback is an amazing and powerful tool, one that the internet has provided us with unprecedented access too, but more opinion doesn't always mean more valid opinion.

Below are RateBeer.com's 10 Best Brewers 2009. You can see the entire list for yourself here.

Three Floyds Brewing Company
Port Brewing/Lost Abbey
AleSmith Brewing Company
Hair of the Dog Brewing Company
Russian River Brewing
Mikkeller
Founders Brewing Company
Stone Brewing Company
Surly Brewing Company
Struise

[via RateBeer Best 2009 -- Award Results]

Source

Filed Under: Lists, Drink Recipes
Tags: beer, beeradvocate, ratebeer, Speakin suds, Three Floyds, Three Floyds Brewing, top beers, top breweries

Chew on This

Royal Wedding Cake Details Revealed! - YumSugar

Make Your Own Pop Tarts - CHOW
Denny's Introduces a Maple Bacon Sundae - Eater
Whole Foods Opening Bars in Stores - USA Today
Rules Of Bar Etiquette -- For The Bartender - HuffPost Food

Reader comments (Page 1 of 1)

2-17-2009 @3:58PM foxdude0486 said... Makes me wonder what the leanings would be at beersuggest.com But I'd also consider the number of users and involvement of them considering beersuggest is still a younger site with a smaller userbase.
Reply

Vote upVote downReport

2-17-2009 @4:16PM Jason said... As far as I know, none of these guys taste beers blindly, so it
really doesn't matter what they say. They could be rating beers by
"coolest label" for all we know, even if they're doing it
subconsciously.
Reply

Vote upVote downReport

2-17-2009 @11:31PM scott said... so what's the point of your post? in general, that top ten are some fine breweries. what's your beef? you don't like some of the top ten? which ones and why? what's your top ten? it's not much of a rant if you don't back it up with your own list. you have a valid criticism with rate beer and those Swedish beers since they are virtually impossible to find even within Sweden, but overall, the top 25 are reasonable beers to include understanding the readership skews towards stronger beers that are more consistent with the emerging American palate.
Reply

Vote upVote downReport

2-17-2009 @11:59PM Mike Pomranz said... Hey scott,

No "beef" whatsoever. As I said in my opening paragraph, the point of this post was fairness. After providing a critical analysis of BeerAdvocate's best of lists a couple months back, I felt it only fair to give the RateBeer lists the same treatment (especially since it reiterated some points from that previous post).

For the record, I am a big fan of both sites and utilize them all the time to determine consumer beer opinion. I think RateBeer's top ten breweries make a fine list. And to answer your question, I'd be hard-pressed to come up with a better list myself being that I don't have the breadth of knowledge that can be compiled through the hundreds upon hundreds of RateBeer.com users.

But if you hadn't read my original post on BeerAdvocate's lists, I see how this post might be a bit confusing. My original point: Though these consumer-generated lists may make great debate fodder, I am skeptical that they create definitive lists of the "best beers in the world." I would rather see lists compiled from expert opinions, in the same way that I value the AP or Coaches poll in determining the top 25 college basketball teams over a "fan poll."

Hope that clarifies things a little.
Reply

Vote upVote downReport

2-18-2009 @12:14PM Greg Koch said... Great post! I would disagree with your assertion that BA has a "bias" towards us. I would say instead it's a "simple reflection of the fact of our greatness." Heh. I kid, I kid.

"Best" is, of course, up to the individual. Unless you are perfectly aligned with "the wisdom of crowds" (I sure as hell am not...but will readily agree that it does exist and that it usually has a strong validity...especially in the RB & BA crowd due to their relative enthusiasm and education on the subject), then you'll have personal points of disagreement. I mention what I think of "Best" in this RateBeer interview I did waaaay back in 2002: http://tinyurl.com/b6gtpl

RateBeer has, in past years, named Stone as "Best," as they did in 2002 for the (I think) 2nd time in a row: http://www.ratebeer.com/txt/features/RB1102-Brewers.htm (doh! That link no longer works...guess that Stone WAS 'Best' in 2002 but that is now just hearsay...heh). Stone was again named 'Best' in 2003, but that link is dead too, so maybe it's just a fake memory I planted in my own head. ;-]-=

In 2003, BeerAdvocate also gave Stone top honors: http://beeradvocate.com/boba/072003/breweries.php

So, I think that the "of all time" nod from the recent BA Magazine was aimed at the fact that Stone's been somewhat highly regarded for a long time. So, we don't get the "Most Improved" or "Best New Brewery" or "Rising Star" awards. Perhaps we get the "Best Focus Over A Long Period Of Time" award? Hell, I don't know...that's not for me to decide.

Our job is to do what we do the best that we can do it. Then, folks get to decide for themselves if they like it or not. Fair enough. As it should be.

Yes, some absolutely fantastic breweries, brewers and beers have shot up and down on the various lists over the years...including my own personal list. You'll see many of these breweries and beers represented on the tap list at the Stone World Bistro & Gardens, as well as on the Stone Distributing price sheet (many don't know, but we proudly distribute a fantastic collection of highly respected craft breweries in SoCal).

Perhaps Stone will end up as merely "highly regarded among the best breweries on the planet." I can live with that. That will mean that there are many other fantastic beers out there. I can definitely live with that!

Cheers!

Greg Koch, CEO & co-founder
Stone Brewing Co.
Stone Brewing World Bistro & Gardens
Reply

Vote upVote downReport

2-18-2009 @12:29PM Andy said... Both RB and BA suffer from what you could call an entourage effect: largely-populated regions with enthusiastic RB/BA users tend to artificially pump up beers from particular region or brewery. Combined with extremely limited distribution for special releases and lore begins to affect ratings in a way that obscures reality.

Lost Abbey is a perfect example of this, enjoying a huge entourage (their Patron Sinners/Saints club nurtures this) and an unusually strong regional BA/RB user base. This explains why some of their less remarkable offerings have unusually high ratings.

Until one of these sites finds a way to normalize this type of bias, I'll continue to take the reviews with a grain of salt.
Reply

Vote upVote downReport

2-18-2009 @7:58PM EyeChartBrew said... In his post, Mike Pomranz said, "Though these consumer-generated lists may make great debate fodder, I am skeptical that they create definitive lists of the "best beers in the world."

With all due respect Mike, this seems to be a bit of a no-brainer. Who out there is really and truly looking at these lists as being truly definitive of the world of beer, circa 2009?

Maybe I'm a bit out-of-the-loop re: the latest doings of BA, RB, et al in the beer world, but I just can't see too many folks taking these lists as gospel.
Reply

Vote upVote downReport

2-19-2009 @1:32PM Mike Pomranz said... First things first,

Thank you, Greg, for taking the time to comment on this post. I am humbled just thinking how some of those I truly respect in the industry are familiar with my work -- to have them take the time to chime in on the discussion is an honor.

Your comment once again gives me a chance to reiterate that the two posts discussed here are in no way intended to reflect negatively on the breweries listed in these polls. What is NOT in question is that every brewery on these lists has a reason to be on these lists. Obviously, Stone Brewery, which continues to create amazing beers, deserves recognition as a truly outstanding brewery.

I also want to reiterate that I have the utmost respect for the purveyors and users of both RateBeer and BeerAdvocate. These sites are marvelous communities that provide a deeper insight into beer than has ever been available previously.

My only point is that in someway, these lists feel flawed. This "flaw" has nothing to do with the beers, the breweries, the websites or the users: It's a result of the methodology.

Still, I'm not even suggesting we change the methodology. I'm just asking that we acknowledge this flaw, and as Andy commented, take these things "with a grain of salt."

To which EyeChartBrew asks, "Who out there is really and truly looking at these lists as being truly definitive of the world of beer, circa 2009?"

Maybe EyeChartBrew is right? Maybe I am the one reading too much into this. But my sense is that many people don't take the time to dissect what they read. Many of us do, but many don't. Many people take what they see or what they read at face value. When they read "Best in the World," they think best in the world.

I'd like to believe that all my readers are sympathetic to my perspective before I present it, but as someone who's job it is to divulge opinion, I guess then I would be out of the job!

Thanks again to those who've contributed to the discussion above.
Reply

Vote upVote downReport

2-19-2009 @4:02PM Monika said... I'm right there with you, Mike.

I have always been edgy about "Best" lists, unless the title is "Some of the Best," or the person making the list is familiar with every single thing in a given group. These are rarely, if ever, accurate, and always seem to use "Best" as a way to seem definitive and grab viewers.

Now "RateBeer's Best," or "RateBeer's Favorite," or "RateBeer's Highest Rated Beers" would work.

And as for people not taking it was the best out there: Maybe it's different for beer, but writing lists on Cinematical, someone ALWAYS takes issue when a "Best" list comes out and doesn't contain a certain film the reader admires.

Best means best. Unless it's qualified, it should be THE best.

But not The Best of the Best. That's reserved for awesome Eric Roberts movies.
Reply