There seems to be some sort of background discussion on coffee scores happening this week, with Mark Prince weighing in, as well as a few other bloggers and reviewers. Some are saying "simplify the system," others are saying "be more critical." I thought I'd just add my thoughts to the conversation.

Metrics

I don't care if you use the SCAA score sheet, a Q Grader rubric, or your own invented system, just be transparent about it. Let us know what goes into your scores and why, to give them context and meaning. But here's the deal - make your metric actually mean something.

Mark Prince suggested that he'd only ever scored coffees about a 95 a few times, which sounds to me like a broken metric. If 100 is unattainable, why use it at all? Hell, if 96 is unattainable, then your system is basically just 0-95. It's understandable that a coffee would have to be pretty damn terrible to score a 0, but shouldn't there be a 100 coffee out there? How can you judge something as almost 100 if you truly believe there's nothing better? Basically, I see no reason for a score to be present on a system if it can not be used. Perfection may not be possible, but that means you should redefine your ceiling criteria. A 100 should not represent the be-all-end-all of coffee for all time, but rather the best in class at its time. Provide scope, whether temporal or restricted by region (the American scale, or the Colombian Coffee scale). Or, possibly, make your scale a bell curve - stick 100 right smack in the middle, like the IQ distribution. The median IQ is always 100, because the scale is always adjusted, allowing for mass increase or decrease in intelligence of the population. That sort of metric would allow for mass increase or decrease of overall coffee quality, no?

Descriptions and Reasons

Everybody's palate is different, and heavily influenced by culture, sensory aptitude, and personal subjectivity. You don't have to be truly objective in reviewing coffee, I think, as you are no King Review - there are other options. Your reviews should represent your tastes, so they resonate with your audience better. But don't just say "Hey I liked that one, yum." Provide some reasoning, like "I really enjoyed the acidity, because it made the juicy cherry flavors pop." Providing flavor and palate descriptors means people know just what to expect, but I'd even suggest going further. If you're assigning a score at all, break it down for your readers. Tell them what points were lost where, and what that means to you. Sweet Marias even has the "Cupper's Correction" score added in to say "I'm fluffing the score a bit because it all turned out well in the end," or "Maybe I was a bit harsh on the details, and this isn't exactly an 85-point coffee." Make known your reasoning for your decisions, and people will understand that the number is hardly arbitrary.

And that's basically it. Scores, numbers, whatever are useful, but they need some sort of context to help the reader out. Numbers in a vacuum are about as helpful as saying this blog post is a 10. 

Blog Category: 

Comments

I agree with what you re saying for the most part that a review system needs to have clear metrics and there should be a perfect coffee out there besides the one that Plato believes there is. On a sie would you mind posting where you reading this? I normally can find what people ar referring to but in this case I could not.

Submitted by jbviau on
Yeah, a little less murkiness in this domain would be nice. I was going to say that I don't ever remember a coffee being rated lower than 70ish by Kenneth Davids, but of course when I went to check the site was down. I give his server a 0. ;)

Submitted by EricBNC on
<br>I am in the habit of assigning scores since the sheet I use for grading/cupping uses numbers. It helps more if you follow a reviewer with similar tastes to yours and that isn't easy to find but isn't impossible either. Once located you figure if they throw a 90 on the bag then it is going to be good - anything more than that is meaningless since the differences in a 91 vs a 93 (or 85 vs 93) can be lost by grind, dose, temp, age, carelessness, etc... <br>I think if a coffee is really bad now days Coffee Review will not review it - too much good coffee deserving of ink to waste on talking about bad coffee is my guess why.

Submitted by Chamie on
I definitely favor reviews to ratings -- I learned a long time ago that everyone has different tastes and I'd rather know what someone tasted than how they rated the taste. About rubrics and perfect scores -- depending on the rubric, it's understandable that a perfect score is close to impossible. If the breakdown of the rubric is granular enough, it would be a rare coffee indeed that scored every possible point available. Thanks for making me expand my ways of thinking about coffee yet again.

Submitted by rwgamer on
I think coffee reviewers like to leave room for that magical, mystical coffee experience that they've been searching their whole life for. If the coffee doesn't transport them to an alternate dimension, it doesn't deserve 100.

Yeah it doesn't make sense to assign a number to score since everyone has different palate and preference. I agree with Chamie that coffee review should be more descriptive and less numerical.

Submitted by wakeknot on
Seriously though, I don't mind the idea that 100 is above the awarded scores because no coffee is perfect and there is always room for improvement so I like to leave room for that in the scores. I agree though that in general ratings are a broken system.

I really agree with your idea about grading coffee. Sometime numbers are useful but in the case of coffee, it sometime taken for too much importance.

On Coffee Scores

| by

There seems to be some sort of background discussion on coffee scores happening this week, with Mark Prince weighing in, as well as a few other bloggers and reviewers. Some are saying "simplify the system," others are saying "be more critical." I thought I'd just add my thoughts to the conversation.

Metrics

I don't care if you use the SCAA score sheet, a Q Grader rubric, or your own invented system, just be transparent about it. Let us know what goes into your scores and why, to give them context and meaning. But here's the deal - make your metric actually mean something.

Mark Prince suggested that he'd only ever scored coffees about a 95 a few times, which sounds to me like a broken metric. If 100 is unattainable, why use it at all? Hell, if 96 is unattainable, then your system is basically just 0-95. It's understandable that a coffee would have to be pretty damn terrible to score a 0, but shouldn't there be a 100 coffee out there? How can you judge something as almost 100 if you truly believe there's nothing better? Basically, I see no reason for a score to be present on a system if it can not be used. Perfection may not be possible, but that means you should redefine your ceiling criteria. A 100 should not represent the be-all-end-all of coffee for all time, but rather the best in class at its time. Provide scope, whether temporal or restricted by region (the American scale, or the Colombian Coffee scale). Or, possibly, make your scale a bell curve - stick 100 right smack in the middle, like the IQ distribution. The median IQ is always 100, because the scale is always adjusted, allowing for mass increase or decrease in intelligence of the population. That sort of metric would allow for mass increase or decrease of overall coffee quality, no?

Descriptions and Reasons

Everybody's palate is different, and heavily influenced by culture, sensory aptitude, and personal subjectivity. You don't have to be truly objective in reviewing coffee, I think, as you are no King Review - there are other options. Your reviews should represent your tastes, so they resonate with your audience better. But don't just say "Hey I liked that one, yum." Provide some reasoning, like "I really enjoyed the acidity, because it made the juicy cherry flavors pop." Providing flavor and palate descriptors means people know just what to expect, but I'd even suggest going further. If you're assigning a score at all, break it down for your readers. Tell them what points were lost where, and what that means to you. Sweet Marias even has the "Cupper's Correction" score added in to say "I'm fluffing the score a bit because it all turned out well in the end," or "Maybe I was a bit harsh on the details, and this isn't exactly an 85-point coffee." Make known your reasoning for your decisions, and people will understand that the number is hardly arbitrary.

And that's basically it. Scores, numbers, whatever are useful, but they need some sort of context to help the reader out. Numbers in a vacuum are about as helpful as saying this blog post is a 10. 

Category: BLOG

very interesting idea!

March 3, 2012 | by sontondaman

I really agree with your idea about grading coffee. Sometime numbers are useful but in the case of coffee, it sometime taken for too much importance.

this post is an 11!

February 23, 2012 | by wakeknot

Seriously though, I don't mind the idea that 100 is above the awarded scores because no coffee is perfect and there is always room for improvement so I like to leave room for that in the scores. I agree though that in general ratings are a broken system.

Yeah it doesn't make sense

February 22, 2012 | by hoonchul@hotmail.com

Yeah it doesn't make sense to assign a number to score since everyone has different palate and preference. I agree with Chamie that coffee review should be more descriptive and less numerical.

Leave room for magic.

February 21, 2012 | by rwgamer

I think coffee reviewers like to leave room for that magical, mystical coffee experience that they've been searching their whole life for. If the coffee doesn't transport them to an alternate dimension, it doesn't deserve 100.

Transparency, yes!

February 21, 2012 | by Chamie

I definitely favor reviews to ratings -- I learned a long time ago that everyone has different tastes and I'd rather know what someone tasted than how they rated the taste. About rubrics and perfect scores -- depending on the rubric, it's understandable that a perfect score is close to impossible. If the breakdown of the rubric is granular enough, it would be a rare coffee indeed that scored every possible point available. Thanks for making me expand my ways of thinking about coffee yet again.

scorless

February 20, 2012 | by EricBNC


I am in the habit of assigning scores since the sheet I use for grading/cupping uses numbers. It helps more if you follow a reviewer with similar tastes to yours and that isn't easy to find but isn't impossible either. Once located you figure if they throw a 90 on the bag then it is going to be good - anything more than that is meaningless since the differences in a 91 vs a 93 (or 85 vs 93) can be lost by grind, dose, temp, age, carelessness, etc...
I think if a coffee is really bad now days Coffee Review will not review it - too much good coffee deserving of ink to waste on talking about bad coffee is my guess why.

Hmm...

February 20, 2012 | by jbviau

Yeah, a little less murkiness in this domain would be nice. I was going to say that I don't ever remember a coffee being rated lower than 70ish by Kenneth Davids, but of course when I went to check the site was down. I give his server a 0. ;)

I agree with what you re

February 20, 2012 | by intrepid510

I agree with what you re saying for the most part that a review system needs to have clear metrics and there should be a perfect coffee out there besides the one that Plato believes there is. On a sie would you mind posting where you reading this? I normally can find what people ar referring to but in this case I could not.

Categories

  • http://coffeekind.com/sites/default/files/icon_blog_on.png

    BLOG

  • http://coffeekind.com/sites/default/files/icon_knowledge1_on.png

    KNOWLEDGE

  • http://coffeekind.com/sites/default/files/icon_news_on.png

    NEWS

  • http://coffeekind.com/sites/default/files/icon_guides_on_1.png

    BREWING GUIDES

  • http://coffeekind.com/sites/default/files/icon_buying_guides_on_1.png

    BUYING GUIDES

  • http://coffeekind.com/sites/default/files/drink_guide_1.png

    DRINK GUIDES

  • http://coffeekind.com/sites/default/files/recipehover.png

    RECIPES

  • http://coffeekind.com/sites/default/files/latest%20reviews.png

    LATEST REVIEWS