Philosopher Senior Member Joined: 13 Feb 2007 Posts: 188 Location: Australia Expertise: I like coffee
Espresso: Silvia Grinder: Rocky
Posted Mon Oct 8, 2007, 3:11am Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
JasonBrandtLewis Said:
It's not just the essays you point to. I've often said that you can't have a 100-point scale, that it promotes the concept of "perfection" --and what happens when that perfect wine, or that perfect "god shot" is surpassed by another one? 104 points?
Moreover, it assumes that you can mark coffee on a linear scale which will be scored the same way no matter who, when and how many times it is marked. Considering that even green beans will vary batch to batch and season to season, I am not sure if it is possible to achieve this degree of consistency.
In reality every cup has its own particularly unique character which has to be enjoyed on its own merits and interpreted through the preferences of the person drinking it.
The best advice I have heard about how to use wine scores is to actually find a wine judge whose preferences line up with your own. More useful are the tasting notes which give more descriptive comments about the vintage. It is sad when people slavishly follow other's opinions rather trying to develop their own palate and preferences. It might sound impressive to have a cellar of Parker 95+s but if the big, complex, high alcohol reds do not suit your style then you have really wasted your money (even though you have impressed your wine-geek friends). Even then your moods will vary about what you prefer drinking on any given day, at what time and what food you happen to be serving with it.
It is the same with coffee. If we were honest with ourselves, we tend toward preferences for particular characteristics e.g. medium or heavy body, chocolate or citrus/floral-like notes, medium or high acidity. If you are going to use some rating system, it has to be combined with more detailed information to have any meaning.
Actually the most important thing is to find a wine critic who is consistent. This is true whether or not you agree with him/her. The more consistent the critic is, the more it is possible to "calibrate" your own palate preferences to his/hers. Parker is perhaps the most consistent critic. As such, I know that when he uses certain words, regardless of the point score, I know I'm absolutely going to not like that wine at all . . . regardless of the fact it received 95+ points. Certain other words, and I know I'm going to enjoy the wine, whether it got a "92," an "82," or a "72"! Someone who is inconsistent in their palate preferences is useless (IMHO, of course).
But, back to coffee:
Philosopher Said:
Moreover, it assumes that you can mark coffee on a linear scale which will be scored the same way no matter who, when and how many times it is marked. Considering that even green beans will vary batch to batch and season to season, I am not sure if it is possible to achieve this degree of consistency.
In reality every cup has its own particularly unique character which has to be enjoyed on its own merits and interpreted through the preferences of the person drinking it . . . .
It is the same with coffee. If we were honest with ourselves, we tend toward preferences for particular characteristics e.g. medium or heavy body, chocolate or citrus/floral-like notes, medium or high acidity. If you are going to use some rating system, it has to be combined with more detailed information to have any meaning.
I absolutely agree! But there is yet another factor to consider: unlike wine -- even if we all agreed that the extraordinarily rare "Siberian Lentilberry" coffee roasted by "Mooncoins" of Vladivostok was a "100-point" coffee -- there is the variable of the barista. Whether you go to Mooncoins for an espresso, or if you go to your house . . . or mine! . . . the skill of the individual pulling the shot can instantly turn that alleged 100-point coffee into a 72 in quite a hurry . . . .
Within the wine trade, there are so many variables than can affect a wine -- from temperatures during shipping to storage to food pairings to the allergies or cold that a taster may be sufferring -- we often say, "There is no such thing as a great wine, only a great bottle of wine." Perhaps there is no such thing as a great coffee, but only a great cup of coffee (or espresso).
Posted Mon Oct 8, 2007, 8:00am Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
I think issue is more how a roast is evaluated not what kind of rating system is used. The evaluation process would probably evolve into a rating system. If you're evaluation is subjective and under what you consider is common extraction conditions like 200F across the board, same grind/dose/distribution for all roasts your rating would be more subjectively based regardless of what system you're using.
If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective. Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control. The results of a test like that would be more reflective of actual roast quality. Yes, tasting is subjective for sure and could be influenced by the sampler's preference. You could give a biography of the sampler with their personal preferences in a roast. That would help somewhat with putting a sampler's subjective evaluation into perspective.
Whatever is done roasts need to be evaluated fairly with essential variables of espresso extraction addressed and controlled with a protocol that is not arbitrary or unreasonably inconsistent. You could really skew your results if you were more focussed on satisfying specific elements of a rating system rather than focussing on the best extraction. Extraction first then rating system.
With respect to extraction quality, since there is no way to satisfy all possible conditions a roast might be extracted under from equipment capabiltiy to barista skill, evaluations need to be geared towards what a roast is capable of under generally very good to ideal conditions. There's no other way to know what a roast is capable of, IMHO, and report on it. Rather than discourage espresso interest I think it would encourage it. When I started I read about the honey thick pours from dialed in grinds and the deep charactered extractions of extraction temps tuned in to a specific roast it encouraged me to get deeper into espresso rather than give it up. It makes espresso much more interesting to know how complex and dynamic it is and more rewarding when you approach very good to ideal espresso for yourself. Espresso's evaluation needs to be uncompromising not only to be competently informative but to encourage appreciation.
JasonBrandtLewis Senior Member Joined: 9 Dec 2005 Posts: 6,099 Location: Berkeley, CA Expertise: I live coffee
Espresso: Elektra T1 - La Valentina -... Grinder: Mahlkönig K30 Vario -... Vac Pot: Yama 5-cup Drip: CCD, Chemex Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 12:47pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
mrgnomer Said:
I think issue is more how a roast is evaluated not what kind of rating system is used. The evaluation process would probably evolve into a rating system. If you're evaluation is subjective and under what you consider is common extraction conditions like 200F across the board, same grind/dose/distribution for all roasts your rating would be more subjectively based regardless of what system you're using.
If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective.
Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control. The results of a test like that would be more reflective of actual roast quality. Yes, tasting is subjective for sure and could be influenced by the sampler's preference. You could give a biography of the sampler with their personal preferences in a roast. That would help somewhat with putting a sampler's subjective evaluation into perspective.
But given the realities of the space-time continuum . . . just how much time does one reviewer have to spend on each coffee sample to pull shots? to tweak grind and temperature for each coffee*? to write up the results? to write up his/her CV to accompany the article? to meet his/her deadlines for the next publication date? (and so on and so on and so on)
Let alone the roast date of the beans, the freshness (or lack thereof), etc. And IS the "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru the same blend all year 'round?
Posted Mon Oct 8, 2007, 1:31pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
JasonBrandtLewis Said:
Within limits, it's SUBjective, either way.
In an ideal world, perhaps . . .
But given the realities of the space-time continuum . . . just how much time does one reviewer have to spend on each coffee sample to pull shots? to tweak grind and temperature for each coffee*? to write up the results? to write up his/her CV to accompany the article? to meet his/her deadlines for the next publication date? (and so on and so on and so on)
Let alone the roast date of the beans, the freshness (or lack thereof), etc. And IS the "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru the same blend all year 'round?
Philisophically, everything is subjective. Space-time continuum? Only if Dr. Who were pulling the shots, I'd imagine or Picard found himself in a wormhole after firing up his GS3. Maybe you could get Q to do the reviews?
As far as time goes, it's experience that would be more of a factor than time. We're talking knowledgeable baristas doing the evaluation not casual home users who happen to be writers and general critics.
The evaluation can be broken down, I believe, to objective factors. How well those factors are controlled as variables depends on the experience of the evaluator. Saying it can't be done is non sense. You'd have to be a top ranked world class barista to say that with any credibility.
The point is general critics are putting forward reviews that read like they're expert evaluations. Au contraire, mon frère, they're turning out not to be experts. But the average consumer wouldn't know that just like the average consumer who reads the top 10 espresso picks from Consumer Reports wouldn't know that those picks are not even on the radar for really good espresso. Really good espresso along with the good roasts that make it possible needs to be fairly evaluated and promoted by good, knowledgeable reviews. Doing otherwise is just marketing.
Yeah, some roasts could be off from ideal for any number of reasons. Good roasters wouldn't suffer as much, I'd imagine, the inconsistencies of variables like green quality or freshness. They'd be sourcing out good greens and making sure their packaging and shipping ensures their roasts reach consumers in a way that preserves their quality. If their blend varies from time to time so what. It's the quality of the roast/blend that's being assessed, not what's in it.
If big roaster wants to ship their roasts over by barge and it takes months to be accessible to North American consumers, that's their choice. If a small/mid sized artisan roaster dates their roasts and ships it right after roasting to arrive anywhere in North America within 3-4 days, that's their choice too. If the decision to source top quality greens and ship right after roasting makes for better espresso than a stale overseas roast an evaluation that's based on predominantly subjective opinion isn't going to be definitive. The conclusions of an objective evaluation by qualified baristas is the only way to know with any credible certainty.
JasonBrandtLewis Senior Member Joined: 9 Dec 2005 Posts: 6,099 Location: Berkeley, CA Expertise: I live coffee
Espresso: Elektra T1 - La Valentina -... Grinder: Mahlkönig K30 Vario -... Vac Pot: Yama 5-cup Drip: CCD, Chemex Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 2:18pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
So -- out of curiousity, Mr. Gnomer -- where do you draw the line?
Do you never review such brands as Illycafe? What about Terroir? After all, you'll get it a lot quicker if you like in Portland, Maine than you will of you live in Portland, Oregon. Conversely, one would expect to receive a shipment from Stumptown more quickly if you lived in Portland, Oregon than in Portland, Maine . . . let alone, how fresh is it if you walk in?
What about reviewing coffees from Starbuck's? Peet's? Peerless? Caribou? Or do you only review coffees roasted from "micro-roasters" like Blue Bottle or Ecco Caffe? Is Intellegensia too large?
It's your publication. You set up the criteria . . .
* * * *
Now then, on another tangent, what are the prerequisites that qualify one to serve as a judge in a professional competition? There are standards put forth by the World Barista Championship. But rarely are these professional judges the ones who write critiques of various coffees -- such as may be found on coffeereview.com or in the forums of coffeegeek.com. I don't know -- are only certified judges permitted to write articles for magazines, trade journals, newspapers, etc.? I rather doubt it, but if the editor of Globe & Mail wants to hire a professional WBC judge to write a column on coffee, rather than a journalist on staff in the "Food" section . . . .
(Note: if I read the website correctly, Mark Prince is on the 2006-2007 WBC roster as a Certified Judge for Sensory Evaluation only -- not Technical -- having passed the written test with 90%, and the "Sensory Skills Test" with a score of "100/90/70" . . . whatever that means. I know that doesn't mean that another judge's evaluation will agree exactly with Mark's, nor does it mean my evaluation will, either. And, no disrespect whatsoever intended -- I know I would probably fail the WBC judges' test -- but whose opinion is more important: Mark's, another judge's, or my own? Hey, if I had Mark's taste buds inside my mouth . . . but I don't; I have my own.)
None of this should be taken as being arguementative. Rather, these are ALL issues I've dealt with for some 25 years as a professional wine judge/writer.
MarkPrince Moderator Joined: 19 Dec 2001 Posts: 5,462 Location: Vancouver, BC Expertise: Professional
Espresso: KvdW Speedster Grinder: Compak K10 WBC Vac Pot: A bit too many Drip: Clive Coffee Drip Stand Roaster: Hario Glass Retro Roaster
Posted Mon Oct 8, 2007, 3:02pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
JasonBrandtLewis Said:
(Note: if I read the website correctly, Mark Prince is on the 2006-2007 WBC roster as a Certified Judge for Sensory Evaluation only -- not Technical -- having passed the written test with 90%, and the "Sensory Skills Test" with a score of "100/90/70" . . . whatever that means.
They have me listed as sensory only? Yikes. I didn't see that. Maybe it's because I've told Justin over and over again that I'm not comfortable doing technical... ;)
Re the scores, 100/90/70 (actually, it's 72, or was supposed to be) refers to the FDA-styled Sensory Skills Test that the SCAA does - three rounds, each one harder than the last, the last one pretty freakin' hard. Those are my scores in each round (also, the minimum required to "pass".
I took the sensory skills test informally a second time to see if I fluked the first one, and got 100/100/74 the second time around.
Philosopher Senior Member Joined: 13 Feb 2007 Posts: 188 Location: Australia Expertise: I like coffee
Espresso: Silvia Grinder: Rocky
Posted Mon Oct 8, 2007, 3:39pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
Actually the most important thing is to find a wine critic who is consistent.
Agree
JasonBrandtLewis Said:
Within the wine trade, there are so many variables than can affect a wine -- from temperatures during shipping to storage to food pairings to the allergies or cold that a taster may be sufferring -- we often say, "There is no such thing as a great wine, only a great bottle of wine." Perhaps there is no such thing as a great coffee, but only a great cup of coffee (or espresso).
You could take the analogy further. The bean is like the un-picked grape from the vine. The roaster and barista are the wine-makers.
Riesling variety can make some very impressive wines. However, the final result is very dependent on where that grape is grown, which ones selected and what happens before it gets into the bottle. There are not many wine-makers who can turn grapes into exceptional wines.
This is why this espresso rating system loses its practicability for most people. We are rating a bean or blend when what we are really rating is a final composite of many factors, some which we have no proper control over.
Philosopher Senior Member Joined: 13 Feb 2007 Posts: 188 Location: Australia Expertise: I like coffee
Espresso: Silvia Grinder: Rocky
Posted Mon Oct 8, 2007, 4:15pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
mrgnomer Said:
If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective. Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control. The results of a test like that would be more reflective of actual roast quality.
It seems that if we are just talking about espresso, any rating system either becomes too generic (and potentially meaningless) or too overly prescriptive (and too unwieldy and impractical). I was wondering if you were to prepare a cup by other methods e.g. vac-brew or french press that you could get such a wide scatter of results.
The rating system also needs to be directed to the appropriate target market. For the majority of people, they want to know if the blend can give good results but is fairly forgiving even when they are preparing for a party of 10 people. The minority are those caffeine-toxic from tweaking their machine the whole morning trying to find the right zone for their blend.
Symbols: = New Posts since your last visit = No New Posts since last visit = Newest post
Forum Rules: No profanity, illegal acts or personal attacks will be tolerated in these discussion boards. No commercial posting of any nature will be tolerated; only private sales by private individuals, in the "Buy and Sell" forum. No cross posting allowed - do not post your topic to more than one forum, nor repost a topic to the same forum. Who Can Read The Forum? Anyone can read posts in these discussion boards. Who Can Post New Topics? Any registered CoffeeGeek member can post new topics. Who Can Post Replies? Any registered CoffeeGeek member can post replies. Can Photos be posted? Anyone can post photos in their new topics or replies. Who can change or delete posts? Any CoffeeGeek member can edit their own posts. Only moderators can delete posts. Probationary Period: If you are a new signup for CoffeeGeek, you cannot promote, endorse, criticise or otherwise post an unsolicited endorsement for any company, product or service in your first five postings.
= No New Posts since last visit 