We’re almost ready to start the Battle Royal of Italian vs. North American espresso blends, and provide an examination and some thoughts into new and possibly different ways that espresso can be evaluated by the industry, by coffee reviewers, and even by consumers.
In the first article that brought this subject up, Peets, Baskets and Blends, I was fairly harsh towards Ken Davids and his August, 2007 feature on CoffeeReview. In the second article, called Evaluating Espresso, the discussion moved on to how the industry currently evaluates espresso, and how there really isn’t a set standard, akin to the “cupping standard” that the coffee industry follows (albeit with some variances).
The subsequent discussions about the points raised in Evaluating Espresso were almost all that I hoped they would be – some great ideas and debates bounced back and forth. We had comments from dissenters (those who didn’t agree with the methodology I proposed), from certified wine geeks, and from many people very passionate about great espresso.
The only thing missing from the discussion was Ken himself, which I think is still a shame – and mostly my fault. In the original article, looking back on it now, I know I could have been more polite and welcoming to Davids, and the tone of the original article, more than anything else probably precludes his participation in these discussions. But never say never. I hope Davids will join the community discussions here and if for no other reason, help advance a more standardised form of testing, evaluating, ranking and rating espresso.
In this last article, I’ll get much more into detail about how we did our testing, who did the testing (and barista duties), and what the results were. This is by no means the final say on how to test and evaluate espresso blends – if anything, it’s barely a starting point. But ultimately , I think what we’ve come up with is more “fair” to the wide range of blends than any other test documented online, at least for the moment.
Testing Methodology (and Transparency)
In Italy and around the world, those serious about espresso know the “Four ‘M’ Rule” almost by heart. Put simply, the four Ms are “Macchina” or Machine, “Macinzaione”, or grinder, “Miscela”, or the blend, and “Mano”, the hand (or person) running the show. (For more detail about the 4Ms, read an excerpt from an Imbibe Magazine article I wrote last year). All are equally important to producing a great shot of espresso, and all must be paid attention to if the goal is to be as fair as possible to each and every blend (miscela) you are evaluating.
We had the tools here – on the macchina side, a well tuned La Marzocco Linea modified machine (essentially a GS/3 inside, with bigger boilers); for the macinzaione, a choice between the Anfim Super Caimano with titanium burrs, a digital timer (for great accuracy), and one of the best dosing grinders out there, or a Mazzer Super Joly with fairly fresh (100lbs so far) burrs. The miscela? well, the blends were what we were judging, so one would hope they’d be up to spec. And the last element, the mano – that was the one I was most concerned with. I had to find a person (or persons) who were up to matching the first two Ms.
I mentioned in the previous article that in Vancouver we have a person that I consider one of the world’s top baristas, if not the best barista practicing the art today. A person who, if he ever entered a barista competition, everyone else might as well just be playing for second place. That person is Vince Piccolo, formerly of Caffe Artigiano, currently owner of 49th Parallel Roasters.
I don’t say this lightly, for a variety of reasons. I know many great baristas around the world, and I’d struggle to come up with a top five or top tend list, because I know dozens and dozens that would be in those lists. I also don’t make this statement about Vince lightly because he and I have had a wild ride in our friendship, including a long period where we didn’t talk, except to have harsh words for each other. But in all my travels, all my witnessing the worlds’ best baristas on the stage and in their comfort zones (read: their cafes), I have not seen anyone come close to having the complete package Vince has: the intrinsic machine skill, indepth knowledge, and almost arcane sixth sense about espresso. Because of this, Vince was my first choice to be the “barista” in this shot pulling evaluation.
I approached him about it, and he did agree to do the test, with a caveat: he would only do the Italian side of the challenge. He refused to pull the N. American side, for one simple reason – he didn’t want any accusations or bias to be levelled against the test or him, because one of the seven US/Canada blends is is a blend he’s very familiar with – 49th’s Epic Espresso.
That changed things a bit. Initially, I wanted to pull all 14 coffees over two days (7+7), mixing up the Italian blends with the N. American blends. And I wanted one barista to be pulling everything. But I also wanted Vince to be that barista, because I felt out of anyone, he’d be the most fair and most skilled to get the most out of all the coffees, especially the Italian offerings.
Fortunately, not only do we have Vince Piccolo in Vancouver, but we have a 3 time, top three WBC finalist in town: his brother Sammy Piccolo. And Sammy was up for doing the barista duties, for which I’m eternally grateful. There was no real bias or conflict because Sammy has very limited experience with Epic, a blend that is very different from Caffe Artigiano’s main blend, the one he works with every day.
Having two baristas did mean changing up the testing situation a bit. We moved from intermixing the blends to doing Battle Italy on one day, and Battle North America on the second day. Vince Piccolo would pull all the Italian blends, and Sammy Piccolo would pull all the N. American blends. The tasters would remain the same.
To make this test as fair as possible, we did this evaluation ‘double blind’. This means the barista would not know what coffee they were using – they got the coffees in a numbered bag. The tasters would also not know what coffee they were scoring. They got them in numbered sequence. To continue the fairness to the coffees, I did not open the coffee bags until moments before the barista started their shot pulls, thus giving as much advantage to the (much older) Italian blends as possible. Most were in one way valve bags, and the hope was, pulling the shots soon after opening would give them a boost in terms of their output.
Barista Variables
I gave two absolutes to the baristas: the shots must be WBC standard volumes (that is, a double pull off a double basket, delivering a total of 45-60mls per shot pull, or about 22.5-30mls per espresso cup), and they had to use the same filter basket for all the coffees. That filter basket is the Synesso-sourced 14g basket (which everyone seems to be confusing with the 1mm taller 18 gram basket these days). The 14g basket can be stuffed and updosed with as much as 19g or more of coffee when used on a La Marzocco machine.
The machine used is a highly customized La Marzocco Linea that has PID controls, a .6 gicleur valve inside, and is extremely stabilized, using many of the tricks found in the GS/3 and other top technology machines. Out of the two grinders the baristas had to choose from, both went with the Anfim model.
Every other variable was up to the barista. They were presented with three or more pounds (up to 2kilos) of each coffee and told to dial it in as much as they could, then once they were happy with the shots, serve the judges. The baristas could change or modify any of the following:
- grind,
- volume of grind (dose),
- dosing technique,
- tamping technique,
- preheating,
- preinfusion (ie, wet the pucks if they wanted),
- grouphead temperature,
- cup temperature,
- machine pump pressure (which we did change several times),
- any other variable they could think of.
And both baristas did just that – doing modifications to the brewing temperature frequently, and choosing to updose some coffees, normal dose others, and change the grind just about every blend. Sometimes, the tamp and pack were modified, bringing out maximum sweetness and body in some shots.
Tasting Evaluators
For tasters, there were many people in Vancouver who would have been ideally suited to the task, but I decided to go with the Jones brothers – Colter (the 2006 Canadian Barista Champion) and Barrett (2nd place in the 2006 CNBC) for a variety of reasons, including their exposure to a lot of world calibre espresso, and both brothers’ serious foodie backgrounds. Both Colter and Barrett were fresh off the 2007 WBC, and had freshly experienced a lot of the top espresso shots ever pulled.
I stepped in as the third tester, mainly because of my own WBC, USBC, and CNBC experience. But there’s a caveat. I did test the Italian blends, but opted out at the last moment on testing the N. American blends because I was suffering from allergies, and on medication that destroyed my palate.
My backup was Andrea Piccolo – Sammy’s wife, and involved in the QC and cupping going on at her place of work: Swiss Water. In the past, I noticed that Andrea’s likes and palate were very similar to my own likes and tastes, and after doing some independent tasting and evaluation on her scoring vs. my scores on the Italian side of things, I was confident she was calibrated to my scores, and would score almost identical to how I would in the N. American side of the equation.
In addition, we had several “ghost” tasters who were there to provide backup should one judge be way off calibration. Their scores weren’t officially part of the test and ratings, though some of their comments are included in the tasting results, and in the N. American tasting session, influenced some of Andrea’s scores, only minutely (about 4 individual scores were adjusted, in most cases, about a half point down).
How We Scored The Espresso
Prior to the evaluation, I had a meeting with Colter Jones, Vince Piccolo and Barrett Jones about how we should score. I also had some discussions with world-calibre espresso judges from my WBC contacts on how to best score and evaluate espresso, from a judges’ standpoint.
On the overall philosophy, it was a no-brainer that for something as subjective as espresso (which, in my opinion, is even more subjective than cupping coffee), nothing short of a panel of judges would suit – one person, operating in a vacuum, is not fair to an espresso blend, and is too liable to allow for tunnel-vision in the scoring, not mention showcasing personal biases and such. A minimum of three tasting judges seemed a good starting point – two is almost as arbitrary as one judge is, but with three (or more), then you start to get better on the averages on tastes, likes, dislikes, good shots, lucky shots, bad shots, etc.
On how to actually score, the natural tendency was to go with a 100 point system: it’s what most people are familiar with and it is natural to see 100 points as “perfect”. I did have some reservations about this because, as the conversations about how and what to score progressed, I knew that it would be very difficult for any blend, no matter how good, to score higher than the mid-eighties. With the coffee cupping world as of late handing out 92s, 93s, 95s, and more and more 97s (which I believe is a serious problem, as documented in the last article), how would a blend that scored 75 in our evaluation come across? As lacking? As substandard? As a good score?
Because of this, I briefly considered not using a 100 point scale, but instead go for a 10 point scale, or a word-based scale, but only briefly. 100 points is a bit of a standard across the coffee evaluation world, and in many other foodie worlds as well, and it’s something we should preserve. But I also knew it was important to stress that, in our kind of espresso evaluation, nothing’s sacred or out of bounds (ie, if something deserves a 2 or 3 in a category, it gets it – no “5” baselines for us, like they do in cuppings). Because of this, some blends could score below 50 points (something never seen in specialty coffee evaluations), and a good score would be dramatically different than the typical “Coffee Review” score. I’ll detail more below what our panel felt were “acceptable”, “good”, “really good”, “great”, and “must buys” below.
So with that preamble, here’s what we came up with, representing a 100 point maximum score:
Sweetness Balance: 10 points. Not a test or score for maximum sweetness, but more an evaluation on how well the sweets in a shot of espresso balance out with all other factors, including citrus and sparkling acidity (or pursing, puckering acidty), body, and other elements.
Body Balance: 10 points. Overall body, mouth-feel, substance of the shot and how it coated the tongue, how it impacted the overall flavour.
Aftertaste: 10 points. I fought a bit to get this one in – for me, aftertaste is a key element of a great espresso (or a horrible one) – this was balanced between immediate aftertaste and lingering aftertaste. Because of this scoring, judges were encouraged to swallow (ie, not spit) the espressos. This limits the number of espresso shots a judge can have in one evaluation session.
Acidity Balance: 10 points. There’s good acidity, and bad acidity. There’s acidity confused with sourness, and sourness confused with acidity. We were looking for how acidity balanced in the cup – this score does not represent how acidic the shot was, but how the balance was.
Aroma: 10 points. This score was based on three things – whole bean smell, ground coffee smell, and the aroma from crema and the espresso as a whole when it was imbibed.
Crema: 10 points. Multiple layer evaluation involving colour, density, structure, depth of the crema, and also how it impacted the overall cup. Taste was a big component in this score, more so than any visual scoring.
Espresso with milk: 10 points. Using the WBC standard, the baristas built two cappuccinos after the espresso evaluation for each blend was done. The cups were 4.5-5.5oz total volume, with 4.5oz beverages built (all with latte art, natch). The drink was evaluated pretty much identically to the WBC standard, but with one 10 point score instead of multiple scoring elements. We looked at taste balance, how the espresso cut through the milk as well as interacted with it, and how the aftertaste was. Both “evident espresso in the drink” and “espresso’s taste changes to the milk” were part of this scoring.
Judges’ Overall Score: 10 points. This is the judges’ overall impression of the espresso and cappuccino, of the coffee as a whole. It is scored after a round table discussion about the coffee with the barista.
Barista Score: 20 points. I wanted something that reflected the barista’s impression of the coffee, since they tasted more of the coffees than any of the testers. The score reflected how easy (or difficult) the blend was to work with (a big impact on this score), how it was to “dial in” and how it tasted as they tried to get the most out of the coffee blend.
What Our Scoring Means
Because no point was sacred (ie, I encouraged the tasters to score accurately – if something deserved a 0 or a 1, give it that score. If it deserved a 10 (or a 20), then give it out), I knew going in there would probably be scores below 50 points, sometimes much less. I also knew that a score above 80 points would be pretty darned near impossible, unless the barista was able to pull off two “god shot” doubles in a row – those kinds of espresso shots where the stars lined up, the heavens sang, and everything fell into place. So we had some discussion on what would qualify as good, great, bad, okay, shots, and here’s what we came up with.
Below 40 points: Unacceptable. Not a blend that is worth buying.
40 to 50 points: Unacceptable, but with potential to become acceptable. A bit cryptic but what it means is a blend scoring in this area may have some “5”s and “6”s in some scoring categories, and probably was having a very bad day on the grinder and espresso machine. It could go up, but it could also go down. If the barista score is 10 or more, it’s a blend you may want to consider trying anyway.
50 to 60 points: Acceptable blend. It won’t offend, but it won’t stand out either. In skilled hands it may even do better on some days.
60 to 70 points: A good blend, and one recommended to try. In this point range, the blend isn’t “one of the world’s best”, but it’s certainly not offensive, and would have some interesting taste notes that may appeal more to some people. Look for barista scores, milk scores in blends that fall in this category – some may be even better in milk than in straight shots.
70 to 75 points: A very good blend, one that stands out in taste nuances, straight shot quality, and ability to consistently deliver a pleasing shot of espresso. Again, look at barista scores, milk scores, but also judges’ overall scores in blends that fall into this point range to find out how it delivered. A recommended buy.
75 to 80 points: A great blend, highly recommended. Anything scoring an 80 on our testing sheets is a standout blend, and what I’d consider the equivalent of a 90 to 93 in a Cup of Excellence scoring system. These are blends that feature a lot of 7s, 8s and 9s on the scoring sheet. Particularly forgiving blends get above 15 points in the Barista score, and this is another sign of the blends’ great qualities.
80 to 85 points: A “top” blend, must try. Any blends hitting this mark are the equivalent of a “95” in the cupping world. We had some discussion about whether or not any blend could consistently score higher than an 85 on our scoring sheet if you were pulling shot after shot after shot of it, and evaluating each shot independently, and the answer was “no”. Anything above an 85, at this point in time with current technology, current barista skills, current understanding of the espresso process, was deemed impossible. Maybe some day as the technology, equipment, and skills continue to refine, we can be even more fair and friendly towards the blend, but for now, in this scoring system at least, a score in the 80 to 85 range would be the gold standard.
I want to be clear on this – I believe that any blend that scored 75 or higher in our scoring methodology is a potential “100 point” blend when compared to the more old school ways of scoring – in other words, these blends have more potential to deliver that magical “god shot” than anything scoring below that threshold. We evaluated what we tasted, and giving a blend a 79.5 or a 73 or a 72 was a very high score on this scoring system. When pulling shots informally, after the official tastings were over, comments like “wow, this is even better than the scored shot” were made at least once or twice. This shows the nature of espresso and how damned difficult it is to evaluate. Our knowledge of espresso, and performance in delivering it is still so much in the cradle, that we still all seem to enjoy the occasional (and rare) “god shot” calibre drink that you simply cannot produce in a regimented test.
With all of that out the way, let’s get to the results of this testing!
Battle Italy
Here are the results for the Italian blends, listed in the order they were tested (overall placing is listed next to the Average Score). NB – reviewers comments are separated by a semicolon, and are pulled directly from the scoresheets.
Lavazza Qualitia Rossa
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 2.0 | 7.0 | 4.0 | 7.0 | 4.0 | 5.0 | 7.0 | 5.5 | 11.0 | 52.5 |
| Barrett Jones | 2.0 | 7.0 | 4.0 | 2.0 | 4.0 | 7.0 | 6.0 | 4.0 | 11.0 | 47.0 |
| Mark Prince | 4.5 | 5.0 | 3.0 | 3.0 | 6.0 | 6.0 | 8.0 | 4.0 | 11.0 | 50.5 |
Lavazza‘s Qualitia Rossa is described on their own website as being, “the perfect blend for breakfast because it goes so well with milk and is excellent with bakery products (bread, rusks and biscuits).” We did find it went fairly good with milk (scoring a 6, 7 and 8 by the three judges), but that was about it. Surprisingly, it did much better than Lavazza’s other blend in this testing, the supposedly superior (and all arabica) “Oro” blend. The Qualitia Rossa is a Robusta / Arabica blend and our tasters could distinguish the rubber tastes of the robusta in the shots.
Reviewer comments: Bitter, bitter, burnt, bitter; full body but no sweetness; aftertaste is not pleasing; slight acidity balances out the body, but needs sweetness to complete; not much flavours popping out at all; crema looks good, made me think it would taste better than it did; blend comes alive in milk; tastes much better in milk; milk rounds out the espresso
Pellini Aroma d'Oro
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 3.0 | 5.0 | 5.0 | 2.0 | 7.0 | 8.0 | 8.0 | 6.0 | 13.0 | 57.0 |
| Barrett Jones | 5.0 | 7.0 | 7.0 | 4.0 | 6.0 | 5.0 | 7.0 | 6.0 | 13.0 | 60.0 |
| Mark Prince | 4.0 | 5.0 | 6.0 | 2.5 | 5.0 | 8.0 | 8.5 | 5.0 | 13.0 | 57.0 |
Pellini is a brand I’ve never heard of before this test, but came highly recommended. The Pellini Aroma Oro blend is their premium robusta arabica blend (90% arabica), and actually did quite well even though the coffee was showing serious problems with age of beans. We found, with most of the Italian blends that age was unkind to two things our tasters placed some value on: sweetness in the cup, and a good balance of acidity. The Oro blend suffered in both of these regards.
Reviewer comments: distinct lack of sweetness and vibrancy; barely an edge of sweets in the cup; hard to distinguish any unique flavours; big body, nice touch; good body and aftertaste; struggling to get some chocolate, some vanilla in this shot; overall aroma and crema very pleasing; crema’s very nice; aftertaste is one of the better today; not much individual flavours – just coffee; seems somewhat dulled out; great with milk, I’d buy the blend for cappuccinos; great espresso for milk; becomes a bit more chocolate and balanced in milk.
Segefredo Intermezzo
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 2.0 | 7.0 | 0.5 | 2.0 | 4.0 | 7.0 | 4.0 | 1.0 | 7.0 | 34.5 |
| Barrett Jones | 1.0 | 7.0 | 1.0 | 2.0 | 6.0 | 7.0 | 4.0 | 1.0 | 7.0 | 36.0 |
| Mark Prince | 1.5 | 4.0 | 1.5 | 3.0 | 4.0 | 5.0 | 3.5 | 2.0 | 7.0 | 31.5 |
One o the biggest sellers in Italy, and the worst coffee in this roundup, I’m not sure what that says, but Segafredo’s Intermezzo didn’t do well; in fact, some of the tasters refused to have more than a sip or two.
Reviewer comments: very offensive; burnt rubber; tastes like skunk; actually caused a bit of vomit action I had to suppress (Colter Jones’ comment); skunky taste and smell; that it has body isn’t necessarily a good thing; crema is average; aroma – horrid; has a flavour – burning rubber; barely becomes acceptable with milk; less offensive with milk; difficult blend to work with (barista’s comment)
Pellini "Top" Espresso Blend
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 6.0 | 6.0 | 5.0 | 5.0 | 6.0 | 7.5 | 7.5 | 7.0 | 12.0 | 62.0 |
| Barrett Jones | 6.0 | 6.0 | 5.0 | 5.0 | 6.0 | 7.5 | 8.0 | 6.5 | 12.0 | 62.0 |
| Mark Prince | 6.0 | 6.5 | 6.0 | 6.0 | 5.0 | 8.0 | 8.0 | 7.5 | 12.0 | 65.0 |
The top scoring coffee in the Italian side of the battle is one that all the participants agreed as being the best cup, even before the scores were calculated. We also felt that Pellini Top, a 100% organic, all arabica blend, could cross into the “very good” or better category if we’d had it fresher. It too was showing signs of bean age, with flavours flattening out and a slightly dulled overall taste and feel, but it was a very pleasing espresso, and certainly the best we had that day.
Reviewer comments: hints of chocolate in the aroma; bean quality looks fantastic; beans look great; initial taste is very pleasing; hard to distinguish flavours, but I’ll say bittersweet chocolate, vanilla and some spice; good body, mouth-feel, overall balance; really nice shot of espresso; I’d be happy to have this in a random cafe; some vanilla and mild spices in the aftertaste; hints of vanilla, some faded soft mints; crema looks great; good crema; acidity’s a bit low, but nice – could use a kick; really nice espresso in milk; milk makes this in the very good category.
Lavazza Qualita Oro
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 2.0 | 5.0 | 2.0 | 3.0 | 4.0 | 5.0 | 4.0 | 4.0 | 8.0 | 37.0 |
| Barrett Jones | 2.0 | 6.0 | 2.0 | 2.0 | 5.0 | 3.5 | 6.0 | 4.0 | 8.0 | 38.5 |
| Mark Prince | 1.5 | 4.0 | 3.0 | 3.0 | 4.0 | 3.5 | 6.0 | 4.0 | 8.0 | 37.0 |
Oh Lavazza. Qualita Oro is supposed to be your best “consumer” coffee on the market (their blue bag commercial brands are supposed to be better), but this one scored pretty bad, and was our second last coffee in the roundup. It was another blend that the reviewers had a hard time drinking.
Reviewer comments: very bitter and sour at the same time; bordering on offensive; stale, drek, stanky; I’m not enjoying this cup (Colter Jones comment); nothing much in this cup except for some harsh tastes; aroma is better than the taste; aftertaste gets worse by the second; barely acceptable in milk; gets lost in milk, which is agood thing.
Illy Medium (red can) Roast
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 3.0 | 4.0 | 2.0 | 3.0 | 5.0 | 4.0 | 8.0 | 3.5 | 11.0 | 43.5 |
| Barrett Jones | 1.0 | 6.0 | 2.0 | 2.0 | 7.0 | 3.0 | 8.0 | 2.0 | 11.0 | 42.0 |
| Mark Prince | 2.0 | 4.0 | 3.0 | 3.0 | 5.0 | 4.0 | 9.0 | 4.0 | 11.0 | 45.0 |
When the reviewers were told at the end of the tasting that this coffee was Illy’s famed coffee, everyone was surprised, and everyone admitted they’d had much better Illy coffee in the past. We looked at the remaining beans in the samples, and were surprised at the number of broken beans and even more surprised at finding some defects – illy is supposed to have a conveyor belt, laser system that scans every single roasted bean for defects and brokens. All the reviewers agreed that we most likely had a “bad batch” of especially stale Illy, and sure enough, after the test, I bought more illy from a second source, and found the “best by” date to be 9 months later than the ones we used in the testing. And our cans still hadn’t come close to expiry yet, according to Illy! C’est la vie. Roasted on dates, not “best by” dates, Illy!
Reviewer comments: no sweetness at all in this cup; some sweetness, but barely there; very dull, flat espresso; it seems like a hull with nothing adorning it; aftertaste is pretty offensive, though not the worst of the day; aroma is surprisingly nice; aftertaste is very bad; aroma is okay, very neutral; most excellent with milk; very weird – in milk this is fantastic. milk is the best thing that could have happened to this blend.
Segefredo Metropol
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 4.5 | 6.0 | 5.0 | 6.0 | 5.0 | 8.0 | 8.0 | 7.0 | 14.0 | 63.5 |
| Barrett Jones | 4.0 | 5.0 | 5.0 | 5.0 | 6.0 | 7.5 | 7.5 | 6.5 | 14.0 | 60.5 |
| Mark Prince | 6.0 | 5.0 | 5.0 | 6.0 | 5.0 | 7.0 | 7.0 | 7.0 | 14.0 | 62.0 |
Segafredo’s other offering in this test, their Metropol blend, was called their best consumer blend by several espresso experts over in Italy, and for us, it was a night and day difference compared to the very skunky Intermezzo. Metropol is also a blend of arabica and robusta, and while it is not readily available yet in BC, my supplier had just received a sample shipment and offered me two bags. I think one of the reasons why Metropol did so well is because it was obviously fresher than any of the other Italian blends tested.
Reviewer comments: A fairly silky smooth espresso; vibrant, has potential; even balance between sweetness, body and acidity; body is light, but pleasing; body could be a bit more; aroma is fairly good; aroma aids the overall taste sensations; getting some slight fruit, nice acidity, some hazelnut in the shot; really nice in milk; gets lost a bit in milk; nice blend to work with (barista comments).
Battle: Italy Followup Comments
By and large, the testers were all of the belief that age of beans was the worst enemy to these blends. There was a desire around the table to taste Pellini Top and Aroma D’Oro fresh, because we all felt it could be a 75-80 point coffee if pulled just right. Segafredo’s Metropol probably had the greatest advantage by being a blend “fresh off the air freight”, and probably scored at least 10 points higher than it would have if I’d bought it off the grocery shelf.
We were all quite disappointed with the scoring of Illy’d Medium Roast whole bean, once we found out which coffee it was. In fact, speculation around the table during testing was that the Pellini Top was illy, and the Illy was one of the Segafredos or Lavazzas. Everyone agreed that in the past, we’ve had 65, 70 point Illy espresso, and sure enough, when I bought another can a week later, not only did I notice the “Best by” date was nine months later than our test samples, but the resulting shots made with that can were indeed back to the Illy standard that one can enjoy in North America. I did notice in the newer can some broken beans, but not as many as in the older cans – something we were also quite surprised about.
Our barista, Vince Piccolo, really went out of his way to get the most out of these blends – so much so that in some cases, he went through the entire supply of coffee (reflected in both his score and the fact that he had to “make do” and just serve shots, without hope of getting anything better than a 35 or 40 point shot of espresso. Some of the blends were quite easy to work with, and that too is reflected in his “barista score”.
Battle: North America
Here are the results from the five US blends and one Canadian blends we tested, in the order they were blind-tasted in.
Ecco Caffe Ecco Reserve N. Italian Blend
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 6.0 | 6.0 | 6.5 | 7.0 | 7.0 | 8.0 | 8.0 | 7.5 | 15.0 | 71.0 |
| Barrett Jones | 6.0 | 6.0 | 6.6 | 6.0 | 6.5 | 7.5 | 7.5 | 7.0 | 15.0 | 70.0 |
| Andrea Piccolo | 6.0 | 6.5 | 6.0 | 6.0 | 6.5 | 7.0 | 7.0 | 6.0 | 15.0 | 66.0 |
Ecco Caffe‘s Ecco Reserve blend ($14/lb) was the standout espresso of the entire evaluation, beating the second place coffee by 6 full points. It shined both as a straight espresso and in cappuccinos. Highly recommended by all the tasters and the barista pulling the shots.
Reviewer comments: gentle and pleasant; chocolaty finish; lingering aftertaste that pleases; milk chocolate notes throughout; bright tones that don’t offend, with a deep chocolate finish; syrupy chocolate that morphs into caramel, aftertaste pleasing; chocolate dominates; crema was resilient and lasted for minutes; doesn’t dominate milk, but cuts through nicely; aroma exudes comfort; milk balance is amazing; super chocolate throughout, amplified by milk.
49th Parallel Coffee Roasters' Epic Espresso
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 8.0 | 8.0 | 7.0 | 8.0 | 8.0 | 8.5 | 6.0 | 7.5 | 15.5 | 76.5 |
| Barrett Jones | 4.0 | 6.0 | 6.0 | 7.0 | 8.0 | 8.5 | 6.0 | 7.0 | 15.5 | 68.0 |
| Andrea Piccolo | 8.0 | 8.0 | 7.0 | 7.0 | 7.0 | 9.0 | 7.0 | 7.5 | 15.5 | 76.0 |
Our second place coffee comes from 49th Parallel Roasters, the new roasting works of former Caffe Artigiano owner Vince Piccolo and his roast master, Mike Piccolo. Epic Espresso ($14.95/lb CAD) is their signature blend, and had mixed results on the table, with two testers liking it more than the third.
Reviewer Comments: Sweet finish; fruity and citrus notes; tasting grapefruit with orange bitters; creamy aftertaste; slightly ashy beginning; bright as a short espresso; excellent crema; aroma lingers; slight chalk in the taste; cuts through small milk well; aftertaste mingles well; not so great in milk as some others; fruity syrup cuts through milk.
Intelligentsia Coffee Roasters' Black Cat
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 7.0 | 8.0 | 5.0 | 6.5 | 7.0 | 7.5 | 8.0 | 7.0 | 16.0 | 72.0 |
| Barrett Jones | 7.5 | 7.0 | 8.0 | 6.5 | 7.0 | 7.0 | 7.0 | 7.5 | 16.0 | 73.5 |
| Andrea Piccolo | 6.5 | 7.0 | 7.0 | 7.0 | 7.0 | 6.0 | 7.0 | 7.0 | 16.0 | 70.5 |
Intelligentsia Coffee Roasters of Chicago is one of the US’s best known “super specialty” roasters, and their $12.95 (US/lb) Black Cat blend was one of the three American blends evaluated by Ken Davids in his espresso review. It finishes third by virtue of having one higher individual score than 4th place Counter Culture.
Reviewers’ Comments: Bittersweet chocolate throughout; slight hint of chili in the finish; chocolate tastes with a bit of fruity notes at the start of taste; a bit roasty; long lasting, beautiful crema; aroma is very pleasant; acidity balance is good; tastes a bit dulled, not dull; chocolaty aftertaste that lingers long; good milk balance, not too strong, just right; very nice in milk, another comfort drink; a bit bitter in the finish.
Ecco Caffe Ecco Organic N. Italian Blend
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 6.0 | 6.0 | 6.5 | 7.0 | 7.0 | 8.0 | 8.0 | 7.5 | 15.0 | 71.0 |
| Barrett Jones | 6.0 | 6.0 | 6.6 | 6.0 | 6.5 | 7.5 | 7.5 | 7.0 | 15.0 | 70.0 |
| Andrea Piccolo | 6.0 | 6.5 | 6.0 | 6.0 | 6.5 | 7.0 | 7.0 | 6.0 | 15.0 | 66.0 |
Ecco Caffe’s Ecco Organic ($13US/lb) blend was second last in the N. American battle, but still fifth place overall. People around the table were surprised it was from the same roaster as the winning coffee.
Reviewers Comments: caramel afternotes; a bit bland; pretty much says “coffee” to me, without much else; a bit sweet, but not very much; lacks character, but still decent; aroma could be better; crema was fantastic; struggled to find flavors, found caramel and a bit of chocolate; much better in milk; very nice in milk; nice coffee, nothing jumping out.
Allegro Organic Bel Canto N. Italian Blend
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 6.0 | 6.0 | 5.0 | 6.0 | 7.0 | 6.0 | 7.0 | 6.0 | 14.0 | 63.0 |
| Barrett Jones | 7.5 | 7.0 | 7.5 | 6.0 | 7.5 | 7.0 | 7.5 | 6.5 | 14.0 | 70.5 |
| Andrea Piccolo | 6.0 | 7.0 | 7.0 | 6.5 | 6.5 | 6.0 | 7.5 | 7.0 | 14.0 | 67.5 |
Allegro Coffee is a fairly large specialty roaster with a wide range of offerings, including organics, award winners, and other ethical and quality driven choices. They sent us their Organic Bel Canto blend to use (price NA) in this testing. Even though it scored last in the N. American offerings, there were many good comments about this coffee. Interestingly enough, it was the only coffee in this group without a roasted on date I could see.
Reviewers Comments: tasting currant flavours in the shot; sweet finish, chocolate-berry flavours; nice balance with the milk; crema dissipated too fast; crema thin but persistent for a short while; aroma not too bad; aftertaste could be better; cuts through milk nicely, much better in milk than as a straight shot; decent body, fruit I can’t identify in the finish; best with milk.
Counter Culture Espresso Aficionado Blend
| Sweet | Body | Aftertaste | Acidity | Aroma | Crema | With Milk | Judges' Overall | Barista Score (20) | Total (100) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Colter Jones | 7.0 | 8.0 | 6.5 | 6.5 | 7.0 | 8.0 | 8.0 | 6.5 | 15.0 | 72.5 |
| Barrett Jones | 6.0 | 8.0 | 6.0 | 7.0 | 7.0 | 8.0 | 8.0 | 6.5 | 15.0 | 71.5 |
| Andrea Piccolo | 6.0 | 7.0 | 7.0 | 7.0 | 8.0 | 7.0 | 8.0 | 7.0 | 15.0 | 72.0 |
Counter Culture Coffee Roasters in Durham, North Carolina are one of those continually improving roasters that are fronted by one of the most recognized (by name) cuppers in all of the US – Peter Guiliano. They sent us their Espresso Aficionado Espresso Blend ($10/12oz USD) for this evaluation. This coffee tied for 3rd overall, but is ranked 4th because it has lower individual scores than Black Cat.
Reviewers Comments: Superb in milk; big body on the shot; Very sweet, one of the sweeter shots today; body and sweetness work well; a bit ashy in the start and middle; crema quality is very high; acidity balance okay; sweet, but not many individual flavours coming through; hard to identify anything but “coffee” flavour; milk balance is fantastic; very pleasing, start to finish; slightly difficult to work with; acidity could be better, leads to a bit of sourness with sweetness.
Battle: N. America Followup Comments
Again, the main topic of discussion was age of beans, and how much of a difference maker it was to our palates. Just to cross check a few things, I informally pulled some shots of the Pellini Top and Pellini Aroma D’Oro after the test, and some of the tasters noted something pretty interesting: they believed that if I had done the Italian blends intermixed with the US / Canada blends, the Pellinis would have scored lower than they did.
Initially, we were all surprised that nothing scored over 80 points – in fact, we were surprised we didn’t have an 85+ point blend in the roundup, but, as I detailed earlier on in this article and in the previous article, it was felt that anything between 80 and 85 points would be a darned near perfect blend, and anything above 85 points in our testing scale would be a “fluke, a god shot”.
We felt that, at 79.5 points on average, Ecco Caffe’s Ecco Reserve was an “ultra premium” espresso blend, and worthy of first place. If given a grading score, it’d get an A-.
How The Evaluation Could Be Better
Our evaluation protocol and testing is by no means perfect; in fact, it may not even be ideal. There’s many ways this could be improved, there were mistakes we made in this evaluation, and there are completely different approaches one could take to evaluating espresso fairly and accurately. In our own testing, we ran into some problems, which I want to document here.
On milk, the Italian blends had a slightly unfair advantage. During the Italian round, I had a milk that I consider one of the best I’ve ever used for cappuccinos, and the testers and barista concurred. It was Valley Pride Organic 3.25% (homogenized) milk, a dairy owned by Avalon Milk in British Columbia, but still independently produced. This milk is my benchmark milk. I’ve tasted various milks from all over the world thanks to my WBC travels, and this is one of the best I’ve ever tried.
The problem is, I ran out of it after the Italian side of the battle, which took place during the Labour Day weekend. I drove to 6 different stores trying to find it, and couldn’t find a store open on the Labour Day Monday that stocked it. So I went with my second choice, and another excellent milk: Olympic Dairy 3.25% (homogenized) milk – another organic milk. Olympic is a really great milk for coffee as well, but if the Valley Pride is a “10”, Olympic is probably a 9 or 8.5.
Secondly, because I was worried that my opting out of the N. American testing would influence the scoring, We made use of our “ghost” scorers more in the N. American side a bit more and I calibrated their scores against Andrea’s scoring, making some minor (1/2 points in most cases) adjustments. Almost all the adjustments brought the scores down slightly for Andrea’s scoresheet.
Third, we didn’t record something crucial. I forgot to get the baristas to record their shot pulling temperatures for each blend, or estimate the amount of coffee used for each double pull. Bad omission on my part, and I will collect this data if we ever do this test again. Suffice to say, each blend was dialed into different temperatures, ones that best suited that particular blend. Ditto on the doses of ground coffee.
There’s other ways to make this evaluation process better, and by no means am I saying this evaluation is perfect. It’s probably too complex, but it was the best we could come up with. I’ve had some discussion on other things to improve our testing – pulling shots on both a temperature tuned machine, and a consumer machine and presenting both results to the tasting judges was one idea. We did actually do something like this. On the N. American tasting day, after the official evaluation was over, I pulled the top three blends from a Rancilio Silvia, using an Anfim “Best” consumer grinder, and got the judges to taste these shots. In future evaluations, we may look at incorporating two machines – a consumer machine and the professional commercial machine.
A completely different way to rank espresso blends
Here’s another idea that cropped up that I thought was very cool, and very doable. Every year, national and world barista championships have their six finalists. An idea bounced around was that the World Barista Championship (or the United States Barista Championship, or the Australian Barista Championship, etc etc) set up a new division / promotional arm. It would be a tasting “board” made up of the six finalists from the previous year’s competition. A website would be built, a testing program would be written and established, and then these organizations would roll out a structured ranking, rating and evaluation system to promote great espresso blends – in fact, be a genuine counter for Coffee Review. How would it work and how would it help promote the WBC? (or USBC, or whatever?)
Make it possible for espresso blend roasters to submit their coffees for a fair and open (read: public) evaluation by the six finalists. I’ll just talk briefly about the WBC level for this idea. Every month, have an open invite for roasters to submit a blend. They would have to mail at least 2 kilos of their blends to the six finalists, and it would be up to the six finalists to pull and evaluate the blends following a basic set of parameters, but otherwise, they’re free to pull the shots any way they want, with the goal to get the best possible shot from those blends. Have the baristas taste the blends “blind” (ie, the don’t know the coffee roaster, or the blend name). Have the six baristas score the blends, and submit the scores to whoever is running this thing for the WBC. The person running it would be a coordinator and would ensure that the six scoring baristas get the coffee in a timely manner, and evaluate the coffee within a time frame.
Then at the end of each month, post the results on a special website built specifically for this task of promoting great blends (and evaluating not-so-great blends). It can be as little as a couple blends per month, or five, ten, or more per month.
Pay for things in several ways. First, pay the six finalists for their work. It’s only fair. And the WBC should pay them, not the roasters. Raise money by charging an administration fee to the submitting roasters (nothing too big), and selling normal advertising on the website displaying these tests. Google Ads, even ads sold directly to roasters is fine, as long as it is clear that a) being an advertiser is not a guarantee of getting evaluated or being evaluated well, b) the reader understands this. Heck, you could even charge a fee, akin to what Fair Trade does, for use of the official “WBC evaluated” branding the roasters put on their bags – 2 cents per sticker can add up!
It’s an idea worth exploring. It would promote the six finalists heavily in the public eye. It would promote the WBC (or the USBC or whatever) as an authority on great espresso. It would promote the WBC period. And it would be a heckuva lot more fair to the blends, giving a global (or national) perspective on tastes, instead of one individuals’ likes and dislikes. And I think it could be a big money maker, without causing concerns about motives and such (ie, because every fee is even and up front, and the WBC doesn’t sell these coffees through a second website, no one would have reason to believe they rank coffees high for monetary gain).
