Our Valued Sponsor
OpinionsConsumer ReviewsGuides and How TosCoffeeGeek ReviewsResourcesForums
opinions and articles
State of Coffee by Mark Prince
The Controversial WBC
Author: Mark Prince
Posted: July 31, 2006
Article rating: 8.5
feedback: (33) comments | read | write
Click for larger image

It's kind of strange to go from the highest highs of the World Barista Championships competition (the reports of each of the finalists' performance) to controversial stuff, but hey, that's reality. The WBC this year was a great event because at its most basic point it is a celebration of great espresso and a gathering of seriously passionate people, but intense passions can also bring out controversial things. If you haven't had the chance yet, make sure you read our Finalists Report first.

Judging

A lot of discussion has gone on about who the finalist judges were. There was a head judge (Fritz Storm) who was also a paid consultant for several competitors. You had judges who were from the same countries as many of the competitors. And you had judges who, on the scoring sheets, showed seemingly strange scoring choices, lik docking a few points on appearance for one competitor, because they felt his fingernails were a tad "too long".

A lot of issues surrounded Fritz Storm being the head judge, because Storm makes his living from training Baristas and especially Barista competitors. I've given this a lot of thought, and talked to several competitors about this specific situation, and personally, I think Storm is an excellent judge with an amazing palate and an eye for many technical details. He's meticulous judge, but I also believe he's a very fair judge, and I believe he has the ability to put aside any alliances he may have when it comes to judging.

Where I believe Storm fell flat was in his performance as a head judge. According to one competitor in the finals, they were given a "5" by one judge on appearance, and given "2" by another judge on the same category.

This may be just me, but if I were a head judge, at that point I'd call both judges over and say, "okay the two of you, you need to explain these scores, and tell me where the disparity exists". This is what I see the role of head judge doing - making sure that calibration constantly exists in the back, and when there's a huge difference on a subjective (and non-taste) judging point between judges, its their job to reel those differences in. In speaking to one of the finalist judges, this did not happen behind the curtain during the between-competitor calibrations.

There's been some whispered allegations that Storm influenced some judges to steer their scores a certain way. I don't believe this for a second, and in speaking to one of the finalist judges about this, I was told categorically that nothing like that happened behind the curtains. If anything, Storm was seemingly too quiet as a head judge between competitors, and that was probably his worst mistake.

Big drops in points

Click for larger image
Current Board
Members of the current board being introduced on stage.

There were major drops in some (not all) of the competitors scores between the first round and the final round, leading to some speculation about how the finalist judges scored certain competitors. Canada saw a 120 point drop, even though Piccolo felt it was his best shot pours of any competition ever. Iceland also saw a 120 point drop, and Sweden saw an 80 point drop. The UK saw their points go down by around 70. The only difference was the US and Denmark competitors, who saw only minor point drops between rounds.

I won't speculate much on this other than to say in my eyes, being up and close photographing the event, there's no way that Sweden, Canada, the UK and Iceland had such a major difference between rounds. If anything Canada's entrant was better in the second round as compared to the first, and I thought UK's entrant was just brilliant in his second round; though his signature drink visuals were not as good. I also saw the Icelandic competitor's both rounds and felt she was very even between them.

One could argue "different judges, different scoring". But we're supposed to have calibrated judges that give absolute fairness to all competing Baristas, no matter which round or flight they're in. If a Barista does a 800 point performance in one round, and his next round is literally better, better structured, better shots, better flow, no matter who the judges are, the points should be going up, not down.

In reality, I can understand how a 800 point first round performance may have a 775 points second round performance, even if it is better overall. Reality is, the finalist judges would be more careful and perhaps catch more than they do in the first round. Reality is also that all six competitors in the final flight are good - the top six in the world in fact - and the scores should be calibrated to that. So a first round 800 score, in a flight where the average is maybe 650 or lower for the other competitors in that flight, may end up being a 780 or 770 in the finals because the level of competition is that much higher.

The reality for me at least, as a WBC certified judge and someone who's been involved in judging, evaluating, and criticising Barista competitions for five years now is this - I think we need to move to a point where judges are much more accountable for how they judge. Right now, there's very little accountability, except for perhaps how and who gets picked to be a judge in a finals round. This is usually an arbitrary decision made by each competitions' judges trainer or a seasoned head judge.

That's just not good enough. The WBC, and every national body below it, needs to move to a system where we literally do get the best judges picked from real data, experience and feedback from all involved, not just a select few saying "these are the top judges".

Identifying judge issues, improving the pool

We need to move to a situation where there is some kind of peer review evaluation of judges' performance by the competing Baristas. We put those competition Baristas under the microscope, but the performances of their judges year to year have not really been evaluated fairly and equitably by those with most of the stake in having quality, unbiased and fair judges. I firmly believe that the final six Baristas in any competition should have a say and be able to rate how their judges performed. When judges make questionable calls (like fingernail length, as an example, which supposedly happened this year), they should be called on to explain these judgments and how many points were affected by it,

I'm not quite sure how to go about structuring this so it is fair to all parties involved (the Baristas and the judges), but there has to be something in place to make judges more accountable for what they do.

Along with that, I suggest that the WBC create a database system to find out who the fairest judges are, and use this more scientific, emotionally detached process to select who will be judging in the most crucial rounds.

This is actually very easy to do. In every flight round, there's invariably a high and low scoring judge. If the points difference is only a dozen or so points, then the judges are fair and calibrated. But if one judge consistently scores a substantial amount of points less (or more) than any other judge in that flight, there's a problem. That judge should be looked at, and if necessary, have their scoring methodology corrected through retraining, or at worst, should lose their accreditation.

A judge who consistently scores radically outside the median average for their judging pool isn't calibrated, and they present an unfair situation to any Barista who has the (mis) fortune to be in that judge's flight round. Other competitions for other foodie and artistic events around the world do this kind of evaluation of judges, but, as far as I know, the WBC, the USBC, the CNBC, and other competitions do not.

At these competitions, it's a well known fact that there is this bizarro bragging rights thing about being known as a tough judge. To be known as the lowest scorer. Then there's other judges who hear this and decide "well, someone has to be fair to these competitors" so they score up. That's not the way to do it. It's unfair to the competitors, it's stupid to judge tough just so you can be called a tough judge.

How some judges approach the score sheets is even different. Some approach is as thinking every competitor starts at zero and has to move up with their performance. Others start at five or six, and say competitors can only lose points.

Click for larger image
Riddle's leftovers
The leftovers in Riddle's signature drink. Doesn't even look stirred, and the bottom layer untouched.
Click for larger image
Thomsen's Sig Drink
A bit more of Thomsen's drink was had, by the same judge, but still too much left.

In my opinion, both of those judging methodologies are extremely unfair to the competitor. You're at a Barista competition, so you expect the Baristas to have a certain minimum skillset - everything should start out even on the judging sheet - perhaps a 3 (ie, "very good" on the score sheet). Then score up or down depending on what you see. Every single judge should have the mindset of "I will judge fairly and evenly in this competition" and nothing else - no toughest, and no easiest. It's time to set up a comprehensive judges' scoring database to determine who are in fact, via deeds, the fairest judges, and these should be the candidates for judging the most, judging in the finals, and becoming head judge.

Another point - maybe minor, but just another point to emphasise that there a many issues with judging. I've noticed that there are some judges who will barely taste a signature drink, even when the entire beverage is 60mls or less in size (2oz).

A lot of work and effort goes into these beverages. Many times, they are full of complex layers and taste sensations. Having one sip will introduce certain flavours, but then having subsequent sips may change the structure of the taste; many times, this is what the Barista intended. But take the WBC finals this year as an example - Matt Riddle's signature drink was a 2oz "aperitif" style drink; yet at least two of the judges barely had 10mls out of the cup, getting mostly the foam on top of the multi level beverage.

Trust me, I understand the chemistry and caffeine hit of having to drink the drinks from six competitors in a space of 2 hours. I don't expect judges to finish cappuccinos, or even the espressos - that alone is 12 beverages (and add another six for the sig round). But on the signature drink, especially when they are small, I don't believe you can develop a fair and accurate assessment of many of these beverages by only having 5 or 10mls out of the cup. I speak from experience. When I judged, more times than naught I would have at least half or more of a sig drink, no matter the size - and if it was under 90mls, I'd finish it off. If you watch Andrew Barnett in competitions, he does the same thing - in fact, in my flickr account, I have lots of photos of Barnett polishing off signature drinks. One of many reasons why he's considered one of the best judges.

Other controversies

Click for larger image
First Place
The first place trophy, primarily plastic, with a bland, non-descript announcing plaque on front.
Click for larger image
6th Place
Looks identical to first place, save the writing. Competitors have told me they still haven't received the cups displayed.

Moving on from judging, much has also been made about the trophies given to the final six competitors, and the closing party for the Baristas and volunteers. I have my own strong opinions on both subjects, but they've also been tempered by a month or so of thoughts on the subjects.

There was a major gaff on the trophies handed out to the finalist Baristas: the host was responsible for having these made, but they didn't realise that. It was outlined in the contract the WBC had with the host, but it was missed. So on Sunday morning, Samuel Zenger, WBC chair and main organizer of the event arranged for some last minute replacements made up hastily - the plastic stands with mountain goats on them, and a brass plaque in front signifying the finishing position of the Barista.

Here's the thing. At the time, several finalists privately indicated it was unprofessional and and a poor, shoddy indicator of the efforts they just went through, but my take is, these things happen. This was an innocent honest mistake and oversight.

But what I would have done if I were in the host's position, is I would have explained the slip-up fully to the Baristas, and then made plans to get proper, professional, and elegant trophies made up to make up for this innocent mistake. I would have contacted someone like Reg Barber or another artisan to put their creative minds to work in making a trophy suitable and respectful for the efforts these Baristas went through.

These trophies are meant to be a displayable example of excellence in the craft, and what was handed out that day was not in that class. There was no indication of "coffee" or "espresso" in the trophies, and that's something that needs to be corrected. I don't know if the host has taken these steps yet, but they should be doing so - if not, an innocent mistake becomes something less so.

The closing party? Well, it was not good. Terrible location, terrible food, scary singers, terrible audio, and the worst part - they were charging 50 euros (about $$80 Cdn) to attend this thing, and you still had to pay for your drinks.

Last year in Seattle, a true standard was set for the closing party. Sponsored by La Marzocco, it was an amazing event, one I'll remember for a long time. The discussion was all good. No bad vibes at all. The food was excellent and definitely an indicator of how prestigious the WBC had become, and the beverages flowed. It wasn't cheap - it cost La Marzocco thousands and thousands of dollars.

This wrap party serves several purposes. It is meant to be a celebration of the events just finished. It is meant to be a social gathering of the most passionate and avant guard minds in espresso. And it is meant, probably more than anything else, to be a thank you for the substantial work, expense and effort the volunteers, judges, trainers and others involved in the competition. Last year in Seattle, La Marzocco achieved this in spades with their hosted event.

Click for larger image
Tireless efforts
Tireless efforts, like those of Bronwen Serna, making machines factory fresh clean between rounds, deserved better thanks.

This year, the host for the event made a decision to not have sponsorship for the closing party, but instead charge 50 euros for attending, with only the competing Baristas and a guest (their coach in most cases) getting free admittance.

Listen: 50 euros is a lot of money for me, and I have a (albeit limited) expense budget. For a young Barista who, on their boss's frugal dime is attending this event and toiling for four days, running waste buckets, cleaning tables, washing dishes as a volunteer, and going back to their budget hostel in the evening, this was just too much.

A good time was had at the event, but almost in spite of. More people seemed to be outside the tent the event was held at than inside. Things really started rolling when the Russians showed up and had their annual tradition - the praise to Mother Russia table dance, where every Barista who competed gets up on a rickety table, hoists a shot of vodka, and shouts, "Praise to Mother Russia!" before downing the shot. Fun times. But as I said, almost in spite of.

The party, its cost, the shoddy conditions and other elements were a frequent topic of conversation that evening (instead of just talking about the good times from the competition). For 50 euros, people expected good food and free drinks. The drinks were not free, and the bland food was of the fast food, push cart vendor variety.

One really nice thing though - a sponsor did step up anyway, and on their own initiative bought all the remaining tickets to the event. They in turn told the host that these tickets were intended for the volunteers, judges, families and associates of the Baristas and those involved in the competition. That last minute sponsor? The fine folks at La Marzocco.

Again, this is another thing I've been giving a lot of thought to. I could rant and rave even worse about the event and how the host dropped the ball on this one. But I've had time to think, and I think the main problem with the party this year was most likely perception of what's suitable. Cultural differences also probably play a role. Not just Swiss vs other parts of the world, but the culture of the business person vs. the person making barely above minimum wage who has just given dozens and dozens of volunteer hours to the WBC, not to mention probably spending every single dime in their savings account.

Click for larger image Click for larger image Click for larger image
Outside the tent
Early on at the Barista party where control of passage into the tent was fairly tightly controlled.
Outside
Soon after dark, it was evident that more people were outside than in - most only went in to get drinks or food, then came right back out.
Carl and Nick
Carl Sara and Nick Cho having a little tete a tete outside.
Click for larger image Click for larger image Click for larger image
Kenya salutes Russia
The Kenyan Barista comp entrant salutes Mother Russia
Anna Lunell
Anna Lunell does the same - twice!
Sammy Salutes
Sammy Piccolo from Canada salutes Mother Russia!

I'll say this - for all the trip ups and gaffs the host of the WBC made this year, they also worked their asses off. You cannot believe the amount of work and effort that goes into these kinds of things. A senior rep from La Marzocco, for instance, specifically told me that he felt Zenger went above and beyond the call of duty on many things, and that he shouldn't be harshly criticised for any mistakes made. And as I've had a few months to reflect on this, I do agree with it - mostly.

I do think these mistakes are important to bring up and discuss though, so that they can be learned from for future WBC competitions. Som minor mistakes like misprinting the names of competitors or judges on their appreciation awards (mine, for example, had in bold, blazing letters, "Mark Princ" - no "e" at the end of it - also, I wasn't the only one this happened to - I believe Jose Riviera had a similar problem). Mistakes like the trophies (which I hope the WBC is taking steps to correct as I type this). Mistakes like the party and understanding what it's purpose is.

But I also want to applaud Zenger, the SCAE, the WBC board and especially all the amazing volunteers for doing many things right as well, and especially for the effort they put forth in bringing this community of the world's top espresso people together.

Retiring

Prince reading the judging rules

This competition in Bern also marks an occasion for me as well - it will be my last time judging a Barista competition of this format, at least for some time to come. I've been judging now for over five years, starting way back in 2001 at a local informal Barista jam, and in that time I've traveled the world seeing amazing Baristas do their art. I've been instrumental in starting up a national Barista championship program, and best of all, I've met many quality people in this business - almost all of whom are under the age of thirty, which just... I don't have words for their passion and devotion. Tremendous stuff.

But five years has also taken its toll, both financially and mentally I was set to retire after this year's USBC, but felt an obligation to go represent Canada in Bern and be a volunteer there one more time - my last time for Barista competitions, at least in this structure.

I don't know where the current state of Barista competitions is going, but I wish everyone involved great success with it. I hope it grows and evolves, and if I have one bit of advice for anyone who plans on judging in the future, it's this: don't get too wrapped up in it all, understand that there are often different agendas for different people, and keep your eye on the best reason for being a volunteer - to advance the art of the Barista, and the public's perception of coffee as something culinary.

That was always my goal and the reason why I've spent thousands of dollars and traveled for the equivalent of 2.5 months up to this point being a volunteer as a judge. Five years is a long run, and I'm so glad for the opportunity to meet so many young people so passionate about coffee, espresso, and the culture behind it. I'm also so glad I had a chance to meet some of the people involved in judging who are there for the best of reasons.

So many names come to mind, but just off the top of my head, there's Justin Metcalf, a fellow who I consider absolutely brilliant and tireless; Andrew Barnett, often considered the best judge in the entire competition (and to know that Andrew thinks of me that way shows maybe he isn't such a great judge LOL!); David Cooper and Scott Conary, who I think of as great, detailed judges. Dawn Dennis who is a staple at these competitions, but so soft spoken and genuinely nice; Chris White from New Zealand who's been judging for a long time (and the guy who donated an espresso machine for me to give away on the podcast!); and Jose Arreola who always talks watches with me, which is very cool. I could go on - Erik Johnsen, Ave Pilt, Sonja Grant, Arturo Hernandez, Emma Markland, Emily Oak... there are many judges who are just in it for all the right reasons.

But you know what? It's supposed to be about the Barista, and the coffee. They are the true stars of this entire process, from the last place competitors on up to the winners. They've definitely earned my respect, and I hope I've earned theirs in the last five years.

Article rating: 8.5
Author: Mark Prince
Posted: July 31, 2006
feedback: (33) comments | read | write
State of Coffee Column Archives email author
Mark PrinceColumn Description
This regular column will tackle the world of espresso and coffee, including all the theories, controversies, changes and structures that make up this world. A heavy emphasis is placed on the online coffee community, and one thing this column won't do is pull any punches. Every week we'll feature the up's and downs, a quick yet detailed rundown of things that are good and not so good in the coffee world.

Read Author Bio

La Fenice - the Induction Heating Brewer
05.09.2014
Can These Filters Change Espresso?
04.29.2011
Promoting Spro Downs
09.20.2010
State of Espresso 2010
08.16.2010
Espresso 2010
08.14.2010
Nuova Simonelli Aurelia Part 2
04.20.2009
Nuova Simonelli Aurelia Part 1
04.17.2009
N. America vs. Italy - Results
10.08.2007
Evaluating Espresso
09.29.2007
Peets, Baskets, and Blends
09.01.2007
Kickstarting Rants and Raves
08.21.2007
Controversies at the WBC
07.31.2006
What the heck is wrong
02.06.2005
Espresso Glossary
07.20.2004
The Canadian Coffee Scene
06.04.2004
illy cups revisited
12.12.2003
Welcome to CoffeeGeek V2
11.03.2003
The Coffee Crisis
11.27.2002
Survey Results!
05.31.2002
Five Tampers
05.28.2002
Questions and Answers
04.27.2002
Canadian Machines
04.12.2002
Why I like Bodum
03.04.2002
Increase Espresso Sales
02.19.2002
Scarce God Shots
01.27.2002
Espresso Nirvana, Pt.2
01.10.2002
Espresso Nirvana
01.03.2002
Happy Holidays!
12.22.2001
CoffeeGeek Launch Pt.2
12.11.2001
CoffeeGeek Launch
12.01.2001
Short Sighted Businesses
11.27.2001
 
Interactive
Search
Login Password
forgot pw | signup
quickNav
advertisement
sponsorad
Commercial Equipment
Nuova Simonelli, La Marzocco, Rancilio. Nationwide installation. Instant financing options.
www.seattlecoffeegear.com
sponsorad
Coffee Kids
Help folks who help folks in coffee producing nations.
coffeekids.org
sponsorad
Find the Right Machine...
Package deals on the best machines from Izzo, Quick Mill, Rocket, La Marzocco & more.
www.clivecoffee.com
Home | Opinions | Consumer Reviews | Guides & How Tos | CoffeeGeek Reviews | Resources | Forums | Contact Us
CoffeeGeek.com, CoffeeGeek, and Coffee Geek, along with all associated content & images are copyright ©2000-2014 by Mark Prince, all rights reserved, unless otherwise indicated. Content, code, and images may not be reused without permission. Usage of this website signifies agreement with our Terms and Conditions. (0.323255062103)
Privacy Policy | Copyright Info | Terms and Conditions | CoffeeGeek Advertisers | RSS | Find us on Google+