Brewer's Cup Judging, July 9 & 10, Indy State Fair Grounds

[john mills]

I can certainly agree with you Jimmy. Judging is still subjective. I believe it would help judging to bring back a standard for judges to focus their palate against.
I too have many sheets with contradictions even on the same sheet. My belgian 8.9% ABV 93 IBU IPA was recommended for APA style. Don't think of the sheets as a slam, but rather how good the balance, and clean flavors can pass as something much less. If someone else uses the same grain bill winds up with jet fuel.

[sirgiovanni]

I guess it gets a bit scary to me when esters become an infection. They also seemed to not grasp the alcohol content of some of the beers at times when they were describing an unusual warmth. I think they might have been hinting at fussels, which was innapropriate, but it's a bit difficult to mask 10 to 12%.

I thought it was interesting how MECCA was given so much attention and even had the names of their beers called out. I have to admit I found it offensive when my name was called and it was said for me to enjoy my 15 seconds of fame after all that. I'm sure that was just meant to be a joke like the sexist comments on stage but it struck me as strange. After all, OVHA was the third largest contributor to the cash kitty for their event but we set at the back.

I would have like to have seen the award given to John to include the brew pot. The alternative just seems like a result of how much more money you give them through entries which is kind of equivelant to buying more lottery tickets. I think if you could rely on consistent judging to standards across the board, it might make more sense. Just my observations. I'd kind of like to look around at some of the other competitions you guys discuss from time to time and see if it's different or not.

Like I said, I hope it doesn't come across as sour grapes. I really just don't see the point of that much time and money to obtain the quality of feedback given, even in regards to winning entries. Though I might just be a dick that way.

[Jim Dippel]

As anyone that was at the compitition could attest I have been ragging on the judging for the past few years. I think that they have pushed too many under qualified judges through simply because it was so hard to get enough to fill the slate. If you had been to the compitition you would see that a lot of the new judges ape the more qualified judges. If they are with another equally unqualified judge they make a big show of acting as if they know what they are doing. I told someone this weekend that there are probably 30% good judges, and if you are lucky enough to be judged by them you can expect to have a fair shake. Unfortunatly they cannot judge all the beers, so it is just the luck of the draw.

The only recourse we have is to educate ourselves to the poiint that we can get some good judges from this part of the country to take up the slack.

[sirgiovanni]

It's interesting again how they are pushing for entries which was obvious by praising the other group. If their goal is indeed to make more entries/cash for the event, they seem to be working against themselves in regards to your point of judging quality.

I tried to find out if I could be involved in the judging. I have a lot of experience but I'd need help in moving into the BJCP culture. They basically called and asked for my badge number and I ended up on pitcher duty. I don't mind the work and I was able to learn quite a bit about some things with their event from there. But by taking that post, I was assuming they had a quality assurance process in place through the training they provide for the judges they did assign. But I know better now. I'm guessing there is a need for practical applications testing periodically to find out what is happening in each judges head.

In the time being, I think I can get the 20 gallon boilermaker a bit easier by saving some money each week.

[Chris Norrick]

There are several ranks of judges through the BJCP. The judges rank should appear on your score sheet. I take into account the rank when reading the feedback.

At minimum, a BJCP ranked judge must pass a three-hour written and tasting exam, which is very rigorous. The exam score, combined with experience points gained by judging in competitions, helps determine judge’s rank. The ranking system includes ranks for Apprentice, Recognized, Certified, National, Master and Grand Master judges. Higher ranks are earned through higher exam scores, more practical experience, and in the case of Grand Master ranks, service to the BJCP.

APPRENTICE - someone who has taken the BJCP exam, but failed to score at least 60 percent. No experience points are required, but the judge must remain active.
RECOGNIZED - minimum score of 60 percent on the exam. No experience points are necessary to attain this level.
CERTIFIED - minimum score of 70 percent on the exam and at least five experience points, 2.5 of which must be judging points.
NATIONAL - minimum score of 80 percent on the exam and at least 20 experience points, 10 of which must be judging points.
MASTER - minimum score of 90 percent on the exam and at least 40 experience points, 20 of which must be judging points.
GRAND MASTER I - minimum score of 90 percent on the exam and at least 100 experience points, 50 of which must be judging points. A service requirement for the BJCP must also be fulfilled

See the BJCP Study Guide for more info: http://www.bjcp.org/study.php

[sirgiovanni]

It really does sound intense looking it over, even to just get started. Why do you suppose there is a disconnect between the actual guidelines described for each style? Do you think this is more about being in the outer fringe categories that maybe just don't have the same experience levels because of lower entries? For instance, I'm assuming there is pretty good judging going on for an American Pale Ale. But then again, how could any judge have recommended both John's and my IPA for that category based on the IBUs and alcohol contents? And again, I did not expect this beer to win anything in that category but for different reasons. No one seemed to notice the belgium esters as much as they wanted a hardcore bitter beer face. ;)

[sirgiovanni]

It appears to me that the forms were not filled out properly in regards to their ranks.

[BREWsmith]

Timely subjects in today's AHA TechTalk:

From: Joseph Klinck [mailto:]
Sent: Tuesday, July 13, 2010 6:30 PM
Subject: Re: A Question About the BJCP Judging Philosophy?

In theory you could say that for every beer there is a correct score much in the same way that there is a correct answer to a equation in math. Scores are effected by judging experience, palate sensitivity, style knowledge and many other things. Being that in my opinion there is a correct score for each beer then if two people are judging a beer and one scores it a 40 and one scores it a 25 then one of them or both of them are partially wrong. I've had experiences where I had scores that were way off of other judges, in the ensuing discussion one of us usually points something out that the other didn't notice thus making one of us wrong or both of us partially wrong. Lets say we are judging a German Pilsner and there is a high level of diacetyl (this style shouldn't have any) and one judge has a diacetyl blind spot (they can't taste it no matter how strong it is) then that judge will most likely give the beer a higher score than a judge that can detect diacetyl. Thus you can say that the judge with the diacetyl blind spot should lower their score (if it is much higher than the other judge).

[back to top]
--------------------------------------------------------------------------------------------
From: David Houseman [mailto:]
Sent: Tuesday, July 13, 2010 5:34 PM
Subject: Re: A Question About the BJCP Judging Philosophy?

Mario,
There is still quite a bit of subjectivity in how one scores a beer. There are also judges with differing experience and skill levels. So when two of us, perhaps one much more experienced than another, arrive at a score for a beer, it might vary quite a bit. Perhaps one didn't recognize a fault that the other did. Or perhaps one just felt the beer was worse, or better, than than the other. To provide widely differing scores to the entrant is not nearly as informative as when the judges agree and their scores are close to each other. Any judge should be willing to work with their partner and raise or lower their scores somewhat so that they are close to one another and are sending a similar message to the entrant. Things just aren't so black and white that there isn't room for adjusting scores. Frankly as an organizer and judge I try to have judges be within 3 points whenever possible; I think 7 is still to wide a variance. So think of scoring, not just for winning something but in the feedback that score conveys. Is this a Poor, Fair, Good, Very Good or Excellent beer. Scoring is sending that message. Agreeing with your partner on where the beer falls is another way of telling the entrant what you thought of the beer. Widely divergent scores are confusing. Converging scores re-enforce the opinion.

David Houseman
BJCP Competition Director
Grand Master IV

[sirgiovanni]

Thanks for posting jeff.

What do people think about drinking while judging? As far as dealing with subjectively, the judging sheets are not as detailed obviously as the manual. Do people read through on what they are judging or are they purely working through memory?

[Kenny Lucas]

First of all a BIG congrats to John and Jimmy, you make us proud!
Next I have a question that maybe someone who was involved with judging or stewarding can answer. I received my scoresheets in the mail today and going over the cover sheet on my American Amber it reads a score of 40, seems somewhat inflated in relationship to what I am used to seeing. It also states that this beer advanced to a mini best of show round, yet no ribbon. I was thinking that a beer had to receive a 1st to be eliglble for BOS. Am I overlooking something here or just wishful thinking?

[sirgiovanni]

Obviously I have more questions than answers but I think I can answer that one.

In the first round, they go through an initial scoring. Then they have to pick the top three for awards and if they have more than 3 over the point value for the award, they go through a second round tasting to pick the top 3.

It sounds like you made a hell of an amber. ;)

[Chris Norrick]

That check box if for when the beer advances to a "Mini-BOS". There will be a Mini-BOS anytime there are more than one panel of judges per category. There are just too many beers for some categories for one panel of judges. So what they do is each panel (say there where 30 IPA's, 3 panels will judge 10 beers each) will send their top 0-3 scoring beers on the Mini-BOS. The lead judge from each panel will sit down and pour all the advancing beers in a grid. If each panel sent 2 to the Mini-BOS there would be 6 beers competing for 1st, 2nd, and 3rd. That way, a 1st, 2nd, and 3rd is determined from all 30 beers in that category. They don't fill out a new score sheet, they just pick which they feel best represents the Category (or Sub-Category).

There is a strange rule that the "Final Assigned Score" on the cover sheet for 1st place must be higher than 2nd which must be higher than 3rd. So sometimes they have to adjust the score on your Cover Sheet. This is why that score may not be the average of scores on the judging sheets. So it can be true that the highest scoring beer may not place 1st.

Don't forget that they are also judging subcategories at the same panel of judges. You may have had the best 10:B Amber Ale, but there could have been 3 10:A Pale Ales they liked better.

Kenny, you are correct, only the 1st place beers are judged in the main BOS round.

_____________

Here is the question that was asked on TechTalk that sparked the answers provided by Jeff Smith above. This has also been a concern of mine.

Subject: A Question About the BJCP Judging Philosophy?

To whom it may concern,
My name is Mario Vargas and I'm preparing to take the BJCP judge certification exam and in doing so I'm reading the BJCP Exam Study Guide, on page 14 it states "when the other judges have finished scoring the beer, discuss the technical and stylistic merits of the beer, and arrive at a consensus score. Be prepared to adjust your score to make them fall within 7 points of the other judges at your table".?

I don't understand why a judge would change the original score to make it fit within the other judges’ scores, within 7 points?

I started this thread on the AHA website a week ago and I am still uncertain how this BJCP judging philosophy was adopted. I would appreciate any thoughts you would like to add so that I could better understand the, benefits of modifying a score card so that they look aligned. We at the forum have been in discussion on this topic and would love to have some insight from the people who crafted and or understand this Judging system.

http://www.homebrewersassociation.org/f ... 0#msg31720

Thank you,
Mario D. Vargas

[Kenny Lucas]

Thanks for all of your answers...I think I got it now! I do recall how the scores do not always reflect the final outcome and I did forget that other beers are also in the same category.
So, yes, on my part it was just wishful thinking!

[Dwayne_Delaney]

One of the main problems that I have seen with the judging is the lack of respect from most judges for balance in a beer. After sitting with many different judges over the past 3 years, I have seen this over and over. Beers seem to be scored by how many of the primary characteristics of a style are pushed forward in-your-face with very few points given for the overall synergy of a beer.

[Chris Norrick]

I forgot to mention that I stewarded Category 10. THERE WERE 70 ENTIRES!!! It still sucks that a 40 point beer didn't get a medal.