WINETASTER ON 12/03/07 WITH 5 JUDGES AND 8 WINES BASED ON RANKS, IDENT=Y
Copyright (c) 1995-2007 Richard E. Quandt, V. 1.65
FLIGHT 1:
Number of Judges = 5
Number of Wines = 8
Identification of the Wine: The judges' overall ranking:
Wine A is Adelsheim Elisabeth Reserve 2005 ........ 7th place
Wine B is Archery Summit Arcus Estate 2001 ........ 8th place
Wine C is Argyle Nuthouse Willamette 2000 ........ 4th place
Wine D is Chehalem 2000 (Ridge Crest Vineyard) ........ 1st place
Wine E is Chehalem 2002 (Dundee Hills Vineyard) ........ 2nd place
Wine F is Chehalem 2004 (Vineyard) ........ 5th place
Wine G is Domaine Drouhin 2003 ........ 3rd place
Wine H is Mt. Difficulty 2005 (New Zealand) ........ 6th place
The Judges's Rankings
Judge Wine -> A B C D E F G H
Mike 5. 7. 6. 3. 2. 1. 8. 4.
Orley 8. 6. 5. 1. 7. 2. 3. 4.
Bob 5. 4. 1. 3. 2. 8. 7. 6.
John 6. 7. 8. 2. 4. 5. 1. 3.
Dick 4. 5. 2. 6. 3. 7. 1. 8.
Table of Votes Against
Wine -> A B C D E F G H
Group Ranking -> 7 8 4 1 2 5 3 6
Votes Against -> 28 29 22 15 18 23 20 25
( 5 is the best possible, 40 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1543
The probability that random chance could be responsible for this correlation
is rather large, 0.6113. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R. We also show the correlation between
each judge's rankings and the prices of the wines.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R Correlation Price
John 0.1719 -0.8503
Orley 0.0482 -0.4072
Mike -0.1334 0.4431
Bob -0.2381 -0.4311
Dick -0.2994 0.6587
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine D is Chehalem 2000
2. ........ 2nd place Wine E is Chehalem 2002
3. ........ 3rd place Wine G is Domaine Drouhin 2003
4. ........ 4th place Wine C is Argyle Nuthouse Willamette 2000
5. ........ 5th place Wine F is Chehalem 2004
6. ........ 6th place Wine H is Mt. Difficulty 2005 (New Zealand)
7. ........ 7th place Wine A is Adelsheim Elisabeth Reserve 2005
8. ........ 8th place Wine B is Archery Summit Arcus Estate 2001
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 5.4000. The probability that this could
happen by chance is 0.6113
We now test whether the group ranking of wines is correlated with the
prices of the wines. The rank correlation between them is -0.2515. At the
10% level of significance this would have to exceed the critical value of
0.5240 to be significant.
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 0.74 for significance at the 0.05
level and must exceed 0.64 for significance at the 0.1 level
Mike Orley Bob
Mike 1.000 0.214 -0.048
Orley 0.214 1.000 -0.357
Bob -0.048 -0.357 1.000
John 0.095 0.548 -0.405
Dick -0.571 -0.333 0.357
John Dick
Mike 0.095 -0.571
Orley 0.548 -0.333
Bob -0.405 0.357
John 1.000 -0.071
Dick -0.071 1.000
Pairwise correlations in descending order
0.548 Orley and John Not significant
0.357 Bob and Dick Not significant
0.214 Mike and Orley Not significant
0.095 Mike and John Not significant
-0.048 Mike and Bob Not significant
-0.071 John and Dick Not significant
-0.333 Orley and Dick Not significant
-0.357 Orley and Bob Not significant
-0.405 Bob and John Not significant
-0.571 Mike and Dick Not significant
COMMENT:
Overall, these were surprisingly enjoyable and uniform wines. The Archery
Summit Arcus wine was the most expensive and received the worst
score, while the Chehalem wines did extremely well, garnering first and second place.
But it would be very difficult for most people to distinguish the Archery Summit
from wines one half or one third of the price. A question was asked: could
anyone really tell the New Zealand wine apart from the others? Answer:
no. The significance of the agreement among the judges was about the
lowest ever among the judges. The Chehalem wines were from designated
vineyards and would be very hard to buy in any vintage except off their
mailing list. Embedded in our tasting, without the tasters knowing it,
were two wines bottled with screwcaps. They were the 2004 Chehalem
3Vineyard Pinot Noir and the Mt. Difficulty 2004 New Zealand wine.
These were the two least expensive wines in the group. For the price,
one taster said he would rather be drinking French pinot noirs. But it is noteworthy
that ten years ago Oregon Pinot Noirs acidic and unpleasant, whereas they are now
very good and their price is largely the same, in the $30 - $40 range.
Return to previous page