WINETASTER ON 12/18/07 WITH 4 JUDGES AND 5 WINES BASED ON RANKS, IDENT=N Copyright (c) 1995-2007 Richard E. Quandt, V. 1.65
FLIGHT 1: A Tasting of Pinot Noirs (R-T denotes Rossignol-Trapet) Number of Judges = 4 Number of Wines = 5
Identification of the Wine: The judges' overall ranking:
Wine A is Rossignol-Trapet Bourgogne 2005 ........ 3rd place Wine B is Gevrey Chambertin (R-T) 1er cru 2005 ........ 2nd place Wine C is Latricieres Chambertin (R-T) Grand cru 2005 ........ 1st place Wine D is Trinity Oaks Pinot Noir 2006 (Calif.) tied for 4th place Wine E is Talus Pinot Noir 2005 (France) tied for 4th place
The Judges's Rankings
Judge Wine -> A B C D E Mark 4. 5. 1. 2. 3. Andy 5. 1. 3. 2. 4. Guillaume 1. 2. 4. 5. 3. Karl 3. 2. 1. 5. 4.
Table of Votes Against Wine -> A B C D E
Group Ranking -> 3 2 1 4 4 Votes Against -> 13 10 9 14 14
( 4 is the best possible, 20 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1375
The probability that random chance could be responsible for this correlation is rather large, 0.6990. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Correlation Price Karl 0.6325 0.9000 Andy -0.3000 0.3000 Mark -0.4000 0.1000 Guillaume -0.5000 0.1000
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine C is Latricieres Chambertin (R-T) Gr, c 2. ........ 2nd place Wine B is Gevrey Chambertin (R-T) 1er cru 20 3. ........ 3rd place Wine A is Rossignol-Trapet Bourgogne 2005 4. tied for 4th place Wine D is Trinity Oaks Pinot Noir 2006 (Cali 5. tied for 4th place Wine E is Talus Pinot Noir 2005 (France) We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 2.2000. The probability that this could happen by chance is 0.6990
We now test whether the group ranking of wines is correlated with the prices of the wines. The rank correlation between them is 0.9747. At the 10% level of significance this would have to exceed the critical value of 0.8000 to be significant.
We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 1.00 for significance at the 0.05 level and must exceed 0.90 for significance at the 0.1 level Mark Andy Guillaume Mark 1.000 -0.100 -0.800 Andy -0.100 1.000 -0.400 Guillaume -0.800 -0.400 1.000 Karl 0.000 0.100 0.300 Karl Mark 0.000 Andy 0.100 Guillaume 0.300 Karl 1.000 Pairwise correlations in descending order 0.300 Guillaume and Karl Not significant 0.100 Andy and Karl Not significant 0.000 Mark and Karl Not significant -0.100 Mark and Andy Not significant -0.400 Andy and Guillaume Not significant -0.800 Mark and Guillaume Not significant
COMMENT: There was very little agreement among the judges, as can be seen from the Rank Table. As a result, no wine was rated significantly good or significantly bad. This is surprising in the light of the fact that the price differences among the wines were enormous: the least expensive wine (Talus) was a $6 wine, wheras the Latricieres Chambertin cost $135. The best a wine could have scored was a 4 and the worst it could have scored was a 20: in fact, no wine scored higher than 9 or lower than 14. But it should be said that the Latricieres Chambertin did have the best score.
Return to previous page