WINETASTER ON 01/08/01 WITH 8 JUDGES AND 5 WINES BASED ON RANKS, IDENT=N Copyright (c) 1995-2000 Richard E. Quandt
A Tasting of Hermitage La Chappelle
FLIGHT 1: Number of Judges = 8 Number of Wines = 5
Identification of the Wine: The judges' overall ranking:
Wine A is La Chapelle 1990 ........ 2nd place Wine B is Penfolds Grange 1982 ........ 1st place Wine C is La Chapelle 1988 ........ 5th place Wine D is La Chapelle 1985 tied for 3rd place Wine E is La Chapelle 1983 tied for 3rd place
The Judges's Rankings
Judge Wine -> A B C D E John 2. 1. 5. 4. 3. Michael 1. 3. 4. 2. 5. Bob 5. 1. 2. 4. 3. Grant 1. 5. 4. 2. 3. Ed 1. 5. 4. 3. 2. Burt 4. 2. 3. 5. 1. Frank 2. 1. 5. 3. 4. Dick 4. 1. 2. 3. 5.
Table of Votes Against Wine -> A B C D E
Group Ranking -> 2 1 5 3 3 Votes Against -> 20 19 29 26 26
( 8 is the best possible, 40 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1156
The probability that random chance could be responsible for this correlation is rather large, 0.4481. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R John 0.8721 Frank 0.8721 Michael 0.3000 Grant -0.1000 Ed -0.1000 Dick -0.2000 Burt -0.3000 Bob -0.4000
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine B is Penfolds Grange 1982 2. ........ 2nd place Wine A is La Chapelle 1990 3. tied for 3rd place Wine E is La Chapelle 1983 4. tied for 3rd place Wine D is La Chapelle 1985 5. ........ 5th place Wine C is La Chapelle 1988 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 3.7000. The probability that this could happen by chance is 0.4481 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 1.00 for significance at the 0.05 level and must exceed 0.90 for significance at the 0.1 level John Michael Bob John 1.000 0.300 0.100 Michael 0.300 1.000 -0.600 Bob 0.100 -0.600 1.000 Grant -0.100 0.600 -1.000 Ed 0.000 0.300 -0.900 Burt 0.300 -0.800 0.600 Frank 0.900 0.600 0.000 Dick 0.100 0.100 0.700 Grant Ed Burt John -0.100 0.000 0.300 Michael 0.600 0.300 -0.800 Bob -1.000 -0.900 0.600 Grant 1.000 0.900 -0.600 Ed 0.900 1.000 -0.200 Burt -0.600 -0.200 1.000 Frank 0.000 -0.100 -0.100 Dick -0.700 -0.900 -0.100 Frank Dick John 0.900 0.100 Michael 0.600 0.100 Bob 0.000 0.700 Grant 0.000 -0.700 Ed -0.100 -0.900 Burt -0.100 -0.100 Frank 1.000 0.300 Dick 0.300 1.000 Pairwise correlations in descending order 0.900 Grant and Ed Significantly positive 0.900 John and Frank Significantly positive 0.700 Bob and Dick Not significant 0.600 Bob and Burt Not significant 0.600 Michael and Grant Not significant 0.600 Michael and Frank Not significant 0.300 John and Michael Not significant 0.300 Frank and Dick Not significant 0.300 John and Burt Not significant 0.300 Michael and Ed Not significant 0.100 John and Dick Not significant 0.100 Michael and Dick Not significant 0.100 John and Bob Not significant 0.000 John and Ed Not significant 0.000 Grant and Frank Not significant 0.000 Bob and Frank Not significant -0.100 John and Grant Not significant -0.100 Burt and Dick Not significant -0.100 Ed and Frank Not significant -0.100 Burt and Frank Not significant -0.200 Ed and Burt Not significant -0.600 Grant and Burt Not significant -0.600 Michael and Bob Not significant -0.700 Grant and Dick Not significant -0.800 Michael and Burt Not significant -0.900 Bob and Ed Significantly negative -0.900 Ed and Dick Significantly negative -1.000 Bob and Grant Significantly negative
COMMENT: The reason for an substantial lack of agreement was that the wines were extraordinary. An additional problem stems from the introduction of a Penfolds Grange which, irrespective of quality, possesses significant stylistic differences in comparison with the La Chapelle wines. Some of the panel responded to that difference by ranking the Penfolds highly, while others appeared to have downgraded the Penfolds based on its lack of similarity to the other wines. The degree of lightness of the 1988 was a surprise, given its reputation for developing slowly. Compare with another tasting of Hermitage La Chappelle May 2011.
Return to previous page