WINETASTER ON 4/25/94 WITH 6 JUDGES AND 8 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2011 Richard E. Quandt, V. 1.65
FLIGHT 1:
Number of Judges = 6
Number of Wines = 8
Identification of the Wine: The judges' overall ranking:
Wine A is Hermitage La Chappelle 1976 ........ 7th place
Wine B is Hermitage La Chappelle 1978 tied for 2nd place
Wine C is Hermitage La Chappelle 1979 ........ 1st place
Wine D is Hermitage La Chappelle 1980 tied for 2nd place
Wine E is Hermitage La Chappelle 1981 ........ 6th place
Wine F is Hermitage La Chappelle 1982 ........ 5th place
Wine G is Hermitage La Chappelle 1983 tied for 2nd place
Wine H is Hermitage La Chappelle 1984 ........ 8th place
The Judges's Rankings
Judge Wine -> A B C D E F G H
Dick 7. 3. 6. 1. 2. 8. 4. 5.
Bill 6. 3. 1. 2. 5. 4. 8. 7.
John 8. 5. 1. 6. 7. 4. 2. 3.
Frank 4. 5. 2. 8. 6. 1. 3. 7.
Ed 3. 4. 2. 6. 7. 8. 1. 5.
Burt 6. 5. 3. 2. 4. 1. 7. 8.
Table of Votes Against
Wine -> A B C D E F G H
Group Ranking -> 7 2 1 2 6 5 2 8
Votes Against -> 34 25 15 25 31 26 25 35
( 6 is the best possible, 48 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1892
The probability that random chance could be responsible for this correlation
is rather large, 0.3375. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
Bill 0.4097
John 0.3333
Burt 0.2156
Frank 0.0599
Ed -0.1677
Dick -0.3571
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine C is Hermitage La Chappelle 1979
---------------------------------------------------
2. tied for 2nd place Wine B is Hermitage La Chappelle 1978
3. tied for 2nd place Wine G is Hermitage La Chappelle 1983
4. tied for 2nd place Wine D is Hermitage La Chappelle 1980
5. ........ 5th place Wine F is Hermitage La Chappelle 1982
6. ........ 6th place Wine E is Hermitage La Chappelle 1981
7. ........ 7th place Wine A is Hermitage La Chappelle 1976
8. ........ 8th place Wine H is Hermitage La Chappelle 1984
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 7.9444. The probability that this could
happen by chance is 0.3375
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 0.74 for significance at the 0.05
level and must exceed 0.64 for significance at the 0.1 level
Dick Bill John
Dick 1.000 0.143 -0.238
Bill 0.143 1.000 0.048
John -0.238 0.048 1.000
Frank -0.762 0.048 0.429
Ed -0.095 -0.190 0.429
Burt -0.024 0.762 -0.095
Frank Ed Burt
Dick -0.762 -0.095 -0.024
Bill 0.048 -0.190 0.762
John 0.429 0.429 -0.095
Frank 1.000 0.238 0.262
Ed 0.238 1.000 -0.548
Burt 0.262 -0.548 1.000
Pairwise correlations in descending order
0.762 Bill and Burt Significantly positive
0.429 John and Ed Not significant
0.429 John and Frank Not significant
0.262 Frank and Burt Not significant
0.238 Frank and Ed Not significant
0.143 Dick and Bill Not significant
0.048 Bill and Frank Not significant
0.048 Bill and John Not significant
-0.024 Dick and Burt Not significant
-0.095 John and Burt Not significant
-0.095 Dick and Ed Not significant
-0.190 Bill and Ed Not significant
-0.238 Dick and John Not significant
-0.548 Ed and Burt Not significant
-0.762 Dick and Frank Significantly negative
COMMENT:
This tasting, devoted to Hermitage La Chappelle, gives us an opportunity to
compare it with others in which La Chappelle played a prominent role.
Specifically, including the present tasting, there have been altogether
six:
But the degree of overlap among La Chappelle vintages is only modest, so that an
exhaustive comparison of what the group though over time about the various vintages
is not feasible.
There was significant agreement in the group that the 1979 was of
significantly low quality and the 1982 of significantly high quality. The most extensive comparison
can be made between Report -2 and Report -16. The following broad conclusions emerge from these.
1. In Report -2, ´82 is preferred to ´83, which in turn is preferred to ´79.
2. In Report 150, ´82 is preferred to ´79, which is preferred to ´ 83> Moreover,
´79 is preferred ´89 in Report 150, but the reverse is true in Report -2.
3. In Report -16, ´83 is preferred to 82 and ´79 is preferred to ´83.
4. ´79 is deemed worse than any other vintage in Report -2 but better than any other vintage in Report -16.
5. ´90 is preferred to ´83 in Report 40, while the reverse is true in Report -2.
It seems therefore that there is some but but basically not much intertemporal consistency among the tasters, although the same wine
tasted on different occasions may not be exactly the same. Intertemporal consistency is one of the criteria by
which one may judge the reliability of the tasting process, the other being the degree to which differenmt tasters
agree with one another. On neither of these criteria can we say that the group has made reliable judgments.
Return to previous page