WINETASTER ON 12/06/99 WITH 7 JUDGES AND 7 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-98 Richard E. Quandt
FLIGHT 1:
Number of Judges = 7
Number of Wines = 7
Identification of the Wine: The judges' overall ranking:
Wine A is Ch. Palmer 1990 ........ 2nd place
Wine B is Ch. Pichon Longueville, Lalande 1990 ........ 3rd place
Wine C is Ch. Leoville las Cases 1990 ........ 4th place
Wine D is Ch. La Conseillante 1990 ........ 1st place
Wine E is Ch. Ausone 1990 ........ 7th place
Wine F is Ch. Margaux 1990 tied for 5th place
Wine G is Ch. Cos d'Estournel 1990 tied for 5th place
The Judges's Rankings
Judge Wine -> A B C D E F G
John 2. 3. 4. 1. 7. 6. 5.
Burt 1. 2. 6. 3. 5. 4. 7.
Frank 3. 1. 4. 2. 6. 7. 5.
Bob 2. 5. 6. 1. 7. 3. 4.
Orley 5. 4. 3. 1. 2. 6. 7.
Grant 5. 4. 1. 3. 6. 7. 2.
Dick 2. 3. 5. 1. 6. 4. 7.
Table of Votes Against
Wine -> A B C D E F G
Group Ranking -> 2 3 4 1 7 5 5
Votes Against -> 20 22 29 12 39 37 37
( 7 is the best possible, 49 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.4665
The probability that random chance could be responsible for this correlation
is quite small, 0.0033. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
John 0.9370
Dick 0.7928
Frank 0.7857
Burt 0.6071
Bob 0.3214
Orley 0.2857
Grant 0.0714
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine D is Ch. La Conseillante 1990
---------------------------------------------------
2. ........ 2nd place Wine A is Ch. Palmer 1990
3. ........ 3rd place Wine B is Ch. Pichon Longueville, Lalande 19
4. ........ 4th place Wine C is Ch. Leoville las Cases 1990
---------------------------------------------------
5. tied for 5th place Wine F is Ch. Margaux 1990
6. tied for 5th place Wine G is Ch. Cos d'Estournel 1990
7. ........ 7th place Wine E is Ch. Ausone 1990
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 19.5918. The probability that this could
happen by chance is 0.0033
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 0.79 for significance at the 0.05
level and must exceed 0.71 for significance at the 0.1 level
John Burt Frank
John 1.000 0.607 0.857
Burt 0.607 1.000 0.571
Frank 0.857 0.571 1.000
Bob 0.679 0.500 0.286
Orley 0.286 0.179 0.357
Grant 0.393 -0.429 0.429
Dick 0.821 0.857 0.643
Bob Orley Grant
John 0.679 0.286 0.393
Burt 0.500 0.179 -0.429
Frank 0.286 0.357 0.429
Bob 1.000 -0.107 -0.071
Orley -0.107 1.000 0.107
Grant -0.071 0.107 1.000
Dick 0.714 0.393 -0.143
Dick
John 0.821
Burt 0.857
Frank 0.643
Bob 0.714
Orley 0.393
Grant -0.143
Dick 1.000
Pairwise correlations in descending order
0.857 Burt and Dick Significantly positive
0.857 John and Frank Significantly positive
0.821 John and Dick Significantly positive
0.714 Bob and Dick Significantly positive
0.679 John and Bob Not significant
0.643 Frank and Dick Not significant
0.607 John and Burt Not significant
0.571 Burt and Frank Not significant
0.500 Burt and Bob Not significant
0.429 Frank and Grant Not significant
0.393 Orley and Dick Not significant
0.393 John and Grant Not significant
0.357 Frank and Orley Not significant
0.286 Frank and Bob Not significant
0.286 John and Orley Not significant
0.179 Burt and Orley Not significant
0.107 Orley and Grant Not significant
-0.071 Bob and Grant Not significant
-0.107 Bob and Orley Not significant
-0.143 Grant and Dick Not significant
-0.429 Burt and Grant Not significant
COMMENT:
Fact 1: Prices of the wines, which we did not know during the tasting,
Palmer $70, Pichon Lalande $90, Leoville Las Cases $145, Conseillante
$200, Ausone $145, Margaux $375, Cos d'Estournel $100. Fact 2: in a
previous tasting Margaux was ranked first, and that is why it was included
in this tasting.Fact 3: prior to seeing the rankings, it was widely agreed
that these were all outstanding wines and it was not expected that there
would be significant differences in the rankings. To the surprise of
everyone, there was overwhelming agreement on the first-place wine-which
was not the wine that won the previous tasting. Fact 4: Despite the fact
that these were very young wines, they were all accessible and delicious.
We note that price is not necessarily an indicator of quality at this time.
An explanation for why the wines that did well did so is that in our
tasting group the wines are judged according to their quality for drink-
ing today. The group agreed on the qualities of the most favored wines,
but expressed this in varying ways, describing them alternately as soft,
spicy, and balanced, without too much tannin.
Return to previous page