Report

WINETASTER ON 12/01/03 WITH 8 JUDGES AND 7 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2003 Richard E. Quandt, V. 1.65

FLIGHT 1:
Number of Judges = 8
Number of Wines = 7

Identification of the Wine: The judges' overall ranking:

Wine A is Ch. Lafite Rothschild 1990 tied for 5th place
Wine B is Ch. Cheval Blanc 1990 ........ 1st place
Wine C is Ch. La Conseillante 1989 ........ 2nd place
Wine D is Ch. La Conseillante 1990 ........ 3rd place
Wine E is Ch. Margaux 1989 ........ 4th place
Wine F is Ch. Cheval Blanc 1989 tied for 5th place
Wine G is Ch. Lafite Rothschild 1989 ........ 7th place

The Judges's Rankings

Judge Wine -> A B C D E F G

Ed 7. 2. 4. 3. 1. 6. 5.
Grant 7. 6. 3. 2. 5. 4. 1.
Bob 3. 2. 5. 4. 1. 7. 6.
Frank 5. 1. 4. 6. 7. 3. 2.
John 4. 1. 2. 6. 3. 5. 7.
Burt 5. 4. 1. 2. 3. 7. 6.
Orley 4. 1. 3. 2. 5. 6. 7.
Dick 5. 3. 1. 4. 6. 2. 7.

Table of Votes Against

Wine -> A B C D E F G

Group Ranking -> 5 1 2 3 4 5 7
Votes Against -> 40 20 23 29 31 40 41

( 8 is the best possible, 56 is the worst)

Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):

W = 0.2478

The probability that random chance could be responsible for this correlation
is quite small, 0.0644. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.

Correlation Between the Ranks of
Each Person With the Average Ranking of Others

Name of Person Correlation R Correlation Price

Burt 0.6667 -0.0591
Orley 0.6307 0.0985
Ed 0.6071 0.5123
John 0.5714 0.4926
Dick 0.2523 0.2364
Bob 0.2342 0.0591
Frank -0.0541 0.6305
Grant -0.3929 0.0197

The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.

1. ........ 1st place Wine B is Ch. Cheval Blanc 1990
---------------------------------------------------
2. ........ 2nd place Wine C is Ch. La Conseillante 1989
3. ........ 3rd place Wine D is Ch. La Conseillante 1990
4. ........ 4th place Wine E is Ch. Margaux 1989
5. tied for 5th place Wine A is Ch. Lafite Rothschild 1990
6. tied for 5th place Wine F is Ch. Cheval Blanc 1989
7. ........ 7th place Wine G is Ch. Lafite Rothschild 1989

We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 11.8929. The probability that this could
happen by chance is 0.0644

We now test whether the group ranking of wines is correlated with the
prices of the wines. The rank correlation between them is 0.4076. At the
10% level of significance this would have to exceed the critical value of
0.5710 to be significant.

We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.

Pairwise Rank Correlations

Correlations must exceed in absolute value 0.79 for significance at the 0.05
level and must exceed 0.71 for significance at the 0.1 level

Ed Grant Bob

Ed 1.000 0.036 0.643
Grant 0.036 1.000 -0.607
Bob 0.643 -0.607 1.000
Frank -0.214 0.071 -0.393
John 0.429 -0.643 0.571
Burt 0.571 0.107 0.429
Orley 0.429 -0.321 0.500
Dick -0.071 -0.107 -0.286

Frank John Burt

Ed -0.214 0.429 0.571
Grant 0.071 -0.643 0.107
Bob -0.393 0.571 0.429
Frank 1.000 0.107 -0.464
John 0.107 1.000 0.429
Burt -0.464 0.429 1.000
Orley 0.000 0.607 0.643
Dick 0.214 0.500 0.286

Orley Dick

Ed 0.429 -0.071
Grant -0.321 -0.107
Bob 0.500 -0.286
Frank 0.000 0.214
John 0.607 0.500
Burt 0.643 0.286
Orley 1.000 0.464
Dick 0.464 1.000

Pairwise correlations in descending order

0.643 Ed and Bob Not significant
0.643 Burt and Orley Not significant
0.607 John and Orley Not significant
0.571 Bob and John Not significant
0.571 Ed and Burt Not significant
0.500 John and Dick Not significant
0.500 Bob and Orley Not significant
0.464 Orley and Dick Not significant
0.429 Ed and John Not significant
0.429 Ed and Orley Not significant
0.429 John and Burt Not significant
0.429 Bob and Burt Not significant
0.286 Burt and Dick Not significant
0.214 Frank and Dick Not significant
0.107 Grant and Burt Not significant
0.107 Frank and John Not significant
0.071 Grant and Frank Not significant
0.036 Ed and Grant Not significant
0.000 Frank and Orley Not significant
-0.071 Ed and Dick Not significant
-0.107 Grant and Dick Not significant
-0.214 Ed and Frank Not significant
-0.286 Bob and Dick Not significant
-0.321 Grant and Orley Not significant
-0.393 Bob and Frank Not significant
-0.464 Frank and Burt Not significant
-0.607 Grant and Bob Not significant
-0.643 Grant and John Not significant

COMMENT:

The wines tasted different later in the tasting than earlier. The wines
that seemed quite open at the outset appeared to be more closed in later
on. These wines are all very young and there are not many meaningful
differences. Everyone in the room agreed with this; we all agreed that the
wines are likely to get better in the future.

We tasted these wines two years ago and it is interesting to compare the results
to what we concluded in September of 2001. At that tasting (see Report47.html
one of our regular members was absent and was replaced by a guest. As a result,
we can examine tasters' intertemporal consistency for only seven tasters. Moreover,
the present tasting did not have a Ch. Margaux 1990; hence the rankings of the September 2001
tasting were reranked by omitting the wine missing from the later tasting.

The Spearman Rho rank correlation coefficient between the aggregate tasting results in the
two tastings is 0.6667,which may superficially seem impressive, but is not statistically
significant from zero.

We next computed the Spearman Rho coefficients for each individual taster who took part in
each of the two tastings. Five of the seven coefficients were positive, two were negative.
With one exception, none of them was statistically significantly different from zero. The
one exception was the coefficient 0.9286, with a z-value of 2.2745, obtained by Orley.

It is our intention to retaste these wines every two years as long as we can.

Return to previous page