Report

WINETASTER ON 02/05/07 WITH 9 JUDGES AND 9 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2007 Richard E. Quandt, V. 1.65

FLIGHT 1:
Number of Judges = 9
Number of Wines = 9

Identification of the Wine: The judges' overall ranking:

Wine A is Ch. Talbot 1997 ........ 3rd place
Wine B is Mondavi Reserve 1997 ........ 9th place
Wine C is Ch. Lagrange 1997 ........ 7th place
Wine D is Ch. Lynch Bages 1997 ........ 6th place
Wine E is Heitz Martha's Vineyard 1997 ........ 1st place
Wine F is Ch. Troplong Mondot 1997 ........ 8th place
Wine G is Ch. Grand Puy Lacoste 1997 ........ 2nd place
Wine H is Jarvis Reserve 1997 ........ 5th place
Wine I is Ch. Gruaud Larose 1997 ........ 4th place

The Judges's Rankings

Judge Wine -> A B C D E F G H I

Mike 2. 9. 4. 5. 1. 6. 3. 7. 8.
Bob 1. 8. 7. 6. 3. 2. 4. 5. 9.
Ed 7. 3. 6. 4. 1. 9. 8. 5. 2.
John 5. 4. 6. 7. 1. 2. 3. 8. 9.
Burt 4. 7. 6. 5. 8. 9. 1. 2. 3.
Orley 8. 9. 4. 1. 2. 5. 7. 6. 3.
Alan 7. 9. 6. 5. 3. 8. 4. 1. 2.
Frank 6. 5. 9. 7. 1. 8. 2. 3. 4.
Dick 1. 9. 5. 6. 2. 8. 4. 7. 3.

Table of Votes Against

Wine -> A B C D E F G H I

Group Ranking -> 3 9 7 6 1 8 2 5 4
Votes Against -> 41 63 53 46 22 57 36 44 43

( 9 is the best possible, 81 is the worst)

Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):

W = 0.2395

The probability that random chance could be responsible for this correlation
is quite small, 0.0277. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.

Correlation Between the Ranks of
Each Person With the Average Ranking of Others

Name of Person Correlation R

Frank 0.6276
Dick 0.6103
Alan 0.5272
Mike 0.5167
Burt 0.3833
Orley 0.1500
Ed 0.1333
Bob 0.0924
John 0.0586

The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.

1. ........ 1st place Wine E is Heitz Martha's Vineyard 1997
---------------------------------------------------
2. ........ 2nd place Wine G is Ch. Grand Puy Lacoste 1997
3. ........ 3rd place Wine A is Ch. Talbot 1997
4. ........ 4th place Wine I is Ch. Gruaud Larose 1997
5. ........ 5th place Wine H is Jarvis Reserve 1997
6. ........ 6th place Wine D is Ch. Lynch Bages 1997
7. ........ 7th place Wine C is Ch. Lagrange 1997
8. ........ 8th place Wine F is Ch. Troplong Mondot 1997
---------------------------------------------------
9. ........ 9th place Wine B is Mondavi Reserve 1997

We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 17.2444. The probability that this could
happen by chance is 0.0277

We now test whether the group ranking of wines is correlated with the
prices of the wines. The rank correlation between them is 0.1167. At the
10% level of significance this would have to exceed the critical value of
0.4830 to be significant.

We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.

Pairwise Rank Correlations

Correlations must exceed in absolute value 0.70 for significance at the 0.05
level and must exceed 0.60 for significance at the 0.1 level

Mike Bob Ed

Mike 1.000 0.683 -0.167
Bob 0.683 1.000 -0.533
Ed -0.167 -0.533 1.000
John 0.500 0.600 -0.217
Burt -0.033 -0.167 -0.117
Orley 0.200 -0.167 0.417
Alan 0.083 -0.167 0.383
Frank 0.183 0.067 0.433
Dick 0.717 0.317 0.167

John Burt Orley

Mike 0.500 -0.033 0.200
Bob 0.600 -0.167 -0.167
Ed -0.217 -0.117 0.417
John 1.000 -0.567 -0.167
Burt -0.567 1.000 -0.200
Orley -0.167 -0.200 1.000
Alan -0.433 0.583 0.450
Frank 0.183 0.383 -0.050
Dick 0.017 0.283 0.217

Alan Frank Dick

Mike 0.083 0.183 0.717
Bob -0.167 0.067 0.317
Ed 0.383 0.433 0.167
John -0.433 0.183 0.017
Burt 0.583 0.383 0.283
Orley 0.450 -0.050 0.217
Alan 1.000 0.617 0.367
Frank 0.617 1.000 0.333
Dick 0.367 0.333 1.000

Pairwise correlations in descending order

0.717 Mike and Dick Significantly positive
0.683 Mike and Bob Significantly positive
0.617 Alan and Frank Significantly positive
0.600 Bob and John Significantly positive
0.583 Burt and Alan Not significant
0.500 Mike and John Not significant
0.450 Orley and Alan Not significant
0.433 Ed and Frank Not significant
0.417 Ed and Orley Not significant
0.383 Ed and Alan Not significant
0.383 Burt and Frank Not significant
0.367 Alan and Dick Not significant
0.333 Frank and Dick Not significant
0.317 Bob and Dick Not significant
0.283 Burt and Dick Not significant
0.217 Orley and Dick Not significant
0.200 Mike and Orley Not significant
0.183 Mike and Frank Not significant
0.183 John and Frank Not significant
0.167 Ed and Dick Not significant
0.083 Mike and Alan Not significant
0.067 Bob and Frank Not significant
0.017 John and Dick Not significant
-0.033 Mike and Burt Not significant
-0.050 Orley and Frank Not significant
-0.117 Ed and Burt Not significant
-0.167 Bob and Alan Not significant
-0.167 Mike and Ed Not significant
-0.167 Bob and Burt Not significant
-0.167 Bob and Orley Not significant
-0.167 John and Orley Not significant
-0.200 Burt and Orley Not significant
-0.217 Ed and John Not significant
-0.433 John and Alan Not significant
-0.533 Bob and Ed Not significant
-0.567 John and Burt Not significant

COMMENT:

Sadly, one wine in this tasting was corked. The people who ranked it ninth all
thought it was corked. It was the Mondavi Reserve 1997 which cost $139 per bottle.
This is not a judgment on Mondavi Reserve 1997, but a reflection on this particular
bottle. One taster does not think the qualitative determination of one wine
being corked is dispositive. The ranking should be the ultimate judge. With 22
points against for the Heitz wine, this is the equivalent to being ranked second
place by every one on average, an incredibly high agreement for this group. Some
members said that this underlines what a great vintage 1997 was in California.

A new significance test has become available for testing the sum of ranksums for
one subset of wines against the sum of ranksums for the complementary subset of wines.
The test is based on the ratio R defined as (R₁/n₁)/(R₂/n₂), where R₁ and R₂ are the
rank sums in the two subgroups and n₁ and n₂ are the respective number of items in
the groups. Letting the subscript 1 refer to the California wines, the ratio R is not
significantly small. However, if we remove wine B from the ranking (because it was corked)
and rerank the remaining wines, the ratio becomes 0.7548, which is significantly small at the
0.05 level, since the critical value for 9 judges and 8 wineswith 3 in one group and
5 in the other is 0.7934. Hence, on the full sample we cannot say which subgroup was liked better,
but with the indicated adjustment, the California wines were preferred as a group.

Return to previous page