WINETASTER ON 02/05/07 WITH 9 JUDGES AND 9 WINES BASED ON RANKS, IDENT=N Copyright (c) 1995-2007 Richard E. Quandt, V. 1.65

FLIGHT 1: Number of Judges = 9 Number of Wines = 9

Identification of the Wine: The judges' overall ranking:

Wine A is Ch. Talbot 1997 ........ 3rd place Wine B is Mondavi Reserve 1997 ........ 9th place Wine C is Ch. Lagrange 1997 ........ 7th place Wine D is Ch. Lynch Bages 1997 ........ 6th place Wine E is Heitz Martha's Vineyard 1997 ........ 1st place Wine F is Ch. Troplong Mondot 1997 ........ 8th place Wine G is Ch. Grand Puy Lacoste 1997 ........ 2nd place Wine H is Jarvis Reserve 1997 ........ 5th place Wine I is Ch. Gruaud Larose 1997 ........ 4th place

The Judges's Rankings

Judge Wine -> A B C D E F G H I Mike 2. 9. 4. 5. 1. 6. 3. 7. 8. Bob 1. 8. 7. 6. 3. 2. 4. 5. 9. Ed 7. 3. 6. 4. 1. 9. 8. 5. 2. John 5. 4. 6. 7. 1. 2. 3. 8. 9. Burt 4. 7. 6. 5. 8. 9. 1. 2. 3. Orley 8. 9. 4. 1. 2. 5. 7. 6. 3. Alan 7. 9. 6. 5. 3. 8. 4. 1. 2. Frank 6. 5. 9. 7. 1. 8. 2. 3. 4. Dick 1. 9. 5. 6. 2. 8. 4. 7. 3.

Table of Votes Against Wine -> A B C D E F G H I

Group Ranking -> 3 9 7 6 1 8 2 5 4 Votes Against -> 41 63 53 46 22 57 36 44 43

( 9 is the best possible, 81 is the worst)

Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):

W = 0.2395

The probability that random chance could be responsible for this correlation is quite small, 0.0277. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.

Correlation Between the Ranks of Each Person With the Average Ranking of Others

Name of Person Correlation R Frank 0.6276 Dick 0.6103 Alan 0.5272 Mike 0.5167 Burt 0.3833 Orley 0.1500 Ed 0.1333 Bob 0.0924 John 0.0586

The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.

1. ........ 1st place Wine E is Heitz Martha's Vineyard 1997 --------------------------------------------------- 2. ........ 2nd place Wine G is Ch. Grand Puy Lacoste 1997 3. ........ 3rd place Wine A is Ch. Talbot 1997 4. ........ 4th place Wine I is Ch. Gruaud Larose 1997 5. ........ 5th place Wine H is Jarvis Reserve 1997 6. ........ 6th place Wine D is Ch. Lynch Bages 1997 7. ........ 7th place Wine C is Ch. Lagrange 1997 8. ........ 8th place Wine F is Ch. Troplong Mondot 1997 --------------------------------------------------- 9. ........ 9th place Wine B is Mondavi Reserve 1997 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 17.2444. The probability that this could happen by chance is 0.0277

We now test whether the group ranking of wines is correlated with the prices of the wines. The rank correlation between them is 0.1167. At the 10% level of significance this would have to exceed the critical value of 0.4830 to be significant.

We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.70 for significance at the 0.05 level and must exceed 0.60 for significance at the 0.1 level Mike Bob Ed Mike 1.000 0.683 -0.167 Bob 0.683 1.000 -0.533 Ed -0.167 -0.533 1.000 John 0.500 0.600 -0.217 Burt -0.033 -0.167 -0.117 Orley 0.200 -0.167 0.417 Alan 0.083 -0.167 0.383 Frank 0.183 0.067 0.433 Dick 0.717 0.317 0.167 John Burt Orley Mike 0.500 -0.033 0.200 Bob 0.600 -0.167 -0.167 Ed -0.217 -0.117 0.417 John 1.000 -0.567 -0.167 Burt -0.567 1.000 -0.200 Orley -0.167 -0.200 1.000 Alan -0.433 0.583 0.450 Frank 0.183 0.383 -0.050 Dick 0.017 0.283 0.217 Alan Frank Dick Mike 0.083 0.183 0.717 Bob -0.167 0.067 0.317 Ed 0.383 0.433 0.167 John -0.433 0.183 0.017 Burt 0.583 0.383 0.283 Orley 0.450 -0.050 0.217 Alan 1.000 0.617 0.367 Frank 0.617 1.000 0.333 Dick 0.367 0.333 1.000 Pairwise correlations in descending order 0.717 Mike and Dick Significantly positive 0.683 Mike and Bob Significantly positive 0.617 Alan and Frank Significantly positive 0.600 Bob and John Significantly positive 0.583 Burt and Alan Not significant 0.500 Mike and John Not significant 0.450 Orley and Alan Not significant 0.433 Ed and Frank Not significant 0.417 Ed and Orley Not significant 0.383 Ed and Alan Not significant 0.383 Burt and Frank Not significant 0.367 Alan and Dick Not significant 0.333 Frank and Dick Not significant 0.317 Bob and Dick Not significant 0.283 Burt and Dick Not significant 0.217 Orley and Dick Not significant 0.200 Mike and Orley Not significant 0.183 Mike and Frank Not significant 0.183 John and Frank Not significant 0.167 Ed and Dick Not significant 0.083 Mike and Alan Not significant 0.067 Bob and Frank Not significant 0.017 John and Dick Not significant -0.033 Mike and Burt Not significant -0.050 Orley and Frank Not significant -0.117 Ed and Burt Not significant -0.167 Bob and Alan Not significant -0.167 Mike and Ed Not significant -0.167 Bob and Burt Not significant -0.167 Bob and Orley Not significant -0.167 John and Orley Not significant -0.200 Burt and Orley Not significant -0.217 Ed and John Not significant -0.433 John and Alan Not significant -0.533 Bob and Ed Not significant -0.567 John and Burt Not significant

COMMENT: Sadly, one wine in this tasting was corked. The people who ranked it ninth all thought it was corked. It was the Mondavi Reserve 1997 which cost $139 per bottle. This is not a judgment on Mondavi Reserve 1997, but a reflection on this particular bottle. One taster does not think the qualitative determination of one wine being corked is dispositive. The ranking should be the ultimate judge. With 22 points against for the Heitz wine, this is the equivalent to being ranked second place by every one on average, an incredibly high agreement for this group. Some members said that this underlines what a great vintage 1997 was in California. A new significance test has become available for testing the sum of ranksums for one subset of wines against the sum of ranksums for the complementary subset of wines. The test is based on the ratio R defined as (R_{1}/n_{1})/(R_{2}/n_{2}), where R_{1}and R_{2}are the rank sums in the two subgroups and n_{1}and n_{2}are the respective number of items in the groups. Letting the subscript 1 refer to the California wines, the ratio R is not significantly small. However, if we remove wine B from the ranking (because it was corked) and rerank the remaining wines, the ratio becomes 0.7548, which is significantly small at the 0.05 level, since the critical value for 9 judges and 8 wineswith 3 in one group and 5 in the other is 0.7934. Hence, on the full sample we cannot say which subgroup was liked better, but with the indicated adjustment, the California wines were preferred as a group.

Return to previous page