WINETASTER ON 05/07/01 WITH 8 JUDGES AND 7 WINES BASED ON RANKS, IDENT=N Copyright (c) 1995-2001 Richard E. Quandt
FLIGHT 1: Number of Judges = 8 Number of Wines = 7
Identification of the Wine: The judges' overall ranking:
Wine A is Côte Rôtie La Landonne 1985 tied for 5th place Wine B is Côte Rôtie La Mouline 1985 ........ 2nd place Wine C is Côte Rôtie La Landonne 1983 ........ 7th place Wine D is Côte Rôtie La Mouline 1988 ........ 1st place Wine E is Côte Rôtie La Turque 1988 tied for 3rd place Wine F is Côte Rôtie La Landonne 1988 tied for 3rd place Wine G is Côte Rôtie La Mouline 1983 tied for 5th place
The Judges's Rankings
Judge Wine -> A B C D E F G Orley 4. 6. 7. 1. 2. 3. 5. Burt 6. 2. 5. 3. 1. 4. 7. Ed 7. 4. 5. 2. 3. 1. 6. Grant 2. 3. 5. 4. 6. 7. 1. Frank 6. 1. 7. 2. 3. 5. 4. Bob 6. 5. 7. 1. 2. 4. 3. John 3. 1. 5. 4. 6. 2. 7. Dick 4. 2. 7. 1. 6. 3. 5.
Table of Votes Against Wine -> A B C D E F G
Group Ranking -> 5 2 7 1 3 3 5 Votes Against -> 38 24 48 18 29 29 38
( 8 is the best possible, 56 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.3382
The probability that random chance could be responsible for this correlation is quite small, 0.0126. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Frank 0.7857 Dick 0.7500 Ed 0.6071 Burt 0.6071 Orley 0.5714 Bob 0.5714 John 0.1982 Grant -0.2143
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine D is Côte Rôtie La Mouline 1988 --------------------------------------------------- 2. ........ 2nd place Wine B is Côte Rôtie La Mouline 1985 3. tied for 3rd place Wine F is Côte Rôtie La Landonne 1988 4. tied for 3rd place Wine E is Côte Rôtie La Turque 1988 5. tied for 5th place Wine A is Côte Rôtie La Landonne 1985 6. tied for 5th place Wine G is Côte Rôtie La Mouline 1983 --------------------------------------------------- 7. ........ 7th place Wine C is Côte Rôtie La Landonne 1983 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 16.2321. The probability that this could happen by chance is 0.0126 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.79 for significance at the 0.05 level and must exceed 0.71 for significance at the 0.1 level Orley Burt Ed Orley 1.000 0.393 0.571 Burt 0.393 1.000 0.643 Ed 0.571 0.643 1.000 Grant -0.321 -0.571 -0.786 Frank 0.357 0.643 0.393 Bob 0.821 0.393 0.536 John -0.071 0.286 0.286 Dick 0.429 0.250 0.429 Grant Frank Bob Orley -0.321 0.357 0.821 Burt -0.571 0.643 0.393 Ed -0.786 0.393 0.536 Grant 1.000 0.107 -0.107 Frank 0.107 1.000 0.643 Bob -0.107 0.643 1.000 John -0.179 0.214 -0.321 Dick 0.107 0.643 0.393 John Dick Orley -0.071 0.429 Burt 0.286 0.250 Ed 0.286 0.429 Grant -0.179 0.107 Frank 0.214 0.643 Bob -0.321 0.393 John 1.000 0.643 Dick 0.643 1.000 Pairwise correlations in descending order 0.821 Orley and Bob Significantly positive 0.643 Burt and Ed Not significant 0.643 Burt and Frank Not significant 0.643 Frank and Dick Not significant 0.643 Frank and Bob Not significant 0.643 John and Dick Not significant 0.571 Orley and Ed Not significant 0.536 Ed and Bob Not significant 0.429 Ed and Dick Not significant 0.429 Orley and Dick Not significant 0.393 Bob and Dick Not significant 0.393 Ed and Frank Not significant 0.393 Orley and Burt Not significant 0.393 Burt and Bob Not significant 0.357 Orley and Frank Not significant 0.286 Ed and John Not significant 0.286 Burt and John Not significant 0.250 Burt and Dick Not significant 0.214 Frank and John Not significant 0.107 Grant and Frank Not significant 0.107 Grant and Dick Not significant -0.071 Orley and John Not significant -0.107 Grant and Bob Not significant -0.179 Grant and John Not significant -0.321 Bob and John Not significant -0.321 Orley and Grant Not significant -0.571 Burt and Grant Not significant -0.786 Ed and Grant Significantly negative
COMMENT: Even before the ranking were obtained, one taster said that these wines were awesomely majestic. Although it was extremely difficult to rank-order these wines, it was quite surprising that the correlation among the tasters was statistically highly significant. Several tasters thought that the 1983s were fading ever so slightly. Nevertheless, these wines were deemed to be about as good as wines ever get. The 1983s should probably be drunk now. The 1985s have none of the herbal character that some tasters find annoying (and others like), while the 1988s seem very concentrated. Robert Parker says "the 1988s are even richer than the extraordinary 1985s and more concentrated than the magnificent 1983s." We agree with this assessment, and we don't always agree by any means. These wines combine the extraordinary softness and texture of the greatest Burgundies with the intensity and extract of the greatest French, Italian and California cabernets. Note that all the wines received 100 points from Parker, except for the 1983 La Landonne, which received 98+. We finally note that some people expected big differences across the vineyards, while the big differences were actually across the vintages. Addendum on 1/10/2012: While it is true that there are fairly big differences among vintages, there is also a significant difference in the rank sums of La Mouline and La Landonne, with the latter faring much worse.
Return to previous page