WINETASTER ON 04/01/13 WITH 8 JUDGES AND 8 WINES BASED ON RANKS, IDENT=Y Copyright (c) 1995-2013 Richard E. Quandt, V. 1.65
FLIGHT 1: A Tasting of Guigal Côte Rôtie Number of Judges = 8 Number of Wines = 8
Identification of the Wine: The judges' overall ranking:
Wine A is La Landonne 2004 ........ 6th place Wine B is La Turque 2001 ........ 7th place Wine C is La Landonne 2001 ........ 3rd place Wine D is La Mouline 2001 tied for 4th place Wine E is La Turque 2003 ........ 1st place Wine F is La Mouline 2004 ........ 2nd place Wine G is La Mouline 2003 tied for 4th place Wine H is La Landonne 2003 ........ 8th place
The Judges's Rankings
Judge Wine -> A B C D E F G H Ed 7. 8. 3. 6. 1. 4. 2. 5. Orley 5. 8. 3. 1. 4. 2. 7. 6. Zaki 8. 4. 6. 7. 1. 2. 3. 5. Mike 3. 2. 5. 7. 1. 4. 6. 8. Burt 4. 5. 7. 3. 1. 2. 6. 8. Bob 3. 5. 1. 6. 7. 4. 2. 8. Frank 8. 5. 3. 1. 4. 6. 2. 7. Dick 3. 6. 5. 4. 1. 2. 7. 8.
Table of Votes Against Wine -> A B C D E F G H
Group Ranking -> 6 7 3 4 1 2 4 8 Votes Against -> 41 43 33 35 20 26 35 55
( 8 is the best possible, 64 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.2984
The probability that random chance could be responsible for this correlation is quite small, 0.0194. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Dick 0.6228 Burt 0.6190 Ed 0.5952 Zaki 0.4762 Mike 0.3593 Orley 0.3571 Frank 0.1198 Bob 0.0240
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine E is Turque 2003 --------------------------------------------------- 2. ........ 2nd place Wine F is Mouline 2004 3. ........ 3rd place Wine C is Landonne 2001 4. tied for 4th place Wine D is Mouline 2001 5. tied for 4th place Wine G is Mouline 2003 6. ........ 6th place Wine A is Landonne 2004 7. ........ 7th place Wine B is Turque 2001 --------------------------------------------------- 8. ........ 8th place Wine H is Landonne 2003 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 16.7083. The probability that this could happen by chance is 0.0194 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.74 for significance at the 0.05 level and must exceed 0.64 for significance at the 0.1 level Ed Orley Zaki Ed 1.000 0.190 0.619 Orley 0.190 1.000 -0.143 Zaki 0.619 -0.143 1.000 Mike 0.024 -0.167 0.381 Burt 0.143 0.476 0.381 Bob 0.119 0.000 -0.214 Frank 0.381 0.286 0.095 Dick 0.214 0.595 0.238 Mike Burt Bob Ed 0.024 0.143 0.119 Orley -0.167 0.476 0.000 Zaki 0.381 0.381 -0.214 Mike 1.000 0.595 0.071 Burt 0.595 1.000 -0.214 Bob 0.071 -0.214 1.000 Frank -0.238 0.071 0.190 Dick 0.643 0.905 -0.024 Frank Dick Ed 0.381 0.214 Orley 0.286 0.595 Zaki 0.095 0.238 Mike -0.238 0.643 Burt 0.071 0.905 Bob 0.190 -0.024 Frank 1.000 -0.071 Dick -0.071 1.000 Pairwise correlations in descending order 0.905 Burt and Dick Significantly positive 0.643 Mike and Dick Not significant 0.619 Ed and Zaki Not significant 0.595 Orley and Dick Not significant 0.595 Mike and Burt Not significant 0.476 Orley and Burt Not significant 0.381 Zaki and Mike Not significant 0.381 Zaki and Burt Not significant 0.381 Ed and Frank Not significant 0.286 Orley and Frank Not significant 0.238 Zaki and Dick Not significant 0.214 Ed and Dick Not significant 0.190 Ed and Orley Not significant 0.190 Bob and Frank Not significant 0.143 Ed and Burt Not significant 0.119 Ed and Bob Not significant 0.095 Zaki and Frank Not significant 0.071 Burt and Frank Not significant 0.071 Mike and Bob Not significant 0.024 Ed and Mike Not significant 0.000 Orley and Bob Not significant -0.024 Bob and Dick Not significant -0.071 Frank and Dick Not significant -0.143 Orley and Zaki Not significant -0.167 Orley and Mike Not significant -0.214 Zaki and Bob Not significant -0.214 Burt and Bob Not significant -0.238 Mike and Frank Not significant
COMMENT: These were unbelievably wonderful wines, which we tasted three years ago almost to the day. They have the classic aroma and flavor of Syrah from Northern Rhone. Some felt that they could detect the Viognier grapes and they gave a certain elegance and refinement to the wines (in the 2001 La Turque and La Mouline.) One person felt that there was a hint Percaptans in wine H which he ranked 8. This tasting provides a good opportunity to test our intertemporal consistency. Six of the tasters in today's tasting were the same as three years ago, so for each of those tasters we can compare their rankings today with those three years ago. The easiest way to do this is to compute for each taster the Spearman rho rank correlation coefficient, and here they are: Ed 0.1667 Burt 0.2740 Bob 0.1072 Mike 0.3095 Zaki 0.3453 Dick -0.1070 None of the tasters achieved what one might call a fantastic correlation with the results three years ago, although three scored creditably. One taster managed to achieve a mild negative correlation with the earlier results. We did not compute the correlation between the two aggregate rankings because they are "contaminated" by the presence of two tasters who did not attend the later tasting. It is also noteworthy that the Kendall W coefficient of concordance was 0.058 in the earlier tasting, which had a 0.8610 chance of occurring by chance alone, whereas in the present tasting the Kendall W is 0.2984, which is highly significant. One final calculation we performed was the following. We dropped the two people in the calculations who did not become repeat tasters in the second tasting and the two were not present in the first tasting, and recalculated the results with only the six people who were in both and then calculated the Spearman reho rank correlation between the aggregate rankings. The results are as follows: First Tasting Second Tasting Kendall's W 0.1640 0.3823 Probability 0.4405 0.0246 Rank correlation 0.3343 The upshot of all this is the the degree of concordance among the remaining six tasters is greater when the two "non-repeaters" are omitted, buit the reank correlation between the two aggregate rankings is not significant at any reasonable level of significance.
Return to previous page