WINETASTER ON 05/03/04 WITH 7 JUDGES AND 7 WINES BASED ON RANKS, IDENT=N Copyright (c) 1995-2004 Richard E. Quandt, V. 1.65
FLIGHT 1: Number of Judges = 7 Number of Wines = 7
Identification of the Wine: The judges' overall ranking:
Wine A is Cotarella's Montiano 1997 ........ 7th place Wine B is Cotarella's Montiano 2000 ........ 1st place Wine C is Cotarella's Montiano 1999 ........ 6th place Wine D is Cotarella's Montiano 1995 ........ 2nd place Wine E is Cotarella's Montiano 1998 ........ 4th place Wine F is Cotarella's Montiano 1996 ........ 3rd place Wine G is Cotarella's Montiano 1994 ........ 5th place
The Judges's Rankings
Judge Wine -> A B C D E F G Bob 7. 2. 6. 3. 5. 1. 4. Ed 7. 3. 4. 2. 6. 1. 5. Frank 7. 1. 6. 2. 4. 5. 3. John 7. 2. 4. 5. 1. 6. 3. Burt 6. 1. 4. 2. 5. 3. 7. Mike 7. 1. 6. 4. 3. 2. 5. Dick 7. 2. 6. 1. 3. 5. 4.
Table of Votes Against Wine -> A B C D E F G
Group Ranking -> 7 1 6 2 4 3 5 Votes Against -> 48 12 36 19 27 23 31
( 7 is the best possible, 49 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.6093
The probability that random chance could be responsible for this correlation is quite small, 0.0003. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Mike 0.8929 Frank 0.8571 Burt 0.8214 Bob 0.7928 Dick 0.7928 Ed 0.5357 John 0.3571
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine B is Cotarella's Montiano 2000 --------------------------------------------------- 2. ........ 2nd place Wine D is Cotarella's Montiano 1995 3. ........ 3rd place Wine F is Cotarella's Montiano 1996 4. ........ 4th place Wine E is Cotarella's Montiano 1998 5. ........ 5th place Wine G is Cotarella's Montiano 1994 6. ........ 6th place Wine C is Cotarella's Montiano 1999 --------------------------------------------------- 7. ........ 7th place Wine A is Cotarella's Montiano 1997 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 25.5918. The probability that this could happen by chance is 0.0003 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.79 for significance at the 0.05 level and must exceed 0.71 for significance at the 0.1 level Bob Ed Frank Bob 1.000 0.857 0.643 Ed 0.857 1.000 0.429 Frank 0.643 0.429 1.000 John 0.107 -0.143 0.571 Burt 0.643 0.750 0.536 Mike 0.857 0.607 0.679 Dick 0.571 0.429 0.929 John Burt Mike Bob 0.107 0.643 0.857 Ed -0.143 0.750 0.607 Frank 0.571 0.536 0.679 John 1.000 0.071 0.464 Burt 0.071 1.000 0.679 Mike 0.464 0.679 1.000 Dick 0.536 0.571 0.643 Dick Bob 0.571 Ed 0.429 Frank 0.929 John 0.536 Burt 0.571 Mike 0.643 Dick 1.000 Pairwise correlations in descending order 0.929 Frank and Dick Significantly positive 0.857 Bob and Mike Significantly positive 0.857 Bob and Ed Significantly positive 0.750 Ed and Burt Significantly positive 0.679 Frank and Mike Not significant 0.679 Burt and Mike Not significant 0.643 Bob and Burt Not significant 0.643 Bob and Frank Not significant 0.643 Mike and Dick Not significant 0.607 Ed and Mike Not significant 0.571 Frank and John Not significant 0.571 Burt and Dick Not significant 0.571 Bob and Dick Not significant 0.536 John and Dick Not significant 0.536 Frank and Burt Not significant 0.464 John and Mike Not significant 0.429 Ed and Frank Not significant 0.429 Ed and Dick Not significant 0.107 Bob and John Not significant 0.071 John and Burt Not significant -0.143 Ed and John Not significant
COMMENT: When we began this tasting, our first reaction was that there was not much difference among these wines. But as time went on, we decided that there were noticeable differences and we ended up with a very sharp discrimina- tion among them. This is interesting because the Parker ratings of these wines are highly clustered (93-96) but we had to differentiate among them which the Parker ratings do not have to do. The tasting may have been influenced by what was possibly a bad bottle, namely the 1997, which had the highest Parker rating but was very murky and exceptionally acidic. It should also be noted that all these wines are ready to drink now. It is also the case that we have had no wine tasting, as far as we remember, in which the degree of agreement was this high. This is particularly significant in that as a vertical tasting style differences are insignificant. Because there was such agreement that wine A, the 1997, might have been a bad bottle, on which everybody agreed, we decided to redo the calculations as if wine A had been absent. The agreement among the jdges under this scenario continued to be high. The Kendall W coefficient of concordance declines to 0.4169, but this figure is still significant at the 0.0123 level, which indicates an extremely strong level of agreement. Wine B, the 2000, is rated as significantly good, and now, since wine A is absent, wine C, the 1999, is rated significantly bad. On the whole, the agreement of the judges is strong and cannot be attributed to the presence of the potentially bad bottle of the 1997.
Return to previous page