WINETASTER ON 12/03/07 WITH 5 JUDGES AND 8 WINES BASED ON RANKS, IDENT=Y Copyright (c) 1995-2007 Richard E. Quandt, V. 1.65
FLIGHT 1: Number of Judges = 5 Number of Wines = 8
Identification of the Wine: The judges' overall ranking:
Wine A is Adelsheim Elisabeth Reserve 2005 ........ 7th place Wine B is Archery Summit Arcus Estate 2001 ........ 8th place Wine C is Argyle Nuthouse Willamette 2000 ........ 4th place Wine D is Chehalem 2000 (Ridge Crest Vineyard) ........ 1st place Wine E is Chehalem 2002 (Dundee Hills Vineyard) ........ 2nd place Wine F is Chehalem 2004 (Vineyard) ........ 5th place Wine G is Domaine Drouhin 2003 ........ 3rd place Wine H is Mt. Difficulty 2005 (New Zealand) ........ 6th place
The Judges's Rankings
Judge Wine -> A B C D E F G H Mike 5. 7. 6. 3. 2. 1. 8. 4. Orley 8. 6. 5. 1. 7. 2. 3. 4. Bob 5. 4. 1. 3. 2. 8. 7. 6. John 6. 7. 8. 2. 4. 5. 1. 3. Dick 4. 5. 2. 6. 3. 7. 1. 8.
Table of Votes Against Wine -> A B C D E F G H
Group Ranking -> 7 8 4 1 2 5 3 6 Votes Against -> 28 29 22 15 18 23 20 25
( 5 is the best possible, 40 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1543
The probability that random chance could be responsible for this correlation is rather large, 0.6113. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R. We also show the correlation between each judge's rankings and the prices of the wines.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Correlation Price John 0.1719 -0.8503 Orley 0.0482 -0.4072 Mike -0.1334 0.4431 Bob -0.2381 -0.4311 Dick -0.2994 0.6587
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine D is Chehalem 2000 2. ........ 2nd place Wine E is Chehalem 2002 3. ........ 3rd place Wine G is Domaine Drouhin 2003 4. ........ 4th place Wine C is Argyle Nuthouse Willamette 2000 5. ........ 5th place Wine F is Chehalem 2004 6. ........ 6th place Wine H is Mt. Difficulty 2005 (New Zealand) 7. ........ 7th place Wine A is Adelsheim Elisabeth Reserve 2005 8. ........ 8th place Wine B is Archery Summit Arcus Estate 2001 We now test whether the ranksums AS A WHOLE provide a significant ordering. The Friedman Chi-square value is 5.4000. The probability that this could happen by chance is 0.6113
We now test whether the group ranking of wines is correlated with the prices of the wines. The rank correlation between them is -0.2515. At the 10% level of significance this would have to exceed the critical value of 0.5240 to be significant.
We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.74 for significance at the 0.05 level and must exceed 0.64 for significance at the 0.1 level Mike Orley Bob Mike 1.000 0.214 -0.048 Orley 0.214 1.000 -0.357 Bob -0.048 -0.357 1.000 John 0.095 0.548 -0.405 Dick -0.571 -0.333 0.357 John Dick Mike 0.095 -0.571 Orley 0.548 -0.333 Bob -0.405 0.357 John 1.000 -0.071 Dick -0.071 1.000 Pairwise correlations in descending order 0.548 Orley and John Not significant 0.357 Bob and Dick Not significant 0.214 Mike and Orley Not significant 0.095 Mike and John Not significant -0.048 Mike and Bob Not significant -0.071 John and Dick Not significant -0.333 Orley and Dick Not significant -0.357 Orley and Bob Not significant -0.405 Bob and John Not significant -0.571 Mike and Dick Not significant
COMMENT: Overall, these were surprisingly enjoyable and uniform wines. The Archery Summit Arcus wine was the most expensive and received the worst score, while the Chehalem wines did extremely well, garnering first and second place. But it would be very difficult for most people to distinguish the Archery Summit from wines one half or one third of the price. A question was asked: could anyone really tell the New Zealand wine apart from the others? Answer: no. The significance of the agreement among the judges was about the lowest ever among the judges. The Chehalem wines were from designated vineyards and would be very hard to buy in any vintage except off their mailing list. Embedded in our tasting, without the tasters knowing it, were two wines bottled with screwcaps. They were the 2004 Chehalem 3Vineyard Pinot Noir and the Mt. Difficulty 2004 New Zealand wine. These were the two least expensive wines in the group. For the price, one taster said he would rather be drinking French pinot noirs. But it is noteworthy that ten years ago Oregon Pinot Noirs acidic and unpleasant, whereas they are now very good and their price is largely the same, in the $30 - $40 range.
Return to previous page