WINETASTER ON 06/08/12 WITH 9 JUDGES AND 10 WINES BASED ON GRADES, IDENT=N Copyright (c) 1995-2012 Richard E. Quandt, V. 1.65
FLIGHT 1: Number of Judges = 9 Number of Wines = 10
Identification of the Wine: The judges' overall ranking:
Wine A is Heritage Chardonnay 2010 ....... 3rd place Wine B is Unionville Pheasant Hill Single Vineyard 2010 ........ 2nd place Wine C is Puligny Montrachet Domaine Leflaive 2009 ........ 5th place Wine D is Clos des Mouches Drouhin 2009 ........ 1st place Wine E is Silver Decoy "Black Feather" 201 ........ 4th place Wine F is Bellview Chardonnay 2010 tied for 6th place Wine G is Ventimiglia Chardonnay 2010 ........ 9th place Wine H is Meursault-Charmes Jean Latour-Labille2008 ........ 10th place Wine I is Amalthea Chardonnay 2008 ........ 8th place Wine J is Bâtard Montrachet Marc-Antonin Blain 2009 tied for 6th place
The Judges' Grades
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 10.0 13.0 14.0 15.0 8.0 13.0 15.0 11.0 9.0 12.0 Tyler Colman 16.0 14.0 14.0 16.0 12.0 11.0 11.0 14.0 11.0 14.0 John Foy 16.0 17.0 16.0 15.0 14.5 14.5 16.0 17.0 15.0 17.5 Olivier Gergaud 14.0 19.0 12.0 10.0 19.0 18.0 17.0 16.0 18.0 14.0 Robert Hodgson 17.0 11.0 13.0 14.0 14.0 10.0 9.0 9.0 10.0 10.0 Linda Murphy 15.5 15.0 17.0 18.0 16.0 17.0 15.0 14.0 16.0 17.0 Daniele Meulders 10.0 15.0 12.0 12.0 15.0 14.0 15.0 12.0 15.0 12.0 Jamal Rayyis 16.0 15.0 14.5 17.5 16.5 14.0 12.0 15.0 13.0 12.0 Francis Schott 17.0 16.0 12.0 18.0 15.0 16.0 15.0 14.0 17.0 15.0
The Judges' Rankings
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 8.0 4.5 3.0 1.5 10.0 4.5 1.5 7.0 9.0 6.0 Tyler Colman 1.5 4.5 4.5 1.5 7.0 9.0 9.0 4.5 9.0 4.5 John Foy 5.0 2.5 5.0 7.5 9.5 9.5 5.0 2.5 7.5 1.0 Olivier Gergaud 7.5 1.5 9.0 10.0 1.5 3.5 5.0 6.0 3.5 7.5 Robert Hodgson 1.0 5.0 4.0 2.5 2.5 7.0 9.5 9.5 7.0 7.0 Linda Murphy 7.0 8.5 3.0 1.0 5.5 3.0 8.5 10.0 5.5 3.0 Daniele Meulders 10.0 2.5 7.5 7.5 2.5 5.0 2.5 7.5 2.5 7.5 Jamal Rayyis 3.0 4.5 6.0 1.0 2.0 7.0 9.5 4.5 8.0 9.5 Francis Schott 2.5 4.5 10.0 1.0 7.0 4.5 7.0 9.0 2.5 7.0
Group Ranking -> 3 2 5 1 4 6 9 10 8 6 Votes Against -> 45.5 38.0 52.0 33.5 47.5 53.0 57.5 60.5 54.5 53.0
( 9 is the best possible, 90 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1017
The probability that random chance could be responsible for this correlation is rather large, 0.5476. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Robert Hodgson 0.7299 Jamal Rayyis 0.5321 Tyler Colman 0.2750 Francis Schott 0.2037 Linda Murphy 0.0502 Jean-M Cardebat -0.1774 Olivier Gergaud -0.3303 Daniele Meulders -0.3729 John Foy -0.4318
We now compute for each wine the mean (average) grade it received from the judges and the standard deviation of those grades. (The mean of, say, 3 grades is the sum of the grades divided by 3. The standard deviation is a measure of the "dispersion" of the grades around the mean. The numbers 5, 6, and 7 and the numbers 3, 6, and 9 both have the same mean of 6, but the second set has a higher dispersion, i.e., strandard deviation.
Summary Grade Statistics for Wines A B C D E F G H I J Mean 14.61 15.00 13.83 15.06 14.44 14.17 13.89 13.56 13.78 13.72 StdDev 2.60 2.16 1.70 2.59 2.90 2.47 2.47 2.36 3.01 2.35
We now compute the mean and the standard deviation for each judge, from which you can see which judge is harsh and which is lenient, which thinks that the wines are very different and which thinks they are pretty much the same.
Summary Grade Statistics for Judges Name Mean Std.Dev. Jean-M Cardebat 12.00 2.32 Tyler Colman 13.30 1.85 John Foy 15.85 1.03 Olivier Gergaud 15.70 2.93 Robert Hodgson 11.70 2.53 Linda Murphy 16.05 1.15 Daniele Meulders 13.20 1.72 Jamal Rayyis 14.55 1.75 Francis Schott 15.50 1.63
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine D is Clos des Mouches 2009 2. ........ 2nd place Wine B is Unionville Single Vineyard 2010 3. ........ 3rd place Wine A is Heritage Chard 2010 4. ........ 4th place Wine E is Silver Decoy "Black Feather" 201 5. ........ 5th place Wine C is Puligny Montrachet 2009 6. tied for 6th place Wine F is Bellview Chard 2010 7. tied for 6th place Wine J is Batard Montrachet 2009 8. ........ 8th place Wine I is Amalthea Chard 2008 9. ........ 9th place Wine G is Ventimiglia Chard 2010 10. ........ 10th place Wine H is Meursault-Charmes 2008 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.65 for significance at the 0.05 level and must exceed 0.56 for significance at the 0.1 level Jean-M Cardebat Tyler Colman John Foy Jean-M Cardebat 1.000 0.115 0.140 Tyler Colman 0.115 1.000 0.393 John Foy 0.140 0.393 1.000 Olivier Gergaud -0.455 -0.638 -0.250 Robert Hodgson -0.174 0.602 -0.319 Linda Murphy 0.261 0.146 -0.394 Daniele Meulders -0.081 -0.763 -0.314 Jamal Rayyis -0.153 0.617 -0.312 Francis Schott -0.031 0.194 -0.363 Olivier Gergaud Robert Hodgson Linda Murphy Jean-M Cardebat -0.455 -0.174 0.261 Tyler Colman -0.638 0.602 0.146 John Foy -0.250 -0.319 -0.394 Olivier Gergaud 1.000 -0.209 -0.464 Robert Hodgson -0.209 1.000 0.396 Linda Murphy -0.464 0.396 1.000 Daniele Meulders 0.816 -0.329 -0.263 Jamal Rayyis -0.098 0.733 0.075 Francis Schott -0.006 0.409 0.258 Daniele Meulders Jamal Rayyis Francis Schott Jean-M Cardebat -0.081 -0.153 -0.031 Tyler Colman -0.763 0.617 0.194 John Foy -0.314 -0.312 -0.363 Olivier Gergaud 0.816 -0.098 -0.006 Robert Hodgson -0.329 0.733 0.409 Linda Murphy -0.263 0.075 0.258 Daniele Meulders 1.000 -0.260 0.000 Jamal Rayyis -0.260 1.000 0.314 Francis Schott 0.000 0.314 1.000 Pairwise correlations in descending order 0.816 Olivier Gergaud and Daniele Meulders Significantly positive 0.733 Robert Hodgson and Jamal Rayyis Significantly positive 0.617 Tyler Colman and Jamal Rayyis Significantly positive 0.602 Tyler Colman and Robert Hodgson Significantly positive 0.409 Robert Hodgson and Francis Schott Not significant 0.396 Robert Hodgson and Linda Murphy Not significant 0.393 Tyler Colman and John Foy Not significant 0.314 Jamal Rayyis and Francis Schott Not significant 0.261 Jean-M Cardebat and Linda Murphy Not significant 0.258 Linda Murphy and Francis Schott Not significant 0.194 Tyler Colman and Francis Schott Not significant 0.146 Tyler Colman and Linda Murphy Not significant 0.140 Jean-M Cardebat and John Foy Not significant 0.115 Jean-M Cardebat and Tyler Colman Not significant 0.075 Linda Murphy and Jamal Rayyis Not significant 0.000 Daniele Meulders and Francis Schott Not significant -0.006 Olivier Gergaud and Francis Schott Not significant -0.031 Jean-M Cardebat and Francis Schott Not significant -0.081 Jean-M Cardebat and Daniele Meulders Not significant -0.098 Olivier Gergaud and Jamal Rayyis Not significant -0.153 Jean-M Cardebat and Jamal Rayyis Not significant -0.174 Jean-M Cardebat and Robert Hodgson Not significant -0.209 Olivier Gergaud and Robert Hodgson Not significant -0.250 John Foy and Olivier Gergaud Not significant -0.260 Daniele Meulders and Jamal Rayyis Not significant -0.263 Linda Murphy and Daniele Meulders Not significant -0.312 John Foy and Jamal Rayyis Not significant -0.314 John Foy and Daniele Meulders Not significant -0.319 John Foy and Robert Hodgson Not significant -0.329 Robert Hodgson and Daniele Meulders Not significant -0.363 John Foy and Francis Schott Not significant -0.394 John Foy and Linda Murphy Not significant -0.455 Jean-M Cardebat and Olivier Gergaud Not significant -0.464 Olivier Gergaud and Linda Murphy Not significant -0.638 Tyler Colman and Olivier Gergaud Significantly negative -0.763 Tyler Colman and Daniele Meulders Significantly negative
WINETASTER ON 06/08/12 WITH 9 JUDGES AND 10 WINES BASED ON GRADES, IDENT=N Copyright (c) 1995-2012 Richard E. Quandt, V. 1.65
FLIGHT 2: Number of Judges = 9 Number of Wines = 10
Identification of the Wine: The judges' overall ranking:
Wine A is Ch. Montrose 2004 ........ 4th place Wine B is Ch. Mouton Rothschild 2004 ........ 1st place Wine C is Silver Decoy Cab. Franc 2008 ........ 8th place Wine D is Heritage Estate BDX 2010 ........ 3rd place Wine E is Bellview Lumiere 2010 ........ 7th place Wine F is Tomasello Oak Reserve 2007 ........ 5th place Wine G is Ch. Leoville Las Cases 2004 ........ 6th place Wine H is Amalthea Europa VI 2008 ........ 9th place Wine I is Four JG's Cab Franc 2008 ........ 10th place Wine J is Ch. Haut Brion 2004 ........ 2nd place
The Judges' Grades
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 15.0 11.0 12.0 16.0 14.0 11.0 14.5 13.0 10.0 14.5 Tyler Colman 14.0 11.0 16.0 12.0 14.0 13.0 14.0 12.0 13.0 11.0 John Foy 17.5 19.0 18.0 18.0 15.0 16.0 18.0 18.0 17.0 17.5 Olivier Gergaud 10.0 17.0 9.0 14.0 19.0 12.0 15.0 10.0 11.0 18.0 Robert Hodgson 13.0 17.0 13.0 16.0 12.0 15.0 10.0 12.0 8.0 11.0 Linda Murphy 13.0 14.0 17.0 16.0 15.0 17.0 14.0 15.5 13.0 18.0 Daniele Meulders 14.0 16.0 11.0 16.0 14.0 15.0 13.0 11.0 10.0 15.0 Jamal Rayyis 15.0 19.5 14.0 12.0 13.0 16.0 14.5 15.0 16.0 16.0 Francis Schott 19.0 18.0 8.0 15.0 15.0 12.0 15.0 16.0 7.0 17.0
The Judges' Rankings
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 2.0 8.5 7.0 1.0 5.0 8.5 3.5 6.0 10.0 3.5 Tyler Colman 3.0 9.5 1.0 7.5 3.0 5.5 3.0 7.5 5.5 9.5 John Foy 6.5 1.0 3.5 3.5 10.0 9.0 3.5 3.5 8.0 6.5 Olivier Gergaud 8.5 3.0 10.0 5.0 1.0 6.0 4.0 8.5 7.0 2.0 Robert Hodgson 4.5 1.0 4.5 2.0 6.5 3.0 9.0 6.5 10.0 8.0 Linda Murphy 9.5 7.5 2.5 4.0 6.0 2.5 7.5 5.0 9.5 1.0 Daniele Meulders 5.5 1.5 8.5 1.5 5.5 3.5 7.0 8.5 10.0 3.5 Jamal Rayyis 5.5 1.0 8.0 10.0 9.0 3.0 7.0 5.5 3.0 3.0 Francis Schott 1.0 2.0 9.0 6.0 6.0 8.0 6.0 4.0 10.0 3.0
Group Ranking -> 4 1 8 3 7 5 6 9 10 2 Votes Against -> 46.0 35.0 54.0 40.5 52.0 49.0 50.5 55.0 73.0 40.0
( 9 is the best possible, 90 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1543
The probability that random chance could be responsible for this correlation is rather large, 0.2039. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Daniele Meulders 0.7963 Francis Schott 0.3877 Robert Hodgson 0.3272 Jean-M Cardebat 0.2778 John Foy 0.1443 Olivier Gergaud 0.0823 Linda Murphy -0.1074 Jamal Rayyis -0.1296 Tyler Colman -0.6927
We now compute for each wine the mean (average) grade it received from the judges and the standard deviation of those grades. (The mean of, say, 3 grades is the sum of the grades divided by 3. The standard deviation is a measure of the "dispersion" of the grades around the mean. The numbers 5, 6, and 7 and the numbers 3, 6, and 9 both have the same mean of 6, but the second set has a higher dispersion, i.e., strandard deviation.
Summary Grade Statistics for Wines A B C D E F G H I J Mean 14.50 15.83 13.11 15.00 14.56 14.11 14.22 13.61 11.67 15.33 StdDev 2.47 3.00 3.28 1.89 1.83 2.02 1.97 2.49 3.20 2.59
We now compute the mean and the standard deviation for each judge, from which you can see which judge is harsh and which is lenient, which thinks that the wines are very different and which thinks they are pretty much the same.
Summary Grade Statistics for Judges Name Mean Std.Dev. Jean-M Cardebat 13.10 1.91 Tyler Colman 13.00 1.48 John Foy 17.40 1.09 Olivier Gergaud 13.50 3.44 Robert Hodgson 12.70 2.61 Linda Murphy 15.25 1.66 Daniele Meulders 13.50 2.06 Jamal Rayyis 15.10 1.93 Francis Schott 14.20 3.82
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine B is Ch. Mouton Rothschild 2004 2. ........ 2nd place Wine J is Ch. Haut Brion 2004 3. ........ 3rd place Wine D is Heritage Estate BDX 2010 4. ........ 4th place Wine A is Ch.Montrose 2004 5. ........ 5th place Wine F is Tomasello Oak Reserve 2007 6. ........ 6th place Wine G is Leoville Las Cases 2004 7. ........ 7th place Wine E is Bellview Lumiere 2010 8. ........ 8th place Wine C is Silver Decoy Cab. Frfanc 2008 9. ........ 9th place Wine H is Amalthea Europa 2008 10. ........ 10th place Wine I is Four JG's Cab Franc 2008 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.65 for significance at the 0.05 level and must exceed 0.56 for significance at the 0.1 level Jean-M Cardebat Tyler Colman John Foy Jean-M Cardebat 1.000 0.053 0.120 Tyler Colman 0.053 1.000 -0.279 John Foy 0.120 -0.279 1.000 Olivier Gergaud 0.159 -0.394 -0.189 Robert Hodgson 0.061 -0.215 0.363 Linda Murphy 0.095 -0.209 -0.022 Daniele Meulders 0.336 -0.542 0.168 Jamal Rayyis -0.601 -0.538 0.019 Francis Schott 0.481 -0.414 0.318 Olivier Gergaud Robert Hodgson Linda Murphy Jean-M Cardebat 0.159 0.061 0.095 Tyler Colman -0.394 -0.215 -0.209 John Foy -0.189 0.363 -0.022 Olivier Gergaud 1.000 -0.055 0.095 Robert Hodgson -0.055 1.000 0.203 Linda Murphy 0.095 0.203 1.000 Daniele Meulders 0.554 0.710 0.291 Jamal Rayyis 0.074 -0.006 -0.121 Francis Schott 0.246 0.290 -0.155 Daniele Meulders Jamal Rayyis Francis Schott Jean-M Cardebat 0.336 -0.601 0.481 Tyler Colman -0.542 -0.538 -0.414 John Foy 0.168 0.019 0.318 Olivier Gergaud 0.554 0.074 0.246 Robert Hodgson 0.710 -0.006 0.290 Linda Murphy 0.291 -0.121 -0.155 Daniele Meulders 1.000 0.106 0.484 Jamal Rayyis 0.106 1.000 0.224 Francis Schott 0.484 0.224 1.000 Pairwise correlations in descending order 0.710 Robert Hodgson and Daniele Meulders Significantly positive 0.554 Olivier Gergaud and Daniele Meulders Not significant 0.484 Daniele Meulders and Francis Schott Not significant 0.481 Jean-M Cardebat and Francis Schott Not significant 0.363 John Foy and Robert Hodgson Not significant 0.336 Jean-M Cardebat and Daniele Meulders Not significant 0.318 John Foy and Francis Schott Not significant 0.291 Linda Murphy and Daniele Meulders Not significant 0.290 Robert Hodgson and Francis Schott Not significant 0.246 Olivier Gergaud and Francis Schott Not significant 0.224 Jamal Rayyis and Francis Schott Not significant 0.203 Robert Hodgson and Linda Murphy Not significant 0.168 John Foy and Daniele Meulders Not significant 0.159 Jean-M Cardebat and Olivier Gergaud Not significant 0.120 Jean-M Cardebat and John Foy Not significant 0.106 Daniele Meulders and Jamal Rayyis Not significant 0.095 Jean-M Cardebat and Linda Murphy Not significant 0.095 Olivier Gergaud and Linda Murphy Not significant 0.074 Olivier Gergaud and Jamal Rayyis Not significant 0.061 Jean-M Cardebat and Robert Hodgson Not significant 0.053 Jean-M Cardebat and Tyler Colman Not significant 0.019 John Foy and Jamal Rayyis Not significant -0.006 Robert Hodgson and Jamal Rayyis Not significant -0.022 John Foy and Linda Murphy Not significant -0.055 Olivier Gergaud and Robert Hodgson Not significant -0.121 Linda Murphy and Jamal Rayyis Not significant -0.155 Linda Murphy and Francis Schott Not significant -0.189 John Foy and Olivier Gergaud Not significant -0.209 Tyler Colman and Linda Murphy Not significant -0.215 Tyler Colman and Robert Hodgson Not significant -0.279 Tyler Colman and John Foy Not significant -0.394 Tyler Colman and Olivier Gergaud Not significant -0.414 Tyler Colman and Francis Schott Not significant -0.538 Tyler Colman and Jamal Rayyis Not significant -0.542 Tyler Colman and Daniele Meulders Not significant -0.601 Jean-M Cardebat and Jamal Rayyis Significantly negative
Comments by Karl Storchman and Orley Ashenfelter:At its Annual Conference in Princeton, the American Association of Wine Economists AAWE organized a wine tasting called "The Judgment of Princeton." It was modeled after the 1976 "Judgment of Paris." In 1976, British wine merchant Steve Spurrier organized a blind wine tasting with 9 French judges who were associated with the wine industry in various ways (wine journalists, critics, sommeliers, merchants or winemakers). In the first flight the judges rated 10 white wines, 6 from Napa and 4 from Burgundy.
In the second flight, the judges rated 10 reds, 6 from Napa and 4 from Bordeaux, France. In both tastings a wine from Napa, a then relatively unknown wine region, was declared the winner. George Taber of TIME magazine, the only attending journalist, reported the results to the world. The results caused considerable surprise in France and the USA, and helped to put Napa wines on the global wine map.
At the Princeton tasting, now led by George Taber, 9 wine judges from France, Belgium and the U.S. tasted French against New Jersey wines. The French wines selected were from the same producers as in 1976 including names such as Chateau Mouton-Rothschild and Haut Brion, priced up to $650/bottle. New Jersey wines for the competition were submitted to an informal panel of judges, who then selected the wines that would compete. These judges were not eligible to taste wines at the final competition. The results were surprising. Although, the winner in each category was a French wine (Beaune Clos des Mouches for the whites and Chateau Mouton-Rothschild for the reds) NJ wines barely differed in their average rank from those of France. Three of the top four whites were from New Jersey. The best NJ red was ranked 3rd place. Prices for the NJ wines are typically one-third to one-twentieth of their French competitors.
A statistical evaluation of the tasting, conducted by Princeton Professor Richard Quandt, which was similar to an earlier analysis of the Judgment of Paris ( http://www.liquidasset.com/tasting.html), further shows that the rank order of the wines was mostly insignificant. That is, if the tasting were repeated, the results would most likely be different. From a statistical viewpoint, most wines were therefore undistinguishable. Only the best white and the lowest ranked red were significantly different from the other wines.
There was a third similarity to the Paris tasting. In Paris, after the identity of the wines was revealed, Odette Kahn, editor of "La Revue du Vin de France," demanded her scorecard back. Apparently, she was not happy with having rated American wines number one and two.
At the Princeton blind tasting, both French judges preferred NJ red wines over their counterparts from Bordeaux. After disclosing the wines’ identity the French judges were surprised but did not complain. In contrast, some tasters from the U.S. did not want their wine ratings to be published. Here are some additional links: Wikipedia ( http://en.wikipedia.org/wiki/Judgment_of_Paris_%28wine%29, TV clip, JancisRobinson.com ( http://www.jancisrobinson.com/articles/a201206101.html), The New Yorker, Marginal Revolution, BlindTaste
One final set of computations was performed. Mark Censits was present as an observer and himself rated the wines, although he was not one of the official tasters. It was possibly interesting to redo the calculations by including him as if he had been an official taster, and these computations follow below.
Return to previous page
WINETASTER ON 07/01/12 WITH 10 JUDGES AND 10 WINES BASED ON GRADES, IDENT=N Copyright (c) 1995-2012 Richard E. Quandt, V. 1.65
FLIGHT 1: Number of Judges = 10 Number of Wines = 10
Identification of the Wine: The judges' overall ranking:
Wine A is Heritage Chardonnay 2010 tied for 3rd place Wine B is Unionville Pheasant Hill Single 2010 ........ 2nd place Wine C is Puligny Montrachet Leflaive 2009 ........ 8th place Wine D is Clos des Mouches Drouhin 2009 ........ 1st place Wine E is Silver Decoy "Black Feather" 2010 tied for 3rd place Wine F is Bellview Chardonnay 2010 ........ 6th place Wine G is Ventimiglia Chardonnay 2010 ........ 9th place Wine H is Meursault Charmes Latour-Labille 2008 ........ 10th place Wine I is Amalthea Chardonnay 2008 ........ 7th place Wine J is Batard Montrachet 2009 ........ 5th place
The Judges' Grades
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 10.0 13.0 14.0 15.0 8.0 13.0 15.0 11.0 9.0 12.0 Tyler Colman 16.0 14.0 14.0 16.0 12.0 11.0 11.0 14.0 11.0 14.0 John Foy 16.0 17.0 16.0 15.0 14.5 14.5 16.0 17.0 15.0 17.5 Olivier Gergaud 14.0 19.0 12.0 10.0 19.0 18.0 17.0 16.0 18.0 14.0 Robert Hodgson 17.0 11.0 13.0 14.0 14.0 10.0 9.0 9.0 10.0 10.0 Linda Murphy 15.5 15.0 17.0 18.0 16.0 17.0 15.0 14.0 16.0 17.0 Daniele Meulder 10.0 15.0 12.0 12.0 15.0 14.0 15.0 12.0 15.0 12.0 Jamal Rayyis 16.0 15.0 14.5 17.5 16.5 14.0 12.0 15.0 13.0 12.0 Francis Schott 17.0 16.0 12.0 18.0 15.0 16.0 15.0 14.0 17.0 15.0 Mark Censits 15.0 14.0 14.0 18.0 16.0 15.0 14.0 12.0 15.0 17.0
The Judges' Rankings
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 8.0 4.5 3.0 1.5 10.0 4.5 1.5 7.0 9.0 6.0 Tyler Colman 1.5 4.5 4.5 1.5 7.0 9.0 9.0 4.5 9.0 4.5 John Foy 5.0 2.5 5.0 7.5 9.5 9.5 5.0 2.5 7.5 1.0 Olivier Gergaud 7.5 1.5 9.0 10.0 1.5 3.5 5.0 6.0 3.5 7.5 Robert Hodgson 1.0 5.0 4.0 2.5 2.5 7.0 9.5 9.5 7.0 7.0 Linda Murphy 7.0 8.5 3.0 1.0 5.5 3.0 8.5 10.0 5.5 3.0 Daniele Meulder 10.0 2.5 7.5 7.5 2.5 5.0 2.5 7.5 2.5 7.5 Jamal Rayyis 3.0 4.5 6.0 1.0 2.0 7.0 9.5 4.5 8.0 9.5 Francis Schott 2.5 4.5 10.0 1.0 7.0 4.5 7.0 9.0 2.5 7.0 Mark Censits 5.0 8.0 8.0 1.0 3.0 5.0 8.0 10.0 5.0 2.0
Group Ranking -> 3 2 8 1 3 6 9 10 7 5 Votes Against -> 50.5 46.0 60.0 34.5 50.5 58.0 65.5 70.5 59.5 55.0
(10 is the best possible, 100 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1202
The probability that random chance could be responsible for this correlation is rather large, 0.3248. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Robert Hodgson 0.6087 Mark Censits 0.4861 Francis Schott 0.4428 Jamal Rayyis 0.4207 Tyler Colman 0.3326 Linda Murphy 0.3127 Jean-M Cardebat -0.2195 Olivier Gergaud -0.2936 Daniele Meulders -0.3891 John Foy -0.4831
We now compute for each wine the mean (average) grade it received from the judges and the standard deviation of those grades. (The mean of, say, 3 grades is the sum of the grades divided by 3. The standard deviation is a measure of the "dispersion" of the grades around the mean. The numbers 5, 6, and 7 and the numbers 3, 6, and 9 both have the same mean of 6, but the second set has a higher dispersion, i.e., strandard deviation.
Summary Grade Statistics for Wines A B C D E F G H I J Mean 14.65 14.90 13.85 15.35 14.60 14.25 13.90 13.40 13.90 14.05 StdDev 2.47 2.07 1.61 2.61 2.79 2.36 2.34 2.29 2.88 2.43
We now compute the mean and the standard deviation for each judge, from which you can see which judge is harsh and which is lenient, which thinks that the wines are very different and which thinks they are pretty much the same.
Summary Grade Statistics for Judges Name Mean Std.Dev. Jean-M Cardebat 12.00 2.32 Tyler Colman 13.30 1.85 John Foy 15.85 1.03 Olivier Gergaud 15.70 2.93 Robert Hodgson 11.70 2.53 Linda Murphy 16.05 1.15 Daniele Meulders 13.20 1.72 Jamal Rayyis 14.55 1.75 Francis Schott 15.50 1.63 Mark Censits 15.00 1.61
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine D is Clos des Mouches Drouhin 2009 ----------------------------------------------------------------------- 2. ........ 2nd place Wine B is Unionville Pheasant Hill Single 2010 3. tied for 3rd place Wine A is Heritage Chardonnay 2010 4. tied for 3rd place Wine E is Silver Decoy "Black Feather" 2010 5. ........ 5th place Wine J is Batard Montrachet 2009 6. ........ 6th place Wine F is Bellview Chardonnay 2010 7. ........ 7th place Wine I is Amalthea Chardonnay 2008 8. ........ 8th place Wine C is Puligny Montrachet Leflaive 2009 9. ........ 9th place Wine G is Ventimiglia Chardonnay 2010 10. ........ 10th place Wine H is Meursault Charmes Latour-Labille 2008 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.65 for significance at the 0.05 level and must exceed 0.56 for significance at the 0.1 level Jean-M Cardebat Tyler Colman John Foy Jean-M Cardebat 1.000 0.115 0.140 Tyler Colman 0.115 1.000 0.393 John Foy 0.140 0.393 1.000 Olivier Gergaud -0.455 -0.638 -0.250 Robert Hodgson -0.174 0.602 -0.319 Linda Murphy 0.261 0.146 -0.394 Daniele Meulders -0.081 -0.763 -0.314 Jamal Rayyis -0.153 0.617 -0.312 Francis Schott -0.031 0.194 -0.363 Mark Censits -0.119 0.202 -0.375 Olivier Gergaud Robert Hodgson Linda Murphy Jean-M Cardebat -0.455 -0.174 0.261 Tyler Colman -0.638 0.602 0.146 John Foy -0.250 -0.319 -0.394 Olivier Gergaud 1.000 -0.209 -0.464 Robert Hodgson -0.209 1.000 0.396 Linda Murphy -0.464 0.396 1.000 Daniele Meulders 0.816 -0.329 -0.263 Jamal Rayyis -0.098 0.733 0.075 Francis Schott -0.006 0.409 0.258 Mark Censits -0.207 0.500 0.741 Daniele Meulders Jamal Rayyis Francis Schott Jean-M Cardebat -0.081 -0.153 -0.031 Tyler Colman -0.763 0.617 0.194 John Foy -0.314 -0.312 -0.363 Olivier Gergaud 0.816 -0.098 -0.006 Robert Hodgson -0.329 0.733 0.409 Linda Murphy -0.263 0.075 0.258 Daniele Meulders 1.000 -0.260 0.000 Jamal Rayyis -0.260 1.000 0.314 Francis Schott 0.000 0.314 1.000 Mark Censits -0.133 0.231 0.544 Mark Censits Jean-M Cardebat -0.119 Tyler Colman 0.202 John Foy -0.375 Olivier Gergaud -0.207 Robert Hodgson 0.500 Linda Murphy 0.741 Daniele Meulders -0.133 Jamal Rayyis 0.231 Francis Schott 0.544 Mark Censits 1.000 Pairwise correlations in descending order 0.816 Olivier Gergaud and Daniele Meulders Significantly positive 0.741 Linda Murphy and Mark Censits Significantly positive 0.733 Robert Hodgson and Jamal Rayyis Significantly positive 0.617 Tyler Colman and Jamal Rayyis Significantly positive 0.602 Tyler Colman and Robert Hodgson Significantly positive 0.544 Francis Schott and Mark Censits Not significant 0.500 Robert Hodgson and Mark Censits Not significant 0.409 Robert Hodgson and Francis Schott Not significant 0.396 Robert Hodgson and Linda Murphy Not significant 0.393 Tyler Colman and John Foy Not significant 0.314 Jamal Rayyis and Francis Schott Not significant 0.261 Jean-M Cardebat and Linda Murphy Not significant 0.258 Linda Murphy and Francis Schott Not significant 0.231 Jamal Rayyis and Mark Censits Not significant 0.202 Tyler Colman and Mark Censits Not significant 0.194 Tyler Colman and Francis Schott Not significant 0.146 Tyler Colman and Linda Murphy Not significant 0.140 Jean-M Cardebat and John Foy Not significant 0.115 Jean-M Cardebat and Tyler Colman Not significant 0.075 Linda Murphy and Jamal Rayyis Not significant 0.000 Daniele Meulders and Francis Schott Not significant -0.006 Olivier Gergaud and Francis Schott Not significant -0.031 Jean-M Cardebat and Francis Schott Not significant -0.081 Jean-M Cardebat and Daniele Meulders Not significant -0.098 Olivier Gergaud and Jamal Rayyis Not significant -0.119 Jean-M Cardebat and Mark Censits Not significant -0.133 Daniele Meulders and Mark Censits Not significant -0.153 Jean-M Cardebat and Jamal Rayyis Not significant -0.174 Jean-M Cardebat and Robert Hodgson Not significant -0.207 Olivier Gergaud and Mark Censits Not significant -0.209 Olivier Gergaud and Robert Hodgson Not significant -0.250 John Foy and Olivier Gergaud Not significant -0.260 Daniele Meulders and Jamal Rayyis Not significant -0.263 Linda Murphy and Daniele Meulders Not significant -0.312 John Foy and Jamal Rayyis Not significant -0.314 John Foy and Daniele Meulders Not significant -0.319 John Foy and Robert Hodgson Not significant -0.329 Robert Hodgson and Daniele Meulders Not significant -0.363 John Foy and Francis Schott Not significant -0.375 John Foy and Mark Censits Not significant -0.394 John Foy and Linda Murphy Not significant -0.455 Jean-M Cardebat and Olivier Gergaud Not significant -0.464 Olivier Gergaud and Linda Murphy Not significant -0.638 Tyler Colman and Olivier Gergaud Significantly negative -0.763 Tyler Colman and Daniele Meulders Significantly negative
COMMENT: The addition of one extra taster does not make much difference. The rank sums, when adjusted for the larger number of tasters, are extremely similar to what we found in the case of the 9 original tasters, and the W coefficient of concordance hardly changes.
WINETASTER ON 07/01/12 WITH 10 JUDGES AND 10 WINES BASED ON GRADES, IDENT=N Copyright (c) 1995-2012 Richard E. Quandt, V. 1.65
FLIGHT 2: Number of Judges = 10 Number of Wines = 10
Identification of the Wine: The judges' overall ranking:
Wine A is Ch. Montrose 2004 ........ 5th place Wine B is Ch. Mouton Rothschild 2004 ........ 1st place Wine C is Silver Decoy Cab. Franc 2008 ........ 9th place Wine D is Heritage Estate BDX 2010 tied for 3rd place Wine E is Bellview Lumiere 2010 ........ 8th place Wine F is Tomasello Oak Reserve 2007 tied for 3rd place Wine G is Ch. Leoville Las Cases 2004 ........ 6th place Wine H is Amalthea Europa VI 2008 ........ 7th place Wine I is Four JG's Cab. Franc 2008 ........ 10th place Wine J is Ch. Haut Brion 2004 ........ 2nd place
The Judges' Grades
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 15.0 11.0 12.0 16.0 14.0 11.0 14.5 13.0 10.0 14.5 Tyler Colman 14.0 11.0 16.0 12.0 14.0 13.0 14.0 12.0 13.0 11.0 John Foy 17.5 19.0 18.0 18.0 15.0 16.0 18.0 18.0 17.0 17.5 Olivier Gergaud 10.0 17.0 9.0 14.0 19.0 12.0 15.0 10.0 11.0 18.0 Robert Hodgson 13.0 17.0 13.0 16.0 12.0 15.0 10.0 12.0 8.0 11.0 Linda Murphy 13.0 14.0 17.0 16.0 15.0 17.0 14.0 15.5 13.0 18.0 Daniele Meulder 14.0 16.0 11.0 16.0 14.0 15.0 13.0 11.0 10.0 15.0 Jamal Rayyis 15.0 19.5 14.0 12.0 13.0 16.0 14.5 15.0 16.0 16.0 Francis Schott 19.0 18.0 8.0 15.0 15.0 12.0 15.0 16.0 7.0 17.0 Mark Censits 15.0 18.0 14.0 14.0 15.0 19.0 17.0 17.0 15.0 16.0
The Judges' Rankings
Judge Wine -> A B C D E F G H I J Jean-M Cardebat 2.0 8.5 7.0 1.0 5.0 8.5 3.5 6.0 10.0 3.5 Tyler Colman 3.0 9.5 1.0 7.5 3.0 5.5 3.0 7.5 5.5 9.5 John Foy 6.5 1.0 3.5 3.5 10.0 9.0 3.5 3.5 8.0 6.5 Olivier Gergaud 8.5 3.0 10.0 5.0 1.0 6.0 4.0 8.5 7.0 2.0 Robert Hodgson 4.5 1.0 4.5 2.0 6.5 3.0 9.0 6.5 10.0 8.0 Linda Murphy 9.5 7.5 2.5 4.0 6.0 2.5 7.5 5.0 9.5 1.0 Daniele Meulder 5.5 1.5 8.5 1.5 5.5 3.5 7.0 8.5 10.0 3.5 Jamal Rayyis 5.5 1.0 8.0 10.0 9.0 3.0 7.0 5.5 3.0 3.0 Francis Schott 1.0 2.0 9.0 6.0 6.0 8.0 6.0 4.0 10.0 3.0 Mark Censits 7.0 2.0 9.5 9.5 7.0 1.0 3.5 3.5 7.0 5.0
Group Ranking -> 5 1 9 3 8 3 6 7 10 2 Votes Against -> 53.0 37.0 63.5 50.0 59.0 50.0 54.0 58.5 80.0 45.0
(10 is the best possible, 100 is the worst)
Here is a measure of the correlation in the preferences of the judges which ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.1501
The probability that random chance could be responsible for this correlation is rather large, 0.1564. Most analysts would say that unless this probability is less than 0.1, the judges' preferences are not strongly related. We now analyze how each taster's preferences are correlated with the group preference. A correlation of 1.0 means that the taster's preferences are a perfect predictor of the group's preferences. A 0.0 means no correlation, while a -1.0 means that the taster has the reverse ranking of the group. This is measured by the correlation R.
Correlation Between the Ranks of Each Person With the Average Ranking of Others
Name of Person Correlation R Daniele Meulders 0.7485 Francis Schott 0.4877 Robert Hodgson 0.3476 Olivier Gergaud 0.2652 Mark Censits 0.2284 Jamal Rayyis 0.1790 Jean-M Cardebat 0.1774 John Foy 0.0944 Linda Murphy -0.0612 Tyler Colman -0.7237
We now compute for each wine the mean (average) grade it received from the judges and the standard deviation of those grades. (The mean of, say, 3 grades is the sum of the grades divided by 3. The standard deviation is a measure of the "dispersion" of the grades around the mean. The numbers 5, 6, and 7 and the numbers 3, 6, and 9 both have the same mean of 6, but the second set has a higher dispersion, i.e., strandard deviation.
Summary Grade Statistics for Wines A B C D E F G H I J Mean 14.55 16.05 13.20 14.90 14.60 14.60 14.50 13.95 12.00 15.40 StdDev 2.35 2.92 3.12 1.81 1.74 2.42 2.05 2.57 3.19 2.47
We now compute the mean and the standard deviation for each judge, from which you can see which judge is harsh and which is lenient, which thinks that the wines are very different and which thinks they are pretty much the same.
Summary Grade Statistics for Judges Name Mean Std.Dev. Jean-M Cardebat 13.10 1.91 Tyler Colman 13.00 1.48 John Foy 17.40 1.09 Olivier Gergaud 13.50 3.44 Robert Hodgson 12.70 2.61 Linda Murphy 15.25 1.66 Daniele Meulders 13.50 2.06 Jamal Rayyis 15.10 1.93 Francis Schott 14.20 3.82 Mark Censits 16.00 1.61
The wines were preferred by the judges in the following order. When the preferences of the judges are strong enough to permit meaningful differentiation among the wines, they are separated by -------------------- and are judged to be significantly different.
1. ........ 1st place Wine B is Ch. Mouton Rothschild 2004 ---------------------------------------------------------------- 2. ........ 2nd place Wine J is Ch. Haut Brion 2004 3. tied for 3rd place Wine F is Tomasello Oak Reserve 2007 4. tied for 3rd place Wine D is Heritage Estate BDX 2010 5. ........ 5th place Wine A is Ch.Montrose 2004 6. ........ 6th place Wine G is Ch. Leoville Las Cases 2004 7. ........ 7th place Wine H is Amalthea Europa VI 2008 8. ........ 8th place Wine E is Bellview Lumiere 2010 9. ........ 9th place Wine C is Silver Decoy Cab. Franc 2008 ----------------------------------------------------------------- 10. ........ 10th place Wine I is Four JG's Cab. Franc 2008 We now undertake a more detailed examination of the pair-wise rank correla- tions that exist between pairs of judges. First, we present a table in which you can find the correlation for any pair of judges, by finding one of the names in the left hand margin and the other name on top of a column. A second table arranges these correlations in descending order and marks which is significantly positive significantly negative, or not significant. This may allow you to find clusters of judges whose rankings were particularly similar or particularly dissimilar. Pairwise Rank Correlations Correlations must exceed in absolute value 0.65 for significance at the 0.05 level and must exceed 0.56 for significance at the 0.1 level Jean-M Cardebat Tyler Colman John Foy Jean-M Cardebat 1.000 0.053 0.120 Tyler Colman 0.053 1.000 -0.279 John Foy 0.120 -0.279 1.000 Olivier Gergaud 0.159 -0.394 -0.189 Robert Hodgson 0.061 -0.215 0.363 Linda Murphy 0.095 -0.209 -0.022 Daniele Meulders 0.336 -0.542 0.168 Jamal Rayyis -0.601 -0.538 0.019 Francis Schott 0.481 -0.414 0.318 Mark Censits -0.388 -0.410 0.045 Olivier Gergaud Robert Hodgson Linda Murphy Jean-M Cardebat 0.159 0.061 0.095 Tyler Colman -0.394 -0.215 -0.209 John Foy -0.189 0.363 -0.022 Olivier Gergaud 1.000 -0.055 0.095 Robert Hodgson -0.055 1.000 0.203 Linda Murphy 0.095 0.203 1.000 Daniele Meulders 0.554 0.710 0.291 Jamal Rayyis 0.074 -0.006 -0.121 Francis Schott 0.246 0.290 -0.155 Mark Censits 0.263 0.068 0.006 Daniele Meulders Jamal Rayyis Francis Schott Jean-M Cardebat 0.336 -0.601 0.481 Tyler Colman -0.542 -0.538 -0.414 John Foy 0.168 0.019 0.318 Olivier Gergaud 0.554 0.074 0.246 Robert Hodgson 0.710 -0.006 0.290 Linda Murphy 0.291 -0.121 -0.155 Daniele Meulders 1.000 0.106 0.484 Jamal Rayyis 0.106 1.000 0.224 Francis Schott 0.484 0.224 1.000 Mark Censits 0.222 0.687 0.263 Mark Censits Jean-M Cardebat -0.388 Tyler Colman -0.410 John Foy 0.045 Olivier Gergaud 0.263 Robert Hodgson 0.068 Linda Murphy 0.006 Daniele Meulders 0.222 Jamal Rayyis 0.687 Francis Schott 0.263 Mark Censits 1.000 Pairwise correlations in descending order 0.710 Robert Hodgson and Daniele Meulders Significantly positive 0.687 Jamal Rayyis and Mark Censits Significantly positive 0.554 Olivier Gergaud and Daniele Meulders Not significant 0.484 Daniele Meulders and Francis Schott Not significant 0.481 Jean-M Cardebat and Francis Schott Not significant 0.363 John Foy and Robert Hodgson Not significant 0.336 Jean-M Cardebat and Daniele Meulders Not significant 0.318 John Foy and Francis Schott Not significant 0.291 Linda Murphy and Daniele Meulders Not significant 0.290 Robert Hodgson and Francis Schott Not significant 0.263 Olivier Gergaud and Mark Censits Not significant 0.263 Francis Schott and Mark Censits Not significant 0.246 Olivier Gergaud and Francis Schott Not significant 0.224 Jamal Rayyis and Francis Schott Not significant 0.222 Daniele Meulders and Mark Censits Not significant 0.203 Robert Hodgson and Linda Murphy Not significant 0.168 John Foy and Daniele Meulders Not significant 0.159 Jean-M Cardebat and Olivier Gergaud Not significant 0.120 Jean-M Cardebat and John Foy Not significant 0.106 Daniele Meulders and Jamal Rayyis Not significant 0.095 Jean-M Cardebat and Linda Murphy Not significant 0.095 Olivier Gergaud and Linda Murphy Not significant 0.074 Olivier Gergaud and Jamal Rayyis Not significant 0.068 Robert Hodgson and Mark Censits Not significant 0.061 Jean-M Cardebat and Robert Hodgson Not significant 0.053 Jean-M Cardebat and Tyler Colman Not significant 0.045 John Foy and Mark Censits Not significant 0.019 John Foy and Jamal Rayyis Not significant 0.006 Linda Murphy and Mark Censits Not significant -0.006 Robert Hodgson and Jamal Rayyis Not significant -0.022 John Foy and Linda Murphy Not significant -0.055 Olivier Gergaud and Robert Hodgson Not significant -0.121 Linda Murphy and Jamal Rayyis Not significant -0.155 Linda Murphy and Francis Schott Not significant -0.189 John Foy and Olivier Gergaud Not significant -0.209 Tyler Colman and Linda Murphy Not significant -0.215 Tyler Colman and Robert Hodgson Not significant -0.279 Tyler Colman and John Foy Not significant -0.388 Jean-M Cardebat and Mark Censits Not significant -0.394 Tyler Colman and Olivier Gergaud Not significant -0.410 Tyler Colman and Mark Censits Not significant -0.414 Tyler Colman and Francis Schott Not significant -0.538 Tyler Colman and Jamal Rayyis Not significant -0.542 Tyler Colman and Daniele Meulders Not significant -0.601 Jean-M Cardebat and Jamal Rayyis Significantly negative
Comment: As in the case of the white win es, the addition of one extra taster makes very little difference, with the exception that the Ch. Moutin Rothschild now turns out to be significantly "good."
Return to previous page