WINETASTER ON 01/03/09 WITH 15 JUDGES AND 3 WINES BASED ON RANKS, IDENT=N
Copyright © 1995-2009 Richard E. Quandt, V. 1.65
This tasting, which took place at the annual meeting of the Oenonomy Society, presents
the menu of the meal that was served, as well as the results of tasting four flights of wines,
a total of 14 wines.
MENU BY CHEF HUBERT KELLER AT FLEUR DE LYS
HORS D'OEUVRES
Champagne "Dom Pérignon" (Moët et Chandon), 1999
CHILLED DUNGENESS CRAB SALAD
with Shavings of Young Vegetables.
Lobster Infused Vinaigrette, and Lobster Fondant with Caviar
Hermitage Blanc (J.–L. Chave), 1985
Chevalier Montrachet (Louis Latour), 1989
Chablis “Les Clos“ (René and Vincent Dauvissat), 1990
CLASSIC SALMON SOUFFLÉ
with Red Wine Essence, Choucroute Soup with Toasted Spätzle
and Alsatian Blood Sausage over Choucroute
Châteauneuf du Pape (Château de Beaucastel), 1989
Latricières Chambertin (Joseph Drouhin), 1989
Volnay–Santenots (Joseph Drouhin), 1989
BONELESS QUAIL STUFFED WITH RIS DE VEAU
presented with Roasted Parsnips,
Young Leeks & Foie Gras, Lightly Smoked Apple Veal Jus, Pine Nuts
Zinfandel "Geyserville" (Ridge Vineyards), 1974
Cabernet Sauvignon "Montebello" (Ridge Vineyards), 1981, 1984, and 1991
SEARED FILET MIGNON
with Lobster Truffled Mac & Cheese "en Brioche"
Accented with a Red Wine, Shallot, Thyme "Bordelaise" Sauce
Cabernet Sauvignon "Montebello" (Ridge Vineyards), 1969
Cabernet Sauvignon "Eisele" (Ridge Vineyards). 1971
Petite Sirah "York Creek" (Ridge Vineyards), 1971
"Vin Mystère" (Ridge Vineyards), date unknown
PISTACHIO GATEAU
Roasted Apricot Cooled Off with Almond and Amaretto Ice Cream
Château Climens, 1986
ASSORTMENT OF MINIATURE PATRIES & CHOCOLATES
Annual Meeting of the Oenonomy Society
San Francisco, Saturday, January 3rd, 2009
FLIGHT 1:
Number of Judges = 15
Number of Wines = 3
Identification of the Wine: The judges' overall ranking:
Wine A is Chave Hermitage Blanc 1985 ........ 1st place
Wine B is Louis Latour Chevalier Montrachet 1989 ........ 3rd place
Wine C is Dauvissat Chablis "Les Clos" 1990 ........ 2nd place
The Judges's Rankings
Judge Wine -> A B C
Rich 1. 2. 3.
Frank 3. 1. 2.
Larry 1. 2. 3.
Mark 1. 2. 3.
Chip 1. 3. 2.
Marie 1. 3. 2.
Dwight 2. 3. 1.
John 1. 3. 2.
Orley 3. 1. 2.
Rob 3. 2. 1.
Bruce 2. 3. 1.
Paul 3. 1. 2.
Jan 1. 3. 2.
Roman 1. 2. 3.
Bronwyn 3. 2. 1.
Table of Votes Against
Wine -> A B C
Group Ranking -> 1 3 2
Votes Against -> 27 33 30
(15 is the best possible, 45 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.0400
The probability that random chance could be responsible for this correlation
is rather large, 0.5488. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
John 1.0000
Chip 1.0000
Marie 1.0000
Jan 1.0000
Rich 0.5000
Larry 0.5000
Mark 0.5000
Bruce 0.5000
Dwight 0.5000
Roman 0.5000
Bronwyn -0.5000
Rob -0.5000
Orley -1.0000
Frank -1.0000
Paul -1.0000
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine A is Chave Hermitage Blanc 1985
2. ........ 2nd place Wine C is Dauvissat Chablis "Les Clos" 1990
3. ........ 3rd place Wine B is Louis Latour Chevalier Montrachet 1989
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 1.2000. The probability that this could
happen by chance is 0.5488
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 1.00 for significance at the 0.05
level and must exceed 1.00 for significance at the 0.1 level
Rich Frank Larry
Rich 1.000 -0.500 1.000
Frank -0.500 1.000 -0.500
Larry 1.000 -0.500 1.000
Mark 1.000 -0.500 1.000
Chip 0.500 -1.000 0.500
Marie 0.500 -1.000 0.500
Dwight -0.500 -0.500 -0.500
John 0.500 -1.000 0.500
Orley -0.500 1.000 -0.500
Rob -1.000 0.500 -1.000
Bruce -0.500 -0.500 -0.500
Paul -0.500 1.000 -0.500
Jan 0.500 -1.000 0.500
Roman 1.000 -0.500 1.000
Bronwyn -1.000 0.500 -1.000
Mark Chip Marie
Rich 1.000 0.500 0.500
Frank -0.500 -1.000 -1.000
Larry 1.000 0.500 0.500
Mark 1.000 0.500 0.500
Chip 0.500 1.000 1.000
Marie 0.500 1.000 1.000
Dwight -0.500 0.500 0.500
John 0.500 1.000 1.000
Orley -0.500 -1.000 -1.000
Rob -1.000 -0.500 -0.500
Bruce -0.500 0.500 0.500
Paul -0.500 -1.000 -1.000
Jan 0.500 1.000 1.000
Roman 1.000 0.500 0.500
Bronwyn -1.000 -0.500 -0.500
Dwight John Orley
Rich -0.500 0.500 -0.500
Frank -0.500 -1.000 1.000
Larry -0.500 0.500 -0.500
Mark -0.500 0.500 -0.500
Chip 0.500 1.000 -1.000
Marie 0.500 1.000 -1.000
Dwight 1.000 0.500 -0.500
John 0.500 1.000 -1.000
Orley -0.500 -1.000 1.000
Rob 0.500 -0.500 0.500
Bruce 1.000 0.500 -0.500
Paul -0.500 -1.000 1.000
Jan 0.500 1.000 -1.000
Roman -0.500 0.500 -0.500
Bronwyn 0.500 -0.500 0.500
Rob Bruce Paul
Rich -1.000 -0.500 -0.500
Frank 0.500 -0.500 1.000
Larry -1.000 -0.500 -0.500
Mark -1.000 -0.500 -0.500
Chip -0.500 0.500 -1.000
Marie -0.500 0.500 -1.000
Dwight 0.500 1.000 -0.500
John -0.500 0.500 -1.000
Orley 0.500 -0.500 1.000
Rob 1.000 0.500 0.500
Bruce 0.500 1.000 -0.500
Paul 0.500 -0.500 1.000
Jan -0.500 0.500 -1.000
Roman -1.000 -0.500 -0.500
Bronwyn 1.000 0.500 0.500
Jan Roman Bronwyn
Rich 0.500 1.000 -1.000
Frank -1.000 -0.500 0.500
Larry 0.500 1.000 -1.000
Mark 0.500 1.000 -1.000
Chip 1.000 0.500 -0.500
Marie 1.000 0.500 -0.500
Dwight 0.500 -0.500 0.500
John 1.000 0.500 -0.500
Orley -1.000 -0.500 0.500
Rob -0.500 -1.000 1.000
Bruce 0.500 -0.500 0.500
Paul -1.000 -0.500 0.500
Jan 1.000 0.500 -0.500
Roman 0.500 1.000 -1.000
Bronwyn -0.500 -1.000 1.000
Pairwise correlations in descending order
1.000 Chip and John Significantly positive
1.000 Rich and Larry Significantly positive
1.000 Rich and Mark Significantly positive
1.000 John and Jan Significantly positive
1.000 Frank and Paul Significantly positive
1.000 Chip and Jan Significantly positive
1.000 Rich and Roman Significantly positive
1.000 Frank and Orley Significantly positive
1.000 Orley and Paul Significantly positive
1.000 Marie and John Significantly positive
1.000 Rob and Bronwyn Significantly positive
1.000 Larry and Roman Significantly positive
1.000 Chip and Marie Significantly positive
1.000 Mark and Roman Significantly positive
1.000 Marie and Jan Significantly positive
1.000 Larry and Mark Significantly positive
1.000 Dwight and Bruce Significantly positive
0.500 Mark and Chip Not significant
0.500 Rich and John Not significant
0.500 Dwight and Rob Not significant
0.500 Dwight and John Not significant
0.500 Marie and Roman Not significant
0.500 Rich and Marie Not significant
0.500 Larry and Jan Not significant
0.500 Bruce and Jan Not significant
0.500 Chip and Dwight Not significant
0.500 Frank and Bronwyn Not significant
0.500 Frank and Rob Not significant
0.500 Larry and Chip Not significant
0.500 Rich and Chip Not significant
0.500 John and Roman Not significant
0.500 Larry and John Not significant
0.500 Orley and Rob Not significant
0.500 Bruce and Bronwyn Not significant
0.500 Marie and Dwight Not significant
0.500 Dwight and Jan Not significant
0.500 Paul and Bronwyn Not significant
0.500 Rich and Jan Not significant
0.500 Rob and Bruce Not significant
0.500 Rob and Paul Not significant
0.500 Mark and Marie Not significant
0.500 Mark and Jan Not significant
0.500 Mark and John Not significant
0.500 Chip and Bruce Not significant
0.500 Dwight and Bronwyn Not significant
0.500 Chip and Roman Not significant
0.500 Jan and Roman Not significant
0.500 John and Bruce Not significant
0.500 Marie and Bruce Not significant
0.500 Larry and Marie Not significant
0.500 Orley and Bronwyn Not significant
-0.500 Mark and Orley Not significant
-0.500 Rich and Orley Not significant
-0.500 Mark and Dwight Not significant
-0.500 Larry and Paul Not significant
-0.500 Rich and Paul Not significant
-0.500 Frank and Dwight Not significant
-0.500 Chip and Rob Not significant
-0.500 Rich and Frank Not significant
-0.500 Orley and Bruce Not significant
-0.500 Larry and Bruce Not significant
-0.500 Rich and Bruce Not significant
-0.500 Larry and Dwight Not significant
-0.500 Rich and Dwight Not significant
-0.500 Mark and Bruce Not significant
-0.500 John and Rob Not significant
-0.500 Frank and Larry Not significant
-0.500 Frank and Mark Not significant
-0.500 Orley and Roman Not significant
-0.500 Bruce and Paul Not significant
-0.500 Frank and Roman Not significant
-0.500 Bruce and Roman Not significant
-0.500 Chip and Bronwyn Not significant
-0.500 Dwight and Paul Not significant
-0.500 Frank and Bruce Not significant
-0.500 Dwight and Roman Not significant
-0.500 Dwight and Orley Not significant
-0.500 Jan and Bronwyn Not significant
-0.500 Larry and Orley Not significant
-0.500 Rob and Jan Not significant
-0.500 John and Bronwyn Not significant
-0.500 Marie and Bronwyn Not significant
-0.500 Mark and Paul Not significant
-0.500 Marie and Rob Not significant
-0.500 Paul and Roman Not significant
-1.000 Chip and Paul Significantly negative
-1.000 John and Paul Significantly negative
-1.000 Rich and Bronwyn Significantly negative
-1.000 Marie and Orley Significantly negative
-1.000 Rich and Rob Significantly negative
-1.000 Larry and Bronwyn Significantly negative
-1.000 Marie and Paul Significantly negative
-1.000 Chip and Orley Significantly negative
-1.000 Rob and Roman Significantly negative
-1.000 Frank and Chip Significantly negative
-1.000 Frank and Marie Significantly negative
-1.000 Mark and Rob Significantly negative
-1.000 Frank and John Significantly negative
-1.000 Larry and Rob Significantly negative
-1.000 Paul and Jan Significantly negative
-1.000 Orley and Jan Significantly negative
-1.000 Mark and Bronwyn Significantly negative
-1.000 Frank and Jan Significantly negative
-1.000 John and Orley Significantly negative
-1.000 Roman and Bronwyn Significantly negative
COMMENT:
In two of the flights (Flights 2 and 3) a small number of tasters entered tied ranks
which WINETASTER does not permit (because we made an early decision that
tied ranks encourage "lazy" tasting). In Flight 2, one taster had two wines tied and
in Flight 3 three tasters had two wines tied each. We therefore broke the ties by
assigning the lexicographically earlier wine the lower and the
lexicographically later wine the higher of the tied ranks. To test the
impact of this, we also reranked the wines using the opposite convention.
Return to previous page
Report
WINETASTER ON 01/14/09 WITH 15 JUDGES AND 3 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2003 Richard E. Quandt, V. 1.65
FLIGHT 2:
Number of Judges = 15
Number of Wines = 3
Identification of the Wine: The judges' overall ranking:
Wine A is Beaucastel CDP 1989 ........ 1st place
Wine B is Drouhin Latricieres Chambertin 1989 ........ 2nd place
Wine C is Drouhin Volnay-Santenots 1989 ........ 3rd place
The Judges's Rankings
Judge Wine -> A B C
Rich 1. 2. 3.
Frank 1. 2. 3.
Larry 1. 2. 3.
Mark 1. 2. 3.
Chip 1. 2. 3.
Marie 1. 2. 3.
Dwight 1. 2. 3.
John 1. 2. 3.
Orley 1. 3. 2.
Rob 1. 2. 3.
Bruce 1. 3. 2.
Paul 1. 3. 2.
Jan 1. 2. 3.
Roman 1. 2. 3.
Bronwyn 1. 2. 3.
Table of Votes Against
Wine -> A B C
Group Ranking -> 1 2 3
Votes Against -> 15 33 42
(15 is the best possible, 45 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.8400
The probability that random chance could be responsible for this correlation
is quite small, 0.0000. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
Rich 1.0000
Frank 1.0000
Larry 1.0000
Mark 1.0000
Chip 1.0000
Marie 1.0000
Dwight 1.0000
John 1.0000
Bronwyn 1.0000
Rob 1.0000
Roman 1.0000
Jan 1.0000
Orley 0.5000
Bruce 0.5000
Paul 0.5000
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine A is Beaucastel CDP 1989
---------------------------------------------------
2. ........ 2nd place Wine B is Drouhin Latricieres Chambertin 1989
---------------------------------------------------
3. ........ 3rd place Wine C is Drouhin Volnay-Santenots 1989
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 25.2000. The probability that this could
happen by chance is 0.0000
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 1.00 for significance at the 0.05
level and must exceed 1.00 for significance at the 0.1 level
Rich Frank Larry
Rich 1.000 1.000 1.000
Frank 1.000 1.000 1.000
Larry 1.000 1.000 1.000
Mark 1.000 1.000 1.000
Chip 1.000 1.000 1.000
Marie 1.000 1.000 1.000
Dwight 1.000 1.000 1.000
John 1.000 1.000 1.000
Orley 0.500 0.500 0.500
Rob 1.000 1.000 1.000
Bruce 0.500 0.500 0.500
Paul 0.500 0.500 0.500
Jan 1.000 1.000 1.000
Roman 1.000 1.000 1.000
Bronwyn 1.000 1.000 1.000
Mark Chip Marie
Rich 1.000 1.000 1.000
Frank 1.000 1.000 1.000
Larry 1.000 1.000 1.000
Mark 1.000 1.000 1.000
Chip 1.000 1.000 1.000
Marie 1.000 1.000 1.000
Dwight 1.000 1.000 1.000
John 1.000 1.000 1.000
Orley 0.500 0.500 0.500
Rob 1.000 1.000 1.000
Bruce 0.500 0.500 0.500
Paul 0.500 0.500 0.500
Jan 1.000 1.000 1.000
Roman 1.000 1.000 1.000
Bronwyn 1.000 1.000 1.000
Dwight John Orley
Rich 1.000 1.000 0.500
Frank 1.000 1.000 0.500
Larry 1.000 1.000 0.500
Mark 1.000 1.000 0.500
Chip 1.000 1.000 0.500
Marie 1.000 1.000 0.500
Dwight 1.000 1.000 0.500
John 1.000 1.000 0.500
Orley 0.500 0.500 1.000
Rob 1.000 1.000 0.500
Bruce 0.500 0.500 1.000
Paul 0.500 0.500 1.000
Jan 1.000 1.000 0.500
Roman 1.000 1.000 0.500
Bronwyn 1.000 1.000 0.500
Rob Bruce Paul
Rich 1.000 0.500 0.500
Frank 1.000 0.500 0.500
Larry 1.000 0.500 0.500
Mark 1.000 0.500 0.500
Chip 1.000 0.500 0.500
Marie 1.000 0.500 0.500
Dwight 1.000 0.500 0.500
John 1.000 0.500 0.500
Orley 0.500 1.000 1.000
Rob 1.000 0.500 0.500
Bruce 0.500 1.000 1.000
Paul 0.500 1.000 1.000
Jan 1.000 0.500 0.500
Roman 1.000 0.500 0.500
Bronwyn 1.000 0.500 0.500
Jan Roman Bronwyn
Rich 1.000 1.000 1.000
Frank 1.000 1.000 1.000
Larry 1.000 1.000 1.000
Mark 1.000 1.000 1.000
Chip 1.000 1.000 1.000
Marie 1.000 1.000 1.000
Dwight 1.000 1.000 1.000
John 1.000 1.000 1.000
Orley 0.500 0.500 0.500
Rob 1.000 1.000 1.000
Bruce 0.500 0.500 0.500
Paul 0.500 0.500 0.500
Jan 1.000 1.000 1.000
Roman 1.000 1.000 1.000
Bronwyn 1.000 1.000 1.000
Pairwise correlations in descending order
1.000 Rich and Frank Significantly positive
1.000 Rich and Larry Significantly positive
1.000 Rich and Mark Significantly positive
1.000 Rich and Chip Significantly positive
1.000 Rich and Marie Significantly positive
1.000 Rich and Dwight Significantly positive
1.000 Rich and John Significantly positive
1.000 Chip and Bronwyn Significantly positive
1.000 Rich and Rob Significantly positive
1.000 Marie and John Significantly positive
1.000 Larry and Jan Significantly positive
1.000 Rich and Jan Significantly positive
1.000 Rich and Roman Significantly positive
1.000 Rich and Bronwyn Significantly positive
1.000 Frank and Larry Significantly positive
1.000 Frank and Mark Significantly positive
1.000 Frank and Chip Significantly positive
1.000 Frank and Marie Significantly positive
1.000 Frank and Dwight Significantly positive
1.000 Frank and John Significantly positive
1.000 Larry and Rob Significantly positive
1.000 Frank and Rob Significantly positive
1.000 Dwight and Jan Significantly positive
1.000 Dwight and Roman Significantly positive
1.000 Frank and Jan Significantly positive
1.000 Frank and Roman Significantly positive
1.000 Frank and Bronwyn Significantly positive
1.000 Larry and Mark Significantly positive
1.000 Larry and Chip Significantly positive
1.000 Larry and Marie Significantly positive
1.000 Larry and Dwight Significantly positive
1.000 Larry and John Significantly positive
1.000 Chip and Roman Significantly positive
1.000 Orley and Bruce Significantly positive
1.000 Orley and Paul Significantly positive
1.000 Mark and Roman Significantly positive
1.000 Mark and Bronwyn Significantly positive
1.000 Larry and Roman Significantly positive
1.000 Larry and Bronwyn Significantly positive
1.000 Mark and Chip Significantly positive
1.000 Mark and Marie Significantly positive
1.000 Mark and Dwight Significantly positive
1.000 Mark and John Significantly positive
1.000 Bruce and Paul Significantly positive
1.000 Mark and Rob Significantly positive
1.000 Dwight and Rob Significantly positive
1.000 Chip and John Significantly positive
1.000 Mark and Jan Significantly positive
1.000 Chip and Rob Significantly positive
1.000 John and Jan Significantly positive
1.000 Chip and Marie Significantly positive
1.000 Chip and Dwight Significantly positive
1.000 Jan and Bronwyn Significantly positive
1.000 Marie and Jan Significantly positive
1.000 Marie and Dwight Significantly positive
1.000 Marie and Roman Significantly positive
1.000 John and Roman Significantly positive
1.000 Chip and Jan Significantly positive
1.000 John and Bronwyn Significantly positive
1.000 John and Rob Significantly positive
1.000 Rob and Jan Significantly positive
1.000 Dwight and Bronwyn Significantly positive
1.000 Marie and Bronwyn Significantly positive
1.000 Marie and Rob Significantly positive
1.000 Jan and Roman Significantly positive
1.000 Roman and Bronwyn Significantly positive
1.000 Rob and Roman Significantly positive
1.000 Dwight and John Significantly positive
1.000 Rob and Bronwyn Significantly positive
0.500 Rich and Paul Not significant
0.500 Rich and Bruce Not significant
0.500 Frank and Orley Not significant
0.500 Mark and Paul Not significant
0.500 Dwight and Paul Not significant
0.500 Marie and Orley Not significant
0.500 Larry and Paul Not significant
0.500 Larry and Orley Not significant
0.500 Mark and Bruce Not significant
0.500 Dwight and Bruce Not significant
0.500 Chip and Orley Not significant
0.500 Frank and Bruce Not significant
0.500 Frank and Paul Not significant
0.500 Chip and Paul Not significant
0.500 Marie and Bruce Not significant
0.500 Orley and Rob Not significant
0.500 Rich and Orley Not significant
0.500 Larry and Bruce Not significant
0.500 Orley and Jan Not significant
0.500 Orley and Roman Not significant
0.500 Dwight and Orley Not significant
0.500 Rob and Bruce Not significant
0.500 Marie and Paul Not significant
0.500 John and Bruce Not significant
0.500 John and Paul Not significant
0.500 Chip and Bruce Not significant
0.500 Mark and Orley Not significant
0.500 Bruce and Jan Not significant
0.500 Bruce and Roman Not significant
0.500 Bruce and Bronwyn Not significant
0.500 Paul and Jan Not significant
0.500 Paul and Roman Not significant
0.500 Paul and Bronwyn Not significant
0.500 Orley and Bronwyn Not significant
0.500 John and Orley Not significant
0.500 Rob and Paul Not significant
COMMENT:
As we indicated before, there was one pair of wines that were ranked a tie by one
of the judges and we broke the tie by assigning the lexicographically earlier wine
the lower and the lexicographically later wine the higher of the tied ranks. To test
the robustness of this procedure, we also computer the result for the reverse way of
ranking the tied wines. The Kendall W coefficient then becomes 0.8044, the probability
that this could have occurred by chance is 0.0000, as before, and wine A is still
significantly of high quality and C of low quality. We conclude that the tied rank and
our way of breaking the tie had no impact on the results.
Return to previous page
Report
WINETASTER ON 01/14/09 WITH 15 JUDGES AND 4 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2003 Richard E. Quandt, V. 1.65
FLIGHT 3:
Number of Judges = 15
Number of Wines = 4
Identification of the Wine: The judges' overall ranking:
Wine A is Ridge Geyserville 1974 ........ 1st place
Wine B is Ridge Montebello 1981 (cabernet) ........ 4th place
Wine C is Ridge Montebello 1984 ........ 3rd place
Wine D is Ridge Montebello 1991 ........ 2nd place
The Judges's Rankings
Judge Wine -> A B C D
Rich 1. 4. 2. 3.
Frank 2. 1. 3. 4.
Larry 1. 4. 3. 2.
Mark 2. 3. 4. 1.
Chip 1. 2. 4. 3.
Marie 2. 4. 3. 1.
Dwight 1. 2. 4. 3.
John 1. 3. 4. 2.
Orley 1. 4. 2. 3.
Rob 1. 4. 2. 3.
Bruce 1. 4. 3. 2.
Paul 1. 4. 3. 2.
Jan 1. 4. 3. 2.
Roman 1. 2. 3. 4.
Bronwyn 1. 3. 4. 2.
Table of Votes Against
Wine -> A B C D
Group Ranking -> 1 4 3 2
Votes Against -> 18 48 47 37
(15 is the best possible, 60 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.5164
The probability that random chance could be responsible for this correlation
is quite small, 0.0000. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
Bruce 0.9487
Paul 0.9487
Larry 0.9487
Jan 0.9487
John 0.8000
Bronwyn 0.8000
Marie 0.7379
Mark 0.6000
Chip 0.4000
Dwight 0.4000
Orley 0.4000
Rich 0.4000
Rob 0.4000
Roman 0.2000
Frank -0.4000
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine A is Ridge Geyserville 1974
---------------------------------------------------
2. ........ 2nd place Wine D is Ridge Montebello 1991
---------------------------------------------------
3. ........ 3rd place Wine C is Ridge Montebello 1984
4. ........ 4th place Wine B is Ridge Montebello 1981 (cabernet)
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 23.2400. The probability that this could
happen by chance is 0.0000
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 1.00 for significance at the 0.05
level and must exceed 1.00 for significance at the 0.1 level
Rich Frank Larry
Rich 1.000 -0.200 0.800
Frank -0.200 1.000 -0.400
Larry 0.800 -0.400 1.000
Mark 0.000 -0.400 0.600
Chip 0.200 0.600 0.400
Marie 0.400 -0.800 0.800
Dwight 0.200 0.600 0.400
John 0.400 0.000 0.800
Orley 1.000 -0.200 0.800
Rob 1.000 -0.200 0.800
Bruce 0.800 -0.400 1.000
Paul 0.800 -0.400 1.000
Jan 0.800 -0.400 1.000
Roman 0.400 0.800 0.200
Bronwyn 0.400 0.000 0.800
Mark Chip Marie
Rich 0.000 0.200 0.400
Frank -0.400 0.600 -0.800
Larry 0.600 0.400 0.800
Mark 1.000 0.400 0.800
Chip 0.400 1.000 0.000
Marie 0.800 0.000 1.000
Dwight 0.400 1.000 0.000
John 0.800 0.800 0.600
Orley 0.000 0.200 0.400
Rob 0.000 0.200 0.400
Bruce 0.600 0.400 0.800
Paul 0.600 0.400 0.800
Jan 0.600 0.400 0.800
Roman -0.200 0.800 -0.400
Bronwyn 0.800 0.800 0.600
Dwight John Orley
Rich 0.200 0.400 1.000
Frank 0.600 0.000 -0.200
Larry 0.400 0.800 0.800
Mark 0.400 0.800 0.000
Chip 1.000 0.800 0.200
Marie 0.000 0.600 0.400
Dwight 1.000 0.800 0.200
John 0.800 1.000 0.400
Orley 0.200 0.400 1.000
Rob 0.200 0.400 1.000
Bruce 0.400 0.800 0.800
Paul 0.400 0.800 0.800
Jan 0.400 0.800 0.800
Roman 0.800 0.400 0.400
Bronwyn 0.800 1.000 0.400
Rob Bruce Paul
Rich 1.000 0.800 0.800
Frank -0.200 -0.400 -0.400
Larry 0.800 1.000 1.000
Mark 0.000 0.600 0.600
Chip 0.200 0.400 0.400
Marie 0.400 0.800 0.800
Dwight 0.200 0.400 0.400
John 0.400 0.800 0.800
Orley 1.000 0.800 0.800
Rob 1.000 0.800 0.800
Bruce 0.800 1.000 1.000
Paul 0.800 1.000 1.000
Jan 0.800 1.000 1.000
Roman 0.400 0.200 0.200
Bronwyn 0.400 0.800 0.800
Jan Roman Bronwyn
Rich 0.800 0.400 0.400
Frank -0.400 0.800 0.000
Larry 1.000 0.200 0.800
Mark 0.600 -0.200 0.800
Chip 0.400 0.800 0.800
Marie 0.800 -0.400 0.600
Dwight 0.400 0.800 0.800
John 0.800 0.400 1.000
Orley 0.800 0.400 0.400
Rob 0.800 0.400 0.400
Bruce 1.000 0.200 0.800
Paul 1.000 0.200 0.800
Jan 1.000 0.200 0.800
Roman 0.200 1.000 0.400
Bronwyn 0.800 0.400 1.000
Pairwise correlations in descending order
1.000 Orley and Rob Significantly positive
1.000 Rich and Orley Significantly positive
1.000 Rich and Rob Significantly positive
1.000 Larry and Paul Significantly positive
1.000 Bruce and Paul Significantly positive
1.000 John and Bronwyn Significantly positive
1.000 Chip and Dwight Significantly positive
1.000 Larry and Jan Significantly positive
1.000 Paul and Jan Significantly positive
1.000 Larry and Bruce Significantly positive
1.000 Bruce and Jan Significantly positive
0.800 John and Paul Not significant
0.800 Rich and Jan Not significant
0.800 Rich and Larry Not significant
0.800 Marie and Jan Not significant
0.800 Larry and Marie Not significant
0.800 Marie and Paul Not significant
0.800 Dwight and John Not significant
0.800 Chip and John Not significant
0.800 Chip and Roman Not significant
0.800 Bruce and Bronwyn Not significant
0.800 John and Bruce Not significant
0.800 Mark and John Not significant
0.800 Dwight and Roman Not significant
0.800 Dwight and Bronwyn Not significant
0.800 Marie and Bruce Not significant
0.800 Rob and Paul Not significant
0.800 Larry and Rob Not significant
0.800 Rich and Bruce Not significant
0.800 John and Jan Not significant
0.800 Rich and Paul Not significant
0.800 Larry and John Not significant
0.800 Larry and Orley Not significant
0.800 Frank and Roman Not significant
0.800 Mark and Marie Not significant
0.800 Orley and Jan Not significant
0.800 Mark and Bronwyn Not significant
0.800 Larry and Bronwyn Not significant
0.800 Chip and Bronwyn Not significant
0.800 Orley and Paul Not significant
0.800 Rob and Bruce Not significant
0.800 Paul and Bronwyn Not significant
0.800 Rob and Jan Not significant
0.800 Orley and Bruce Not significant
0.800 Jan and Bronwyn Not significant
0.600 Frank and Chip Not significant
0.600 Mark and Bruce Not significant
0.600 Marie and John Not significant
0.600 Mark and Jan Not significant
0.600 Marie and Bronwyn Not significant
0.600 Larry and Mark Not significant
0.600 Mark and Paul Not significant
0.600 Frank and Dwight Not significant
0.400 Orley and Bronwyn Not significant
0.400 Larry and Chip Not significant
0.400 Mark and Dwight Not significant
0.400 Larry and Dwight Not significant
0.400 John and Rob Not significant
0.400 Roman and Bronwyn Not significant
0.400 Marie and Rob Not significant
0.400 Chip and Jan Not significant
0.400 Dwight and Jan Not significant
0.400 Chip and Paul Not significant
0.400 John and Roman Not significant
0.400 Rich and John Not significant
0.400 Rob and Roman Not significant
0.400 Rich and Marie Not significant
0.400 Dwight and Paul Not significant
0.400 Chip and Bruce Not significant
0.400 Marie and Orley Not significant
0.400 Orley and Roman Not significant
0.400 Rich and Bronwyn Not significant
0.400 Mark and Chip Not significant
0.400 Dwight and Bruce Not significant
0.400 Rob and Bronwyn Not significant
0.400 John and Orley Not significant
0.400 Rich and Roman Not significant
0.200 Paul and Roman Not significant
0.200 Dwight and Rob Not significant
0.200 Dwight and Orley Not significant
0.200 Rich and Dwight Not significant
0.200 Jan and Roman Not significant
0.200 Larry and Roman Not significant
0.200 Chip and Rob Not significant
0.200 Chip and Orley Not significant
0.200 Bruce and Roman Not significant
0.200 Rich and Chip Not significant
0.000 Marie and Dwight Not significant
0.000 Chip and Marie Not significant
0.000 Mark and Rob Not significant
0.000 Rich and Mark Not significant
0.000 Mark and Orley Not significant
0.000 Frank and Bronwyn Not significant
0.000 Frank and John Not significant
-0.200 Frank and Orley Not significant
-0.200 Rich and Frank Not significant
-0.200 Mark and Roman Not significant
-0.200 Frank and Rob Not significant
-0.400 Frank and Paul Not significant
-0.400 Frank and Mark Not significant
-0.400 Marie and Roman Not significant
-0.400 Frank and Larry Not significant
-0.400 Frank and Jan Not significant
-0.400 Frank and Bruce Not significant
-0.800 Frank and Marie Not significant
COMMENT:
Here we repated the procedure employed in Flight 2, i.e., we recalculated the results by breaking the
tied votes in the reverse way. The Kendal W now becomes 0.5342 (vs. the old value of 0.5164); the
probability that this could have occurred by chance is still 0.0000, wine A is still significantly
of high quality and wines C and D of low quality. Hence again the fact that ties were recorded originally
had no impact on the results.
Return to previous page
Report
WINETASTER ON 01/14/09 WITH 15 JUDGES AND 4 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2003 Richard E. Quandt, V. 1.65
FLIGHT 4:
Number of Judges = 15
Number of Wines = 4
Identification of the Wine: The judges' overall ranking:
Wine A is Ridge Montebello 1969 ........ 1st place
Wine B is Ridge Eisele 1971 tied for 2nd place
Wine C is Ridge Petite Sirah York Creek 1971 tied for 2nd place
Wine D is Ridge "Vin Mystere" date unknown ........ 4th place
The Judges's Rankings
Judge Wine -> A B C D
Rich 1. 3. 2. 4.
Frank 1. 2. 3. 4.
Larry 1. 2. 3. 4.
Mark 2. 1. 3. 4.
Chip 1. 2. 3. 4.
Marie 1. 2. 3. 4.
Dwight 1. 2. 3. 4.
John 1. 2. 3. 4.
Orley 1. 3. 2. 4.
Rob 1. 3. 2. 4.
Bruce 1. 3. 2. 4.
Paul 1. 3. 2. 4.
Jan 2. 3. 1. 4.
Roman 1. 2. 3. 4.
Bronwyn 2. 3. 1. 4.
Table of Votes Against
Wine -> A B C D
Group Ranking -> 1 2 2 4
Votes Against -> 18 36 36 60
(15 is the best possible, 60 is the worst)
Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):
W = 0.7920
The probability that random chance could be responsible for this correlation
is quite small, 0.0000. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.
Correlation Between the Ranks of
Each Person With the Average Ranking of Others
Name of Person Correlation R
Rich 0.8000
Frank 0.8000
Larry 0.8000
Bruce 0.8000
Chip 0.8000
Marie 0.8000
Dwight 0.8000
John 0.8000
Orley 0.8000
Rob 0.8000
Roman 0.8000
Paul 0.8000
Jan 0.4000
Mark 0.4000
Bronwyn 0.4000
The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.
1. ........ 1st place Wine A is Ridge Montebello 1969
---------------------------------------------------
2. tied for 2nd place Wine B is Ridge Eisele 1971
3. tied for 2nd place Wine C is Ridge Petite Sirah York Creek 1971
---------------------------------------------------
4. ........ 4th place Wine D is Ridge "Vin Mystere" date unknown
We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 35.6400. The probability that this could
happen by chance is 0.0000
We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.
Pairwise Rank Correlations
Correlations must exceed in absolute value 1.00 for significance at the 0.05
level and must exceed 1.00 for significance at the 0.1 level
Rich Frank Larry
Rich 1.000 0.800 0.800
Frank 0.800 1.000 1.000
Larry 0.800 1.000 1.000
Mark 0.400 0.800 0.800
Chip 0.800 1.000 1.000
Marie 0.800 1.000 1.000
Dwight 0.800 1.000 1.000
John 0.800 1.000 1.000
Orley 1.000 0.800 0.800
Rob 1.000 0.800 0.800
Bruce 1.000 0.800 0.800
Paul 1.000 0.800 0.800
Jan 0.800 0.400 0.400
Roman 0.800 1.000 1.000
Bronwyn 0.800 0.400 0.400
Mark Chip Marie
Rich 0.400 0.800 0.800
Frank 0.800 1.000 1.000
Larry 0.800 1.000 1.000
Mark 1.000 0.800 0.800
Chip 0.800 1.000 1.000
Marie 0.800 1.000 1.000
Dwight 0.800 1.000 1.000
John 0.800 1.000 1.000
Orley 0.400 0.800 0.800
Rob 0.400 0.800 0.800
Bruce 0.400 0.800 0.800
Paul 0.400 0.800 0.800
Jan 0.200 0.400 0.400
Roman 0.800 1.000 1.000
Bronwyn 0.200 0.400 0.400
Dwight John Orley
Rich 0.800 0.800 1.000
Frank 1.000 1.000 0.800
Larry 1.000 1.000 0.800
Mark 0.800 0.800 0.400
Chip 1.000 1.000 0.800
Marie 1.000 1.000 0.800
Dwight 1.000 1.000 0.800
John 1.000 1.000 0.800
Orley 0.800 0.800 1.000
Rob 0.800 0.800 1.000
Bruce 0.800 0.800 1.000
Paul 0.800 0.800 1.000
Jan 0.400 0.400 0.800
Roman 1.000 1.000 0.800
Bronwyn 0.400 0.400 0.800
Rob Bruce Paul
Rich 1.000 1.000 1.000
Frank 0.800 0.800 0.800
Larry 0.800 0.800 0.800
Mark 0.400 0.400 0.400
Chip 0.800 0.800 0.800
Marie 0.800 0.800 0.800
Dwight 0.800 0.800 0.800
John 0.800 0.800 0.800
Orley 1.000 1.000 1.000
Rob 1.000 1.000 1.000
Bruce 1.000 1.000 1.000
Paul 1.000 1.000 1.000
Jan 0.800 0.800 0.800
Roman 0.800 0.800 0.800
Bronwyn 0.800 0.800 0.800
Jan Roman Bronwyn
Rich 0.800 0.800 0.800
Frank 0.400 1.000 0.400
Larry 0.400 1.000 0.400
Mark 0.200 0.800 0.200
Chip 0.400 1.000 0.400
Marie 0.400 1.000 0.400
Dwight 0.400 1.000 0.400
John 0.400 1.000 0.400
Orley 0.800 0.800 0.800
Rob 0.800 0.800 0.800
Bruce 0.800 0.800 0.800
Paul 0.800 0.800 0.800
Jan 1.000 0.400 1.000
Roman 0.400 1.000 0.400
Bronwyn 1.000 0.400 1.000
Pairwise correlations in descending order
1.000 Chip and John Significantly positive
1.000 Frank and Larry Significantly positive
1.000 Larry and Chip Significantly positive
1.000 Larry and Marie Significantly positive
1.000 Larry and Dwight Significantly positive
1.000 Larry and John Significantly positive
1.000 Chip and Roman Significantly positive
1.000 Rich and Orley Significantly positive
1.000 Rich and Rob Significantly positive
1.000 Rich and Bruce Significantly positive
1.000 Rich and Paul Significantly positive
1.000 Larry and Roman Significantly positive
1.000 Rob and Bruce Significantly positive
1.000 Rob and Paul Significantly positive
1.000 Orley and Bruce Significantly positive
1.000 Marie and Roman Significantly positive
1.000 Frank and Chip Significantly positive
1.000 Frank and Marie Significantly positive
1.000 Frank and Dwight Significantly positive
1.000 Frank and John Significantly positive
1.000 Orley and Rob Significantly positive
1.000 Orley and Paul Significantly positive
1.000 Marie and John Significantly positive
1.000 Dwight and Roman Significantly positive
1.000 Chip and Marie Significantly positive
1.000 Frank and Roman Significantly positive
1.000 Chip and Dwight Significantly positive
1.000 Jan and Bronwyn Significantly positive
1.000 Marie and Dwight Significantly positive
1.000 Dwight and John Significantly positive
1.000 John and Roman Significantly positive
1.000 Bruce and Paul Significantly positive
0.800 Rich and Chip Not significant
0.800 Larry and Mark Not significant
0.800 Chip and Rob Not significant
0.800 Frank and Bruce Not significant
0.800 Orley and Roman Not significant
0.800 Rich and Dwight Not significant
0.800 Rich and Larry Not significant
0.800 Frank and Orley Not significant
0.800 Mark and Marie Not significant
0.800 Mark and Dwight Not significant
0.800 Mark and John Not significant
0.800 Rich and Jan Not significant
0.800 John and Rob Not significant
0.800 Rich and Bronwyn Not significant
0.800 Bruce and Bronwyn Not significant
0.800 Frank and Rob Not significant
0.800 Mark and Roman Not significant
0.800 Paul and Bronwyn Not significant
0.800 Bruce and Jan Not significant
0.800 Bruce and Roman Not significant
0.800 Rich and Frank Not significant
0.800 Chip and Orley Not significant
0.800 John and Paul Not significant
0.800 Chip and Bruce Not significant
0.800 Marie and Rob Not significant
0.800 Dwight and Orley Not significant
0.800 Rich and John Not significant
0.800 Larry and Rob Not significant
0.800 Paul and Jan Not significant
0.800 Larry and Paul Not significant
0.800 Marie and Orley Not significant
0.800 Orley and Bronwyn Not significant
0.800 Rich and Roman Not significant
0.800 Marie and Paul Not significant
0.800 Rob and Jan Not significant
0.800 Frank and Mark Not significant
0.800 Rob and Bronwyn Not significant
0.800 Rich and Marie Not significant
0.800 Chip and Paul Not significant
0.800 Dwight and Rob Not significant
0.800 Dwight and Bruce Not significant
0.800 Dwight and Paul Not significant
0.800 Paul and Roman Not significant
0.800 Frank and Paul Not significant
0.800 John and Bruce Not significant
0.800 Marie and Bruce Not significant
0.800 Mark and Chip Not significant
0.800 Rob and Roman Not significant
0.800 Orley and Jan Not significant
0.800 Larry and Bruce Not significant
0.800 Larry and Orley Not significant
0.800 John and Orley Not significant
0.400 Chip and Jan Not significant
0.400 Frank and Jan Not significant
0.400 Chip and Bronwyn Not significant
0.400 John and Bronwyn Not significant
0.400 Larry and Jan Not significant
0.400 Dwight and Bronwyn Not significant
0.400 John and Jan Not significant
0.400 Frank and Bronwyn Not significant
0.400 Marie and Jan Not significant
0.400 Rich and Mark Not significant
0.400 Marie and Bronwyn Not significant
0.400 Mark and Orley Not significant
0.400 Mark and Rob Not significant
0.400 Mark and Bruce Not significant
0.400 Mark and Paul Not significant
0.400 Jan and Roman Not significant
0.400 Dwight and Jan Not significant
0.400 Roman and Bronwyn Not significant
0.400 Larry and Bronwyn Not significant
0.200 Mark and Jan Not significant
0.200 Mark and Bronwyn Not significant
Return to previous page