In October, the Tea Review Tasting Panel — a diverse group of highly respected tea cuppers and industry professionals — participated in the first of a series of group cuppings aimed at developing a substantial degree of consensus on assigning a numerical score to a traditionally subjective sensory experience. This cupping was the first step in a process that will help pioneer 100-point wine-style reviews in the tea industry.
For more information about the broader process, visit our post Quantifying Quality: Establishing Standards and Scoring Methods for 100-point Tea Reviews. Separately, we describe the Tea Review scoring methodology and how to interpret overall scores.
We’re pleased to share the results of our first group cupping [Download Results]. Each member of the Tasting Panel was sent 16 tea samples — four white, four green, four oolong, and four black — to cup and score on a blind basis. We used the Tea Review Cupping Form to capture our sensory observations and score the teas.
In the cupping results document, the sample ID #’s appear in the far left column, followed by tea name, type, and origin. The top-level scoring results are as follows:
- The average score for all samples by all cuppers was 84.7 points. Obviously, there is no right answer but this seems to be reasonable (perhaps at the low end of reasonable) for a set of samples that were expected to range from fair to excellent using a scoring system none of the Tasting Panel members had used before.
- The average high score by all cuppers for all tea samples was 92.7 points (i.e. “very good to outstanding,” according to our qualitative interpretations.
- The average low score by all cuppers for all samples was 2 points (i.e. “not recommended” according to our qualitative interpretations.
- The average score for the nine cuppers ranged from a low of 81.5 points (fair to good) to a high of 87.9 points (good to very good).
In short, in aggregate, the scores seem to be within the range of reason.
However, we do see a significant spread of scores (high score minus low score) among cuppers for several of the tea categories as well as individual tea samples:
- The highest category spread was an average of 20.75 points for white teas
- For oolong teas, it was 17.25 points
- For green teas, it was a modest 10.875 points
- The smallest average category spread was for black teas at just 8.0 points
The spread in cupper scores for individual tea samples ranged from a low of 5.5 points (93.5 – 88) for sample 104 (88th night – first flush green tea) to an extreme 35 points (93 – 58) for sample 202 (Yunan Silver Needle white tea).
Our immediate next step is to focus on areas where we have the greatest degree of consensus on scores, namely black teas. We are in the process of organizing our second cupping, which will focus on a range of black teas. We plan to include the Ceylon OP Sri Lanka black tea, which had a spread of just six points in cupping #1, as a benchmark tea in cupping #2.
We also in the process of drafting a complete sample review of the Ceylon OP tea, which will include a consensus overall score as well as category scores, notes on appearance and liquor color, and a blind sensory assessment, and notes about the tea.
