Well, it’s that time of year again. Seriously, this time last year I had my turn at asking my fellow Whisky Knights a whisky-related question. If memory serves, and it does, the question posed last year was regarding how one developed their whisky tasting “nose” & “palate”.
This time around, I wanted to discuss something that’s been on my mind for a while: Whisky Rating systems.
As a reminder, the valiant knights (and links to their blogs) of this round table are:
Chris – Nonjatta
Keith – Whisky Emporium
Karen & Matt – Whisky For Everyone
Ruben – Whisky Notes
Mark – Glasgow’s Whisky (And Ale)
Neil & Joel – Caskstrength.net
Lucas & Chris – Edinburgh Whisky Blog
Jason – Guid Scotch Drink
Gal – Whisky Israel
Mike – Whisky Party
Peter – The Casks
Joshua – The Jewish Single Malt Whisky Society
Without further ado, I bring you, my question:
Whisky ratings. We all rate whisky and we all do it in a different way. Many of us, as well as other bloggers, critics & whisky writers, have our own technique. The most popular being the 100 point system. Some have a ten point system or a star system. Then again, some do not use a system at all. What value do you see in a rating system? Do you use someone’s rating to help you choose a whisky to buy? To take it a step further, do you feel a 90pt Ardbeg is equal to a 90pt Wild Turkey Rye or Bushmills Irish Whiskey? Can we actually pit an American or Irish whisk(e)y up against a Scotch whisky or are they too different to compare? Lastly, what holes are there in the current rating system (if any) and do you see the possibility for a unified system for all to use?
Chris from Nonjatta — As i think I have said before I am sceptical about the possibility of words capturing the reality of a taste with any accuracy. I am even more sceptical about capturing taste on any sort of number scale. If you think about it, it is really quite absurd. It is then merely a way of expressing approbation or disapprobation, and the whole idea of having a systematic system for judging the merit of whisky on a 100 point scale, in which a 91 Macallan is 1 point more good than a 90 Bourbon reviewed 6 years before, seems to me to be deeply problematic if it is being approached with anything other than a slightly whimsical wish to express oneself. Even if it were possible, I am pretty sure trying to maintain such a critical superstructure in my head and on my tongue over years through all sorts of environments would annihilate my enjoyment of whisky. That said, ratings are very useful for the reader. They are clear and force the writer to come down on the real issue that counts for many people: Which one should I buy? So, I do rate whisky on a simple star system on my blog and a 100 point system has advantage for the reader in that it allows even more clarity on the hierarchy of whisky being presented by the writer. Jim Murray’s Whisky Bible, for instance, would not work on a five star system because it would create massive monotony through the book. So, it seems to me that ratings are impossible but demanded. It is a bit like the stage magician who chops his assistant in half: It is impossible, but there is a willing suspension of disbelief by the audience and everyone goes home happy.
Chris Bunting
Nonjatta
Keith from Whisky Emporium –Hi Joshua and thanks for a very interesting and indeed quite topical question, or even five questions if the fingers on my left hand don’t lie 😉
So, let’s get this over with quickly; 100, difficult – sometimes maybe, yes, yes, why?
OK, is that good enough for you?
No, you aren’t getting away that easily from me with this one!
Firstly, you are correct that many people use different rating systems, some don’t even use any if you’re talking point-wise, so let me try to look at this from my own personal point of view and, in doing so, maybe, just maybe go some way to answering your question(s):
I currently have over 700 whisky reviews on Whisky Emporium and although I score each sample I try, you’ll have to look very hard to find any scores on the site. In fact there are less than a dozen and if I remember correctly they are all to be found on my Aberlour page where I compare A’Bunadh batches. Not publishing my scores on the website is obviously a conscious decision as I prefer people to look at what I write here and not what I score. I make it obvious if I like something. I make it equally obvious when something pretty bad comes along like Loch Dhu-gly. I also very often refer to some whiskies as being ‘greats’ or not quite so, but still excellent.
For me, a ‘great’ whisky is one which I have scored at least 90 points and as I tend to err towards conservatism in my scoring, these are quite rare.
Having said this you will immediately recognise that I use a 100-point system and yes, I use the full scale of it. One often hears complaints that whisky scorers only ever use a fraction of the scale as everything tends to be scored 75+. Well, so far I have managed to use most of the range from 9 to 96 out of 100. Just for your information; Loch Dhu-gly was the 9 and a never bottled cask sample of Dalmore 40y at Cask Strength was the 96.
My own scoring method is not very scientific, I have no sub-sets of scores for different aspects of the whisky, for me it is just a ‘marker’ or indicator of how much I enjoyed the whisky. In this way I see no problem in comparing different styles; whether grains, single malts, blends, Scottish, Japanese, Irish, bourbon ….. whatever. So yes, using my method it is perfectly possible to compare completely different types of whisky. In fact I often do and if you see me score a grain whisky, or perhaps an Irish whiskey at 88 points, then you’ll immediately know it’s every bit as good as a Scottish one with the same score.
Does the whisky world, the whisky drinking public or even the occasional buyers looking for celebratory drams need a unified scoring system? Goodness, wouldn’t that make the (whisky) world a boring place?
Sláinte Mhath,
Keith
Gal from Whisky Israel – Well that’s a tricky one 😉
Rating rating. can’t live with them, and can’t live without them , eh?
I think at some time or another any blogger or whisky lover will debate this issue. How do you rate a whisky? Should we rate a whisky?. When i first started blogging about whisky i chose not to rate whisky by numbers or letters (A,B,B+) and just to give my impression of it. Then after some time when i reviewed more and more whiskies, it got very tough to remember which ones i liked more than others, since “this is a really good whisky” or “this whisky is a fine work of art” are not clear enough when you come back to them later. Also, some blog readers did want to know how much i liked a dram. Also, some people are looking for a “what to buy” and what not (if they are seeking presents ideas, or just do not have the time to read your lengthy notes”.
In my opinion we need a two grade system (as some use, not me yet, but i’m getting there soon) : First : a subjective grade from 0-100, which says how good is the whisky in your opinion regardless of its price/availability. Then a second grade for Bang for your buck. Let’s say you rate an Ardbeg 1974 and it’s amazing, surely it will get a 90+ grade. Now, it costs an arm and a leg, so the 2nd grade should give it a 3/5 ratio for example. A lagavulin 16 which is one of my all time favorites (and a good value!) or the Aberlour Abunad’h will get say 90 / 88 respectively, but their BFB grade will be 5/5. It’s obvious you can not beat an old Ardbeg with an affordable Nikka From The Barrel,but hey, if you are on a budget (almost all of us are) and we can not buy every older and amazing expression, the 2nd grade can help you buy good stuff, for a price you can afford.
Now, the 0-100 system, although better than the 1-10 system, still has many flaws. and Indeed, it’s hard to compare a 90pt Ardbeg, and a 90 pt irish whisky. Having said that, I do think that every 0-100 should reflect how this whisky is among its group . when judging an Irish whisky, we do not look for the same characteristics we look in a peated Islay. but the overall, how it all connects, should be on the same scale.
Holes in the system? many.
1.How can you decide if you give a whisky 84,85 or 86? they are pretty close…
2.My 100 is not your 100. Some people look for different things in a whisky. While i may find baby vomit to be attractive in some whiskies, some people will be disgusted by it, and rate it much lower…
3. Whisky tends to grow on you, many times i taste a whisky a few times and then after going thourgh the bottle i like / dislike it more. so tasting notes are prone to change.
I could go on and on, but i don’t think we have something better (other than the 2 grade system) right now. A unified system? no chance. We are all so different, and experience whisky in different ways. There is no chance to unify it. But, when you do find a whisky “rater” that you find close to your palate and preferences, stick with him. If you disagree with someone’s grading system, just admit you to are too different, and give others a chance. We have different noses, and palates. And that’s what beautiful about it.
Slainte!
Peter from The Casks — I’m constantly bemused by how riled up people get about any kind of scoring system…and increasingly frustrated by my own system. Personally, I like scoring/grading, I’m always glad to see it, it helps me to better understand what someone thinks of a whisky. For readers, I see a scoring system as just another way of qualifying a whisky, basically just another tasting note with the added bonus of relating to other whiskies reviewed in the same fashion/by the same person. I would never buy a whisky just because I saw a random score by some reviewer I rarely pay attention to, a grade on its own is fairly meaningless. I would, however be more inclined to buy after reading a trusted source who gives favorable notes and a high grade. To me, that seems a pretty obvious modus operandi, yet people get their underwear in an uproar over scoring…I don’t get it.
For myself, I find it a valuable tool for cataloging and comparing my reviews, giving myself a constant, consistent baseline that I refer to…at least in theory. The problem I have with my own system (and one I bet most people have with their own) is that even though I thought I started with a broad baseline, I’ve now encountered far more whiskies than I ever expected to and am finding that baseline a little narrow and the interval of quality not quite consistent. In my head, the difference between an 83pt whisky and an 85pt one is perhaps the same as an 85pt and a 90pt. I’ve basically found many more decent, above average…but not truly great whiskies and they’re cramming that end of the grading curve. I think this is more of a problem just for myself, though, as I’d hope that someone reading my reviews will get a good picture from the entire review and previous reviews with the score being just another qualifier. But for myself, I wonder what to do about including a score, I’ll always keep one for myself, but I do consider from time to time ditching it on posts. Perhaps the best solution would be to go back and re-calibrate/re-score things…I’d be interested to know what people think about that.
As for the Ardbeg – Wild Turkey part of the question, I see no problem in using the same scale for wildly disparate whiskies, be they different styles or price ranges, in fact I think, it’s vital to do so. We all know that age is not an indicator of quality, nor is one style of whisky better than another. Wild Turkey is a great bourbon, but it’s one of the worst Islay Scotches out there, just like Laphroaig is a horrid, horrid Canadian whisky. Each whisky has to be judged and scored on its own merits filtered through the knowledge, experience, and consistent evaluative process of the reviewer.
Ruben from Whisky Notes — Oh no, not that question again. Personally I am at peace with scoring as an easy way to compare things. Of course there are the usual comments: never look at the score alone (and never publish a score alone – if you’re in a rush, provide quick tasting notes rather than a score) and get to know a reviewer’s preferences before using a score as a quality indication (context is important). Given these limitations, I think it’s possible to honestly and rather accurately rate whisky based on a few factors.
In my opinion, complexity, maturity and exemplarity are the most important factors for a high score. As such, Irish or American whiskey (or any other drink for that matter) are perfectly able to get an equally high score, although some of the production methods and ingredients don’t allow for the same complexity as Scotch. I don’t think I’ll ever try a vodka that reaches 90 points. That’s also why an 80 points blended whisky can be named “very good” while an 80 points single malt will be mediocre.
I have two more remarks. Firstly, I think most people start scoring too soon, and this is one of the important reasons why some people think scoring is bad as a whole. Beginners think Lagavulin 16 is as good as it gets. Try 100 whiskies before you start scoring, and mirror yourself to more experienced reviewers, after all we’re trying to be comparable, right? Secondly, don’t hesitate to alter a score afterwards. Your palate is never working in the same way, your mood is never the same, and your appreciation will never be the same. Try to revisit well-known whiskies once in a while, make new notes and give a second opinion for your own rating!
Karen & Matt from Whisky For Everyone — To score or not to score? … hmmm, both systems/styles have their pros and cons and there are good and bad examples of both. The argument seems to keep rumbling on and divides people in to one camp or the other. As a reader, you want things to be laid out as clearly as possible so a scoring system appears the perfect answer, as it tells you this whisky scores 85/100 for example. This makes the whisky good, right? 85/100 sounds good … and it is, but it is only when you learn the nuances of a particular blogger’s scoring system that this may mean that it is good, average or down right awful. For example, someone who scores everything in the 90s/100 would consider 85 a poor mark and therefore a poor whisky, or vice versa. When consistently done well, as is the case with some of our fellow Whisky Round Tablers and other high profile whisky writers, a good scoring system can be a valuable tool and guide to consumers.
The key word here has to be ‘consistency’. This consistency surely comes with increased knowledge of whisky. This means that if a writer gives a whisky 85/100, four stars, 4 drams out of 5 etc today, then they should give the same whisky the same score when reviewed on a different day. It can only be relevant if this system is watertight and if so, then there is no reason why a score system should not work across all genre of whiskies (be it Scottish, Irish, Japanese, American, Swedish, blended, single malt or new make) or across other groups of spirits.
We do not score whiskies for one simple reason – we never have. When we were starting as beginners (both to the world of whisky and the world of blogging), we felt that we did not have the experience to score whisky in a consistent way. Three years down the line, we still feel this is right for us – we enjoy using words rather than numbers or a combination of the two. For us the enjoyment and delight of whisky is in the nuances of each dram – things we personally do not find coming through in a numerical system. We tend to review a whisky in isolation, rather than in groups (where a comparative scoring system could be useful), and consider its merits, always trying to find positives even if there may not appear to be any! Will we ever reconsider and start scoring whiskies using stars or scores out of a hundred? – probably not.
Jason from Guid Scotch Drink– I think the main value of a rating system is providing the whisky consumer with a quick reference point that can successfully guide their next purchase. Once you understand whose rating system fits your palate I think it’s quite easy to go forth and experiment with their recommendations. I’m not sure that every whisky consumer understands this aspect of whisky rating and sometimes there’s a tendency to complain that whisky writer X is crazy or blogger Y knows nothing about whisky. Crack the code and whisky bounty awaits.
I like to say that good whisk(e)y is good whisk(e)y. There was a time when we would never have thought to pit an Indian single malt against a Scotch single malt, a small batch bourbon against an old Irish grain whiskey, or an Australian single malt against an American wheat whiskey. I think those days are behind us. There’s so much great whisk(e)y available now that we ought to be discussing each release’s merits across boundaries.
I have no interest in seeing a unified system. I think that while the 100 point scale remains dominant there’s plenty of room for five star reviews, ten point reviews, and others. The key is that every rater explain their system at some point in the proceedings. My biggest question is always how Ruben’s 100 point scale compares to Serge’s and Serge’s to Michael Jackson’s. With that answer in hand I can move from one review to another and not lose anything in translation.
For the time being, however, I will continue on without a system. I prefer to communicate the merits of a whisk(e)y with an overall comment that tries to sum up the pros and cons without numerical intervention.
Neil and Joel from Caskstrength.net — We don’t rate whiskies. In Wittgenstein’s book Philosophical Investigations, he develops a theory known as the Private Language Argument which shows that each and every person has their own ‘private language’, incoherent to the rest of the world. We believe that scoring whisky takes the Private Language Argument one step beyond in to a realm where the score is only relevant when measured against previous scores (a score can not exists purely in isolation). If something scores a 92 / 100, is this good? Maybe the scorer marks nearly everything in the high 90’s, therefore a score of 92 isn’t good at all. Our approach at Caskstrength is to paint a picture of the style of the malt with our tasting notes. If you like the general direction in which the tasting notes are flowing (“smoky bonfire” or “delicate honeysuckle and ginger”) then you might like to try that particular whisky, if it fits your flavour profile. In our overall section we sum up if we believe the whisky is, in our opinion, good or bad, value for money or otherwise, etc. This seems to work for us.
Cheers
Joel + Neil
Mike from Whisky Party — I think this is a good question, and I think it’s especially important for the likes of us and the rest of the whisky-writing world to think about how we attempt to relate our judgements to the public. This topic has come up (at least briefly) on WDJK, and one of the conclusions I drew was that a more transparent rating system is a better rating system (eg, Jeff’s at Scotch Hobbyist). But it also takes work on the readers’ end; from that perspective, it’s necessary to think about the rater and the context a little bit (especially considering their taste but also if they score on a universal scale or within categories or what– it’s not always clear, so figuring those things out is useful).
At WhiskyParty we gave up on rating whiskies within a few months of starting our blog– I had to stop because I realized I was too inexperienced to be able to compare whiskies across the entire range, and thus the numbers were somewhat meaningless; but if I were to rate a whisky (and whenever I give an informal rating such as “pretty good,” “drinkable,” or “excellent”) it usually has something to do with the ratio of price to deliciousness/distinctiveness/overall impression. We do sort of rate whiskies for the 40 Under $40 series, and it’s a simplified version of that concept on a three-tier scale (‘Yes,’ ‘No,’ or ‘Maybe’ as to whether one should actually shell-out for the bottle or not). It is, of course, the subjective opinion of the reviewer, but we hope it goes beyond the abstract and makes sense to the budget-conscious connoisseur/party hound.
It’s especially hard for me to think about more abstract ratings, though, since I really like almost all whiskies, with a preference for certain styles and production methods more than for specific expressions. For example, what’s my favourite Laphroaig? Well, it depends more on mood or occaisson than on which one I think is somehow objectively better (I might even consider things that don’t always explicitly make it into a rating such as rarity or price). Perhaps after ten or fifteen years of tasting whiskies I’ll have a more systematically derived sense of how expressions compare to one another across regions, price-points, and categories, but for now all I can offer are my impressions.
I’ll end with something I’ve had to think about recently: the Whiskey Comparison Tool from FindTheBest.com is based largely on an “expert” rating system. Granted, they’re just starting out and are still working through some issues, but the whiskies are presented as if the average of one or two expert ratings is equivalent to an average from multiple expert ratings, and the overall average rating of the 287 expressions that they’ve considered is 4.1 out of 5 (so, like most systems, the results are skewed toward the top). What’s more, the ratings are derived from “expert rating sources” such as the LA Whisk(e)y Society and Wine Enthusiast’s magazine– they’re not specific people doing the rating, which makes it hard for me to trust the score (it could come from a different “expert” each time, eliminating the option of “knowing your expert”). Due, in part, to the limited number of expressions reviewed and the unbalanced scoring, something like Rebel Yell (one reveiw, 5/5), for example, outclasses Pappy Van Winkle Family Reserve 20 year old (four reviews, 4.5/5). That seems like a missfire to me. Not to unfairly bash on the Whiskey Comparison Tool– they’re working to improve what could be a great resource for both new and seasoned drinkers. But I think it highlights some of the potential problems with any rating system, especially one that strives to balance out naked subjectivity with aggregate scores.
Joshua from Jewish Single Malt Whisky Society — From the consumer standpoint I can see why a rating system of some sort can be so very important. However, the value of a score to the consumer is more related to the consumers’ alignment with a reviewers’ likes & dislikes. I could LOVE an old Bowmore because I think the soapy notes all fall into place with everything else going on. And, while I don’t score in a numeric sense, I do tend to blather on about how good this or that one is (and vice versa) so a reader might pick up on that, buy and old Bowmore and think it’s pure shite.
Did I fail the reader? No. What we have here is a case where the readers’ likes & dislikes do not match up with mine. So, what’s the solution?
Well, as a suggestion to the consumer, what you might want to do is re-familiarize yourself with any whiskies you have and love; then try to taste them along side the notes you might find from one of us Whisky Bloggers (or the many others out there like: Serge of WhiskyFun, Dr. Whisky, Jeff the Scotch Hobbyist, Oli’s Dramming.com blog, Malt Impostor, etc…). Once you find notes & a whisky rating system that aligns with how you feel about the whisky in hand, you’ve made a match! You may match your tastes up with a few whisky reviews out there (I know I have).
So, in sum, I think a whisky rating system can be very helpful but can also be useless to someone who does not share the sames tastes as the reviewer.
“Do you feel a 90pt Ardbeg is equal to a 90pt Wild Turkey Rye or Bushmills Irish Whiskey? Can we actually pit an American or Irish whisk(e)y up against a Scotch whisky or are they too different to compare?”
While there are some phenomenal whiskies out there, be they Scotch whisky, Irish whiskey, Bourbon whiskey, Rye whiskey, etc…, I feel that you simply can not pit them against each other. Yes, they are all “whisk(e)y” but the differences between them are too great to compare them together.
L’chayim/Slainte/Cheers/Kampai!
For all of the Whisky Round Table discussions, click here.
On the nose — Wafts of juniper; fresh & bright. Grinning a grin and stinking of gin.
Far off scents of the railroad.
Very one-sided with its light quality and bright, sweet citrus notes (think lime leaf and fizzy like seltzer with lime).
Finish — Lots of cereal notes – very malty all of the sudden. Honey covered muesli.
It’s all on dark chocolate covered, raspberry nougat chews (if you can imagine a light version of that).
Big-ass bails of hay.
Fruit marmalades (grapefruit, orange and lime).
Finish — Slightly nutty and tannic (think halved walnuts). Faint notes of rye (?).
