Tuesday, May 27, 2008

Ratings and Recommendations

There are so many articles on the topic or rating and reviewing wines that I hesitate to write another one, but one thing I have discovered in my brief research is that almost all of it is based on attacking or defending the 100 point scale.

In fact, in my very first post on this blog (back in June 2006) I said:

This is not a site about wine tasting notes, collecting and investing in 'fine' wines, ranting against the 100-point systems and a certain reviewer (although it may come up from time to time) or matching it with food. When I say "Wine Culture" I am thinking of how the vine, its fruit and the fermented by-product has played some role in our lives for thousands of years, and how even today this agricultural product is present in our digitised, mechanised and hectic lives. Just as well.


However, the European Wine Bloggers' Conference is encouraging us to discuss the issues that ALL wine bloggers face, and this is most certainly one of them. One of the latest posts by Ryan Opaz is:

To Rate or Not to Rate - That is the Question

There is a high likelihood that if you have a wine blog you will be tasting wines and writing up the results in some manner (there are a few weird sites like mine that don't, but we are not common). If so, should you be rating them for your readers?

One argument, for example here on Fermentation, is that wines ought to be judged on some objective criteria, whatever these may be, and if we fail to establish what these are and how they are measured, then the whole business of rating is worthless.

On the other hand, companies like Wine Spectator, rely on a fixed scale to be able to classify the thousands of wines they deal with.

But where do bloggers come into this?

One of the themes that keeps coming up in my thinking about wine blogs is that the majority of them are NOT objective writers trying to be "journalists", but rather they are, and should be, individuals with opinions looking to entertain. This means that we bloggers should be encouraged to say what we think, whether we are experts on the subject at hand or not, just as we would in a conversation with friends, not writing a piece for Encyclopedia Brittanica.

Wine reviews in this context are not supposed to be "objective" reviews. You don't go round talking to your friends about that 88 point wine you drank the other night, do you? In fact, these are a form of word of mouth recommendation and as such a recommendation should be simple; buy it, or don't buy it!

If bloggers could agree on this as the start of a common recommendation system, we would have a way of cooperating and make wine blogs a more influential voice for consumers looking for recommendations and information (as discussed recently on 1 Wine Dude).

And what would this scale look like? It could be simple binary: yes/no; toast/bury; etc. but this is a little too simple even if it is a model that works elsewhere.

I would like to throw out the following for discussion, and I'm sure someone, somewhere, is already doing this, but unfortunately I have not come across this*:

+2 I loved it and would really recommend you buy this
+1 It was OK and largely worth the money
0 OK, reasonable but not memorable and there are probably better options for your money
-1 A bad investment of time and money.
-2 I hated it. Avoid at all costs.

Points to note:
1. It uses +/- numbers so that the aggregate reviews can be summed, and strong feelings have more impact than lukewarm ones, after all there is a difference between a recommendation that sounds like "Yeah, buy it I suppose" and "go out and buy all the bottles you can, now!"

2. It is based on no more than personal opinion at that point in time (taking into account cost, perceived value, quality, context, etc.)

3. It gets around the issue of the "great value" wine that only scored 87 points, compared to the exhorbitantly expensive 93 pointer. Which would you rather recommend more to your friends?

4. It could be refined further, but ought to remain simple

Most bloggers I know who rate wines point out that this is just their point of view and you need to read the rest of the text about quality and context anyway. They also point out that it is only a reference point and that once you get to know them, you can get a feel for whether you have similar tastes, and therefore whether you could use their ratings as a guide to your own shopping.

In this case, let's ditch the intermediary step and go straight to the heart of the matter - buy it, or don't buy it.


* this is not completely true as I read a blog some years ago that had something like this but I cannot locate it any more as the conversation about 100-point scales drowns out all other discussions. Please send me any suggestions for where to find such scales.

18 comments:

Peter F May said...

The rating system I am most comfortable with is the 7 point scale used for the past 30 years by the Central London Wine Society.

Scoring 1 = Faulty, 2 = Poor, 3 = Barely acceptable, 4 = Acceptable and above, 5 = Good, 6 = Very Good, 7 = Fantastic

(why score 1 for a faulty wine? Because scoring is by show of hands at the end, so if someone gives it a 1 they must explain what the fault is.)

I've used the 20 point scale in competitions and the 100 point system at CellarTracker, but in the 20 pointer basically it was whether the wine was a gold, silver, bronze or not, thus only 5 points mattered, and in the 100 point scale, well its only the top 15 that matters...

Wulf said...

Here is my scale:

1. Bleugh! Not a wine I would willingly drink again.

2. Meh! I can take it or leave but given the choice would leave it.

3. Mmmmnnn. Pleasant enough but not a choice I would go out of my way for.

4. Tasty! I would happily buy more of this and will also look out for similar choices.

5. Fantastic! An excellent drink and one that I would gladly return to.

I've used 1-5, as that lets me easily mark up the review using the hreview microformat, which may become a useful tool for aggregating information.

I am of course marking (and drinking) unashamedly for my personal taste and as a guide to my future explorations.

Robert McIntosh said...

thanks Peter, the Central London Wine Society one is interesting (can't remember seeing a 7 point wine rating scale before)

the point with the others is moving away from supposedly objective ratings and medals and focusing on the personal evaluation and recommendation of a blogger, and doing so in a way that it can be aggregated across different bloggers

Taster B said...

I like the binary 'buy it'/'don't buy it' system you suggest. It give the most flexibility to those of us who would prefer to express enthusiasm with prose rather than numbers. :-)

Robert McIntosh said...

thanks Wulf - I'm not familiar with hreview microformat, where can I get more info?

I think your scale matches mine very well, except for the idea of -2 to +2 instead of 1-5.

As I say, I don't think I'm proposing anything radically different, but only that a simple, personal, subjective scale might be easier to share than a more complicated 0-100 (or 20)

Richard Auffrey said...

My own system has 3 choices:

1) Drink & Buy: Wines I consider worthy enjoy to recommend to others to buy. Tasting notes will then mention if the wine is exceptional, an excellent value, or some other special item.

2) Drink, Not Buy: These are wines I would drink but probably would not buy. It often is because the wine is not a good value, too overpriced for what you get.

3) No Drink, No Buy: This is not a wine I would recommend at all, either to drink or buy.

Andrew said...

While I applaud the use of something different - I've not seen a scale using a +/- before - I just wonder why you don't use a 1-5.

I use a unique system 100 point system that can be translated to the 1-5 (which I do) but I have to use .25 and .5... nothing is perfect.

Anonymous said...

5pts for all! Really we might just talk about moving away from the score all together again. Lately I just don't see value in the score, and a +/- really doesn't say much to the reader.

Robert in your proposal wouldn't 0 be neutral, and like our 3 mean "Perfectly fine wine, though nothing interesting" Then the 1 and -1 etc are for wines that go above or below "correct" wine.

Taking this, most wines in the world will rate at 0 or above, thus negating 1- and 2-...I find that we as bloggers, and even mainstream journalists, very rarely write about wines we don't like. I mean who really wants to talk about wines that are flawed? Corked wines definitely should not be talked about as the bottle is flawed, and a second tasting should be made. Unless you see a trend in one producer, flawed wine probably should not be tasted/rated. Bad wine, well, why waste my time drinking/writing about bad wine!

I starting to think I like the Michelin Guide idea - 3 stars for quality(though I would lower the entrance barrier for one star) and then only mention other wines of interest without awarding a star if they are still correct. Thus you only have high quality wines with any stars at all. hmmm I'm starting to realize as I type this that it seems to make the most sense to me...uh oh

BTW on the hReview microformats, talk to Andre from Adegga...

Gabriella Opaz said...

Andrew, I tend to agree with you that the +/- system for me tends to get a bit confusing.

For me, I am torn. On one hand I like our 5 grape system. It's easy, simple and functional. And when the moment calls for me to place a quantitative judgment on a wine, it works.

However, there is also the issue of experience, and can a point encompass all the external, and internal, stimuli that contributed to my take on the wine. When I'm out with friends, I might find a wine I typically dislike, really enjoyable. Same holds true with food. There have been several times when food has made a mediocre wine, fabulous.

As you mentioned Robert, the debate seems to come down to intention. Why, and for whom, are you rating a wine?

For me, wine bloggers are gearing their writing for a dynamic and interactive conversation, not a sterile analysis of the wine. So why not find a common thread to bring us all together, combine our "wine blogging" power ;-)

Robert McIntosh said...

Thank you everyone for your comments.

I have not completed my internal debate with regard to the +/- but I sense that there might be benefits to the ability to sum as well as average scores, and anyway, I was trying to add a new twist to the discussion.

Is there a mathematician or statistician amongst us that can advise as to whether there are benefits to be derived from this?? I feel there must be but can't work it out.

On the other hand, I don't use ratings myself so I realise I am possibly not qualified to say how they would work in practice.

As for not reviewing bad wines, I think that in the context of what I am saying, i.e. word of mouth, that is wrong and a waste of an opportunity.

If you are a consumer faced with 400+ SKUs at your local Tesco it is worth knowing what wines NOT to buy as well as which ones you SHOULD buy, don't you think? Lack of reviews could mean it might be worth buying but hasn't been reviewed yet.

I can understand that a journalist that is reviewing samples or a Press visit may accentuate the positive/interesting/useful as time is limited, but most bloggers are writing up their personal experiences, and a bad one is as useful as a good one. That is exactly what happens with restaurant reviews isn't it?

Robert McIntosh said...

.. and Gabriella, you remind me about a point I considered but failed to deal with.

Some bloggers may object to the idea that reviews are intended as calls to action: i.e. "Buy!"

Many would argue that it is simply a case of "I liked this" or "I didn't like that", and whether or not people agree and do the same is their issue.

This is the point about context - "I liked that wine (at that moment, in that context)" which does not necessarily map directly onto "Buy It".

However, if we could agree to standardise our reviews with a final, separate Buy/Don't Buy or 1-5 buy rating, then it could be useful?

Anonymous said...

The question though Robert is, would you walk into a store, and not buy a wine based on a point score? Would a 2star rating from us(knowing that this is below average) stop you from making a purchase?

I don't think so, since you are a wine lover. If the wine was a style that you typically liked, you may still buy it to see if you agree.

Who are points for? I"m looking at Peter May's score system and I really want to know, what is the difference between "Poor" and "Barely acceptable"?

Would one lead to a purchase and the other not?

Andrew said...

Another point - if we are reviewing wines should we really bother to detail low rated wines? Should we not concentrate on the good stuff? I for one do not have the time, generally, to write up low scorers.

My rating gives an average from 1-5 in drinkability, interest, value and enjoyment. Are these not the crux of what we are trying to impart in a 'score'?

ie 'Drinkability 3 Interest 3 Value 4 Enjoyment 4 ' Scribblings Rating - 88/100 [?? out of 5]

Wulf said...

The hreview format is one of a number of microformats that various key players on the web are experimenting with. It is a set of standards for marking up a review so that not only is it readable to a person but a computer can extract key information, such as the product being reviewed and the score given. That then allows all reviews published on the web to be treated as an open database.

It also improves visibility. Do a search on Google for durius tempranillo and you will find the review I posted yesterday already on the first page. Add the word review to your search and it comes higher (trying just now, it hit second place) - I think it is quite impressive that my tiny little blog that writes about wine once in a while can do that and come second only to corkd.com!

I suspect the magic is not my writing style but my mark-up acumen and that using the hreview microformat is largely responsible. Therefore, it is a demonstrably useful solution to the question of how to post impressions of wine that have some influence.

It might even be possible to set up a system that grades other reviewers based on how much their published reviews agree with yours, weighting their opinions accordingly and thus, by promoting the scores of those who seem to have similar tastes, giving more chance of going straight to what you would call "the good stuff" while still recognising the important point that tastes differ. That would be valuable because, the more you put into sharing your own views, the more "intelligent" the system would seem to become.

Peter F May said...

Wulf - is there a Blogger pug in for generating the code for hreview?

The generator at http://microformats.org/code/hreview/creator doesn't work

Wulf said...

I don't know about a plugin. I'm a web developer by trade, so I marked up my first reviews by hand and now copy and paste them on as a template.

There is not that much code to work with so I think this is almost as easy as any plugin could hope to be, as long as the though of working with HTML-like markup is not too daunting.

Jeanne said...

Speaking as a very occasional wine blogger, this is a very interesting discussion. I find the 100 point system a distraction and let's be real, for most of us, what's to distinguish an 88 point wine from a 89 point wine? Far too many points to play with!

If I do a numerical rating (and I have in the past) I tend to use a simple 20 point scale which is what I was taught by the Cape Wine Academy (3 for colour, 7 for nose and 10 for palate), but that's more from force of habit than a feeling that all reviews must be quantified.

I think that if your intention is to give the average consumer of a bottle of wine an indication of whether or not they should drink, buy or stock up on this wine, a simpler system is the only way forward. Five points (however you delineate them) is good as it gives you scope for expressing slight and strong dislike, slight and strong preference, and indifference.

For what it's worth, as a food blogger I grapple with the same issues when it comes to restaurant reviews. I dislike the star system and prefer ratings for individual attributes like service, food and value. But again, if you are doing this to advise the general public, some sort of simple rating system is needed, which is why the star rating system or something like it has survived.

Anonymous said...

Hi Robert, I think you can use the hReview Creator at http://microformats.org/code/hreview/creator to generate a review that is compatible with hReview format.

I should also say that we're adding hReviews to all tasting notes on Adegga.com and in a way we're making all of them available under a common format.

We support 4 different rating scales (1-5, 1-10, 1-20 and 100 points) but to get an overall picture of a wine we give it in the 5 point scale. We decided to do this based on the fact that 5 points is probably the most used rating scale everywhere. It's at the same time flexible (5 different values, 10 if you use .5!) and simple (easy to read and identify).

I think it's important for wine bloggers to create reviews using their own scale / system but in the end I agree that we have to find a common format to make it easier for other people to follow.