By Finlo Rohrer BBC News Magazine

A publishing giant got into hot water after offering gift vouchers to anyone who would give their textbook a five-star review. It raises one of the key questions in online purchasing, how much can you trust the customer reviews you read?

In the US, it is called "shill reviewing".

You are the owner of a company whose product - Brand X Widget is struggling. On a major online retailer, Brand X Widget has been given a slew of one-star reviews by customers.

How tempting it is to tell all your employees to log on, without revealing their allegiance to the company, and dole out some five-star reviews. Slowly but surely Brand X Widget's star rating rises.

OTHER TERMINOLOGY Amazon bombing: Concerted effort to change Amazon sales rankings by simultaneously buying product Sock puppetry: The act of creating a fake online identity to praise, defend or create the illusion of support for one's self, allies or company (New York Times) Astroturfing: Formal political, advertising, or public relations campaigns seeking to create the impression of being spontaneous "grassroots" behaviour (Wikipedia) Seeding: Process of placing viral marketing such as videos in forums etc

Of course, the customer is the loser.

Read around the issue of these shill reviews and you will see discussion forum melees about books whose Amazon star-rating has mysteriously risen. Everyone knows reviews influence sales.

"If you look at internet retailers one of the drawbacks they have is that they don't have the advice a physical bookshop can give," says Graeme Neill of industry magazine The Bookseller. "[Customer reviews] are almost like a member of staff that you would grab in a bookshop and ask 'what do you think of this book'. It is a good sounding post."

Amazon and other retailers have long recognised the importance of customer reviews. Dedicated reviewers earn status by appearing in the "top reviewers" list, and can even be sent products to review in advance.

It's no surprise that the recent actions of science publisher Elsevier caused a storm. The firm offered a $25 (£15) Amazon voucher to academics who contributed to the textbook Clinical Psychology if they would go on Amazon and Barnes & Noble (a large US books retailer) and give it five stars. Elsevier was quick to disown the actions of its marketing employee and emphasise that it had all been a mistake.

"The company doesn't pay for positive reviews," says Tom Reller, director of corporate relations. "This was a recent employee error. We haven't given out any gift cards under the programme."

POSSIBLE SHILL? Went for a fantastic meal at the Hearty, Happy Miller the other day. This is country pub food at its very, very best (1). The new beer garden made the visit a particular pleasure. A great place to take the kids. And so easy to get to taking the A473 to Trumpton, taking a left at the Blue Man roundabout and continuing for two miles (2). You'll spot it because of the beautiful new beer garden (3). STAR RATING: ***** 1. Gushing praise 2. Advert-like qualities 3. Repeated key phrase

He emphasises that the rogue employee had gone a long way beyond normal publishing practice.

"Encouraging interested parties to post book reviews isn't outside the norm in scholarly publishing, nor is it wrong to offer to nominally compensate people for their time.

"But in all instances the request should be unbiased, with no incentives for a positive review, and that's where this particular e-mail went too far."

Away from publishing, travel review websites also prove a great temptation for shills. If you run a hotel on the brink of bankruptcy you may want to boost your own, or damage a neighbour with allegations of rats, bedbugs and strange smells from the plumbing.

"Obviously, the damage to travellers is to those who rely uncritically on anonymous reviews," says travel writer and blogger Edward Hasbrouck. "It can also reward disreputable travel companies and hotels, and damage the reputations of good ones."

But how can the unwitting customer spot a shill, and what can websites featuring customer reviews do to defend themselves?

Hasbrouck has written on the challenge.

"If there are only a few reviews of a place, assume that there is a good chance they've been planted by friends or foes. If there are many reviews, act like a statistician, and start by dropping the outliers [the most extreme comments]."

In more general terms, watch out for similarities in style between reviews from people with different usernames, particularly if those reviews were placed about the same time.

EUROPEAN LAW Commercial practices which are in all circumstances considered unfair: 22. Falsely claiming or creating the impression that the trader is not acting for purposes relating to his trade, business, craft or profession, or falsely representing oneself as a consumer Unfair Commercial Practices Directive 2005/29/EC

If a series of reviews has been inspired by a manager or owner pushing a particular agenda, you may see variations of the same phrase cropping up again and again - a reference to a new conservatory at the hotel, or the all-you-can eat buffet at a restaurant.

Watch out for people who are new users, or whose only reviews are on the same item. And a sudden wave of five-star reviews after a more sustained run of lower ratings should provoke suspicion.

For those who run the websites, they can spot the cruder shilling attempts if they emanate from the same IP address.

The big customer review websites say they have robust policies to combat shilling.

"Amazon works hard to maintain the integrity of its customer reviews," a spokesman says. "We have very clear guidelines, and when a customer reports a review that they feel is inappropriate, we investigate, and may (or may not) take it down."

The travel reviews website TripAdvisor is also on the case.

"Every review is screened prior to posting and a team of quality assurance specialists investigate suspicious ones; we have proprietary automated tools that help identify attempts to subvert the system; and our 25 million monthly visitors also help police our content.

"When a review is suspected to be fraudulent, it is immediately taken down. Further measures to penalise those attempting to game the system… can include sending a warning letter to the property owners and posting a notice on TripAdvisor to warn travellers that a particular property is being investigated for potentially fraudulent activity."

Shilling damage

But, of course, it is in the interests of the websites to say they have a handle on shilling, says Hasbrouck.

"Knowledge of shilling also damages the business of review web sites, who therefore have an interest in reassuring the public that they have eliminated it."

Ultimately, the sheer weight of numbers of the honest users may be the best defence.

"One of the things Amazon have is a question did you find this review helpful - reviews are ranked," says Neill. "I wouldn't underestimate the humble customer's ability to distinguish between what is a gushing press release and a genuine 'I felt this book was fantastic'."

And of course, there are those who would say the reliable customer reviews can only be maintained in parallel with professionally produced content.

"In terms of travel information there is so much online it can be overwhelming," says Anna Paynton, who works on the Rough Guides and the website TravelDK, which features customer reviews. "How do you wade through it? And how do you know what to trust? A lot is posted by businesses and by friends of the businesses. That's why having a guidebook is so important. It's been researched by travel writers and researched by editors.

"Most people know that user generated content is going to have an element of unreliability. We do check but the problem is that there is so much it is very difficult to really monitor it closely."

So perhaps we should rely on our own shill-hunting abilities.

Send us your comments using the form below.

I glimpsed first-hand what a bad review might do when I was one of the first to buy a 3G phone, a few years ago. Very disappointed with it, I wrote an honest review on a website and a week later saw that 2000 people had viewed it. Assuming a large portion of those would-be customers would not have bought that phone afterward, I estimate that one review carried a potential loss of revenue upward of £180,000 - from one disgruntled customer. Power to the people!
John, Bristol

Ballot stuffing has always been an issue with online reviews however it's generally easy to spot a corporate job because all the bogus ones will be coming from a small range of IP addresses. As for people offering vouchers/Incentives to give a good review they are far more likely to be reported for it. Bear in mind there are still people out there with principles much as the tabloids would have us believe otherwise.
Big Ian, London

I see this kind of thing on sites like Amazon and TripAdvisor all the time. You can spot the ones on Amazon easily enough as they are usually reviewers who only ever review a single product. Amazon/TripAdvisor should flag this fact up and not count ratings by reviewers until they review say, 5 products.
Vinny Hassell, Liverpool

My first experience of shilling (didn't even know there was a word for it!) was reading the reviews for a film on IMDB a few years ago. The reviews were uniformly bad, until I reached a cluster of reviews giving the film 10/10. All the positive reviews had been posted within a few days of each other, and I found that not one of the reviewers had posted a review for any other film. Needless to say, I didn't bother watching the film.
Paul, Southampton

As well as shilling, some sites are very bad for pre-release reviews. A product is given five stars because "it's by X and their last was brilliant". Why do Amazon and others allow this?
Geoff, Southampton, uk

This isn't just a problem with fake reviews, the whole user review process is deeply flawed. Look at reviews on Amazon of products and you will find many with either one or five stars, and nothing in between. I have seen comments where people have given a product one star just so their review stands out, or even reviews of things that have not even been released, or they admit to not even having bought.
Tom, Maidstone, UK

I buy a lot of computer equipment online. As an experienced user myself I do read the reviews, however I also make judgments about the person who wrote the review; for example if I can see they've given a low rating due to errors on *their* part I will ignore their rating. It is harder to do that with high ratings, however I look for reviewers who demonstrate some relevant knowledge of the product they are reviewing; therefore "Go here it's great" or "buy this it's cheap and works!" comments get ignored while those who justify their comments relating them to aspects of the product get my attention.
Julian, Barry, S Wales

What this article doesn't mention is the editorial process of moderated reviews and comments and how the bias of an organisation can be pushed through the selection of favourable comments. When comments are pre-moderated how are users to be sure of the unbiasedness of the site's editors?
The_Reveller, London

I don't think there will every be a way to truly stop fake reviews. I work for FreeIndex and we've tried many different tactics to prevent this from happening but some still get through. The best thing that can happen is for the public to become educated and know how to spot the fakes.
Stephen Bradley, Bristol, UK

It's worth noting that in the UK, the Consumer Protection from Unfair Trading Regulations 2008 would prohibit this type of activity (under general prohibitions on unfair practices, and also because falsely creating the impression you are a consumer, or hiding the fact you are acting in the course of trade is specifically prohibited in all circumstances). That is not to say that it doesn't take place, but brand owners and their marketers should be aware that they can face criminal penalties for this sort of action.
PS, London

Have certainly spotted shilling on the TripAdvisor website. While looking into a hotel in Sardinia, my girlfriend and I saw three reviews giving very similar glowing praise and in VERY poor English, plus a large number of well-written but far more critical feedback. We didn't book there.
Stu, Reading, UK

I took some important clients to a restaurant in Manchester that had excellent reviews on a food website. The service was awful and we were ignored most of the time. I wrote a review the was factual not emotional and submitted it to the same site to be told that the review site owner would not accept any "bad" reviews only the good ones. Yet nowhere did he say this on the site, therefore giving misleading impressions of the restaurants listed.
Louise Wareing, Manchester

The BBC may edit your comments and not all emails will be published. Your comments may be published on any BBC media worldwide.

E-mail this to a friend

Printable version