Forbes India – Marketing: The Internet is awash with positive product reviews. Here’s how to decode the good from the great

Through a computational analysis of hundreds of thousands of online reviews, researchers have discovered that positive stars and numerical ratings do not reliably correlate with how well products and companies ultimately succeed Image: Shutterstock

ever had a great meal in a restaurant that had a rating of four out of five stars, but then had a lackluster meal at another restaurant with the exact same rating?

If so, you may have fallen victim to what Kellogg School researchers have dubbed the “Problem of Positivity”: the vast majority of online reviews are positive, but those positive reviews don’t always translate to real-world quality. “You can have two products with four and a half stars, but they’re not equally good, and they’re not nearly as successful in the market,” explains Derek Rucker, Kellogg Professor of Marketing.

Fortunately, researchers have developed another way to help analyze products that look similar based on ratings. In a new paper, Rucker—along with co-authors Laurent Nordgren, professor of management and organizations at the Kellogg School and Matthew D. Roklage of the University of Massachusetts in Boston—found that emotion in the written text of the review has a much better predictor of success than numerical ranking. or its accompanying star.

Emotion differs from valence – whether the feeling is positive or negative – and from the other extreme – how positive or negative the feeling is. Rather, it is how rooted the reaction is in the emotion. “The word ‘cool’ is a very positive word that also conveys a lot of emotion,” Rucker explains. “Perfection” is also a very positive word, but it doesn’t involve a lot of emotion. The same goes for “awesome” in comparison to “perfect” or “terrible” in comparison to “dumb.”

Through a computational analysis of hundreds of thousands of online reviews, researchers have discovered that positive stars and numerical ratings do not reliably correlate with how well products and companies ultimately succeed. But the sentimentality of the accompanying written reviews did. (Although negative reviews are rare, they are actually more helpful; perhaps a one-star restaurant is really bad and doomed.)

It’s an important finding for consumers, business owners and industry experts.

“If you look at the ratings, you can tell what is a bad restaurant, but you can’t really tell what is the best option for you within a group of good restaurants,” Nordgren says. “Analysis of the language people use allows us to understand emotions in a way that the evaluation system does not.”

Why ‘Great’ Movies Outperform ‘Perfect’ Movies

To understand how people’s emotion affects success, researchers began with film reviews.

They have collected user generated ratings from the popular rating site Metacritic. For every movie on the site released between 2005-2018, they analyzed the numerical ratings, as well as the text of the first 30 reviews. (The researchers hypothesized that these early reactions would be less influenced by the film’s eventual success or by the opinions of other viewers and professional critics.)

The researchers used a computational tool developed by co-author Rocklage called the Evaluative Lexicon (EL) to measure emotionality for written reviews. The EL scans the text and “distinguishes words that tend to convey more emotion than those that convey less emotion,” says Rucker—essentially, separating “awesome” from “ideal”—and then generating an emotional score.

The researchers found that the problem of positivity was very real: Of all the films in the analysis, 81 percent had reviews of five or more on a ten-point scale. However, these positive reviews don’t necessarily translate into box office success.

Instead, emotional outbursts of positive feedback were a more accurate measure of real hit. EL Score for text user reviews predicted box office profits, a pattern that held true even when controlling for film genre, budget, length, and release date.

And it’s not just movies. The researchers repeated the analysis of book reviews on Amazon and found the same pattern: For books that received at least three stars out of five, more stars did not reliably predict higher sales. But more emotionally charged reviews anticipated sales.

Was that bruschetta “amazing” or just “fun”?

Next, the researchers turned their attention to the restaurants, using Yelp to collect the first 30 star and text reviews of all restaurants in Chicago in 2017. They also collected the average number of daily reservations per establishment on OpenTable, the most popular online reservation service. in the United States

As with books and movies, nearly all Yelp reviews have been favorable—92 percent of restaurants have a rating greater than three out of five stars. But unlike movies and books, these star ratings have been associated with daily bookings. However, the reviews’ emotional outbursts as measured by the review lexicon were a better predictor of which restaurant tables would be filled.

From movies to restaurants, “At best, star ratings are an inconsistent signal. Sometimes it predicts success, sometimes it doesn’t,” says Roker. But “in a sea of ​​positivity, one distinguishing factor is the amount of emotion conveyed in the review.”

Leverage the power of online reviews

In theory, online reviews are supposed to give consumers and industry experts new and more accurate ways to select the best products.

“The promise of these online platforms is that we can now see and learn from others, and that should have extraordinary value,” Nordgren says. But the huge number of positive reviews diminished its usefulness, missing an opportunity. One important way to take advantage of this “unfulfilled promise” is to look at emotion.

Of course, that doesn’t mean business owners should ignore numerical ratings, Rucker points out. “If a consumer leaves you a one-star rating, that counts, too,” he says. “We are not suggesting to ignore the reviews, but we have to acknowledge that they are part of the information. We should take advantage of the fact that the data is richer.”

But it cannot be the only way to measure success. “If people are giving you a positive feedback but there is nothing emotionally strong in their language, they are not really showing passion for the brand. Perhaps there is an opportunity for improvement, and our tool can help reveal that,” he says. “Emotion is another piece of data that People have to think about it and look at it.”

Click here to see Forbes India’s comprehensive coverage of the Covid-19 situation and its impact on life, business and the economy

Check the end of season subscription discounts with Moneycontrol pro subscription absolutely free. Use code EOSO2021. Click here for details.

[This article has been republished, with permission, from Kellogg Insight, the faculty research & ideas magazine of Kellogg School of Management at Northwestern University]