Opinion Mining and Sentiment Analysis: The Next Step for Marketers

Evolution of Web Technology has resulted in a huge amount of Data present in the Web for the Internet users. These users not only use the available Resources in the Web, but also give their Feedback/Opinion, thus generating additional useful Information. Due to overwhelming amount of user’s Opinions, Views, Feedback and Suggestions available through the Web Resources, it’s very much essential to Explore, Analyse and Organise their views for better Decision Making. Opinion Mining or Sentiment Analysis is a Natural Language Processing (NLP) and Information Extraction task that identifies the User’s views or Opinions explained in the form of Positive, Negative or Neutral Comments and Quotes underlying the Text. Text categorisation generally classifies the Documents by topic.

Opinion Mining or Sentiment Analysis (SA) involves building a System to explore User’s Opinions made in Blog Posts, Comments, Reviews or Tweets, about the Product, Policy or a Topic. It aims to determine the attitude of a User about some topic. In recent Years, the exponential increase in the Internet usage and Exchange of user’s Opinion is the motivation for Opinion Mining. The Web is a huge Repository of Structured and Unstructured Data. The Analysis of this Data to extract underlying user’s Opinion and Sentiment is a challenging task. An Opinion can be described as a Quadruple consisting of a Topic, Holder, Claim and Sentiment. Here the Holder believes a Claim about the Topic and expresses it through an associated Sentiment. To a Machine, Opinion is a “Quintuple”, an object made up of five different things: (Oj, fjk, SOijkl, hi, tl), where Oj= the Object on which the Opinion is on, fjk = a Feature of Oj, SOijkl = the Sentiment Value of the Opinion, hi = Opinion Holder, tl = the Time at which the Opinion is given.

There are several challenges in the field of Sentiment Analysis (SA). The most common challenges are given here. Firstly, Word Sense Disambiguation (WSD), a classical Natural Language Processing (NLP) problem is often encountered. For example, “an unpredictable plot in the Movie” is a Positive Phrase, while “an Unpredictable Steering Wheel” is a Negative one. The Opinion word unpredictable is used in different senses. Secondly, addressing the problem of sudden deviation from Positive to Negative Polarity, as in “The Movie has a Great Cast, Superb Story line and Spectacular Photography; the director has managed to make a mess of the whole thing”. Thirdly, Negations, unless handled properly can completely mislead. “Not only do I not approve Supernova 7200, but also hesitate to call it a Phone” has a Positive Polarity word approve; but its effect is Negated by many Negations. Fourthly keeping the Target in focus can be a challenge as in “my camera compares nothing to Jack’s Camera which is Sleek and Light, produces life like Pictures and is Inexpensive”. All the Positive words about Jack’s Camera being the constituents of the Document Vector will Produce an overall decision of Positive Polarity which is wrong.

A Subjective-Objective Sentence classifier that does not require Annotated Data as input is built. Such a Classifier may then be used to improve Information Extraction Performance. Subjectivity Classification can prevent the Sentiment Classifier from considering irrelevant or even potentially misleading text. Document Sentiment Classification and Opinion Extraction have often involved Word Sentiment Classification Techniques. The Entropy Weighted Genetic Algorithm (EWGA) is a Hybridised Genetic Algorithm that incorporates the Information gain heuristic for feature selection. SVM and N-gram approaches outperformed the Na?ve Bayes approach when Sentiment classification was applied to the Reviews on Travel Blogs for Seven popular Travel destinations in the US and Europe.

The Figure depicts the Major important steps in order to achieve an Opinion impact. The Web users post their Views, Comments and Feedback about a Product or a Thing through various Blogs, Forums and Social Networking Sites. Data is collected from such Opinion Sources in such a way that only the Reviews related to the topic, i.e. searched is selected. The Input document is then Pre-processed. Pre-Processing, in this context, is the removal of the fact-based sentences, thus choosing only the opinionated sentences. Further refinements are made by removing the Negations and by sensing the Word Disambiguation. Then, the process of extracting relevant Features is done. Feature selection can potentially improve classification Accuracy, narrow in on a Key feature subset of sentiment discriminators, and provide greater insight into important Class Attributes. The extracted features contribute to a Document Vector upon which various Machine Learning Techniques can be applied to classify the Polarity (Positive and Negative opinions) using the obtained Document Vector and finally the Opinion Impact is obtained based on the sentiment of the Web users.

SENTIMENT ANALYSIS

Ordinary Keyword Search will not be suitable for Mining all kinds of Opinions. Hence it becomes necessary that the Sophisticated Opinion Extraction Methods are used. Sentiment Analysis (SA) is a Natural Language Processing (NLP) Technique, helps to identify and extract subjective Information in Source Materials. Sentiment Analysis (SA) aims to determine the attitude of the writer with respect to some Topic or the Overall Contextual Polarity of a Document. The attitude may be his or her Judgement or Evaluation, Affective State, or the intended Emotional Communication. A basic task in Sentiment Analysis (SA) is classifying the Polarity of a given text at the Document, Sentence, or Feature/Aspect level — whether the expressed Opinion in a Document, a Sentence or an Entity Feature/Aspect is Positive, Negative, or Neutral. Beyond Polarity sentiment classification, the emotional states such as "Angry", “Sad" and "Happy” are also identified.

One of the challenges of Sentiment Analysis is to define the Opinions and Subjectivity. Originally, Subjectivity was defined by Linguists. Quirk defines Private State as something that is not open to objective Observation or Verification. These Private States include Emotions, Opinions, and Speculations among others. The very definition of a Private State foreshadows difficulties in analysing Sentiment. Subjectivity is highly context-sensitive, and its expression is often peculiar to each person. Subjectivity Detection and Negation are the most important Pre-processing steps in order to achieve efficient opinion impact.

Subjectivity Detection can be defined as a process of selecting Opinion Containing Sentences. (e.g.,) “India’s Economy is heavily dependent on Tourism and IT Industry. It is an Excellent place to live in.” The First Sentence is a Factual one and does not convey any Sentiment towards India. Hence this should not Play any Role in deciding on the Polarity of the Review, and should be filtered out. Hence, the Polarity Classifier assumes that the Incoming Documents are Opinionated. Joint Topic-Sentiment Analysis is done by collecting only on-topic Documents (e.g., by executing the Topic-based Query using a Standard Search Engine). In Information Extraction, both topic based text filtering and subjectivity filtering are complementary. If a Document contains Information on a variety of topics that may attract the attention of the user, then it will be useful to Classify the Topics and its related Opinions. This type of Analysis can be useful for Comparative Search Analysis of related items and also to discuss on the Texts that contains various Features and Attributes.

The Political Orientation of the Websites can be done by classifying the Concatenation of all the Documents found on that particular site. Analysing Sentiment and Opinions in Political Oriented Text, generally focuses on the Attitude expressed via Texts which are not targeted at a specific issue. In order to Mine Opinion, the main concentration is on non-factual Information in Text. There are various affect types; in specific here the concentration is on the Six “universal” emotions: Anger, Disgust, Fear, Happiness, Sadness and Surprise. These Emotions could be easily associated with an interesting Application of a Human-Computer interaction, where when a System identifies that the user is Upset or Annoyed, the System could change the user interface to a different mode of interaction.

Negation is a very Common Linguistic Construction that affects Polarity and, therefore, needs to be taken into consideration in Sentiment Analysis (SA). When treating Negation, one must be able to correctly determine what part of the meaning expressed is modified by the presence of the Negation. Most of the times, its expression is far from being simple, and does not only contain obvious Negation Words, such as Not, Neither or Nor. Research in the field has shown that there are many other words that invert the Polarity of an Opinion Expressed, such as Diminishers/ Valence Shifters (e.g., I find the functionality of the new phone less practical), connectives (Perhaps it is a great Phone, but I fail to see why), or even modals (In theory, the Phone should have worked even under Water). As can be seen from these examples, Modelling Negation is a difficult yet an important aspect of Sentiment Analysis (SA).

Feature Engineering is an extremely basic and essential task for Sentiment Analysis (SA). Converting a piece of Text to a Feature Vector is the basic step in any Data driven approach to Sentiment Analysis. It is important to convert a piece of Text into a Feature Vector, so as to process Text in a much efficient manner. In Text Domain, effective feature selection is a must in order to make the learning task effective and accurate. In Text Classification, with the Bag of Words Model, each position in the input Feature Vector corresponds to a given Word or Phrase. In the bag of words Framework, the Documents are often converted into Vectors based on predefined Feature Presentation including Feature type and Features Weighting Mechanism, which is critical to classification accuracy. The major feature types contain Unigrams, Bigrams and the mixtures of them, etc. The commonly used features used in Sentiment Analysis and their critiques are Term Presence, Term Frequency, Term Position, Sub-sequence Kernels, Parts of Speech, Adjective-Adverb Combination, Adjectives, n-gram features etc.

Let us consider the n-gram features for Feature Extraction. An n-gram is a Contiguous Sequence of n items from a given sequence of Text or Speech. An n-gram could be any combination of letters (Syllables, Letters, Word, Part-of Speech (POS), Character, Syntactic, and Semantic n-grams). The n-grams typically are collected from a Text or Speech Corpus and n-gram features captures Sentiment Cues in text. Fixed n-grams are exact Sequences. Variable n-grams are extraction patterns capable of representing more sophisticated Linguistic Phenomena. n-gram features can be classified into two categories: 1) Fixed n-grams are sequences occurring at either the Character or Token Level.

2) Variable n-Grams are extraction patterns capable of representing more Sophisticated Linguistic Phenomena. A Plethora of fixed and variable n-grams have been used for Opinion Mining. Documents are often converted into Vectors according to predefined features together with Weighting Mechanisms. Correlation is a commonly used method for feature selection. The process of obtaining n–gram can be given as in the steps below,

1) Filtering – removing URL Links

2) Tokenisation – Segmenting text by splitting it by spaces and punctuation marks, and forming bag of words

3) Removing Stop Words – Removing articles (“a”,”an”, ”the”)

4) Constructing n-grams – from consecutive words

After the extraction of the Features, it would be effective if the Features are reduced.

Feature Reduction is an important part of Optimising the Performance of a (Linear) classifier by reducing the Feature Vector to a size that does not exceed the Number of Training Cases as a starting point. Further reduction of Vector size can lead to more improvements if the features are noisy or redundant. Reducing the number of Features in the feature vector can be done in two different ways:

1) Reduction to the top-ranking n features based on some criterion of “Predictiveness”.

2) Reduction by Elimination of sets of Features (e.g. Elimination of Linguistic Analysis Features etc.)

Now the extracted Features are reduced, and the classification of the Sentiment based on the Polarity and Intensity of the Text using any of the Machine Learning approaches is to be done.

SENTIMENT CLASSIFICATION:

Sentiment Classification broadly refers to Binary Categorisation, Multi-class Categorisation, Regression and Ranking. Sentiment Classification mainly consists of two important Tasks, including Sentiment Polarity Assignment and Sentiment Intensity Assignment. Sentiment Polarity Assignment deals with Analysing, whether a Text has a Positive, Negative, or Neutral Semantic Orientation. Sentiment intensity assignment deals with Analysing, whether the Positive or Negative sentiments are Mild or Strong. There are several tasks in order to achieve the Goals of Sentiment Analysis. These tasks include Sentiment or Opinion Detection, Polarity classification and Discovery of the Opinion’s Target. A wide range of Tools and Techniques are used to Tackle the difficulties in order to achieve Sentiment Analysis. The various Methodologies used in order to achieve Sentiment Classification are:

1) Classification with respect to Term Frequency, n-grams, Negations or Parts of Speech;

2) Identification of the Semantic Orientation of words using Lexicon, Statistical Techniques and Training Documents;

3) Identification of the Semantic Orientation of the Sentences and Phrases;

4) Identification Semantic Orientation of the Documents;

5) Object Feature Extraction;

6) Comparative Sentence Identification;

The Sentiment Polarity Classification is a Binary Classification task where an Opinionated Document is Labelled with an overall Positive or Negative Sentiment. Sentiment Polarity Classification can also be termed as a Binary Decision task. The input to the Sentiment Classifier can be opinionated or sometime not. When a news article is given as an Input, Analysing and Classifying it as a Good or Bad news is considered to be a text Categorisation task. Furthermore, this piece of Information can be Good or Bad news, but not necessarily subjective (i.e., without expressing the view of the author). Summarising Reviews in order to collect information on to why the Reviewers liked or disliked the Product is another way of Mining Opinion. In order to determine the Polarity of the outcomes as described in medical texts is yet another type of categorisation related to the degree of Positivity. Few other problems related to the determination of the degree of Positivity are the Analysis of comparative sentences. Automated Opinion Mining often uses Machine Learning, a component of Artificial Intelligence.

While Sentiment Polarity Assignment deals with Analysing, whether a Text has a Positive, Negative, or Neutral Semantic Orientation, Sentiment Intensity Assignment deals with Analysing, whether the Positive or Negative Sentiments are Mild or Strong. Consider the Two Phrases “I don’t like you” and “I hate you”, where, both the sentences would be assigned a Negative Semantic Orientation but the latter would be considered more Intense than the first. Effectively classifying Sentiment Polarities and Intensities entails the use of classification methods applied to Linguistic Features. While several classification Methods have been Employed for Opinion Mining, Support Vector Machine (SVM) has outperformed various Techniques including Na?ve Bayes, Decision Trees, Winnow, etc.

MACHINE LEARNING APPROACH:

The aim of Machine Learning is to develop an Algorithm so as to Optimise the performance of the System using Example Data or past Experience. The Machine Learning provides a solution to the classification problem that involves two steps:

1) Learning the Model from a Corpus of Training Data

2) Classifying the unseen Data based on the Trained Model.

In general, classification tasks are often divided into several sub-tasks:

1) Data Pre-processing

2) Feature selection and/or Feature reduction

3) Representation

4) Classification

5) Post processing

Feature selection and feature reduction attempt to reduce the Dimensionality (i.e. the number of Features) for the remaining steps of the task. The classification phase of the process finds the actual mapping between patterns and labels (or targets). Active learning, a kind of Machine Learning is a promising way for Sentiment Classification to reduce the Annotation Cost. The following are some of the Machine Learning approaches commonly used for Sentiment Classification:

NAIVE BAYES CLASSIFICATION

It is an approach to text classification that assigns the class c*= argmaxc P(c| d), to a given document d. A naive Bayes classifier is a simple probabilistic classifier based on Bayes' theorem and is particularly suited when the dimensionality of the inputs are high. Its underlying probability model can be described as an "independent feature model".

Despite its simplicity and the fact that its conditional independence assumption clearly does not hold in real-world situations, Naive Bayes-based text categorisation still tends to perform surprisingly well; indeed, Naive Bayes is optimal for certain problem classes with highly dependent features.

MAXIMUM ENTROPY

Maximum Entropy (ME) classification is yet another Technique, which has proven effective in a number of Natural Language Processing (NLP) applications. Sometimes, it outperforms Naive Bayes at standard text classification. Its estimate of P(C | D) takes the exponential form.

For instance, a particular Feature/Class Function might fire if and only if the Bi-gram “Still Hate” appears and the document’s Sentiment is hypothesised to be Negative. Importantly, unlike Naive Bayes, Maximum Entropy makes no assumptions about the relationships between Features and so might potentially perform better when conditional independence assumptions are not met.

SUPPORT VECTOR MACHINES

Support Vector Machines (SVMs) have been shown to be highly effective at traditional Text Categorisation, generally outperforming Naive Bayes. They are large-margin, rather than Probabilistic, Classifiers, in contrast to Naive Bayes and Maximum Entropy. In the two-category case, the basic idea behind the training procedure is to find a Maximum Margin Hyperplane, represented by Vector, that not only separates the Document Vectors in one class from those in the other, but for which the separation, or margin, is as large as possible.

There are quite a large number of Companies, Big and Small, that have Opinion Mining and Sentiment Analysis (SA) as part of their Mission. Review-oriented Search Engines basically use Sentiment Classification Techniques. Opinion Mining proves itself to be an important part of Search Engines. Topics need not be restricted to Product reviews, but could include Opinions about Candidates running for Office, Political Issues, and so forth. Summarising user Reviews is an important problem. One could also Imagine that Errors in user ratings could be fixed: there are cases where users have clearly accidentally selected a low rating when their review indicates a Positive Evaluation.

Sentiment Analysis (SA) and Opinion Mining Systems also have a Potential Role in imparting sub-component Technology for other Systems. Specifically, Sentiment Analysis System is an Augmentation to Recommendation systems; since it might recommend such a system not to suggest items that receive a lot of Negative feedback.

Detection of “Flames” (Overly-Heated Opposition) in Email or other types of Communication is another possible use of Subjectivity Detection. In Online Systems that display ads as sidebars, it is helpful to detect Web Pages that contain Sensitive content inappropriate for ads Placement; for more sophisticated systems, it could be useful to bring up product ads when relevant Positive sentiments are detected. It has also been argued that information extraction can be improved by discarding information found in Subjective Sentences.

Question Answering is another area where Sentiment Analysis can prove useful. For example, Opinion-oriented Questions may require different Treatment. For Definitional Questions, providing an Answer that includes more Information about how an Entity is viewed may better inform the user. Summarising may also benefit from accounting for multiple viewpoints. One effort seeks to use Semantic Orientation to track literary reputation. In general, the Computational Treatment of affect has been motivated in part by the desire to improve Human-Computer interaction.

It's the breadth of Opportunities – promising ways Text Analytics can be applied to Extract and Analyse attitudinal Information from Sources as varied as Articles, Blog Postings, Email, Call-Centre Notes and Survey responses – and the difficulty of the Technical Challenges that make Existing and Emerging applications so interesting. Three other Applications include influence Networks, Assessment of Marketing response and Customer Experience Management/Enterprise Feedback Management.

There are several challenges in Analysing the Sentiment of the Web user Reviews. First, a Word that is considered to be Positive in one situation may be considered Negative in another situation. Take the word "Long" for instance. If a Customer said a Laptop's Battery Life was long, that would be a positive opinion. If the Customer said that the Laptop's start-up time was long, however, that would be a Negative Opinion. These differences mean that an Opinion System trained to gather Opinions on one type of Product or Product feature may not perform very well on another.

Another challenge would arise because; people don't always express Opinions in the same way. Most traditional text processing relies on the fact that small differences between two pieces of Text don't change the meaning very much. In Opinion Mining, however, "the Movie was Great" is very different from "the Movie was not Great”.

People can be contradictory in their statements. Most reviews will have both Positive and Negative comments, which is somewhat Manageable by Analysing sentences one at a time. However, the more informal the Medium (Twitter or Blogs for example), the more likely people are to combine different Opinions in the same sentence. For example: "the movie bombed even though the lead actor rocked it" is easy for a human to understand, but more difficult for a computer to parse. Sometimes even other people have difficulty understanding what someone thought based on a short piece of text because it lacks context. For example, "That movie was as good as his last one" is entirely dependent on what the person expressing the Opinion thought of the previous film.

Pragmatics is a sub-field of Linguistics which studies the ways in which context contributes to meaning. It is important to detect the pragmatics of user Opinion which may change the sentiment thoroughly. Capitalisation can be used with subtlety to denote Sentiment. In the examples given below, the first example denotes a Positive sentiment whereas the second denotes a Negative sentiment.

Just finished watching THE DESTROY.

That Completely Destroyed me.

Another challenge is in identifying the Entity. A Text or Sentence may have multiple Entities associated with it. It is extremely important to find out the Entity towards which the Opinion is directed. Consider the following examples.

Sony is better than Samsung.

Raman defeated Ravanan in football.

The examples are positive for Sony and Raman respectively but Negative for Samsung and Ravanan.

Information from micro-blogs, blogs and forums as well as news source, is widely used in Sentiment Analysis (SA) recently. This media information plays a great role in expressing people’s feelings, or opinions about a certain topic or product. Using social network sites and micro-blogging sites as a source of data still needs deeper analysis. There are some benchmark data sets especially in reviews like IMDB which are used for algorithms evaluation.

In many applications, it is important to consider the context of the text and the user preferences. That is why we need to make more research on context-based Sentiment Analysis (SA). Using TL techniques, we can use related data to the domain in question as a Training Data. Using Natural Language Processing (NLP) tools to reinforce the Sentiment Analysis (SA) process has attracted researchers recently and still needs some enhancements.