Understanding Sentiment Analysis in 5 Minutes

Basics

Alright, for starters: Sentiment Analysis is the process of determining whether a piece of text is positive, negative or neutral. It’s also known as opinion mining, because it ascertains the opinion or attitude of a speaker. Common use cases for this technology are to discover how people feel about a particular person, brand or event.

Now Onto Using It

Okay, I'm gonna launch a fleet of high volume, Sichuan inspired food trucks in the Bay Area (because dandan noodles rock). But first I want to know what people on Twitter think of the Chinese food scene in San Francisco. Knowing this will help guide how I market and run my new business. 

Analyzing tweets for sentiment will answer this question accurately. I can even learn why people think the food is good or bad by extracting the exact words that indicate why people did or didn't like the pre-existing options. If the analysis shows as a common theme, for example, that most Chinese food is too salty then I immediately have an insight into customer demand. I can now craft a low-sodium menu that'll give me an edge over the competition!

This is the kind of insight I'd normally aim to find through market research. But imagine what a headache it would be to hire a traditional market research firm for this task now. First, it would require an enormous budget and countless man-hours to cold call and survey random San Franciscans just to ask them if the local mapo tofu is too salty. Pretty ridiculous, right?  

This is where a company like Lexalytics (where I work!) comes in handy. By deploying these sophisticated text mining tools, I'll get answers in seconds from the comfort of my food truck garage. Before you know it, my low sodium Sichuan menu will be the hottest lunch option in The City!

Companies like Lexalytics provide sentiment analysis solutions directly to businesses. In our case, we offer affordable and enterprise level APIs that easily integrate with many of our clients' other Business Intelligence solutions. Hundreds of companies around the world rely on sentiment analysis to track and monitor public opinion of their products, services or organization in general. If someone is attacking your brand on social media, a sentiment analysis system like ours scores the relevant posts as extremely negative, and a social media monitoring solution flags them for immediate response. The idea is to drive better business decisions for companies of all sizes and budgets around the world by emancipating them from the fog of traditional market research and reputation management.

Multi-level Analysis

Less precise sentiment analysis tools lump together the sentiment expressed at individual entities into a general document sentiment score. Take a Facebook comment that reads:

A weak sentiment analysis system will score this as positive and as negative, but will report the entire comment’s sentiment as neutral (the positive and the negative cancelling each other out). Lexalytics recognizes the importance of that middle word, but, and so we report separate sentiment for the first and second parts of the sentence.

How Does This Work

In order to determine the sentiment of the overall document, we can use our own scoring algorithms – using the weighted phrases from the previous side, and then using our proprietary way of adding them up. We can also take a set of sentiment tagged content and build a document-level sentiment classifier.

First things first, we need to identify the sentiment phrases (and not, say, a proper noun like “Good Morning America”), apply things like “intensification” and “negation” – for stuff like “good” “very good” “not very good”, etc. Then, if we're dealing with English, use the syntax matrix to determine the syntactic effect word order.

Doing this allows a business to have both a high level and granular grasp of exactly what their target audience is feeling. This isn't palm reading and crystal balls, it's hard data science. Deploying sentiment analysis for your business is like a ship sending out radar―you're able to chart the most practical course, avoid mishaps and, when it's paired with intention analysis, it even allows you to predict what's coming next before it happens with 80% accuracy.