How To Use Topic-Based Sentiment Analysis For Customer Insights

You don’t have to be an expert in artificial intelligence to enjoy the benefits of machine learning techniques for better, customer-focussed, business decisions. Topic-based sentiment analysis is perfect for companies who want to take it slow and steady while extrapolating insights they want to derive from consumer comments, reviews, or even just news. You can do away with manual processing of data that can be both expensive as well as error-prone due to human bias and limitations, and switch to automated analysis of your data. This blog articulates how with a simple topic-based sentiment analysis model, you can discover customer insights for incremental benefits without investing too much money.

What is topic-based sentiment analysis?

Topic-based sentiment analysis is a natural language processing (NLP) technique that is used to gain meaningful information from text data derived from various sources. This machine learning task identifies and extracts recurrent topics in a text by using sub-tasks such as named entity recognition (NER) and sentiment analysis. NER recognizes and extracts themes or “entities” from unstructured text data and classifies them into predefined categories. These categories can be names of persons, geo-locations, businesses, buildings, brands, medicines, diseases, or any number of categories that can be custom-fed into a machine learning model.

The algorithm isolates each topic for its sentiment score by running it through a sentiment analysis process. In this way, an organization can protect its brand reputation by keeping a tab on public sentiment around the various facets of its business. When done at scale, topic analysis can help companies extract valuable business intelligence from large volumes of unstructured data from social media comments, news articles, emails, customer service chats, Voice of the Employee data, Voice of the Customer data, healthcare data, and any such source.

Scope of topic-based sentiment analysis

Below are areas where companies can easily use topic analysis to gain valuable business intelligence.

• Monitor brand reputation:

The reputation of an organization’s brand is based on many factors. It’s not just dependent on the quality of the product or service, but also on things such as who are its brand ambassadors, what is the nature of its brand campaign, and what kind of government policies it supports. In the rapidly rising cancel culture brought upon by the uniquity of social media platforms such as Twitter and Facebook, topic analysis gives organizations a huge reprieve by enabling them to thwart any negative opinions through proactive measures.

• Knowledge Management:

A business generates massive amounts of data continuously, and all this information needs to be thematically organized for future reference and use. Topic analysis, combined with semantic search, can help an organization tremendously in search and retrieval of information so that none of the data is ever lost in the archives. Knowledge management is extremely important in industries such as market research, healthcare & pharmaceuticals, automotive and heavy machinery, aviation, and even beauty and well-being.

• Improve products & services:

Positive and negative comments about a product or service need not spell doom and gloom. They can be used to improve product quality and enhance customer experience. New product developments including prices for new products can be researched and understood by directly engaging with consumers through social media listening. When a business can gather and analyse all comments related to itself, it can get a better understanding of what works and doesn’t work for consumers, and thus have data-backed insights for product planning.

• Better customer support:

Having a diligent customer support team that is engaged in improving customer experience is a good thing. But proactively learning from the various conversations, chats, and emails is an even more astute business decision. It can go a long way in finding valuable insights for improved return on investments. Conducting topic-based sentiment analysis on all the customer support data can be a goldmine for companies in this regard.

• Market research:

Market research through opinion mining can give great insights into new target markets and consumers. You can conduct market research through survey data analysis and get consumers’ opinions about their interest in the new product, service, location, and many such aspects. For example, when a healthcare provider decides to open a clinic in a new location, it serves it good to analyse the public sentiment around the new location, doctors it will recruit, office hours, and other such factors. This blog will tell you more about how you can use surveys for sentiment analysis.

• Competitor analysis:

Topic-based sentiment analysis can give you important information about the performance of your competitors. By conducting a market analysis through social media monitoring, you can know what the public sentiment is around your competitors’ products and services in comparison to yours. You can also use this information to improve your offerings or decide a new course of action by analysing the correctness of your target market.

• Employee nurturing:

A simple topic analysis conducted on all Voice of the Employee data can give a company comprehensive insights it can use for employee engagement and attrition management. You can also gather data from external employee review websites such as Glassdoor or Indeed, to understand how your brand is perceived by potential employees and how current employees feel about you. Not only will this help you attract good talent, but also help in developing better employee programs.

• Stock sentiment analysis:

For a financial company, topic-based sentiment analysis can be very productive in reading sentiments in people’s tweets that can easily reflect on stock trading. Negative sentiments can impact sales, while positive sentiments result in financial gains. The daily prediction of the stock index can be ascertained through real-time monitoring of all tweets and news pertaining to a particular stock.

How does Topic Analysis work?

As the name suggests, the topic analysis model works on a machine learning algorithm that is taught and trained to identify certain topics in a collection of texts. When the model identifies a topic, it will extract it and club it into a predetermined classification. As the model is trained, it also learns each time it processes data.

Over a period, it starts identifying frequently recurring words and begins identifying them as important, and extracts them.

For example, the model can be trained to identify the word “Porche” and it will know that Porche is a luxury car. When it sees that “Rolls Royce” is frequently mentioned in the same word patterns (luxury, expensive, classy, extravagant) and in the same manner of speaking as a Porche, it will identify the Rolls Royce as a luxury car too.

Topic-based sentiment analysis models are customized for every individual company or industry, with the relevant topics or themes, as Repustate does. The data can be gathered from any source, be it news, blogs, social media sites, and even videos and podcasts. For audio and video content analysis, speech-to-text transcription is used, and the analysis done as with regular text. Since Repustate understands 23 languages and dialects natively, without the use of translations, the topic analysis model can be easily used by businesses that have operations in multiple countries and languages. All the analysis is then shown on a sentiment analysis dashboard.

Here are the basic steps of how topic analysis actually works:

Step 1: Data collection

Text is collected from all relevant sources. This can be either from internal sources or external sources depending on the nature of the company, industry, and what the organization needs to analyse.

Step 2: Data classification

Data is classified on sentiment scores (positive/negative), topics (food/drinks), or intent (purchase/feedback)

Step 3: Model Training

Training data that will be used to customize the model is imported.

Step 4: Creating Custom Tags

Tags are created for the data. For example, product, location, or service. Once the model is trained, it will automatically start segregating text based on the tags that are created.

Step 5: Topic Classification

The topic classifier is trained by choosing a tag for each sample text. For example, a text that reads, “Although all the shops were closed by the time we arrived, the hot dog vendor was still there, thankfully. Best hot dogs we ever had!” can be tagged as the topic “food”.

Step 6: Data Segmentation

Each topic is isolated and then analysed for the sentiment expressed.

Step 7: Data Visualization

The results are presented on a sentiment analysis dashboard in the form of graphs and charts.

How is topic-based sentiment analysis used in business models?

Topic-based sentiment analysis is used by businesses in manufacturing, finance, hospitality, healthcare, public service departments, etc. across the scope of market research, sales and advertising, knowledge management, and more. Below are real-world examples of how topic analysis has helped companies.

1. Automotive Manufacturing

A vehicle manufacturer that was going through a brand crisis due to a major product recall approached Repustate for help in analysing public sentiment around its brand. It wanted to track social media, automotive blogs, and customer forums to see how people were responding to the news of the recall. Since Repustate offers topic analysis in different languages, the company was able to gauge consumer sentiment around the world. They found some very interesting insights from the reactions of its Arabic, Spanish, English, and German-speaking audiences. Read here to know more

2. Banking & Finance

A bank in South Africa wanted to conduct customer opinion mining to study how they could increase consumer satisfaction. They wanted to know what it was that was influencing its net promoter score rather bleakly when there was nothing obvious that stood out in terms of customer dissatisfaction. Perplexed by this, they collaborated with Repustate for a topic-based sentiment analysis model. They were surprised by their learnings. Based on their insights, they did a complete overhaul of their website for a better digital experience, and increased teller staff to accommodate surges in customer traffic during certain hours of the day. This simple solution solved their outreach bottleneck. Read here to know more

3. Market Research

Topic analysis can be leveraged very smartly for market research, just as a company in the food industry did. The client wanted to conduct a competitor analysis when it decided to enter into the health-snacks market. They wanted to send out multitudes of surveys to different demographics to examine the market but needed Repustate to help them understand the data that they gathered from the surveys. Repustate’s topic-based sentiment analysis solution helped them decipher the comments in the open-ended questions in the survey. The insights gathered from the exercise helped the company decide how to direct its ad-spend during the product rollout as well as position itself in the market. Read here to know more

4. Healthcare & Pharma

A large US-based hospital network wanted to analyse its massive collection of electronic health records (EHR) data but was unable to make sense of it. This was because medical practitioners quickly jotted down a few notes into an EMR, with little to no details and context, before quickly moving on to the next patient. By using topic-based sentiment analysis, the client was able to decipher all the notes not only for their own records but also for reference while conducting studies on results at various dosage levels, different medicines, etc. Read here to know more

5. Government

An Asian government ministry needed to recognize and respond to the needs of its increasingly digital-savvy people in real-time to maintain a positive image. Repustate collaborated with the ministry and custom-built a topic-analysis sentiment solution that could create predictive models to anticipate bottlenecks in various public services. With the new solution in place, the government could anticipate when a particular service may experience downtime, and so deploy sufficient resources to proactively meet such a situation. Read here to know more

6. Hospitality

Data from millions of hotel reviews can be analysed to understand customer opinions. A star rating does not tell the whole story, as different people might give completely different ratings to a hotel based on not just what they experienced at the hotel, but also how their journey or related service was. For example, in a review that reads as the below, it is evident that the person enjoyed their stay at the hotel, even though the rating they gave was a 2-star.

The answer lies in the semantic analysis of hotel reviews. And that can only be done by analysing the sentiment score of each topic that is identified by a topic-based sentiment analysis model. Read here to know more

Topic-based Customer Insights Dashboard

Understand your customers and boost your brand worth with Repustate’s cloud-based topic-based sentiment analysis API with out customer experience dashboard. It can read and analyse data from any source that you choose including surveys, social media platforms such as TikTok, YouTube, Facebook, Twitter, and more. More importantly, it represents all the insights and sentiment scores gained on a visualization dashboard, making the analysis easy for you to understand.

The solution comes pre-loaded for sentiment analysis in English for analysis of standard document volumes. You can add as many of the 23 languages and dialects that Repustate processes natively, as you need, to meet the demands of your demographic audience and geographies. You can deploy it on-premise, or as a cloud-based API.

With a model like this in place, you can be cautious with your financial planning and at the same time get all the advantages of a smart, AI-based platform that can help you reap great dividends.