Spotlight: Text Analytics

Ivy Apr 14, 2014 No Comments

What do you do when you have a huge volume of textual data?

If the data is in structured form as in voter records you make a query that returns information as a number. Using more and different queries, you get detailed  information; such as number of female voters in a particular constituency above the age of 50 and so on. However, when the data does not conform to a common schema as in documents, emails, tweets and blogs i.e. unstructured data, a ‘keyword search’ is made that returns information analysed for contextual significance.

Text mining = analysis of both structured and unstructured data to extract meaningful associations, trends, and patterns in large volumes of different text sources.

While text mining uses algorithms that are more broad-based,  text analytics is more goal driven.

The massive volume of unstructured digital data in social media ecosystems like Twitter, Facebook and Flickr, offers great scope for investigating social, cultural, economic or political behaviour.  For instance, a business may apply text analytics to assess the feedback of a new product launch, while a news channels will apply different techniques to gain insights during elections. Such as, analysing the Twitter response to the mandate of a political candidate /party using the hashtags  #ModiManifesto, #CongressManifesto, and deriving insights into the focus of a manifesto using certain keywords.

So how would you define text analytics?

Text analytics is the process of analysing unstructured text, extracting appropriate information and transforming it into a structured form that can then be leveraged in various ways. The analysis and extraction processes apply techniques that originate in computational linguistics, statistics and other computer science disciplines. For huge volume of data, automated tools are applied, both commercial/proprietary and free to download and use. 

Processes  

Techniques

• Statistical Analysis – quantify text data
• Qualitative Analysis
•  Content or Language Analysis (lexical and syntactic processing, semantic processing) – identify structural elements, extract meaning, reduce dimensions of text data
• Entity extraction and information retrieval
• Application of Computational Linguistics and Statistical NLP (Natural Language Processing)

If you would like to try your hand at text analytics, download a free-to-use tool and analyse information of your interest in a real-time platform like Facebook or Twitter. Discover the value of hidden information, uncover “gems” of information with use of text analytics.

Some Free Tools for text mining and analytics

Philologic – search, analysis and retrieval tool for analysis of large volumes of text

Text Pair – identifying similar passages in large volumes of text.

Text Stat – produces word frequency lists from multiple languages and file formats

Read More at IVY Blog’s Posts on:

Social Media Analytics

Sentimental Analysis

Niche Analytics – various roles in analytics