Data Analyst or Data Scientist?


Well, I don’t blame you, for the industry working definitions of these two job titles are pretty inadequate. Both disciplines (if I may call them such) have overlapping areas and are usually complementary. At the same time, the titles call for a mixed bag of different skill sets, qualifications and operational roles that set them apart.

While the analytics space has many job titles (Data Engineer, Business Analyst, Computer Scientist, Big Data Practitioner, Econometrician, Data Miner, etc.); a thumb rule for distinguishing a Data Analyst from a Data Scientist is the functional role within the data life cycle and the continuum of skills called for.

DATA life cycleWhile the Data Analyst aggregates data for visualization and building organisational database; the Data Scientist extracts useful information from the data for predictive insights and facilitating business decisions.

So let us examine in detail what these titles are about.

Data Analyst

Usually an entry level job, the Data Analyst is nevertheless core to the analytic workspace. He deals with data at every primary and aggregator level, linking business data with the reporting. While a Data Analyst maybe considered a junior title in a smaller company, he would occupy a senior and specialised position in a large organization.

The Data Analyst makes data usable for insights in real-time, often performing multiple roles of database administration, BI and data-on-demand.The job role spans the length and breadth of data, ranging from extraction of statistical information and building of the organisational RDBMS to presentation and visualization.

Data Scientist

Is Data Scientist, a scientist who experiments with data for modelling or a data professional who administers data for further analysis? Well, we’d say, he is both!

A Data Scientist is a practitioner of Data Science – the study of applying machine learning, statistics, computer science or other scientific discipline – to interpret and extract knowledge from large amounts of data for predictive modelling. Although the formal training for a Data Scientist is similar to that of a Data Analyst (see below), what sets him apart is his business insight. He is expected to summarise data and systematically design forecasting models based on research, and validate the same for the decision making process.

When a Data Scientist works in equity – high frequency trading, predicting stock prices – he may be referred to as a Quant! Quaint as this title may sound, the title is associated with high grossing salaries.

There is no strict dictum about the qualifications or expertise requirement for a Data Scientist. Rather it is the nature of job role, the industry, the company policy and projects handled, that define the qualifications or skill sets required of the Data Scientist. As the volume of data increases, the ability to handle Big Data and proprietary software becomes a must-have.

Qualifications and Skills required

Data Analyst

  • Degree in statistics, mathematics or computer science
  • Knowledge of databases, data warehousing and Business Intelligence
  • Knowledge of all file types and importing them into the database
  • Expertise in data storage, retrieval, application of ETL tools
  • Knowledge of Excel, SQL, data modelling
  • Knowledge of Hadoop and R based analytics
  • Ability to analyse data in real-time
  • Working knowledge of the business
  • Communication savvy

Data Scientist

  • Degree in mathematics/ statistics /computer science with preference for a PhD; and/or proficiency in the above disciplines
  • Knowledge of one or more programming languages with ability to code, firm grasp on relational and SQL/ MySQL database systems, distributed architectures,  machine learning, data mining algorithms and data modelling techniques, operations research
  • Knowledge of SAS, SPSS, Python and R
  • Knowledge of Big Data, and ability to analyse large datasets
  • Creative thinking and in-depth knowledge
  • Business acumen and IT savvy

Data Analyst vs. Data Scientist

At the end of the day, it all boils down to jobs and salaries.

Here are some random examples of titles you can expect.

JOB TITLESSalary trends for Data Scientists have been on the rise, while that for Data Analysts have been regular and steady.

JOB TRENDSAccording to PayScale, the average pay for a Data Analyst is Rs 292,953 /year with highest paying skills associated with VBA, SAS, and SQL. “Most move on to other jobs if they have more than 10 years’ experience” is what it says. However, the average salary for a Data Scientist, logs more than double at Rs 680,337.

Many young professionals start their career with the title of Data Analyst and gradually evolve / migrate as show below.

Data analyst career path


Pinned from

If you are STILL wondering what this appellation is all about, have fun reading this thread!

Posted in ARTICLES | Tagged , , , , , , , , , | Leave a comment

How analytics can help fight terrorism

The fight against terrorism – an application area with plenty of scope for development

Terrorist activities are trending strongly across news channels and investigative operations. In this scenario, surely analytics can help fight terrorism by predicting terror attacks and identifying terrorist financing? Oddly enough, it was a lawyer who first advocated wielding the power of data mining and analytics for fighting terrorism – and not a technocrat or member of the IT or data mining intelligence community! According to Philip Bobbitt, one way to combat terrorism is to strengthen the “valuable tool” of data mining and analytics by extracting information from disparate sources, such as “terrorist watch lists, airline reservations, immigration records” and more. Stacking up analytics on this data makes it further possible to identify patterns to support the government intelligence framework. Patterns can point out possible terrorists or even predict terrorist activities, facilitating global efforts at keeping citizens and property safe.

Albeit, it is six years since Philip Bobbitt first mooted the idea in his book Terror and Consent, yet the adoption has sadly been lacking in India. This post from IVY is aimed to bring to the forefront yet another avant-garde application of analytics, in the hope we can see students and professionals of analytics venture into this largely unexplored domain.

Leveraging Analytics to identify terrorist financing, patterns and potential hotspots

The realm of fraud analytics is a well explored area, with the ability to detect various types of fraudulent transactions, unnatural financial activities, systemic suspicious transactions and money laundering. When extended to analyzing trade-based money laundering across geographies, anomalies and patterns can be detected in global underground financial systems that are the conduits of terrorist financing. Data mining, analysis and visualization tools have the ability to gather, connect, track, analyse and distribute intelligence information and leads about terrorist activities. Moving beyond the prediction capabilities, integrated seamless systems can additionally provide notifications and alerts for preventive action.

Areas of countering terrorism, using analytics

  • Identifying money laundering activities used for terrorist financing
  • Following the money trail of terrorist organizations / suspicious individuals
  • Identifying hotspots of terrorist activities for effective countermeasures
  • Correlating terrorist attacks with trends in geo-politics and money trails
  • Identifying potential uprising and terrorist sponsored activities
  • Predicting potential terrorist activities based on any /all of the above

Analytical Techniques in play

Data mining, sentiment analysis, text mining, machine learning techniques and predictive analytics are some of the methodologies being leveraged to identify and combat terrorism. The Memex program also produces instant search results for specific domains and tasks, like patterns in state-wide crime or linkages with a car used for terrorist activity, for mission-critical action.

Data utilised

Different Government, intelligence, and criminal databases; financial systems, social media and internet, are some of the key data that is mined and analysed. Using advanced data analytics patterns can be identified for policy making and measures, predicting organized terrorist activities and cutting off channels of terrorist financing, or even deflecting proxy terrorism or uprising using social media analytics.

User cases

#  A project titled Computational Analysis of Terrorist Groups makes an excellent case of how analytical techniques can be leveraged for combating anti-terrorism.  Developed by researchers at the University of Maryland, the model, known as Temporal-Probabilistic Rule System, helps predict terrorist attacks of a particular terror organisation. It used algorithms to parse mined data on 770 variables from 20 years of a terrorist organisation’s activities, updated monthly for computational analysis.  This helped establish an understanding of the factors that determined frequency of attacks, types of terror strikes used in different geopolitical situations, trends in proxy terrorist activities and other criteria. Algorithms were also used to determine conditions that could lead to attacks.

#  The Los Angeles Police Department and the London’s Metropolitan Police Service use sophisticated data-analysis systems, “designed to identify and connect related pieces of intelligence to help officers deter and respond to terrorist attacks”.

Using a hybrid relational and open text-search database paired with an intelligence engine that compresses data, simultaneously search of multiple databases are possible.

Although the above are mere examples of how programs can be developed to effectively counter terrorism, there are unlimited ways in which private analytic firms and individuals working within the Government system, can use mining and analytic techniques to support the global fight against terrorism. The market as well as scope for such programs, is unlimited.

Bottomline –The ROI of using analytics for countering terrorism cannot be quantified. Terrorism is today an albatross around the neck of every Government, not to forget the trauma associated with terrorist attacks and deaths. Although GIS systems are being used for countering terrorism, there is plenty of opportunity to leverage the power of analytics for tactical strategic solutions to counter terrorism and provide public safety.

Posted in ARTICLES | Tagged , , , , , , , , | Leave a comment

What is all the hype about Disruptive Technologies?

Much is mentioned about the way Disruptive Technologies have stormed the field of analytics, and in particular Big Data Analytics.

Although, the term was originally popularised by the Harvard Business School professor Clayton Christensen in his book The Innovator’s Dilemma, referring largely to innovations in technology, one finds an increasing mention in analytics.

So let us examine how technology came to be referred as ‘disruptive’.

When an existing way of doing things was overturned by an innovative technology product, service or usage, it came to be known as ‘disruptive technology’.

Techopedia defines ‘Disruptive Technology’ as

“any enhanced or completely new technology that replaces and disrupts an existing technology, rendering it obsolete. It is designed to succeed similar technology that is already in use.”

This “applies to hardware, software, networks and combined technologies”.

In the domain of analytics, this definition could be extended to a new, unused, unapplied, untested alternative to existing practices which offers functional advantages for better business ROI.

Disruptive technologies in analytics may not necessarily be a cheaper solution. Rather it suggests that on deployment, the technology offers business value and efficacy by solving problems faster. Such technologies encourage continued exponential improvements in performance and core competency, rewriting rules of businesses, creating new markets and generally fostering deep analytics talent.

Technologies that have ‘Disrupted’ Analytics

  • Cloud computing is an early disruptive technology, that has emerged as an integral system for most industry verticals (BFSI, healthcare, e-governance, and so on) – with embedded analytic engines into the various solutions and service delivery mechanisms
  • Embedded sensors, in smartphone devices, wearables (bracelets, jogging shoes, sports and so on) or POS (point-of-sales) terminals enable collection of user data for analysis.
  • Disruptive Online Advertising like Google Ads, or targeted ads by Flipkart / other digital marketers based on browser behavior, aim to deliver marketing ROI
  • Disruptive CRM (Customer Relationship Management) apps in mobile phones facilitate realtionship building with the customer, for customer retention and building predictive models
  • Web 2.0 – of which social media and blogs is an integral part – is a pervasive disruptive innovation that has made the online community collaborative and interactive for two-way customer engagement, targeted content and advertising. Social media analytics can be leveraged for product innovation, CRM and what-if scenarios.
  • Mobile Internet and apps have revolutionized consumer behaviour, and are central to SMAC.

Why Disruptive Technologies are core to analytics?

Disruptive Technologies shake up the analytics life cycle with ground-breaking products or innovations in applications, computation capabilities and data-driven predictive models to name a few. As many of the disruptive technologies merge seamlessly, analytic applications are also becoming ubiquitous across the various channels and devices.

For instance, banking companies are harnessing cloud computing services for storage and harvesting of data, on-demand BI services and fraud detection with embedded data analytics software; using social media and mobile for CRM and business growth.  In retail and digital marketing sectors, innovative algorithms and progressive applications leverage disruptive technologies (cloud, embedded apps and sensors, social media, disruptive online advertising, mobile internet) for predicting demand, improving products and services, targeting customers, increasing sales and business ROI.

Bottomline –Analytics, especially in the era of Big Data, is unthinkable without disruptive technologies. The constant endeavour to improvise, innovate and devise innovation in technology and applications, are important for high granularity of data, high velocity of data and deeper analytic insights.

Posted in ARTICLES | Tagged , , , , , , , , , | 1 Comment

The latest buzz in Indian Analytics – September

The month of September has been a ‘wait and watch’ period to see how the new Government implements reforms, in particular exchange of technology and FDIs. Although, the India-US talks of September 29-30 will take some time to get off the ground, will it mean we can see more FDIs in the analytics industry and less of the acquisitions we have been seeing the past few years? Will there be more inter-country exchange in analytics training programmes?

Albeit, one thing is a surety, that the digital marketing war is picking momentum with Amazon following in the wake of Snapdeal and Flipkart, and more in the pipeline. This will have a butterfly effect on digital channels usage and analytics deployment. The analytics space is unlimited…. for the smart ones to score a Mangalayam of their own.

Reports and Market Projections

# India to lead in Analytics by 2015? According to a market study by financial services firm Avendus, quoted in the Business Standard, the Indian Analytics sector is predicted to grow to $1.15 billion by 2015. As analytics becomes more pervasive, the demand for analytics from India will increase – driven by availability of huge analytics talent pool, maturity of the industry and a wide spectrum of services.

Applications, Tools & Products

# Myntra, a leading online store for branded lifestyle goods, has teamed up with iProspect Communicate2  to sign up for Google Analytics premium (GAP). It is the first brand in India to tie up with Google Analytcs enterprise version, for its expertise in insights into data-driven audience to narrow customer path for targeted campaigns.

# Openbravo , a provider of commerce solutions for agile retailers,  showcased its new commerce platform with retail analytics at the recently held India Retail Forum at Mumbai (Sep 17-18). The product, Openbravo Commerce Platform, offers embedded Retail Analytics, with Web and mobile point of sale, e-commerce, flexible merchandise management capabilities and a comprehensive supply chain management “to  help retailers be more responsive to customer behavior or market changes thanks to actionable insights about business performance across all channels”.

# For those who want to make use of free cloud services, IBM’s Watson Analytics cloud service allows to upload data and then query and explore results to spot trends, patterns, and conduct predictive analysis. The goal is to make users “become paying customers with access to advanced features such as slicing and dicing larger data sets, correlation with live data feeds, and advanced analysis”.  For SMEs that can’t afford a dedicated business analyst product, Watson Analytics is a useful ”to delivers a powerful self-service tool for understanding complex pools of data”

# Digital Audience has launched the audience marketing platform ARQ, that leverages
the power of SMAC for desired audience outreach, lower cost of customer acquisition & retention.

Acquisitions & investments

# Nielsen makes it foray into India with acquisition of Delhi based economic research and analytics firm, Indicus Analytics. With this, Nielsen can now cater to Harvard Business School, London School of Economics, amongst others, which are part of Indicus’s client roster of 80.

# FirstSource Solutions of Sanjiv Goenka, has invested in Bangalore based year old start-up Nanobi Data and Analytics.  IVY Blog had earlier featured Nanobi Analytics for its innovative analytics app store.

# Chandigarh mobile VAS provider / tech company Altruist buys nine-year old Kolkata based startup iConnectiva. in an all-cash deal.iConnectiva is an analytics product and services company with focus on telecom subscriber data monetisation. It provides a portfolio of analytics solutions that enable telecom, media and utility companies to lower churn, reduce fraud, improve operational effectiveness, boost ARPU and increase overall profitability. iConnectiva.sepcialises in helping its clients predict hidden insights working with mobile operators across 35 countries in the Middle East, Africa, Europe and Asia. With this acquisition, Altruist can now offer the product to its 70 customers base having operations in more than 50 countries.

Awards & Recognition

Amusement park developer, Pan India Paryatan Pvt. Ltd., was awarded Business Intelligence (BI) / Analytics IT category award at the InformationWeek EDGE Award 2014, held on 5th September at Mumbai.

Posted in Uncategorized | Tagged , , , | Leave a comment

Featured Start-up of the month – Analytics and Risk Management Hiring Firm, Rinalytics Advisors

This month we make a detour from our usual analytics start-up profiles. We feature a specialised executive search firm focused exclusively on Analytics and Risk Management talent hiring cutting across the industry sectors.

The traditional placement firms have today evolved in fresh avatars, with add-on services, niche domains serviced, or management hiring ‘only’ like this start-up Rinalytics Advisors.

As its website says, it is an era of hyperspecialisation”.  Hirers want to recruit the best of analytics and risk talent in the shortest curve possible. This is where specialised search firms like Rinalytics step in to bring  a “combination of industry-specific expertise” for key positions.

The firm has a unique take on scouring the best of analytics talent. Unlike most placement firms, Rinalytics leverages its in-depth knowledge across key areas of analytics to find the best-fit people. Befitting the analytics culture, Rinalytics uses the power of analytics, talent mapping and customised executive research methodology to identify the best suited candidate for the position.

Time is valuable and good talent is precious. Filling this gap in the analytics and risk hiring scenario, the firm offers expertise in almost all areas of analytics and risk management. The focus is on a number of priority sectors listed herein for both, anlaytics and risk

So if you have some work experience behind you and are equipped with domain knowledge, then look up their job spotlight sections for risk and analytics. This executive search firm leverages a unique selection proposition to help you find the job of your dreams!

Posted in ARTICLES | Tagged , , , | Leave a comment

What do you need to become a Data Analyst?

It is a data-driven world today, where every organisation or company is working with data. However, data alone is not sufficient for decision-making, unless analysed for insights.

Data analysis is the very first step towards making sense of any data. It is very often an entry level job, which requires good computer skills and an understanding of data management.

Who is a Data Analyst?

At a basic level, a data analyst is one who compiles and analyses data and data-based information from multiple sources, including that contained in the database.

What does he do?

A Data Analyst applies data analysis tools to the unstructured and  structured data he compiles for his employers or clients to facilitate decisions. He may also be entrusted with the task of data accuracy, data compliance and make recommendations for database designs. His work is to present the analysed data in a compact meaningful format for indentification of trends, patterns and other insights.

Who hires Data Analysts?

Any company which collects data,  whether internal (employee performance, marketing ROI, etc.,) or external data (social media conversations, digital footprint of buyers, etc), hires data analysts. Companies hiring can range from government departments, healthcare, banking and financial,  to manufacturing, retail, telecommunications and more. Hirers may also be management consultants, contractual hirers or companies that have a dedicated department of systems analysis.

Salaries of Data Analysts

Salary levels depend upon the qualifications and skills sets of the applicant, as well as the job role, location and company policies and of the recruiting company. For instance, salary trends  are highest in Fiance and Banking, followed by Retail, Healthcare and Marketing.

According to Payscale research, the average pay for a Data Analyst is Rs 292,953 per year. You can even calculate your potential here.

Must-have Qualifications for a Data Analyst

  • A minimum Bachelors degree
  • Area of Qualification can be in any filed -Statistics, Mathematics, Engineering, Information Technology, Computer Science, Social Sciences, Finance, Actuarial Science/ Risk, Management Information Systems or even BBA!
  • Computer savvy
  • Affinity for figures
  • Attention to details
  • A certification in Data Analytics

Skill sets Required

  • Analytical skills
  • Mathematical skills
  • Knowledge of Statistics
  • Knowledge of database, data manipulation, data  warehousing
  • Soft skills

Preferred Add-ons

In addition to the above, if you have quiet a few of these following add-ons, you may well expect to land a great job with a top notch company, with competitive salary and a scope to grow. So while you wait for your interview call, I would recommend you to utilise your time and free internet resources to add as much as you can to your application portfolio. Although these may not be not essential for the job you are applying for, but they will be advantageous to you.

  • Certifications / Diplomas in IT, especially database management
  • Knowledge / experience working with BI or data analysis softwarelike SAS
  • Experience of  0 to 2 yrs
  • Domain knowledge – marketing, econometrics, healthcare, software development, programming, and so on.
Posted in ARTICLES, DATA ANALYTICS (KPO), LIFE SKILLS | Tagged , , , , , , , | Leave a comment

A cheat sheet on how to tackle an Analytics Project

Analytics helps solve many problems across various application areas like healthcare, retail, climate science, crime, banking, fraud and more. Each of these different applications calls for some basic domain knowledge. For instance, to solve a problem of your retailer client, you would need to have an idea of retail operations in general.

However, what happens when you are assigned a problem or project?

Does the approach differ? No. The Analytics Project Life Cycle goes through some typical stages.

Analytics life cycle2

 The Analytics Project Life Cycle

  1. What is the Problem?
  • Understand the type of problem for analysis – predictive analytics, prescriptive analytics, machine-to-machine implementation, root cause analysis, and so on.
  • Define the problem / project objective – the steps, metrics
  • Understand scope of the project – specifics, budgets, time, other considerations
  • Draft schedule and scope
  • How to tackle the project – develop customised in-house solution or implement a vendor product. Benchmark products for the latter.
  1. What are the available data sources?
  • ‘Extract’ or compile available data
  • Evaluate data quality
  • Perform EDA (Exploratory Data Analysis)
  • Populate fields
  • Clean your data, improve quality and consistency
  • Is the available data sufficient to solve the problem?
  1. What additional data do your require?
  • Any historical data required
  • Whether data is required in real-time
  • Address the storage and access of such data
  • Fields needed
  • Granularity desired
  1. What analysis would you implement?
  • Address the next step – how to ‘Transform’ the data
  • Identify and remove outliers
  • Selecting appropriate imputation methodology
  • Conduct cross-correlation
  • Select best-fit model
  • Apply Sensitivity analysis
  • Measure and test the model
  1. How to implement or deploy the model?
  • Address encoding or recoding of model
  • Verification of model – temporal logic, scalability, verification algorithms to use
  • Checking and debugging
  • Frequency of updation
  • Need for an API
  • Workflow analysis tools
  1. How to communicate?
  • Dashboard architecture
  • Modes of Visualisation
  • Integrations of results
  • Reporting
  • Are the key deliverables adequately represented? 
  1. Does it require maintenance or monitoring?
  • Post implementation review and versioning
  • Model monitoring – placing alerts and processes for problem resolution
  • Real-world testing – stress tests, other tools to use
  • Metadata to attach to facilitate future troubleshooting
  • Follow-up on team feedback
  • Recommendations for action
Posted in ARTICLES | Tagged , , | Leave a comment

Protected: Urjent Opening For Statistical Analyst II – India

This content is password protected. To view it please enter your password below:

Posted in DATA ANALYTICS (KPO), JOB ALERTS | Enter your password to view comments.

Protected: Urjent Opening For Research and Data Analyst – New Delhi

This content is password protected. To view it please enter your password below:

Posted in DATA ANALYTICS (KPO), JOB ALERTS | Enter your password to view comments.

Protected: Urjent Opening For Game Analyst – Bangalore

This content is password protected. To view it please enter your password below:

Posted in DATA ANALYTICS (KPO), JOB ALERTS | Enter your password to view comments.