Democratization of Business Analytics Dashboards

I am super impressed with the following visual dashboard from IPL T20 tournament - IPL 2013 in Numbers.  For those of you not so familiar with cricket or IPL, IPL is the biggest, the most extravagant and the most lucrative cricket tournament in the world.  I like the way IPL is bringing sports analytics to the common masses.

What is impressive is that each metric (runs, wickets, or tweets) is live so these numbers get updated automatically, pretty cool for IPL and cricket fans.  Also, each metric is clickable so one can drill down to his or her heart's content.  This is a common roll-up analysis but the visualization and the real time updates make this dashboard pretty appealing.  IPL team, thanks for not putting any dials on this dashboard (LOL).

I have been influencing and now building analytics products that power these sports and various other dashboards/reports for many years.  The most fascinating thing is that these dashboards (or lets call it analytics in general) are reaching the masses like never before.  Everyone has heard of terms like democratization of data and humanization of analytics.  This is it!  The data revolution is underway.  

Now, there are many new frontiers to go after and the existing ones need to be reinvented.  Yes, the analytics market is ready for massive disruption.  This is what keeps me excited about Business Analytics space.

Happy Analyzing and Happy Friday!

Agile BI, Simple BI, Self-Serve BI - Okay, What the Hell This Thing is?

In layman's terms, anyone, including my mom, who is suffering from information overload should be able to analyze any data using simple and easy to use data visualization tools, get insights (like growth in milk usage at our home) and then share the results with my dad who should cut feeding expensive organic milk to his two cats.

Wow, that sounds pretty simple, isn't it? Yes, and precisely for that reason IDC says that this phenomenon presents a big market opportunity:

“We are at the forefront of an evolutionary market that is fraught with opportunity for innovative tools and solutions that can help users handle the information overload plaguing every major organization around the globe.” IDC Market Analysis, Worldwide Interactive Data Visualization Tools Forecast

How big of a market opportunity? $1 Billion big by 2013 and $1.6 Billion by 2015 says Gartner. See this graphics:

So someone asked me few weeks ago, how would you define simple, self-serve BI and I gave him the following definition -

Agile BI is a simple yet power-packed solution which is easy-to-use, cost-effective and offers full 360 degree experience and above all my mom should be able to use it without bugging me...

And here is my definition of a power-packed solution:

There are ZERO products that fulfill this vision today.  Products like QlikTech, Spotfire and Tableau do a pretty good job and therefore enjoy more that 70% of the market share. Where are the big guys?

"Agile" and "Big" doesn't go together I guess!

Here is how I contrasted Qlik against a large enterprise BI player:



This story is universal and gives competitive advantage to younger more agile players over their older and aging brethren because they have offered one single self-serve BI tool that could serve to many personas!





Qlik and Tableau have seen pretty solid growth over the past few years as a result of keeping their strategy simple.  Here is an older blog on Qlik showing its amazing growth: http://goo.gl/cyV7a


The most recent evidence of double digit growth in the Agile BI market was seen in Tableau's 2011 earnings: (http://apandre.wordpress.com/)

• sales doubled year over year to $72M in 2011
•  104% growth in bookings in Q4’11 and 94% growth YoY,
• WW customer base grew by 40% in 2011
•  more than 7,000 organizations use its analytics product
•  big growth with customers in Europe, where base grew by 67 percent

2011 was the year of Agile (Simple) BI and the momentum is gaining further strength. Do you know now what Agile BI a.k.a Simple BI a.k.a self-serve BI is defined as?

Happy Simplifying!

Updated Sentiment Analysis and a Word Cloud for Netflix - The R Way!

The Netflix investors must be happy and cheerful as the stock is up more than 78% since the beginning of the year (YES, 78%, Source: Yahoo Finance!).  I am not going to talk about what turned the stock around after a much talked/hyped about Netflix debacle of the late 2011 that earned Reed Hastings quite a few UNWANTED title and every one demanded his resignation from the top post.  Not so fast, Mr. Bear!  Reed Hastings must be smiling!  After a stellar performance this year including carefully released stats on viewership, streaming hours as well as a solid Q4'11 earnings, Netflix is back and most importantly viewers are back!

Well, is is not coincidental that the sentiment for Netflix is also improving, 68% of the tweets now have positive sentiment.  See the table below:

Total	Positive	Negative	Average	Total	Sentiment
Tweets  Fetched	Tweets	Tweets	Score	Tweets
499	171	80	0.281	251	68%

*Make sure you understand and interpret this analysis correctly. This analysis is not based on NLP. 

I updated the sentiment analysis that I did last year, http://goo.gl/fkfPy ,  (I was then just beginning to play with Twitter and Text Mining packages in R) and used advanced packages like "TM" and  "WordCloud".  The new analysis is based on more than 6,800 words which are most commonly prescribed in various sentiment analysis blogs/books. (Check out Hu and Liu http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html)

I came across this excellent blog by Jeffrey Bean, @JeffreyBean, (http://goo.gl/RPkFX) and his tutorial. Thank you Mr. Bean!  Please follow the instructions from Bean's slides and the R code listed there as well as the R code here:

Here is the updated R code snippets -
#Populate the list of sentiment words from Hu and Liu (http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html)

huliu.pwords <- scan('opinion-lexicon/positive-words.txt', what='character', comment.char=';')

huliu.nwords <- scan('opinion-lexicon/negative-words.txt', what='character', comment.char=';')

# Add some words

huliu.nwords <- c(huliu.nwords,'wtf','wait','waiting','epicfail', 'crash', 'bug', 'bugy', 'bugs', 'slow', 'lie')

#Remove some words

huliu.nwords <- huliu.nwords[!huliu.nwords=='sap']

huliu.nwords <- huliu.nwords[!huliu.nwords=='cloud']

#which('sap' %in% huliu.nwords)

twitterTag <- "@Netflix"

# Get 1500 tweets - an individual is only allowed to get 1500 tweets

 tweets <- searchTwitter(tag, n=1500)

  tweets.text <- laply(tweets,function(t)t$getText())

  sentimentScoreDF <- getSentimentScore(tweets.text)

  sentimentScoreDF$TwitterTag <- twitterTag

# Get rid of tweets that have zero score and seperate +ve from -ve tweets

sentimentScoreDF$posTweets <- as.numeric(sentimentScoreDF$SentimentScore >=1)

sentimentScoreDF$negTweets <- as.numeric(sentimentScoreDF$SentimentScore <=-1)

#Summarize finidings

summaryDF <- ddply(sentimentScoreDF,"TwitterTag", summarise, 

                 TotalTweetsFetched=length(SentimentScore),

                 PositiveTweets=sum(posTweets), NegativeTweets=sum(negTweets), 

                 AverageScore=round(mean(SentimentScore),3))

summaryDF$TotalTweets <- summaryDF$PositiveTweets + summaryDF$NegativeTweets

#Get Sentiment Score

summaryDF$Sentiment  <- round(summaryDF$PositiveTweets/summaryDF$TotalTweets, 2)

Saving the best for the last, here is a word cloud (also called tag cloud) for Netflix built in R-

I will be putting the R code up here for building a word cloud after scrubbing it.

Happy Analyzing!

Business Analytics Market - Ripe for M&A Opportunities?

This is not a recommendation to buy any of the companies I am mentioning here. I am just sharing my opinion on potential M&A opportunity in Business Analytics space. The companies in highlighted rows present could be a target of M&A this year. (Click on the image to enlarge it.)

Who could buy: HP, IBM and Oracle (or may be Dell) will likely acquire them in 2011.

There may be some mergers between INFA/QLIK or TDC/TBX or TDC/MSTR or MSTR/INFA or INFA/TDC to build a stronger company and to offer complete business analytics solutions. 

Also see my other post on emerging and fast growing companies in Analytics space.

(Disclosure: I don't have any position in any of the companies.)

From the World of Business Analytics - Miscellaneous Posting

• Business Analytics landscape is changing - CORDA was acquired
Former Omniture CEO Unveils A-List Investors – Acquires Corda (A dashboard company) http://blogs.wsj.com/digits/2011/05/19/former-omniture-ceo-unveils-a-list-investors/
http://www.corda.com/


• On Demand Predictive Analytics: In2clouds - http://www.in2clouds.com/

• 40 BA Vendors We're Watching in 2011 (All eyes on the agile vendors that gained customer traction in the rebound)
http://www.information-management.com/issues/21_2/40-vendors-were-watching-in-2011-10019878-1.html?zkPrintable=true (Source: Information Management Magazine, 03/01/2011)

Predictive Analytics to the Rescue and Beyond! Predictive Analytics to go pervasive!

This segment of the analytics will explode soon. I have never made a prediction but if I were to make one - this will be it. Predictive Analytics have been the thing of super smart, masters of finance or PHD in statistics and/or mathematicians in an organization, not generally IT. This will soon change just like everything else has changed around us with consumerization of IT. Companies like SAS, SPSS (IBM) and bunch of other smaller niche companies offer predictive solutions to companies to make future decisions by analyzing the patterns in the data (all of the statistics: mean, variance, confidence-intervals, distribution, monte-carlo, seasonality, decision trees etc.)

Here are some anecdotal use-cases from the real would on how predictive is helping companies become smarter and more profitable:

 "Some of the most famous examples of analytics in action come from the world of professional sports, where 'quants' increasingly make the decisions about what players are really worth. Consider these examples from the business world:


--Best Buy was able to determine through analysis of member data that 7% of its customers were responsible for 43% of its sales. The company then segmented its customers into several archetypes and redesigned stores and the in-store experience to reflect the buying habits of particular customer groups.


--Olive Garden uses data to forecast staffing needs and food preparation requirements down to individual menu items and ingredients. The restaurant chain has been able to manage its staff much more efficiently and has cut food waste significantly.


--The U.K.'s Royal Shakespeare Co. used analytics to look at its audience members' names, addresses, performances attended and prices paid for tickets over a period of seven years. The theater company then developed a marketing program that increased regular attendees by more than 70% and its membership by 40%."


Source : Forbes: Why Predictive Analytics Is A Game-Changer?

As always, more on this later.. My approach will be to introduce each topic in the Analytics and then go deeper as the opportunity arise. I want to bring more use-cases in future blogs...

Real Time Business Analytics - A new study coinciding with SAP HANA's greater push?

This is a clip from a news article that I just read - 


A recent study by Oxford Economics, "Real-Time Business: Playing to Win in the New Global Marketplace" [PDF], found that 30 percent of the companies surveyed have implemented some kind of real-time IT system, and 65 percent of the remainder have plans to do so in the near future. In addition, early adopters are reporting significant benefits: On average, companies that have implemented such systems are seeing average revenue gains of 21 percent and cost reductions of 19 percent. In fact, among early adopters, 77 percent report revenue gains with even higher revenue gains in certain industries like oil and gas.


It is not a groundbreaking study and definitely a study on this topic has been done before. This is a survey based study and  focuses primarily on the advantages of real time business intelligence using customers' account. Good read. 


The study was sponsored by SAP and its release coincides with SAP's SAPPHIRE NOW event where real-time (In-Memory) business analytics received a disproportionate amount of coverage.  Here is the full press release with seven customer testimonials - 
http://www.sap.com/index.epx#/news-reader/index.epx?category=ALL&articleID=15202&page=1&pageSize=10


Enjoy!

Big Date Beneficiaries - Sybase, Netezza, Exadata and more!

So my last post was about big data. Let's talk what kind of companies are going to benefit from the data explosion and mind you, the big data opportunity can't be discussed in one single blog. Now, the big data story started in 2010 and there are already few beneficiaries:

• SAP acquired Sybase;
• IBM acquired Netezza; and
• Oracle acquired Exadata.

Going into 2011, the big data story is only going to get bigger. Here is an article that I read earlier (from a series of such articles) that discussed 3 predictions for 2011 based on big data - 3 "Big-Data" Predictions for 2011

Prediction #1: Not all data is created equal. Traditional relational database management systems will be challenged in 2011. (other flavors of repositories including columnar, in-memory, Hadoop/MapReduce, and other NoSQL approaches made popular by Google, Facebook, and other large-scale Internet applications.)

Prediction #2: Cloud architecture deployments will grow, specifically for long-term data storage and retention.

Prediction #3: Enterprises will search for sustainable storage.

Highlights:

data retention and management software will benefit from this trend

Informatica

Teradata