Decision Management and the Cloud

Last week Predictive Analytics World brought together a fantastic collection of minds to share case studies and expertise. A theme echoed in multiple sessions and conversations was that analytics are a necessary but not sufficient ingredient for success. To succeed it is critical to have a strong alignment and integration of technology, processes and corporate strategy. Not an easy task but the ROI tends to be irresistible and mastering this act is at the core of Decision Management.

What does Cloud Computing have to do with effective Decision Management? Well, one of the most obvious aspects is better analytics. Cloud computing offers more storage and more processing power at lower costs. More computing power and more data for less money seems quite attractive. Furthermore, these characteristics make predictive analytics accessible to many more organizations and applications. I like to think about it as the democratization of predictive analytics made possible by Cloud computing.

But that is just one part of the story. Predictive Analytics on the cloud will succeed because of the Cloud's standards and open platforms. Operationalizing analytics behind the firewall of a large corporation can require the custom integration several layers of expensive software and in-house applications, e.g.: point of sales systems, call centers, databases, rule/workflow engines, analytic engines/models, etc. The cost and complexity of these projects can easily challenge the most optimistic ROI models.

Fortunately the Cloud is being designed precisely to simplify these scenarios. For example, let's consider Salesforce.com, one of the leading Cloud platforms available today (along with Netsuite and Intuit among others). Salesforce simplifies integration because it has a rich API based on open standards. Salesforce has a powerful workflow engine to automate business processes a flexible data repository and native support of email, social and mobile channels. All of this functionality is available on demand on a pay as you go basis. Similar functionality is becoming more and more popular among cloud platforms but getting it on-premises is far more complicated.

The Cloud can deliver more powerful analytics and also help make them actionable. Now we just need alignment with the Corporate Strategy but we'll leave that for a future post.

Doing more with less

Christopher Musico wrote a well timed piece for DestinationCRM describing how some companies are leveraging predictive analytics to become more competitive. It comes as no surprise that during challenging economic times companies try to be more efficient by making better use of their assets. And what better asset than their internal databases?

I think it is safe to say that by now most companies (large and small) have deployed relational databases and reporting software of varying sophistication. This reporting tools have been asked to answer questions about historical events. Questions about 'what happened', about 'the impact (how much, how many, how often)' and even more specific questions that require drilling down to granular levels of information to understand where exactly the event took place (place, product, point in time). Some more advanced companies have even deployed alerting systems to be notified as soon as certain conditions occur. The natural next step to develop competitive advantages is to deploy analytic solutions that can predict and optimize future events.

This conclusion might seem obvious to many. Christopher's article includes the results of a survey where more than 50% of the respondents intend to deploy predictive analytics in the next 6 months. Predictive Analytics can bring many benefits but effective deployment is no easy task. The software cost, skills shortage and infrastructure complexity can be significant barriers to entry. Not to mention the necessary changes in culture and business processes.

Traditionally, the successful deployment of predictive analytical solutions has been reserved to a handful of large companies with vast resources. It is nice to see organizations like USTA succeeding with these projects. I believe this is a sign of things to come. I think the intersection of Cloud Computing and predictive analytics will create new possibilities for powerful and accessible insight.

More Blue Analytics

IBM continues to position itself as the absolute leader in Business Analytics and Optimization. With the acquisition of SPSS IBM has the deepest portfolio of products and services available under one roof. Oracle also has quality products but it lacks IBM's depth in Business Consulting. On the other hand SAS software and consulting is top notch but IBM's offering is superior because it can enhance its analytics platform with database, hardware and decision management software. It is almost too much for a single company to handle effectively. For sure some new buyers will be concerned with vendor lock-in but I believe most IBM customers will be glad to have access to all of these capabilities from a trusted advisor.

While Cloud Computing has grabbed most of the headlines this year, the developments in Predictive Analytics should not be ignored. I am convinced that the intersection of Cloud Computing and Predictive Analytics will provide the perfect stage for the future of business innovations.

BI and the Cloud

Wayne Eckerson from The Data Warehouse Institute has an interesting post about Implementing BI in the Cloud. He mentions that BI in the Cloud faces four constraints:

1) Customization or application fit

2) Ongoing cost of transferring data to the Cloud

3) Data Security

4) Vendor viability

Wayne wraps up his post with the following conclusion:

BI for SaaS offers a lot of promise to reduce costs and speed deployment but only for companies whose requirements are suitable to cloud-based computing. Today, these are companies that have limited or no available IT resources, little capital to spend on building compute-based or software capabilities inhouse, and whose BI applications don’t require significant, continuous transfers of data from source systems to the cloud.

I tend to agree with the following high level thoughts:

1) BI on the Cloud is not for everybody (yet)

2) Due diligence is necessary to reduce risks on data security and vendor viability

But Wayne's post raised several questions in my mind:

Integration. There are several ways to customize an application. For example, a multi-tenant architecture like Salesforce.com offers endless possibilities to customize and extend every single instance. Are these customizations unprofitable? No they are not, they are part of the application and they do not require changes to the underlying code. Can the same level of customization apply to BI? Absolutely. Wayne mentions briefly Platform as a Service but his focus is towards custom application development (although his chart shows "DW as a Service"). An intriguing approach to offer a BI platform as a service would be to setup something like MicroStrategy and configure it in a multi-tenant fashion. The underlying layer of IaaS would support the data repository while the top layer of SaaS could support ad-hoc reporting, vertical applications or full customizations on top of their API. Would this be unprofitable? Not at all. Wayne makes another good point regarding integration:

So, unless the SaaS vendor supports a broad range of integrated functional applications, it’s hard to justify purchasing any SaaS application.

But from my experience, successful enterprise wide deployments need to focus on integrating subject areas at the data level. This is an architecture and design challenge. A well integrated data repository will support integrated functional applications seamlessly. It is about the underlying data not only the application.

Ongoing Data Transfers Costs. Is this really a significant constraint? How much data does the typical Data Warehouse has to incorporate on a daily basis? The cost to transfer data to Google's App Engine is $0.10 per GB. Moving a TB a day would cost around $3,000 per month (I'm not suggesting using BigTable as a DWH repository yet). As Data Warehouse costs go, this does not seem unreasonable. Amazon is running a promotion right now that would bring that cost down to $1,000; hardly a deal braker. Latency and complexity can complicate this data transfer. This is to be expected because 99% of them were not designed with the Cloud in mind. Which brings me to my final point.

I mentioned using MicroStrategy as a BI platform on the cloud as an example to make a point. I believe that successful Cloud applications need to do more than just cloning their on-premise counterparts. They need to leverage the Cloud inherent qualities, for example elastic computing power. The nature of the Cloud can enable ongoing ETL: receive a copy of the transaction on the fly via a web hook, cleanse, transform and aggregate in real time or a few times a day at least. How about Map Reduce? I think this technique will allow to create more powerful analysis over more data, faster and easier.

Rigid applications built with yesterday's patterns will struggle to survive, in the Cloud or elsewhere. The Cloud is an open environment by definition, its openness will facilitate the integration of multiple data sources from inside and outside the corporate firewall. This integration will support a next generation of cross-functional applications. Bandwidth and storage costs continue to drop very rapidly and will cease to be a major consideration in the near future. New design principles (e.g. scale out vs. scale up) will enable more sophisticated analysis over ever larger datasets (Google analyzes over a PetaByte of data every day). With over $1B in sales Salesforce.com is the most successful SaaS provider. They host more than 55k customers, well over 1M users and every day execute more than 30M lines of customer code. If they can do it, I'm convinced the next BI leader in the Cloud will do it as well. That is how I see it.

Are we answering the wrong questions?

Lyndsay Wise wrote a good article on about different types of Business Intelligence (BI) and how organizations adopt them based on their level of BI maturity. It reminded me of a recent user group meeting where several people from a Fortune500 company discussed their difficulty managing the ever growing list of sales reports requested by their users. The business users were not finding the answers they were looking for and they hoped that having more reports would help answer their questions. Quantity over quality.

I think this is a reflection of how the evolution of BI has been driven by IT and not Business. This same reason is a common obstacle for BI projects and a contributor to elusive ROI. Looking around it is easy to find many vendors offering "Sales Dashboards". You can get them on a browser with AJAX or Flash, you can get them on your iPhone and even integrate them in your favorite SFA platform or portal. 

This is nice but when you look at the actual reports, they are still pretty basic. These dashboards show charts such as: Revenue and Win Rates Trends, Revenue by Industry/Region/Quarter, Variances over Plan, Count of Deals by Age, etc. These are important questions but companies have been looking at similar reports for the past 20 years. The technical delivery has improved (faster, better, easier) but the actual business content is still lagging.

What would I like to see instead? Well, if I was a sales executive I would be looking for information that can drive action. Something to tell me "what to do" and "what to stop doing" (beyond 'pick up phone and call a Region Manager to ask him why is he/she behind plan'). Knowing that my win rate for last quarter was 25% is fine but I want to know why? What are they key contributing factors? How do I increase it to 30%? 

Technology has had its 15 minutes (years?) in the spotlight. I think it is time to turn our focus on Business. This new focus will drive innovation and will ultimately make companies more effective and more competitive. Of course, I think the solution is Predictive Analytics and I will explain why in a future post.

Customer Experience beyond Customer Feedback

Customer Experience analysis and management has gained a lot of popularity as a business intelligence application. This popularity is due in part to advances in text processing technology as well as the exponential growth of unstructured data (i.e. blogs, email, IM, twitter, etc.)

A common analysis parses customer feedback to identify problems or causes of dissatisfaction. For example measuring the sentiment (positive or negative) of a hotel guest after a stay. This is an important metric for the hotel management along with identifying the root cause of that sentiment. However, I hope that we pay enough justice to these applications and consider all of their capabilities and potential. Otherwise our vision could be too narrow and a narrow vision is risky for adopters, providers and the industry in general. 

The vendors of Customer Experience software and methodologies offer depth that goes beyond the simplistic example of customer sentiment. In my opinion Customer Experience needs to be analyzed in the context of Customer Life-cycle and Customer Value. 

Once we have identified and ranked the key factors that drive customer's sentiment, we need to look at those rankings across a number of dimensions, including time and geography but most importantly customer segment. After tracking both sentiments (both positive and negative) across customer segments we need to overlay financial metrics at the customer level and at the company level. How is this sentiment affecting profitability and how big is the impact. For instance "... because the A/C was too loud in these locations, our business traveler segment reduced their number of stays by X which caused a drop in margins of Y ..." These type of analysis would offer a clear and meaningful ROI analysis to justify and champion initiatives to manage and improve Customer Experience. The next step of course is enhancing these analysis with predictive analytics to create stronger leading indicators and react before the problems appear.

Comment on this blog or email if you have any thoughts on this topic. If you are a vendor and have a case study that touches on these topics let me know as well, I'd love to write about it in this space.

McKinsey and the Cloud

McKinsey&Company just released an interesting document on Cloud Computing: Clearing the air on cloud computing. Very interesting thoughts. I agree with the idea that over hyping Cloud Computing (and any other new technology) is risky and when done on purpose, irresponsible. I also liked their Cloud definition, it seemed pragmatic, down to earth.

I was surprised by their conclusion that AWS would not be cost effective for large corporations. I know AMZN has some large customers and I'm sure they will have some follow up commentary. In terms of the cost analysis, I think the author is missing two points. First, I think that the effort to initiate or further deploy virtualization in the corporate data center has a not zero cost. Starting from training and support. It obviously does not happen overnight either. Secondly and more important in my opinion is the opportunity cost. I believe that the financial rewards offered by the Cloud's speed to market far outweigh the potential incremental cost (assuming they are correct and it is more expensive for large corporations--I have my doubts). 

For example, let's take a hypothetical example of a multi-billion dollar media company, that would be a large corporation in my mind. They need to analyze 4 to 6 TB of clickstream data every month to fine tune their advertising efforts. The ability to execute on their strategy could easily bring additional revenues in the 8 digit range. They have two options: 1) go with their current data center 2) deploy MapReduce/Hadoop at AWS. Option 1 would easily take 6 to 8 months to complete. Option 2 could be up and running in days at most. To me that speed to market is priceless. In the short and long term.

Blue Analytics

IBM is launching a new consulting organization to focus on Business Analytics. This is a very significant move that should bring a lot of positive developments to the industry. IBM has tremendous experience in business consulting and unmatched technology assets to deliver a complete an actionable solution. 

I am convinced that Analytics will be a key differentiator in years to come. Companies will need to compete with Analytics to remain competitive. The technology is available and the current economic conditions - along the need for better risk management - will foster unprecedented innovation. Welcome to the Analytics generation.

AMZN AWSome

Well, Amazon strikes again. MapReduce (Hadoop) on demand. Although AMZN already offered some Hadoop pre-configured AMIs, the simplicity of this new packaging makes it much easier. Furthermore, it is synergistic with EC2 and S3.

I have been using Amazon Web Services for close to a year now and they continue to surpass my expectations. I wouldn't be surprised if AMZN spun off AWS and filed for an IPO sometime next year. It is not easy to isolate AWS's revenues from AMZN financial statements but with customers in 96 countries and a super scalable business model I have to believe this is a cash machine for them. These folks are brilliant. 

Many people often relate the Cloud to pure storage and CPUs as in pure hosting. AMZN goes up one level and provides application services. SimpleDB and SQS are good examples, now Elastic Map Reduce is another one. These are higher level application services on demand, industrial strength and world class.

A quote from Spiderman comes to mind: "... with great power comes great responsibility". What would you do with all this power?

SAS, analytics and the cloud

SAS got a lot of press coverage when it announced it would invest $70M in a new data center. There is no doubt SAS is a world class company, clear market leader. In a 2008 survey by Rexer Analytics, 45% of respondents reported they use SAS. Almost 1 in 2 data miners? not bad.

I did not know SAS had a hosting service. As they said, their hosting business has grown with almost no advertising but late last year I heard they were pitching to one of the top media companies. I was surprised at first but it really makes a lot of sense and their recent announcement confirms the solid traction that business is getting.

After thinking about it a little more I'm very curious to see how their SaaS offering will play out. I believe the Analytics market is prime for a big disruption. The market is dominated by a handful of companies with relatively closed technology (at least in one direction) and significant profit margins. Open Source projects like R have shaken the game a bit but nothing earth shattering yet. I believe the Cloud -with its limitless storage and cpu power- will bring a more disruptive wave.  Will SAS take the lead? Can they embrace the power of Hadoop and in-memory databases to take their business to the next level? Or will they play conservatively, milking their current cash cow and using their market dominance to crush their smaller competitors? 

This will be a fascinating race.

Customer Experience and Analytics

Last year I spent some time learning about Customer Experience and Customer Interaction Management. I enjoy both topics quite a bit. More recently due to my work with predictive analytics I have come across a number of Text Analytics companies that are focusing on improving the Customer Experience. From my experience in the Automotive and Pharmaceutical industry I know there are vast amounts of unstructured data repositories buried in most corporations; and more is generated every day.

I have found however a bit of a disconnect between decision management, text/predictive analytics and customer experience management. I think the effective management of the customer experience starts with a clear strategy that defines what the desired experience should be across customer segments. This strategy needs to be aligned with the corporate goals and have corresponding financial indicators. Once a strategy is in place, analytics can be used to detect specific behaviors or recommend the next best action. These recommendations can be as a predictive score or an alert to a customer service, sales representative or customer touchpoint. But then there is the need for a decision management system that would take the analytics' result and act on it. These decision management system needs to have rules managed by those business users responsible for managing the Customer Experience strategy.

Today I see a chasm where vendors focus just on the analytics or just on the strategy or just on the rules management. It seems to me that there is a big opportunity to bring all those pieces together in a proactive, money making, recession proof solution.

Text Analytics

Sid Banerjee from Clarabridge wrote an interesting post on text analytics. I don't know if Clarabridge has a RESTful API. If it does, I think it would be interesting to use it to create a mashup with the White House blog. The Obama administration has asked the public to submit comments and ideas on a wide variety of topics ranging from health care to the economy. I believe in some cases they have received tens of thousands of emails. I wonder how are they processing them?

With a Clarabridge mashup we could analyze each blog posting from the Obama administration but more importantly we could use it to analyze the citizen's feedback. We could then use this analysis to track the government progress and responsiveness.

In addition to the WH blog I would like to use the API to process the RSS feeds from Recovery.org. Talk about efficiency and transparency, I can't think of a better way of doing it.