The Top 10 Emerging Trends in Business Intelligence, Analytics, and Performance Management

Nenshad D. Bardoliwalla
March 2010

The slides here are a subset of what was presented in person. Much of what was discussed plus additional details that could not be covered due to time constraints can be read in this December 2009 blog post here: The Top 10 Trends for 2010 in Analytics, Business Intelligence, and Performance Management

The Top 10 1 through 5

1.

2.

3.

4.

5.

We will witness the emergence of packaged strategydriven execution applications. The holy grail of the predictive, real-time enterprise will start to deliver on its promises. The industry will put reporting and slice-and-dice capabilities in their appropriate places and return to its decision-centric roots with a healthy dose of Web 2.0 style collaboration. Performance, risk, and compliance management will continue to become unified in a process-based framework and make the leap out of the CFOs office. SaaS / Cloud BI Tools will steal significant revenue from on-premise vendors but also fight for limited oxygen amongst themselves.

The Top 10 6 through 10

6.

7. 8. 9.

10.

The undeniable arrival of the era of big data will lead to further proliferation in data management alternatives. Advanced Visualization will continue to increase in depth and relevance to broader audiences. Open Source offerings will continue to make in-roads against on-premise offerings. Data Quality, Data Integration, and Data Virtualization will merge with Master Data Management to form a unified Information Management Platform for structured and unstructured data Excel will continue to provide the dominant paradigm for end-user BI consumption

Number One
We will witness the emergence of packaged strategy-driven execution applications.

Strategy-Driven Execution
Order Fulfillment Goal Fulfill orders of most profitable customers within 1 day Defection Risk Will this customer leave to a competitor?

... within process

Inventory Forecast Will this be back in inventory?

Creation Purchase Order

Creation Sales Order Fulfillment Request

Request Global ATP

Global ATP Check

Sourcing Control Are suppliers for critical parts delivering on time?

Creation & Release Delivery Request

Delivery Execution

... about process

Receives ASN

Release ASN

Update Inventory Accounting

Update Inventory

Delivery Performance How effective is our fulfillment process?

Managing Performance and Risk in a Down Economy A Practical Approach to Optimizing Business Performance for Everyone , May 2009

Number Two
The holy grail of the predictive, real-time enterprise will start to deliver on its promises.

Surge of Adoption in Advanced Analytics

100 Data Collection
Complex Event Processing Text Mining Audio and Speech Mining

75
Terabytes of Data

Insight Creation High-Volume Data Mining

Time Series Analysis

Social-Network Analysis

50 Decision Support

Data Visualization Forecasting

Intelligent Decision Management Constraint-Based Optimization

25

1960 1970

1980

1990

2000

2010

Time
Adoption Patterns for Advanced Analytical Techniques, Gartner Research, April 2008

Complex Event Processing

Analysis of streams of data flowing from live sources for patterns and significant business indicators for the purpose of taking action
CEP recognizes patterns of events.

Event A followed by event B within 30 seconds Five occurrences of event X within five minutes

CEP handles high event throughput.

Thousands of RFID tags per second Thousands of stock ticker changes per second

CEP can handle events in many business and application contexts.

Airline baggage handling Algorithmic trading Claims processing

Event Processing Breaks Out; These Are the Top Products, Forrester Research, August 13, 2009

Number Three
The industry will put reporting and slice-anddice capabilities in their appropriate places and return to its decision-centric roots with a healthy dose of Web 2.0 style collaboration.

Decision Management Solution Components

Decision Management - A Strategy for Organizationwide Decision Support and Automation, IDC, May 2009

Automating Operational Decisions

Heres how decisions and rules relate, JT on EDM, March 2009

Number Four
Performance, risk, and compliance management will continue to become unified in a process-based framework and make the leap out of the CFOs office.

Unified Strategic Business Process

Bringing it All Together
Understand the Corporate and Departmental Context

Build Strategic Action Plan and Initiatives

Cascade

Develop and Set The Strategy

Assign KPIs Goals and Set Targets KRIs Risks and Set Thresholds Controls Compliance and Set Effectiveness Tests Risk-Adjusted Financial Plan Risk-Adjusted Operational Plan

Model

Optimize

Goals
KPIs

Risks
KRIs

Compliance
Controls

Risk-Adjusted Forecast

Monitor

Analyze

Adjust

Managing Performance and Risk in a Down Economy A Practical Approach to Optimizing Business Performance for Everyone , May 2009

Unified Strategic Business Processes Embedded in Execution Business Processes By Horizontal

Commodity Risk

S&OP

Customer Attrition Risk

Sales Forecasting

Supply Chain

Demand Chain

Finance
SOX

Managing Performance and Risk in a Down Economy A Practical Approach to Optimizing Business Performance for Everyone , May 2009

Number Five
SaaS / Cloud BI Tools will steal significant revenue from on-premise vendors but also fight for limited oxygen amongst themselves.

Do we really need MORE buzzwords?!!

TDWI - Making Sense of Software-as-a-Service BI, Wayne Eckerson, June 25, 2009

What do we mean by the cloud?

Huge Resources: The illusion of infinite computing resources available on demand, thereby eliminating the need for Cloud Computing users to plan far ahead for provisioning No Commitment: The elimination of an upfront commitment by Cloud users, thereby allowing companies to start small and increase hardware resources only when there is an increase in their needs Pay by the Drink: The ability to pay for use of computing resources on a short-term basis as needed (e.g., processors by the hour and storage by the day) and release them as needed
Above the Clouds: A Berkeley View of Cloud Computing, Feb 10, 2009

Peeling back the onion

Toward a Unified Ontology of Cloud Computing, Youseff et al., 2009

Common Concerns for Cloud BI

Security

Integration Data
SLAs Pricing

migration

Number Six
The undeniable arrival of the era of big data will lead to further proliferation in data management alternatives.

We must master parallelism

No 5 GHz chips any time soon. Massive multi-core systems are only way to continue scaling. But parallel programming is REALLY hard. HUGE amounts of data generated by all companies. At these data volumes, calculations need to move to the data or network I/O is saturated.

Hadoop vs. Relational Databases

Hadoop:

A data grid operating system Stores Files (Unstructured) Stores 10s of petabytes Processes 10s of PB/job Weak Consistency Scan all blocks in all files Queries & Data Processing Batch response (>1sec)

Relational Databases:

An ACID Database system Stores Tables (Schema) Stores 100s of terabytes Processes 10s of TB/query Transactional Consistency Lookup rows using index Mostly queries Interactive response

23 How Hadoop Revolutionized Data Warehousing at Yahoo and Facebook, Amr Awadallah, August 5, 2009

Row versus Column-Based Storage

EmpID 1 2 3 Lastname Smith Jones Johnson Firstname Joe Mary Cathy Salary 40000 50000 44000

Serialized As: Row: 1,Smith,Joe,40000;2,Jones,Mary,50000;3,Johnson,Cathy,44000; Column: 1,2,3;Smith,Jones,Johnson;Joe,Mary,Cathy;40000,50000,44000;

Columnar Better Aggregates over many rows with few columns Adding new individual columnar values

Row Better Aggregates with many columns from same row Adding new rows when all columns present

Column-Oriented DBMS, Wikipedia, Snapshot Taken onMarch 8, 2010

Why does this matter to me? $$$$$

If you need to manage100+ TB, Structured and Unstructured Data

Oracle and others - $XXX,000,000

Hadoop + Other Components $XXX,000

Once you hit Multi-TB range, getting an RDBMS to scale becomes VERY expensive!

Number Seven
Advanced Visualization will continue to increase in depth and relevance to broader audiences

Reports dont inspire!

BI must be interactive

Visual Examples, Tableau Software, March 2010

BI must be mobile

RoamBi: Dynamic Data Visualization for the iPhone, Infosthetics, June 2009

Number Eight
Open Source offerings will continue to make in-roads against on-premise offerings

Open Source is a viable alternative in EVERY part of the stack

Full stack: Pentaho, Jaspersoft Charting: JFreeChart Reporting: BIRT OLAP: Mondrian, Palo Statistics: R ADBMS: LucidDB Parallel Processing: Hadoop CEP: EsperTech Data Integration/Data Quality/MDM: Talend

Number 9
Data Quality, Data Integration, and Data Virtualization will merge with Master Data Management to form a unified Information Management Platform for structured and unstructured data

The Information Management Platform

Data Integration Master Data Management Data Virtualization

Data Quality Text Analytics Metadata Management

Driven to Perform: Risk-Aware Performance Management From Strategy Through Execution, April 2009

Number Ten
Excel will continue to provide the dominant paradigm for end-user BI consumption

Excel: Not going anywhere for a while

Excel 2010 with enhanced charting, server option (Excel Services) SharePoint 2010 PowerPivot

PowerPivot

A Peek Inside: The client architecture, PowerPivotGeek, November 11, 2009

Thank You!

Questions? Contact me any time at nenshad@gmail.com or @nenshad on Twitter.