Gartner Business Intelligence & Analytics Summit

30 March-1 April 2015 | Las Vegas, NV

Applying the Big Data Ecosystem

Nick Heudecker
@nheudecker

This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use of the intended Gartner audience or other intended recipients. This presentation may
contain information that is confidential, proprietary or otherwise legally protected, and it may not be further copied, distributed or publicly displayed without the express written permission of Gartner, Inc. or its affiliates.
2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Three Types of Business Decisions

Strategic
Decisions

Tactical
Decisions
Operational
Decisions

What
business
should we
be in?
Which
customers to
target?

What polices
for credit card
approval?

Is the call
center meeting
its SLAs?

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

What prices
to use on our
price list?

Did that
particular order
get lost?

What product
specifications
to use?
Is it time to
order more
material?

Send this
package by
air or
ground?
>Traditional analytics programs primarily address tactical
decisions, not real-time operational decisions.
What
cross-sell offer
to make?

Should we
acquire that
company?

Approve this
credit card
transaction?

Use what
dynamic
price?

Adopt New Perspectives on Data

Real-Time

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Interactive

Batch

Key Issues
1. What technologies comprise a big data ecosystem?
2. How do these pieces fit together?
3. What does IT need to consider when crafting an ecosystem strategy?

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Key Issues
1. What technologies comprise a big data ecosystem?
2. How do these pieces fit together?
3. What does IT need to consider when crafting an ecosystem strategy?

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Spoiled for Choice

Enterprise Data Warehouse

54%

Cloud Computing

42%

Hadoop Distributions

40%

Relational DBMS

34%

In-Memory DBMS

25%

Complex Event or Stream Processing

22%

Search-Based Indexes

22%

NoSQL DBMS

20%

High-Performance Message Infrastructure

17%

In-Memory Data Grids

14%

Column-Store DBMS

13%

dbSaaS

10%

Graph DBMS
dbPaaS
Other
Dont Know

6%
6%
4%
15%

n = 218
Source: "Survey Analysis: Big Data Investment Grows but Deployments Remain Scarce in 2014" (G00263798)
5

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Ecosystems Take Shape

Cloud
Computing

RDBMS

CEP/
DSCP

EDW
Hadoop
NoSQL
In-Memory
DBMS

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Hadoop

Utilities
Interactive

Real-Time

In-Memory DBMS
High-Performance
Messaging
CEP/DSCP

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Batch

Column-Store DBMS
Hadoop Distributions

Manufacturing and Natural Resources

Real-Time

High-Performance
Messaging

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Interactive

Batch

Graph DBMS

Enterprise Data
Warehouse

NoSQL DBMS

Hadoop Distributions

Retail
Real-Time

High-Performance
Messaging
In-Memory DBMS
CEP/DSCP

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Interactive

Column-Store DBMS
Search-Based
Indexes
NoSQL DBMS
Relational DBMS

Batch

Enterprise Data
Warehouse
Enterprise Data
Warehouse
Hadoop Distributions

Key Issues
1. What technologies comprise a big data ecosystem?
2. How do these pieces fit together?
3. What does IT need to consider when crafting an ecosystem strategy?

10

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Building Your Ecosystem

Constraints:

Resources
Skills
SLAs
Dependencies
Governance

Applications:

11

Analytics/Monitoring
Fraud Detection and Prevention
Customer 360
Marketing Optimization
Data Normalization
Recommendations

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Source: Jay Kreps, Confluent

But Why Three Contexts?

Real time/streaming:
Latency
Decision automation

Interactive:
Ad hoc use
Decision support
Agile

Batch:
Throughput
Economics
Post transactional

12

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Most application
features are
multicontext

Lambda Architecture
Data
Sources

Batch Layer

Serving Layer

Compute

Batch
Views

Transactions
Log Data
Geographic
Sensor Data
Images
Audio
Video

Emails
Etc.

13

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Speed Layer
Compute

Streaming
Views

Query

Query

Apache Spark Moves Toward Lambda

General-purpose engine for large-scale data processing

Unifies batch, streaming and interactive data processing

Spark
Streaming

Spark
SQL

GraphX

Spark

14

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

MLlib

Key Issues
1. What technologies comprise a big data ecosystem?
2. How do these pieces fit together?
3. What does IT need to consider when crafting an ecosystem strategy?

15

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Understand IoT Architecture Styles

Thing

Gateway

"Smartness"
(Logic/Rules/Apps)

Cloud/"Internet"

Data

Enterprise/
On-Premises

Analytics

Application logic, data and analytics can be placed anywhere

16

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Recommendations
Remember that big data is a means to an end, not an end in itself.
Engage with business stakeholders first to understand use cases and
decision timeframes.

Iterate through your ecosystem development.

Build your ecosystem with applications and constraints in mind.

Align your deployment environment with your data. This may mean completely
on-premises, cloud or hybrid environments.

17

2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Recommended Gartner Research

Extend Your Portfolio of Analytics Capabilities

Lisa Kart and Others (G00254653)

Harnessing Big Data Velocity With Stream Processing

Nick Heudecker and W. Roy Schulte (G00261533)
Survey Analysis: Big Data Investment Grows but Deployments Remain
Scarce in 2014
Nick Heudecker and Lisa Kart (G00263798)
Applying the Big Data Ecosystem
Nick Heudecker and Hung LeHong (G00252014)
Build Your Blueprint for the Internet of Things, Based on Five
Architecture Styles
Hung LeHong (G00269736)
For more information, stop by Gartner Research Zone.
18

2015 Gartner, Inc. and/or its affiliates. All rights reserved.