Big Data

Past, present and (near) future

Dr.Jay B.Simha
ABIBA Systems

Big Data
Past?

What is Big Data

Cost efficiently
processing the
growing

Volume
50
x

2010

35
ZB

2020

Establishing
the
Veracity of
big data
sources

Responding to the
increasing

Velocity
30
Billion

Collectively
Analyzing the
broadening

Variety

RFID
sensors
and
counting

80%

of
the worlds
data is
unstructure
d

1 in 3 business leaders dont

trust the information they use
to make decisions

Source:IBM

Why Big Data

Big Data
Present

Myths about Big Data

Big Data
Big Data
Volume
Big Data
Big Data
Big Data
Big Data

Is New
Is Only About Massive Data
Means Hadoop
Need A Data Warehouse
Means Unstructured Data
Is for
Social Media & Sentiment
http://mashable.com/2012/06/19/big-data-myths/

Sources of Big Data

12+ TBs
of tweet data
every day

30 billion RFID
tags today
(1.3B in 2005)

4.6
billion
camera
phones
world
wide

? TBs

of
data
every
day

100s of
million
s of
GPS
enable
d
25+ TBs
of
log data
every
day

devices
sold
2+
annually

billion

76 million
smart meters
in 2009

people
on the
Web by
end
2011

Gartner Hype Cycle

Big Data

Sample Case Smart metering

Big Data Big Noise

Social Media Analytics

Big data
(Near) Future

Big data for future

Big Data analytics is not just about managing more or

diverse data, it is about the collection of data and what
you do with it. BIG data is just data.
Smart Data is information that actually makes sense.
Algorithms turn volumes of data into actionable insights.
Fast Data or as it happens information which
enables real time decision making.

Smart meter revisited

Past

Presen
t

Monthly

Interval
Energy Data

Energy Data

Future

Real Time
Energy Data

Smart Data from Big Data

Social Media Analytics - Revisited

Go
Go for
for
the
the best,
best,
DP-2000
DP-2000

Buying
Buying
aa DSLR
DSLR
today
today !!

Buying
Buying
DSLR
DSLR
today!
today!

Thrza
Thrza gr8
gr8
deal
deal on
on ZXZX550
550 @
@ the
the
mall
mall

Prior Social
Business
Transactions
Data
250M tweets/day
Michaels online friends offer lots of advice

Entity Extraction,
Fact Discovery,
Intent &
Sentiment

Influencers

Intent

Millions of tweets yield one

company-specific fact Customer ready to buy a
DSLR camera today,
possibly at a nearby mall

Text Analytics used to extract intent from Social Media

Wifeys
Wifeys birthday
birthday tomorrow,
tomorrow, looking
looking for
for aa killer
killer dslr
dslr

Sarcasm,
Wishful
Thinking
Potential
Locations and
Activity

16

Married, Male, Spouse

Birthdate, Gift Type, Intent
to Purchase, Timeframe

Maybe
Maybe II should
should buy
buy her
her that
that purple
purple
roadster,
while
Im
at
it.
;-)
lol
roadster, while Im at it. ;-) lol

Intent to Purchase,
Gift Type?

In
In NYC
NYC area
area this
this w/e,
w/e, any
any good
good malls
malls
nearby?
nearby?

Region & City Location,

Timeframe, Intent to Shop

Big Data to Connected Intelligence

(multiply it by 10 for the whole

world)

Can Big Data:

Lead us to zero road fatalities? (5M crashes, 35K deaths/year
in US)
Increase life expectancy of our generation by 10%? (90 yrs in
Monaco, Chad 49 yrs)
Eliminate deaths from preventable diseases? (1.5M kids die
each year)
Solve the water shortage problem? (750M lack access to water)
Eliminate the risk/deaths in natural disasters? ($2T damage,
2.9M affected, 1.2M killed)
Make education available universally? (60% dont have college
degree in the US)
Reduce healthcare costs by 80%? ($8T annually in the US)
Raise productivity by 100%?
Source: Chetan Sharma Consulting, 2015
Reduce the impact of global conflicts and war? (project Daniel,

Big Data - problems

Recency effect

80+ percent of data is created in

last one year

Big noise searching for needle in haystack

Rise of Singlualrity Evil twins (Big data, AI)
Privacy issues health, communication, finance ;)
Cognitive dissonance - (too much data, too

less human comprehension)

Acknowledgements

Thanks to all the authors who left

their slides on the Web.

I own the errors of course.