Iot Presentation Day 2

1 IoT data from network’s edge to the IBM IoT database engine © 2016 IBM Corporation
• Agenda
• Introduction, what’s driving IoT data creation and use today?
• Introducing IoT data and requirements to store it efficiently
• The IBM IoT engine, from gateway to enterprise deployments
• A brief look at TimeSeries
• A slightly deeper look at the IBM IoT engine’s capabilities

• A brief look at replication
• Hands-on lab preview
• The operational database is a dynamic market segment evolving
from IMS in the 1960s to fast data and transactional capabilities of
today
• Here are the market trends that have shaped our strategy for this
segment:
• Open Source
• Cloud
• Real time operational
intelligence (HTAP)
• IoT
• Mobile
• Multi-structured
• IBM has been delivering Internet of Things solutions for a Smarter
Planet since 2008
INTELLIGENT
Leveraging the data

generated by digital
technology provides
INTERCONNECTED intelligence to help us
do things better,
In the globalized, improving our
networked world, responsiveness
people, systems, and ability to predict
INSTRUMENTED objects and processes and
are connected, and optimize for future
they events
Digital technologies
(sensors and other are communicating
monitoring devices) with one another in
are being embedded entirely new ways
into many objects,
systems and
processes
• All Internet of Things use cases have a common set of fundamental
requirements
• Easily provision / configure any connected “thing”
• Create a real-time communication channel with the “thing”
• Begin capturing and consolidating data from the “thing” and store it in an
enterprise-class DB
• Provide access to the collected data
• There can be extended requirements as well

• Provide a layer of analytics on the data in both real-time and on historical
trend data
• Be able to use all enterprise data simultaneously, regardless of type
• Proactively monitor sensor data and provide insights into usage
• Trigger events based on specific data conditions
• Interact with the “thing” from business apps and/or from mobile devices
• IoT requires highly optimized spatial and temporal data processing
• Many IoT applications have a spatial component to them
• Vehicles, cell phones, even pets… tracking is common
• In these cases both location and time is important
• Show me the vehicles that have passed by location X in the last hour
• Where has my car been over the last few hours?
• The IBM IoT engine is optimized to allow you to combine time series
and spatial data in the same query along with other data types
• IoT requires the ability to recognize patterns and predict events
• Industrial IoT is reaching a point of stabilization and standardization
• Now looking for predictive analytics for anomalies and deviations
• The IBM IoT engine is optimized to find these patterns
an abnormal power usage pattern
Similar patterns found
• Internet of Things data is highly diverse and requires an engine that
can handle any and all data types
• Can include both standard and non-standard data types, must be able to
store all kinds of data efficiently
• Char or numeric
• Time stamped data
• Spatial
• More
• Being able to analyze it

• Where did it come from? What does it mean? How can I profit from it?
• Integrate it with the systems of record or standard, corporate data

• Must be able to use IoT data in conjunction with traditional data to understand
and make decisions
• Satisfy audit, accountability, privacy requirements

• While system of engagement data has historically operated outside of the
normal enterprise data regulations and control, that is changing
• When talking about IoT data, most people think of NoSQL and
NoSQL databases
• What is a NoSQL database?
• A non-relational, object-like database system
• Some of its terminology will be very familiar to traditional database functions
• Uses a low powered single threaded, process-based architecture
• Uses a dynamic and flexible schema

• Any given “row” in a “table” can look different than the other “row(s)”
• Is focused on single “row” retrieval, there are no joins
• Does not have/enforce ACID rules of transactional integrity

• ACID – Atomicity, Consistency, Isolation, Durability
• Usually only has a single, database-level lock, so multi-row actions are fraught with
risk, though consistency can be achieved
• Has very good horizontal scaling but must use “sharding” to achieve vertical
scaling. However, this introduces latency to the operations
• Other characteristics of a NoSQL engine include
• Can not enforce business rules such as referential integrity or other
constraints easily
• There are options but the end result is either more I/O to return a record or
bloated storage because all the RI’ed data is stored redundantly inside each
object
• Can not perform operations on multiple objects easily (aka multi-row

transactions)
• It’s possible but involves additional programming to invoke and verify that
each target record was, in fact, affected
•where conditional query clauses are strongly discouraged or non-

existent
• For example, select city from zip_info where state = TX has to
be written and executed from within a JavaScript function
• NoSQL databases are appealing to application developers because
• Object “schema” is included as an object within the data itself, not
managed separately. Allows for constant unit test and retest as logic
expands.
• “Data schema” can change quickly as business needs change
• Impact: Applications will have shorter production lives and even shorter life-
cycles
• No need to normalize or design in advance, can just wing it
• Less bureaucratic
• Stripped down IT organization
• Programmer is the DBA
• A large disadvantage is that scalability is very limited, the only solution is

to throw more hardware at the problem (aka “sharding”)
• It’s also extremely difficult to implement or maintain data standards
• Another key differentiator of a NoSQL database engine is that it
doesn’t attempt to do everything a traditional database can do
• It is a bit bucket for storage and retrieval
• All processing is done at the application level
• Advantage: the NoSQL engine is pretty quick
• Disadvantage: the NoSQL engine doesn’t have administrative or
functional refinements
• For example, it doesn’t handle reusing free space well resulting in storage
bloat, if data is moved around a lot
• IBM offers two types of data support for Internet of Things / NoSQL
environments
• Simple NoSQL storage and services with Cloudant
• A silo database, it can only store and retrieve NoSQL data types
• Application-based data processing only
• Informix, IBM’s Internet of Things database engine

• Plug-and-play compatibility with NoSQL databases like MongoDB and
Cloudant, including sharding if requested
• Full integration of NoSQL data types with all standard and non-standard
enterprise data and data types
• Includes spatial and time centric data
• Provides server-based data processing to enhance applications
• Supports advanced analytics with Cognos or other tools in Informix
Warehouse Accelerator
• IBM has positioned Informix as the full Internet of Things database
engine when:
• Simple NoSQL storage is not sufficient
• System of record and system of engagement data must be integrated
together
• Enterprise level rules / requirements must be applied to Internet of
Things data
• Potential use cases:

• Large data volumes
• High velocity of data changes / captures
• Broad range of data types, including standard and non-standard / NoSQL
• Complete integration within the enterprise data stack with all that entails
• ACID, transactions and more
• Deep analytics must be executed on NoSQL data
• The business value of this integration is two-fold
• Hybrid storage – reduces cost and complexity
• IoT and relational data stored in the same engine, in the same database, in
the same table if needed
• Both classes of app can use a single engine
• Utilize the strengths and features of both types of database engines
• Hybrid applications for new business patterns

• An application can use both types of data simultaneously
• Including in the same access statement
• Full join capability within and between the two types of data
• Additional business insight from using both types of data as an integrated
whole
Informix, the ideal database for application development and
deployment
• Supports all the data types and languages for rich, next-gen apps
• With this functionality, the IBM message to the computing world is
simple and straightforward
• To the NoSQL world:

• The NoSQL databases are great and exciting technology and are filling a
valuable place in the computing landscape. But very quickly, you’ll need more,
whether it’s scalability, security, regulatory compliance, data protection,
analytics or a host of other features. Informix enables you to have both, right
now, without having to change your applications. As you want or need more,
you can leverage it without to migrating to a new infrastructure.
• To the structured SQL world:

• Data processing is changing and you need to change with it. Informix is the
IBM solution for integrating unstructured data into and along side traditional
data stores. Informix is the only product that proves the dynamic elasticity and
full data processing, including warehouse analytics, as well as data lifecycle
management for structured and unstructured information.
• Let’s look at a specific use case for IoT data -- edge-of-the network
devices
• Perhaps it’s oven or other mechanical sensors, on-patient medical
devices, scientific / field data capture devices, and so on
• Is it reasonable to expect these devices to communicate directly

with the main corporate data processing systems?
• Is it even possible due to connectivity or data volume issues?
• From IBM’s perspective – usually no

• While there may be situations where the devices must be connected, it
would be better for some data processing to occur closer to the collection
site and only send back what is truly relevant or important to the main
corporate system
• In other words, an intermediary or “gateway” system that can run a

full, enterprise class, IoT solution
• How do smart gateways help IoT solutions?
• Reduces cost
• Reduces cloud storage by filtering / aggregating / analyzing data locally
• Reduces cloud or on-prem hardware requirements by pre-computing values
• Reduces latency
• Intelligent gateways can detect and respond to local events as they happen
rather than waiting for transfer to the cloud
• Performs 80% of simple operations locally
• Security
• Gateways allow customers to capture and get value from their sensors without
sending data to the cloud
• Protocol consolidation
• Cloud does not need to deal with the hundreds of IoT protocols
Over time more and more of the IoT processing will move from the data center to gateway
devices
• What are the requirements for a gateway database?
• The database management system must:
• Have a small install footprint, less than 100 MB
• Run with low memory requirements – less than 256 MB
• Use lossless compression to minimize storage space
• Have built-in support for common types of IoT data like NoSQL, time series
and spatial / GIS data
• Simple application development supporting NoSQL, SQL, and REST
• Require absolutely no administration
• Be able to network multiple gateways together to create a single distributed
logical database
The database must be powerful enough to ingest, process and analyze data in
real-time
• Informix is the database for gateways, to feed cloud / on-prem large
scale operational / analytical systems
• Simple to use
• Hands-free operation – no administration
• Supports popular interfaces such as REST & MongoDB as well as traditional
SQL interfaces (ODBC / JDBC)
• Handles SQL and JSON / NoSQL data in the same database
• Seamless data replication and HA solutions to move or copy data where
needed
• High performance
• Unique support for TimeSeries and Spatial data
• Stream data continuously into the database
• Run analytics as data arrives
• Dynamically add and update analytics when needed
• Time oriented storage is typically 1/3 the size compared to other vendors
IBM Internet of Things
Flexible end-to-end analytics and data management
Deep analytics zone
Streams
Analytics zone
Uses Direct to on-prem or

Smart gateways gateway into the cloud
with real-time analytics
Devices / sensors
• What is time oriented or “TimeSeries” exactly?
• Time series data is a series of events that occur over time that must be
recorded
• Can occur at regular intervals (every second, 5 minutes, hour etc.)
• Can occur at random times
• Can have “on” and “off” periods where events do not occur or are not recorded
• When storing the data, the time value is significant

• The data is accessed based on the time value
• Is usually read in ascending or descending order
• Conceptually, people think of time-oriented events or data as
sequential or array-like
• “A” happens first, followed by “B”, then “C” etc.
• This can’t be modeled in a relational database without creating very

deep and unwieldy tables
Stock Timestamp Price Volume
• For example, to store stock
ABC 2015-01-05 09:30:00.00000 12.34 100
trade information, relational
table would look like this with XYZ 2015-01-05 09:30:00.01000 9.34 1000
multiple indexes ABC 2015-01-05 09:30:00.19000 12.44 10
KCD 2015-01-05 09:30:01.05000 194.00 10000
• Each event has to have its own ABC 2015-01-05 09:30:01.70000 12.54 250
row with redundant data
• Problems with a relational approach
• Slow Performance
• Extremely slow data access, especially for ordered set of rows, due to the
data layout and disk I/O
• DML operations are hard or impossible to do in standard SQL
• High Storage Requirements

• Time series are usually stored as "tall – thin" tables with a very large number
of rows
• May need one index to enforce uniqueness and another for index only read
• More space used for index than data
• Huge space requirements in standard relational layout, due to the volume
(number of rows) and data (length of each row with its redundant data)
• Complex Querying
• Can be difficult to write SQL to work with the data
• Informix TimeSeries provides native support for TimeSeries data
• Store TimeSeries elements as an ordered set
• Uses less space because the "key" is factored out and the time field takes
either 0 (for regular) or 11 ( for irregular) bytes
• Access is faster than index-only reads
• SQL can be made much simpler
• Freedom to manage TimeSeries data:

• Freedom to choose what and how it is stored
• Freedom to choose the TimeSeries interval
• Freedom to choose where the TimeSeries is stored
Stock Ticks
ABC 2015-01-05, daily, {(12.34,1234567), (12.56,950000), (12.34,5555567),..}
KCD 2015-01-20, daily, {(199.08,678900), (198.55,345000), (198.12,850250),..}
XYZ 2014-08-25, daily, {(9.34,890567), (9.56,989000), (9.40,1000780),..}
• When looking at time-oriented data that can be stored in
TimeSeries, there are two components
• “header” data – the data that doesn’t change and is used to enforce
uniqueness
• One or more attributes such as: meter ID, location, meter type etc.
• This data is stored once, not redundantly
• “payload” data – the data that does vary from one captured moment in
time to another
• This data is loaded into the TimeSeries
Header Payload
Stock Ticks
ABC {(12.34,1234567), (12.56,950000), (12.34,5555567),..}
KCD {(199.08,678900), (198.55,345000), (198.12,850250),..}
XYZ {(9.34,890567), (9.56,989000), (9.40,1000780),..}
• TimeSeries provides a significant performance advantage
• A real world benchmark against “another database”
• Comparison of time series data stored in a standard relational table
and in the Informix TimeSeries table
Relational TimeSeries
Storage space Stores one row for every Significant reduction in
record. Duplicates the disk space needed to
information in non-time store the same data. The
series columns. Stores index size on disk is also
timestamps. Null data smaller.Stores all time
takes as much space as series data for a single
actual data. The index source in the same row.
typically includes the time No duplicate information.
stamp column and several Calculates instead of
other columns. stores the time stamp.
Null data does not require
any space. The index
does not include the time
stamp column.
Relational TimeSeries
Query speed Data for a single source Queries that use a time
can be intermixed on criteria require many
multiple data pages in no fewer disk reads and
particular order. significantly less I/O. Data
is loaded very
efficiently.Data for a single
source is stored together
in time stamp order.
Query complexity Queries that aggregate Less application coding

data or apply an and faster queries.Allows
expression can be difficult complex SQL queries and
or impossible to perform analysis. Allows custom
with SQL. Much of the analytics written using the
query logic must be TimeSeries API.
provided by the
application. TimeSeries SQL routines
to run queries on time
series data.
IoT data from network’s edge to the IBM IoT database engine © 2016 IBM Corporation
Advanced Analytic Functions (TimeSeries)
- Advanced analytics functions provide specialized methods of

analyzing time series data for patterns or abnormalities.
- You use advanced analytics functions to perform the following types
of analysis:
• Search based on specific measures like similarity, distance, and
correlation. Find the portions of a sequence which are related to a given
pattern.
• Quantify similarity, distance, and correlation between two sequences
using the Lp-norm, Dynamic Time Warping, or Longest Common
Subsequence method.
• Detect anomalies within time series data.
Requirements for Advanced Analytics Functions
The time series definition must conform to the following requirements:

• The columns in the TimeSeries subtype on which you run advanced
analytics queries must have one of the following data types:
SMALLFLOAT, FLOAT, SMALLINT, INTEGER and DECIMAL.
• The time series must be regular.
Requires the Advanced Analytics Functions extension

• Automatically registered; can be found in
$INFORMIXDIR/extend/TSAFuncs.version/
Examples of Advanced Analytics Functions
• Scan_RangeQuery_Pearson_Correlation() - unction returns time series

data that matches a pattern using a Pearson correlation.
• Scan_DTW_NonConstraint() - function returns matching time series
sequences using dynamic time warping without constraints.
• Scan_Abnormal() - functions return time series data that differ from
nearby sequences
• TSPearson_Correlation_Score() - function generates a Pearson
correlation score for two time series sequences.
• TSPatternMatch() - function returns matches to a specified pattern of
values.
• Etc.
Spatiotemporal Search Functions
Spatiotemporal searches are to track moving objects. You create a

spatiotemporal search index on the time-stamped GPS data that is
stored in a historian database as time series.
• can query on either time or on location to determine the relationship of
one to the other.
• can query when an object was at a specified location, or where an object
was at a specified time.
• can find the path, or trajectory, of a moving object over a range of time or
the relationship between a region and the trajectories of moving objects.
• can find when or where an object was stationary or did not have a signal.
• The spatiotemporal search solution builds upon the data types and
routines from the Informix TimeSeries solution and the data types,
routines, and R-tree indexes from the Informix Spatial solution.
• Spatiotemporal searches return location data as spatial data types

from the Informix Spatial extension.
• Requires the Informix TimeSeries extension, the Informix Spatial

extension, and the R-tree index extension.
Spatiotemporal Search Routines (examples)
• STS_GetPosition() - find the position of an object at a specific time

• STS_GetLastPosition() - find the most recent position of any object in the
time series
• STS_Speed() - find the average speed for a trajectory
• STS_GetLocStationaryWithin - ind the set of objects that were stationary
within a region at a specific time
• Etc.
• Another benchmark on a gateway system
Tests on Intel Quark Informix SQLite

Data loading – high-speed performance 950 / 1050 700 secs
(records per second) secs (Average)2
(DK100 / DK50)1
Storage space that is required for 1 day of data 275 MB 1,200 MB
Aggregation query (seconds) 2 secs 4-25 secs
Moving average (seconds) 25 secs 259,2003 secs
Missing interval search (seconds) 2 secs 14-30 secs
1. The two figures for data loading with Informix reflect a slight difference in performance between the DK100 and
DK50. DK100 had more running components causing a drop in performance vs DK50
2. Data loading with SQLite had significant variations in load performance as the database size increased.
3. The moving average result for SQLite is a projected figure that is based on a partial result after 10 minutes.
What does a true IoT engine
look like?
• Informix uses a communication layer called the “wire listener” to
interface between the database engine and NoSQL applications
• Is a Java layer that can be configured as needed and can allow or
disallow application functionality
• What port number will the app connect from?
• REST or MongoDB style syntax
• Both can be used
• Allow database creation?
• Limit the size of bson objects?
• NoSQL engines are limited to 16 MB, Informix can store up to 2 GB
• Allow or disallow sharding operations
• More
Wire listener
Informix instance
NoSQL app In: 32488 DBSERVER =
Port 32488 60000
DBSERVERALIA
Out: 60001
S = 60001
• With the wire listener properly configured and started, NoSQL
applications like the MongoDB Shell can connect to and use an
Informix instance for data storage / use
• There is an ifmx_stores (structured) database in the instance, is it
accessible?
What databases can the shell

see through the wire listener?
The wire listener will publish

structured / traditional databases
but list them as empty
Can connect to the database but

the wire listener does not publish
structured tables
• Using a NoSQL application against Informix
Can a collection be created

dynamically through the wire listener
by an insert operation? Will it be
created in the Informix database?
Yes it will!
Notice the data type!!
• There is a structured customer table in the database, can a NoSQL
application access data in it through the wire listener?
Yes, it can
How is it the shell or a

NoSQL application can
access a table/collection it
can’t “see”?
• Why does the wire listener act this way with structured databases and
tables?
• Is it security through obscurity?
• No, but it is deliberate

• Traditional Informix tables are highly structured
• They can not arbitrarily change their schema based on an insert statement
• If the Informix database(s) and table(s) were displayed through the wire
listener like collection a NoSQL developer might try to use them as such
• Would get errors back if the data inserted / modified didn’t match the schema of the
structured table
• While databases are displayed as “empty,” at this time a developer has to know of
the existence and schema of a structured table to use it through the wire listener
• This may change in the future
• Using SQL-passthru functionality, it is possible for a NoSQL app to
perform joins and other operations not normally supported by a simple
NoSQL engine
• Can join NoSQL, SQL or both types of data
• For example, this statement includes the use of multiple tables (could
also be collections), a join as well as a group by clause, none of
which can be executed using standard NoSQL operations
• This functionality can be used to enrich NoSQL applications if it can be
deployed securely
• Accessing Informix NoSQL data in collections with SQL
• There are two methods to return NoSQL data through an SQL
statement, both use casting to transform the data:
• One returns the data in the defined data types, aka SQL-oriented results
• The other returns the data in NoSQL key:value pairs
• Either can be used as needed by the application

• Both can have their output “columns” aliased though it’s really of value to SQL-
oriented results
• The syntax is very simple
select data.field_name::data_type from collection_name
• Using a specific data type returns SQL-oriented results
select data.field_name::json from collection_name
• Using the json data type returns NoSQL key:value pairs
• For example, using specific data types to return SQL-style results
• The columns can be aliased if desired
• Option 2 – use the json data type to return in key:value pairs
• In the previous examples, the predicate clauses used structured data
types, e.g. where data.z_ip::int = 79925
• Is it possible to use the JSON type in predicates?

• Yes, but it has to be fully qualified / expanded in JSON key:value syntax
Even though the data value of the

predicate may be a numeric, the
predicate statement should use
quotes as if it was a char string.
Important if zip codes where the first
value is a “0” (zero).
• If there are nested fields, nest the field names in the query like with
named row types
• For example, the manager collection stored in Informix:
{
"name":"John Miller",
"age":21,
"count":27,
"employees": [
{ "firstName":"John" , "lastName":"Doe" },
{ "firstName":"Anna" , “middle”:”Marie”, "lastName":"Smith" },
{ "firstName":"Peter" , "lastName":"Jones" }
] }
select data.name::varchar from manager

returns “John Miller”
select data.employees.middle::varchar from manager

returns “Marie”
• What about joining traditional Informix data to NoSQL data stored in
Informix
• For example, the demo stores database with NoSQL data?
• Yes and no
• It is possible, however the database locales must match
• Most NoSQL databases use UTF-8 as their default locale
• This carries over to default Informix NoSQL functionality
• Ensure that the locale is correctly configured in the database for traditional and
NoSQL objects
• A configuration parameter in the wire listener can help here
• A simple example of using NoSQL and structured data together
• Question – what is the GPS location of the post offices supporting my
customers?
The ifmx_stores database contains

customer information in the customer
table, including mailing city, state and zip
code
• The mon_zip_info collection in the mon_test_db contains post
office location data
• With Informix, write a query that uses both kinds of data together
• In this case, the NoSQL data is in a different database within Informix!
• Part of the functionality Informix provides and makes it a “true”
Internet of Things engine is the ability to integrate NoSQL and SQL
data together
• The previous example of joining a collection and a table in a statement is
one example of this functionality
• But can this data be even more tightly integrated together?
• Instead of storing in two separate objects, can they stored together

in a single object?
• In Informix, the answer is “of course you can!” because NoSQL is
‘just another data type’ like a char, int, node, timeseries and
so on
• The bson data type can used like other types when creating a table
• The bson column will store NoSQL documents like those stored in
collections
• There are a couple of nuances to be aware of however when used this way
• For example, create a table that includes columns with both
structured and bson data types
• Insert a row into the table

• Notice how the bson data is formatted in the statement
• Can the row be queried?
• Yes, but the results will be affected by how the bson column is cast
• Select without casting
• Without casting into a compatible type, the bson data can’t be displayed
• Select with casting
• When cast into a compatible type, the bson data can be displayed
• While data from the structured columns can be selected from a table, is it
possible to query just the JSON data? What is the syntax?
• The syntax is exactly the same as querying from a collection
• Remember, it’s just another data type within Informix!
• What about through the wire listener and a NoSQL application, is the
data available and what is its format?
• Of course it is
• The structured columns are returned as key:value pairs as shown previously
• The bson column is returned as a JSON array type composed of key:value
pairs
• Continuing on, insert another row into the table
• Leveraging “dynamic elasticity”, use a different kind of data in the bson
column
• Reminder – first row had an ID along with first and last name
• This row has meter-oriented data
• Query the table, properly formatting the bson column
• As expected, the bson column is able to have a dynamic structure and

contents from one row to another
What replication options
exist?
• Informix is been at the forefront of providing data replication and
business continuity solutions in the IM marketplace
• NO OTHER DATA SERVER offers the breadth and depth of
technology Informix provides:
• High Availability Clustering (HA Clustering) for business continuity and
failover
• Other data servers have started to mimic portions of this functionality
• Enterprise Replication (ER) for server-driven selective distribution of
data throughout the enterprise
• No other data server offers this type of technology
• Flexible Grid (Grid) easily create small to large clusters providing, server,
database and/or data replication options
• No other data server offers this type of technology
• Sharded tables or collections (Shard) to fragment a table or collection
across multiple nodes
• Also know as “Poor man’s MPP”
• Informix’s replication technologies can be used together to create a
data availability or replication solution that meets customer’s needs
without expensive hardware or software add-ons
• Enterprise Replication (ER) for
data distribution – selective
replication
• Supports active/active updates
• Very low latency
• Selectively replicate data
• All nodes are independent and
active/active
• Flexible scalability
• Subset of data
• Local data design
• Integrated
• Compatible with other availability
solutions
• Any server can also have one or
more H/A secondary instances
• Secure data communications
• With Enterprise Replication, you decide What data, gets replicated
Where and When it replicates
A preview of the lab exercises
• The hands-on exercises will take you through all the steps of
• Installing
• Capturing
• Storing
• Aggregating
• Replicating into the “corporate data center”
• Could be cloud or on-prem
• Using IoT and traditional data together to produce a viable business
result
• The lab will use simple, easy-to-use technology built into Informix to
accomplish these tasks
• Texas Instruments SensorTags have multiple environmental
sensors built into them
• Infrared and ambient temperature, humidity, ambient light level,
barometric pressure and more
• They speak multiple protocols (Bluetooth Smart, 6LoWPAN, ZigBee) and

connect to other devices to send their data
• In this case, Bluetooth to a Raspberry Pi
• Each Raspberry Pi has an Informix timeseries environment
configured and active
• You will create the database objects to capture the data
• Then using Node-RED, you will “wire” a “flow” that links the SensorTag
to the Informix environment so the temperature data is stored in the
database
• Node-RED is installed on the Raspberry Pi
• What is Node-RED?
• From the web site (nodered.org)
• It’s a programming tool for “wiring”

hardware, APIs and services together
to produce a data processing
sequence
• Called “flows”
• It’s browser-based and uses

JavaScript functions
• Built-in or create your own
• Flows are stored in JSON and can be

shared with others
• Node-Red is a browser-based tool
• You will be using the main, “flow building” interface though there are
optional interfaces to create graphs and so on
The “node library”

panel
The diagnostics /
Organized by type, feedback panel
nodes are dragged
from this panel to the Depending on the tab
main work area panel The work area panel selected, this panel
can display
Nodes are placed here information about an
then “wired” together to object OR real-time
create a “flow” that results of operations
processes data
• For example, a node that can connect to and read an environmental
sensor is “wired” to a debug node to check connectivity and if the device
is producing data
• Using the Debug panel, you can see the sensor’s data in real-time
• Once you’ve verified data capture and storage, you’ll configure and
start replication between the Raspberry Pi and an Informix
environment running in the “corporate server”
• With replication enabled, a data aggregation “flow” will be “wired”

using Node-RED
• The aggregation results are inserted into the replicated table and are
automatically sent to the “corporate server”
• Once in the “corporate server” a simple query that joins traditional,
relational data and the replicated NoSQL, timeseries data is
executed to return an answer to a business question
• No other database can execute these types of operations
Lab on!!
ITALIAN HINDI FRENCH JAPANESE BRAZILIAN PORTUGUESE SIMPLIFIED CHINESE
TRADITIONAL CHINESE SPANISH RUSSIAN TAMIL THAI GERMAN ARABIC

Iot Presentation Day 2

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Iot Presentation Day 2

Enviado por

Direitos autorais:

Formatos disponíveis

1 IoT data from network’s edge to the IBM IoT database engine © 2016 IBM Corporation

• A slightly deeper look at the IBM IoT engine’s capabilities

• Hands-on lab preview

Leveraging the data

• There can be extended requirements as well

• The IBM IoT engine is optimized to find these patterns

an abnormal power usage pattern

Similar patterns found

• Being able to analyze it

• Integrate it with the systems of record or standard, corporate data

• Satisfy audit, accountability, privacy requirements

• Uses a dynamic and flexible schema

• Is focused on single “row” retrieval, there are no joins

• Does not have/enforce ACID rules of transactional integrity

• Can not perform operations on multiple objects easily (aka multi-row

•where conditional query clauses are strongly discouraged or non-

• A large disadvantage is that scalability is very limited, the only solution is

• Informix, IBM’s Internet of Things database engine

• Potential use cases:

• Hybrid applications for new business patterns

• To the NoSQL world:

• To the structured SQL world:

• Is it reasonable to expect these devices to communicate directly

• From IBM’s perspective – usually no

• In other words, an intermediary or “gateway” system that can run a

Deep analytics zone

Uses Direct to on-prem or

• When storing the data, the time value is significant

• This can’t be modeled in a relational database without creating very

multiple indexes ABC 2015-01-05 09:30:00.19000 12.44 10

KCD 2015-01-05 09:30:01.05000 194.00 10000

row with redundant data

• High Storage Requirements

• Freedom to manage TimeSeries data:

ABC 2015-01-05, daily, {(12.34,1234567), (12.56,950000), (12.34,5555567),..}

KCD 2015-01-20, daily, {(199.08,678900), (198.55,345000), (198.12,850250),..}

XYZ 2014-08-25, daily, {(9.34,890567), (9.56,989000), (9.40,1000780),..}

ABC {(12.34,1234567), (12.56,950000), (12.34,5555567),..}

KCD {(199.08,678900), (198.55,345000), (198.12,850250),..}

XYZ {(9.34,890567), (9.56,989000), (9.40,1000780),..}

Query complexity Queries that aggregate Less application coding

- Advanced analytics functions provide specialized methods of

The time series definition must conform to the following requirements:

Requires the Advanced Analytics Functions extension

• Scan_RangeQuery_Pearson_Correlation() - unction returns time series

Spatiotemporal searches are to track moving objects. You create a

• Spatiotemporal searches return location data as spatial data types

• Requires the Informix TimeSeries extension, the Informix Spatial

• STS_GetPosition() - find the position of an object at a specific time

Tests on Intel Quark Informix SQLite

Storage space that is required for 1 day of data 275 MB 1,200 MB

Aggregation query (seconds) 2 secs 4-25 secs

Moving average (seconds) 25 secs 259,2003 secs

Missing interval search (seconds) 2 secs 14-30 secs

What databases can the shell

The wire listener will publish

Can connect to the database but

Can a collection be created

Notice the data type!!

How is it the shell or a

• Is it security through obscurity?

• No, but it is deliberate

• The other returns the data in NoSQL key:value pairs

• Either can be used as needed by the application