Is Your Corporate Data Warehouse Shrinking

To a Data Mart
Since the past decade data warehouses have been gaining enormous ground in the
business intelligence (BI) domain. A ‘corporate data warehouse’ was on every
organization’s priority list. Organizations which were wary of making huge
investments but at the same time wanted to be in the race opted for data marts. For
business leaders where innovation was the buzz word, a corporate data warehouse
and then consequently data mining (using complex algorithms and predicted trends
and the why of the trends) seemed to the long term strategy (long term in information
technology means a period of 3-5 years).

Projects were embarked on with great vigor and significant investments were
ploughed in. A giant server that would store terabytes or at least gigabytes of data was
brought installed. And after battling with issues like data quality, tools to be used for
loading and analyzing data, granularity of data to be stored, years to data to be stored,
performance tuning for faster queries and so on and so forth, the corporate data
warehouse finally took shape. It was 15-18 months of persistent effort which had paid
off. Business users and analysts embarked on the journey of discovering the hidden
trends which could save their organization the much so important dollars and also
contribute towards the return on investment of the corporate data warehouse. And
indeed they did. Every issue of the organization newsletter had articles of trends
revealed and savings accrued.

Companies began to rely more and more on these BI systems. Critical business
decisions were based on the current and historical data available in the data
warehouse. Sophisticated OLAP tools which facilitate multidimensional analysis
were used. Business trends are identified using data mining tools and applying
complex business models.

Companies were battling each other out in terms of storing vast data which was being
analyzed day in and day out to make some sense out of it. Standard reports and views
from the sophisticated OLAP tools were generated on a regularly intervals basis for
analysis purposes.

As businesses grow from local to global, the complexities and parameters involved in
decision making and analysis became more complex. The corporate data warehouse
served as the perfect analysis tool.

But with businesses expanding in various business sectors and various locations it
became increasing difficult for the corporate data warehouses to keep pace with the
humungous addition of data.

TCS Confidential
 The corporate data warehouse which at one point of time housed the entire
organization’s business data and provided everything from a holistic view of the
business to low level granular data suddenly ceased to be so. The organization found
the data from a number of new locations and new business sectors missing in the
corporate data warehouse. Critical decision making and analysis work like product
introduction strategy, pricing strategy, marketing strategy and product enhancement
strategy for products in the new business sectors was all being done manually and the
sophisticated analysis tools that the data warehouse boasted of could not be employed
for help.

The organization which once so proudly stated that the corporate data warehouse was
a state-of-the-art system used for BI now began to wonder whether to call the data
warehouse a ‘data warehouse’ or a ‘data mart’ (data mart is a scaled-down version of
a data warehouse that is tailored to contain only information likely to be used by a
specific target group).

Critical business decisions which at one point to time where taken based on the
information provided by the data warehouse and which were made effective globally
suddenly were being looked upon with doubt. Business analysts who took pride in
revealing and analyzing business trends found it difficult to justify their results and
providing global solutions.

Since the integration of data from new locations and business sectors into the data
warehouse needs proper planning and design and requires time, the organizations
found themselves dealing with a situation where the new locations and business
sectors had started developing their own set of home grown analytical systems. The
organization thus ended up with a structure where each location and business sector
had their own system which made the integration process more complex. This further
delayed the process of making the data warehouse a true ‘corporate data warehouse’.
The people maintaining the corporate data warehouse were unhappy with the fact that
inspite of adhering to the best data warehousing standards and norms like
organization wide conformed dimensions, etc. the data warehouse was being devoid
of the status it deserved.

More and more companies today are facing the above situation. Most companies need
to look at global product sales, global product warranty, global product pricing
strategies, global product promotion strategy and global product trends. And there is
no one place where all the required information is available. The obvious choice is to
work on the available data set and extrapolate the results to the locations for which
data is not available. Though this might be acceptable in a sizable number of cases,
extrapolation of data for critical decision making is a risky proposition and should be
avoided. It entirely defeats the purpose of having a corporate data warehouse.
Models which use the data warehouse data for sales forecasting and resource
forecasting purposes need fairly accurate history data. Results of one country or
region cannot be extrapolated to other countries and if so done a significant degree of
fluctuation would be expected. The more the skew ness of data the more it renders

TCS Confidential
 exercises like forecasting, trending, data modeling, etc ineffective. That is when the
big question of the return on investment on the corporate data warehouse crops up.

An amicable solution of such types of problems calls for close interaction between
the business users and the information technology team.

Rome was not built in a day, and so can’t be corporate data warehouses.

Amidst the rapid changes taking place, the needs to be a team (a steering committee)
who looks at the various jobs at hand and prioritizes them based on business needs
and requirements.

The project champion has a major role to play. But it is not only the project champion
who can make them difference. Gone are the days when the business users and
analysts waited patiently for the information technology people to make the system
available to them. And gone are the days when the information technology people got
a ready-made business logic document which they just had to code. Today the
business users are expected to be techno-commercial. The information technology
people are no longer dumb coders but are expected to be business knowledge savvy
and add value to the organization processes. Business analysts and the information
technology people have to work hand-in-hand and understand each other lingo to
churn out effective and optimized processes.

The information technology people should understand the critical business parameters
which are used for making decisions. The business users on the other hand have to
provide continual feedback on the gaps between the expected and actual. Priorities
are to be set up in close coordination in such a way that each step towards integration
should be independently beneficial in some respect. Each step however small should
fit in the entire picture and be capable of providing the organization some benefit in
its own way. This will add value incrementally to the integration work being done
creating a win-win situation.

The project charter is an important and wise way to start. All the intended activities
should be listed and the scope of each activity needs to be detailed out. The
requirement study should comprise brainstorming sessions to identify all possible
areas and bottlenecks that need due consideration. Business analysts and the
information technology people should both participate in these sessions. Scope creep
is something the project champion should watch out for. The brain storming session
can prove to be a forum for throwing up a plethora of ideas. These thoughts and
possibilities need to be looked at carefully and the unnecessary ones should be
weeded out.

The next stage, feasibility analysis, then questions each of these user requirements,
looks at the requirements from the technical standpoint. The project champion could
decide the break the requirements into multiple phases or milestones if necessary.
What follows the feasibility analysis is the software development lifecycle which

TCS Confidential
 comprises Design, Development, Integration Testing, Performance Testing, User
Acceptance Testing, Production Roll out and Maintenance and Enhancements. It’s all
easier said than done and the whole team has to ensure than each requirement is
thoroughly reviewed at all stages of avoid rework and effort overrun at later stages.
The review activity and defect logging though seem to be tedious activities; actually
add a lot of value in making the design scalable and robust.

A slow but sure step by step approach is a must for transforming ‘the shrinking
corporate data warehouse’ back to the ‘corporate data warehouse’.

TCS Confidential