Chapter 8: External Data and the Data Warehouse

http://it-slideshares.blogspot.com/

Agenda
1. 2. Introduction External Data in the Data Warehouse

3.
4. 5. 6.

Metadata and External Data

Storing External Data Different Components of External Data Modeling and External Data

7.
8. 9.

Secondary Reports
Archiving External Data Comparing Internal Data to External Data

10. Summary

http://it-slideshares.blogspot.com/

8.1 Introduction
Most organizations build their first data warehouse efforts on data whose source is existing systems (that is, on data internal to the corporation). A whole host of other data is of legitimate use to a corporation that is not generated from the corporations own systems. This class of data is called external data and usually enters the corporation in an unpredictable format. (Figure 8.1). The data warehouse is the ideal place to store external data. If external data is not stored in a centrally located place, several problems are sure to arise. (Figure 8.2).
http://it-slideshares.blogspot.com/

8.1 Introduction (cont)

http://it-slideshares.blogspot.com/

8.1 Introduction (cont)

http://it-slideshares.blogspot.com/

8.2 External Data in the Data Warehouse

Several issues relate to the use and storage of external data in the data warehouse.
The first problem is the frequency of availability The second problem is totally undisciplined The third problem is unpredictability

http://it-slideshares.blogspot.com/

8.2 External Data in the Data Warehouse (cont)

There are many methods to capture and store external information.
One of the best places to locate external data if it is voluminous is on a bulk storage medium such as near-line storage.

Another technique for handling external data that is sometimes effective is to create two stores of external data.
The external data becomes an adjunct to the data warehouse.

http://it-slideshares.blogspot.com/

8.3 Metadata and External Data

Metadata is vital when it comes to the issue of external data.

http://it-slideshares.blogspot.com/

8.3 Metadata and External Data (cont)

Associated with metadata is another type of data notification data.

http://it-slideshares.blogspot.com/

8.4 Storing External Data

http://it-slideshares.blogspot.com/

8.5 Different Components of External Data

One of the important design considerations of external data is that it often contains many different components, some of which are of more use than others. To manage the data, an experienced DSS analyst or industrial engineer must determine the most important units of data.

8.6 Modeling and External Data

The following question must be answer.
What is the relationship between the data model and external data? (As described in Figure 8.6)

http://it-slideshares.blogspot.com/

8.7 Secondary Reports

When data is repetitive in nature, secondary reports can be created from the detailed data over time.

For example, take the month-end Dow Jones Industrial Average report shown in Figure 8-7.
http://it-slideshares.blogspot.com/

8.9 Archiving External Data

Every piece of informationexternal or otherwisehas a useful lifetime. Once that lifetime is past, it is not economical to keep the information. An essential part of managing external data is deciding what the useful lifetime of the data is. There remains the issue of whether the data should be discarded or put into archives.

http://it-slideshares.blogspot.com/

8.10 Comparing Internal Data to External Data

One of the most useful things to do with external data is to compare it to internal data over a period of time. The comparison allows management a unique perspective.

The following is some problems must be notice when compare internal Data to External Data
The comparison is made on a common key. There needs to be a cleansing of the external data. http://it-slideshares.blogspot.com/

8.11 Summary
The data warehouse is capable of holding much more than internal, structured data. There is much information relevant to the running of the company that comes from sources outside the company. External data is captured, and information about the metadata is stored in the data warehouse metadata. External data often undergoes significant editing and transformation as the data is moved from the external environment to the data warehouse environment. The metadata that describes the external data and the unstructured data serves as an executive index to information.

External and unstructured data may or may not actually be stored in the data warehouse.
http://it-slideshares.blogspot.com/