Report Reader

Version Nine Tutorial

A CaseWare IDEA Document
Copyright © 2013 (v9.1) CaseWare IDEA Inc. All rights reserved. This manual and the data
files are copyrighted with all rights reserved. No part of this publication may be reproduced,
transmitted, transcribed, stored in any retrieval system or translated into any language in any
form by any means without the permission of CaseWare IDEA Inc. CaseWare IDEA Inc. is a
privately held software development and marketing company, with offices in Toronto and
Ottawa, Canada, related companies in The Netherlands and China, and distributor partners
serving over 90 countries. CaseWare IDEA Inc. is a subsidiary of CaseWare International Inc.,
the world leader in business-intelligence software for auditors, accountants, and systems and
financial professionals (www.CaseWare-IDEA.com). IDEA is distributed under an exclusive
license by CaseWare IDEA Inc, 469 King Street West, 2nd Floor, Toronto, CANADA, M5V
1K4. IDEA® is a registered trademark of CaseWare International Inc.
 Contents

Contents

Preface

Preface................................................................................................7
Chapter 1

Report Reader Overview ..................................................................9

Introduction .................................................................................................9
What Kinds of Files Can Be Converted?........................................................9
How does Report Reader Work? ................................................................10

Chapter 2

What’s on the Report Reader Screen.............................................11

Chapter 3

Using Report Reader .......................................................................13

Introduction ...............................................................................................13
Accessing Report Reader ............................................................................13
Creating a Template - The Basic Steps ........................................................14
Examine the Data and its Layout .........................................................15
Select a Sample Line and an Anchor Line.............................................15
Define a Trap.......................................................................................16
Define Field Anchors............................................................................18
Save the Layer .....................................................................................21
Scan for Errors.....................................................................................21
Preview the Database ..........................................................................22
Create an Append Layer ......................................................................23
Rearrange the Field Order....................................................................25
Save the Template ...............................................................................26

Chapter 4

More Examples ................................................................................27

Use Value from Previous Record .................................................................27
Headers, Footers, and Page Breaks.............................................................29
Multi-Line Entries and Multi-line Field Options.............................................32
Pre-Append and Post-Append Layers ..........................................................34
Using Floating Traps...................................................................................36
Create a Floating Trap ................................................................................38
Exclude Line ...............................................................................................48
Filters .........................................................................................................49

Index

Index.................................................................................................51
 Preface

Preface

Welcome to the Report Reader Tutorial.

Report Reader can be used to extract data from Print Report, Text, and Adobe Acrobat PDF files to
IDEA databases (.imd).

IDEA is a general purpose file interrogation tool for auditors, accountants, and investigators. Tests,
analyses, and investigations are carried out using IDEA. However, in order to use IDEA, the relevant
data must be transferred into it and interpreted in terms of records within fields. The Import
Assistant is provided within IDEA to carry out this task and is the usual method to read or import
data into IDEA.

However, in many situations the data may only be available in printout form which can be printed to
a file rather than paper. In these cases, Report Reader can be used to convert the data in a simple
non-technical way.

The wider aspects of how to transfer data from a computer system to IDEA are dealt with in the
guide Auditors Guide to Downloading Data.This guide deals specifically with the use of Report
Reader, explaining the concepts, the menu items, the commands, and the tools. The first section is
an introduction to Report Reader followed by an explanation of the main window and menus. The
guide also includes examples to demonstrate the working of, and the concepts and rules in, Report
Reader.

If you want to follow along with the examples, the data files used can be found in
C:\Users\<username>\My Documents\My IDEA Projects\Tutorial once you have installed IDEA
Version Nine.

In addition, the images used throughout this guide may not reflect what you see on your screen
depending on the IDEA product you are using.

To further understand how Report Reader works, instructional videos based on the examples
described in this guide are provided on the CaseWare IDEA website at www.CaseWare-IDEA.com.

Your feedback is important to us. Send your documentation-related comments to

IDEADocumentation@caseware.com.
 Chapter 1

Report Reader Overview

Introduction The data files used by IDEA have a special file format. The files are encoded in a way that contains
not only the data, but descriptions concerning the arrangement and use of the data.

IDEA includes an Import Assistant that provides importing capabilities for data files that are not in
this format. Report Reader is part of the Import Assistant within IDEA and can be accessed through
it.

Report Reader extracts data from plain text reports, print reports saved to file, and Adobe Acrobat
PDF files and imports it into IDEA. The plain text and print reports might have come from an
application on the computer or from a mainframe computer.

There is extensive online Help available for Report Reader that includes many step-by-step walk-
throughs of tasks. There are also links to the CaseWare IDEA website for additional information,
such as tips and hints, frequently asked questions (FAQs), and technical information.

What Kinds of Files With Report Reader, you can extract data from any Print Report, Text file (for example, .prn, .txt,
Can Be Converted? .asc, .csv, .dat) or Adobe Acrobat PDF file.

A Print Report file is a plain text file or a printer output file. Most DOS applications will produce text
output by a process called “print to disk”. This simply means sending a report to a disk file instead of
printing it. Windows applications usually have a text output option or can print a report to a Text file
with the Generic/Text Only printer driver.

Report Reader can read, display, and extract data from files from almost any source, such as:
10 Report Reader Version Nine Tutorial

• Mainframe text reports

• Minicomputer text reports
• DOS print to disk files
• Generic/Text only files (Windows)
• Word processor Text only files
• Text-only accounting reports
• E-mail reports
• Online communications text captures
• Print to Files (PRN)
• Tab Separated Variables files
• Standard Data Format (SDF)
• Adobe Acrobat PDF

Note: Text from Adobe Acrobat PDF files will be converted into plain text format where information
such as font size, color, text bounding boxes, and rotation angle will be removed. A page break is
inserted at the end of each page and graphics and images will not be converted.

How does Report Report Reader displays your entire original file in a section of the Report Reader window called the
Reader Work? Report View. Using the scroll bars, you can scroll through the report to view the details of the data
and the layout. Data to be extracted can be selected with a simple point-and-click operation. No
special knowledge of the file structure or command language is needed to use Report Reader.

Report Reader extracts the data you select to an IDEA database. Your selections and instructions for
extracting the data can be saved as a template. This template can be re-used to convert other files in
the same layout. Using templates saves you time, particularly if you need to extract data from the
same or similar reports on a regular basis.

Report Reader does more than simply extract data from one file type to another. It distinguishes
between numeric and non-numeric information and handles both appropriately.
 Chapter 2

What’s on the Report Reader Screen

The following image displays the Report Reader window with a report open.
12 Report Reader Version Nine Tutorial

The following image displays the Report Reader window with data selected.

Description

• Menu bar: This section provides access to menu commands, just like other
Windows applications.
• Toolbar: These toolbar buttons have been provided for easy access to frequently
used operations.
• Anchor Editor: Based on the sample line in the Field Editor, a trap is defined here to
anchor similar lines in the report.
• Field Editor: The section between the two thick lines at the top of the page where
a sample line is copied to. Once similar lines have been anchored in the report,
data to be converted is selected in the Field Editor. The data selected here is the
Field Anchor and anchors the fields in the report.
• Report View: When a report is opened in Report Reader, the entire report is
previewed in the Report View.
• Anchored Lines: Highlighted lines in the report that have been anchored by the
trap defined in the Anchor Editor.
• Field Details Window: This window slides out from the right and displays the
properties of a field, such as the name and type once a field is defined.
 Chapter 3

Using Report Reader

Introduction This section provides a simple example to get you started with Report Reader. It demonstrates some
typical tasks which illustrate the use of Report Reader to convert a file to an IDEA database.

If you want to follow along with the examples, the data files used can be found in
C:\Users\<username>\My Documents\My IDEA Projects\Tutorial once you have installed IDEA
Version Nine.

For more information, access the Help system in the software, visit the CaseWare IDEA Support
website at ideasupport.caseware.com or attend one of the many training courses.

To extract data from a report where the data is not arranged in columns, such as in Web logs,
please refer to the Using Floating Traps section in Chapter Four.

Accessing Report 1. From the Windows Start menu, select Programs > IDEA > IDEA.
Reader The IDEA application appears.

2. On the Home tab, in the Import group, click Desktop.

The Import Assistant dialog box appears.

3. Select Print Report and Adobe PDF.

4. Click the Browse button.
5. Navigate to and select the file you want to import through the Report Reader.
6. Optionally, if you have an existing template or a mask (.msk file created with DataImport)
navigate to and select it by clicking the Browse button adjacent to the Optional template file
box, and then select the Edit template check box. Note: You can import a report with an
existing template or mask; however, you cannot create or edit a mask.
7. Click Next.
Report Reader opens and the selected file is displayed.

You can also open a new report directly in Report Reader by selecting File > Open Report or by
clicking the Open Report button on the main toolbar.
14 Report Reader Version Nine Tutorial

The following section uses the Travel_report.prn file to demonstrate the steps required to convert a
file.

Note: The Report Reader window can only display a single file at a time. When a new file is opened,
the current file is automatically closed.

Creating a Report Reader displays the entire report so you can scroll through the report and examine the data
Template - The and the layout. Based on this information, you can define a template to select and extract it to the
Basic Steps required file type.

The choices you make in identifying the data to be extracted are saved as a template. A template is
a way to tell Report Reader what information must be extracted and the position of the information.
The template can then be re-used to extract data from other files with the same layout. You could
also think of the template as a piece of cardboard with holes cut out of it. When you lay this
template on top of a report the information that is visible through the holes is extracted to an IDEA
database.

As most reports contain many levels of detail, the data from each of the levels will be defined in a
separate layer. These layers together then form the template. The Base Layer is the first layer
created and, if required, additional layers called Append Layers can be appended to it to create a
template.

• Base Layer: The first layer defined. The Base Layer must be defined at the detail line or
transaction level. A detail line is the line or lines that contains the data that you want to
identify as records in the resulting IDEA database.

• Append Layer: Additional layers defined after the Base Layer has been defined and saved.
There are two types of Appned Layers: Pre-Append and Post-Append. Data captured in the
Append Layer is appended to the records defined in the Base Layer. A Pre-Append Layer is
defined based on data that precedes the Base Layer. A Post-Append Layer is defined based
on data that follows the Base Layer. The position of the sample line for an Append Layer
relative to the sample line for the Base Layer defines it as a Pre-Append or Post-Append
Layer. For more information, please refer to Chapter Four.

The process of creating a template and converting a file is covered in the following steps:

1. Examine the data and its layout.

2. Select a sample of the required data (called the Anchor Line).
3. Define a trap to anchor similar lines in the report.
4. Capture the data that will become the fields and records.
5. Save the layer.
6. Scan for errors.
7. Preview the database (optional).
8. Re-arrange the field order (optional).
9. Save the template.
10. Convert the file.

Note: Steps 1 through 6 must be completed for creating both a Base Layer and an Append Layer.
 Chapter 3: Using Report Reader 15

Examine the Data and its Layout

As displayed in the previous image, the data in the Travel_report.prn file, like in most reports, is
organized into many levels. It has a title indicating the subject of the report (Report on Travel
Expenditures) and the time period it covers (Month of January 2010). The next level is the category
of travel based on the purpose, such as Conference, Training, and Work. Each category has a total
amount. Lastly, there are detail lines containing data about each trip with the information displayed
in a single line.

In this example, the data will be extracted in two layers. The first layer (Base Layer) will contain data
about individual trips, while the second layer (Append Layer) will capture the data related to the
purpose of the trips.

Select a Sample Line and an Anchor Line

Select a sample of the required data. It can be a single line or multiple lines (up to a maximum of
30). If the sample is a single line, it will also be the Anchor Line. However, if the sample is made up
of multiple lines, one of the lines must be selected as the Anchor Line. Based on the characteristics
of the Anchor Line, a trap must be defined to anchor similar lines in the report.

In the Travel_report.prn file, as each trip is a single-line entry, we want to create one record for each
trip.

1. Highlight the first detail line in the report.

Note: It is not necessary to select the first line or entry as the sample line. Select the entry that
is the best sample of the data to be extracted (for example, the line with the most details). A
trap will capture all lines with matches both above and below the sample line. If the transaction
is a multi-line entry, multiple lines can be selected. The sample line can be up to a maximum of
30 lines.

The Report Reader dialog box appears. The Report Reader dialog box provides you with the
following three layer options:

• Create a standard layer: A standard layer is used when the data in the report is laid
out in columns.
• Create a floating layer: A floating layer is used when the data in the report is not
laid out in columns, such as in a system file or a weblog report. See Using Floating
Traps in Chapter Four.
• Exclude it from the output: When creating Append Layers, after a Base Layer has
been defined, you can exclude highlighted data from being extracted.
2. Select the Create a standard layer option.
16 Report Reader Version Nine Tutorial

3. Click Yes to continue.

The line selected as the sample line now becomes the Anchor Line (indicated by the red arrow
in the left margin) and is copied and displayed in the Field Editor. This is the section at the top
of the page between the two yellow lines. The Field Details window also slides out from the
right side. This window will display field and layer properties when they are defined. You can
click the Auto Hide button to close the window, if desired.

Define a Trap

Using the Anchor Line, a trap must be defined to anchor similar lines in the report. To define a trap,
you must find a characteristic, a character, or a pattern in the Anchor Line that is shared by all the
lines you want to anchor but that is not common to any other lines. In other words, it must be
unique to the lines you want to select.

The trap must be defined in the Anchor Editor just above the Field Editor. It can be defined using
one of the generic traps provided or by entering a specific character.

Generic traps are useful in defining patterns, based on which required lines in a report are anchored.
However, if a unique pattern is not available, entering a specific character as a trap will anchor lines
where the character is found. A combination of a specific character and a generic trap can also be
used to define a trap. Used in a Standard trap (a trap defined in a Standard layer), these traps will
anchor all lines in the report that have the particular type of character in the same column as the
trap.

The pre-defined generic traps are:

• Text Trap: Anchors lines in the report that contain any alphabetical character.

• Numeric Trap: Anchors lines in the report that contain any numeric character.

• Space Trap: Anchors lines in the report that contain a space.

• Non-Blank Trap: Anchors only those lines in the report that do not have a space, but have
either a text or a numeric character.

Using the data in the Anchor Line selected in the previous step, a trap can be defined based on any
of the columns in the report, such as PIN, EMPL_CLASS, and DATE. Scroll through the
Travel_report.prn file and notice that the first character for all of the detail lines is a number. Using
this information, the generic Numeric trap can be selected.
 Chapter 3: Using Report Reader 17

The following steps outline how to anchor the required detailed lines using a generic Numeric trap.
However, there are many other ways to define a trap and you can try other options and view the
results.

1. Click in the Anchor Editor to place your cursor in the position just above the first digit (the
digit 1 in the number 12205605). Click the Numeric Trap button on the toolbar.
The Numeric trap symbol (uppercase N with a yellow background) is inserted in the Anchor
Editor. All of the lines in the report that contain a number in that position are highlighted in the
Report View to visually indicate that these lines have been anchored by the trap.

Note: Typing in the letter N through the keyboard will create a specific trap and will only trap
those lines with the letter N in that column. Only generic traps are displayed with the yellow
background. Specific traps will appear as regular text in the Anchor Editor.

2. Scroll through the report to ensure that all of the required lines and only the required lines are
anchored.
18 Report Reader Version Nine Tutorial

Note: If a trap does not match the data in the Anchor Line, the trap will be displayed with a
red background and none of the lines in the report will be selected. In the following image,
there are no lines selected in the report because a Text trap has been entered over a numeric
character in the Anchor Editor.

Define Field Anchors

Once the required lines are anchored in the report, the data to be defined as fields must be specified
by defining Field Anchors. Field Anchors are defined by highlighting the required data in the Field
Editor (the section at the top of the window between the two yellow lines). As the fields are defined
in the Field Editor, the corresponding data in the report is highlighted as a visual clue to the data
captured.

Note: The maximum length of a field is 500 characters. If the number of characters in a field
exceeds 500, only the first 500 characters will be selected.
 Chapter 3: Using Report Reader 19

1. In the FIeld Editor, highlight 12205605 in order to define the PIN column as the first field.

The selected data in the Field Editor is highlighted in orange to indicate that it is now a Field
Anchor, while corresponding data in the report is highlighted in blue. The Field Details window
on the right-hand side of the screen now also displays field and layer properties, with default
values, in the following categories:

• Layer Information

• Field Information

• Position

• Attributes

• Filters
20 Report Reader Version Nine Tutorial

The default values in the Field Details window can be changed. By default, the data selected as
the Field Anchor becomes the field name. If the data selected as the Field Anchor is numeric,
the letters COL are added as a prefix since a field name cannot begin with number.

2. Define the Field Information and the Position details for each field as outlined in the
following table. Accept the default options for the Attributes and Filters categories.

• PIN - Character - 0 - 8
• EMPL_CLASS - Character - 1 - 22
• DATE - Date - DD/MM/YYYY - 24 - 10
• AMOUNT - Numeric - 23 - 96
• CITY - Character - 476
• AIRLINE - Character - 62 - 17
• AIRMILES - Numeric - 894
• TRAVEL_CLASS - Character - 952
 Chapter 3: Using Report Reader 21

3. Scroll through the report to ensure that all of the data in the fields is selected. Notice that in
the case of the AMOUNT and TOWN fields, some data has not been included. You can edit the
Field Anchor in the Field Editor or in the Field Details window.

To edit a Field Anchor:

The starting point (Offset) and ending point (Actual Width) of a Field Anchor can be changed
in one of the following ways:

• Place the cursor over one end of the Field Anchor until it is displayed as a double-headed
arrow and drag it in either direction to the required position.

• In the Position area of the Field Details window, enter new values for Offset and Actual
Width.

• Use the Up or Down Arrow keys on your keyboard to increase or decrease the length of
the field. To move the field to the left or to the right, use the Left or Right Arrow keys on
the keyboard.

To delete a Field Anchor:

Select the Field Anchor and then click the Remove Field Anchor button on the toolbar.
Alternatively, click the Remove all Field Anchors button to remove all of the Field Anchors.

Save the Layer

Once the required fields are defined, the layer can be saved. Click the Save Layer button on the
toolbar.

Scan for Errors

You can scroll through the report to ensure that the fields are wide enough to capture all the
required data in the report, but a quicker way to do this is to use the Scan for Errors task. This will
scan the report looking for instances where there may be characters adjacent to a field which may
indicate that the field offset or field width needs to be adjusted to include these characters. If the
Scan for Errors task finds any such instance, the field will be highlighted and displayed along with
the Scan for Errors dialog box where you will be presented with a choice of changing the field width
and/or the field offset to capture all of the data in the field or to stop scanning the field.

1. Select Traps > Scan for Errors.

Alternatively, click the Scan for Errors button on the toolbar.

2. In the Field width box, increase the field width to 11.

Notice that as you increase the number in the Field Width box, the highlighted field also widens
in the Report View.
22 Report Reader Version Nine Tutorial

3. Click Change to accept the new field width and to continue scanning.
To ignore a particular instance, click Ignore. The Scan for Errors task will continue to scan the
report and to highlight the next case it finds.

If you have a field that does not start or end on a blank space, select the Stop checking this
field check box and then click Continue so that Scan for Errors will skip the field and move on
to checking the next field.

4. Click Finish to accept any changes made and exit the Scan for Errors dialog box.
5. Click OK.

Preview the Database

Once you have defined a layer, you can preview the report as an IDEA database.

1. Click the Preview Database button on the toolbar.

The report is previewed as an IDEA database in the Preview Data dialog box.

2. Use the horizontal and vertical scroll bars to view all of the records.
The Preview Data dialog box can be resized by dragging any one of the edges.

3. Click Close to return to the Report View.

Alternatively, you can go back to the Report View in Report Reader without closing the Preview
Data dialog box, make changes and then click the Refresh button in the Preview Data dialog
box to view the changes.
 Chapter 3: Using Report Reader 23

Create an Append Layer

In the Travel_report.prn file, you now need to capture data from the next level of the report, which
is the data related to the purpose of travel. In order to extract this data, you will need to define an
Append Layer. Fields defined in an Append Layer are appended to the relevant records in the Base
Layer.

An Append Layer is created the same way as the Base Layer, but the data captured in this layer does
not create a new record in the resulting IDEA database, but is appended to the records defined in
the Base Layer.

1. Select Purpose: CONFERENCE as the sample line.

2. Select the Create a standard layer option and then click Yes to create a new layer.
As this line is above the sample line for the Base Layer, this becomes a Pre-Append Layer and
the data captured in this layer will be appended to the Base Layer records that follow it.

3. In the Anchor Editor, insert a colon over the colon in the Anchor Line in order to define a
specific trap.
4. Verify that the required lines in the report are anchored. Anchored lines appear highlighted in
blue.
5. Define the Field Anchor by selecting CONFERENCE in the Field Editor.

6. Scroll through the report to verify that all of the data related to purpose is selected.
24 Report Reader Version Nine Tutorial

7. In the Field Details window, enter the following details:

• Name: PURPOSE
• Type: Character
• Offset: 9
• Actual Width: 10
• Blank cells: Use value from previous record
As trips in the Travel_report.prn file are categorized by purpose, the data in the PURPOSE field
is common to many trips. Hence, in the Blank Cells box in the Attributes area of the Field
Details window, the Use value from previous record option should be selected. This will
populate all of the records with values in the PURPOSE field. If this option is not selected, the
data in the PURPOSE field would only be included in the first record that it applies to and the
rest of the records would contain a blank field or a zero value depending on the field data type.

Note: The Use value from previous record option works differently on fields defined in the Base
Layer and the Append Layer. Selecting the Use value from previous record option will always
copy the value from the previous record when an empty field is found in a Base Layer.
However, as an Append Layer is associated with a Base Layer, selecting the Use value from
previous record option in an Append Layer will copy the value only to the associated records in
the Base Layer. If Report Reader comes to the next occurrence of the Append Layer and finds a
blank, or no data, the value from the previous Append Layer will not be copied. Instead a blank
field will be created with the Base Layer records. For an example refer to Chapter Four.

8. Click the Save Layer button on the toolbar.

9. Preview the report as an IDEA database by selecting File > Preview Database.
 Chapter 3: Using Report Reader 25

Rearrange the Field Order

When converting the file to an IDEA database, the fields will be positioned in the order they were
defined. However, this order can be changed if required. For example, note that in the image above,
the last field is the one defined in the Append Layer. Through the Field Order dialog box, this field
could be moved to a different position, such as between the CITY and AIRLINE fields.

1. Click the Field Order button on the toolbar.

The Field Order dialog box appears.

• Move Up: Moves the selected fields up one position in the field order list.

• Move Down: Moves the selected fields down one position in the field order list.

• Move to Top: Moves the selected fields to the first positions in the field order list.

• Move to Bottom: Moves the selected fields to the bottom positions in the field order list.

• Swap: Interchanges the positions of two selected fields in the field order list.

Note: You can also change the field order by dragging and dropping the field names within
the the field order list.

2. Click OK to save the changes and exit the Field Order dialog box.
26 Report Reader Version Nine Tutorial

Save the Template

Once the data to be extracted has been selected, the template can be saved. Templates are saved
with the .jpm file name extension. This template can then be re-used to convert files with the same
layout.

1. Select File > Save Template.

Alternatively, click the Save Template button on the main toolbar.

The Save As dialog box appears.

2. Navigate to the required saving location. By default, the template is saved in the Import
Definitions group in the Library.
3. Click Save.
 Chapter 4

More Examples

If you want to follow along with the examples, the data files used can be found in
C:\Users\<username>\My Documents\My IDEA Projects\Tutorial once you have installed IDEA
Version Nine.

Use Value from Here is an example of the difference in the application of the Use value from previous record option
Previous Record to a field in a Base Layer and in an Append Layer. This option must be defined in the Blank Cells box
in the the Attributes area in the Field Details window. The default option is Leave blank which will
create a blank field or a zero value, depending on the data type.

The report displayed below is a monthly shipping report for a distributor of books. Like many
reports, it is organized with multiple sort levels with account number information being the
uppermost level. Within the report are the transactions or detail lines for each account. Each
transaction will be exported as a separate record. The information in the detail lines will be defined
in the Base Layer, while information about the account number will be captured as part of the
Append Layer.
28 Report Reader Version Nine Tutorial

In this report, take note that:

• Account Number 100 has a single date for the three items.
• Account Number 101 has no transactions.
• Company Name is blank for Account Number 102.
 Chapter 4: More Examples 29

Selecting the Use value from previous record option for the date field in the Base Layer and the
account number, the contact name, and the company fields in the Append Layer, results in the
following IDEA database.

When the Use value from previous record option is selected for a field in the Base Layer, the value
from the previous line will be copied. As in this example, the date 01/05/2003 is copied to the
second and third record.

However, in an Append Layer, an empty cell can be created. As the Append Layer is always
associated with a Base Layer, values in an Append Layer will only be copied to the associated records
in the Base Layer.

When Report Reader comes to an Append Layer (Account 101) for which there is no corresponding
Base Layer details, it ignores the information as it has no Base Layer transaction to append it to.

Headers, Footers, If you want to follow along with the examples in this section, the Dave’sBookStore5.txt file is used
and Page Breaks to demonstrate how Report Reader handles headers, footers, and page breaks.

Reports that span multiple pages will have page breaks and may have a header and footer repeated
on each page. Sometimes these page breaks occur in between a multi-line entry or field and you
need a way to block the section out in order to define fields above, below, or around it. If this is the
case, you must define a Header Layer or Footer Layer. By defining a Header Layer and/or Footer
Layer, you are telling Report Reader to skip over those lines.
30 Report Reader Version Nine Tutorial

As headers and footers are always associated with a page break, they are identified in Report
Reader by their position from a page break. Hence, there is no need to define a trap to identify
these lines. The starting point of a Header Layer is always the page break so you only need to define
how far from the page break the header extends. If you define the Header Layer to be five lines
from the page break, the five lines following every page break in the report will be selected as the
header and Report Reader will skip these lines when defining multi-line fields or entries. Similarly, a
Footer Layer ends at a page break, which means you only need to define where it begins. If you
choose three lines before the page break as the Footer Layer, the three lines before every page
break will be marked as part of the Footer Layer.

The Dave’sBookStore5.txt file contains company address information, which could be defined as a
multi-line field, as well as telephone number information after the detail or transaction lines.
Furthermore, the file contains a page break. This page break runs beneath the address lines for
Account Number 102 and is followed by a header.
 Chapter 4: More Examples 31

Note: A Header Layer or a Footer Layer can only be defined after the Base Layer is defined. The first
layer defined cannot be defined as a Header Layer or a Footer Layer. A template cannot have more
than one Header Layer or Footer Layer.

1. Highlight the required lines for the header.

2. Select the Create a standard layer option, and then click Yes.
3. Click the Header Layer button on the toolbar to designate the selected lines as the Header
Layer.
Note: A trap is not required to anchor the header lines. All of the lines up to the page break
are selected as a header.

4. Click the Save Layer button on the toolbar to save the Header Layer.
5. Scroll through the report to ensure that all of the header sections have been captured correctly.
32 Report Reader Version Nine Tutorial

Multi-Line Entries The Dave’sBookStore5.txt file that was used in the previous example will now be used to
and Multi-line Field demonstrate how to select and define multi-line fields.
Options
As the position of a fileld is setoff from the location of the Anchor Line (this is also true in the case
of a multi-line entry) it is easy to define where a multi-line field will start. But you also need a way of
defining where the field will end especially when the length of a multi-line field varies. In this case,
you must find a characteristic that is common to all of these fields, such as ending on a blank line or
ending at the next defined field.

Note: If the report contains header and/or footer information repeated with every page break, then
this information needs to be defined in a Header Layer and/or a Footer Layer in order for multi-line
fields to skip these lines and be defined correctly. A Header Layer and a Footer Layer can be defined
either before or after the layer with the multi-line field.

1. To create a layer with a multi-line entry, highlight the required lines.

A maximum of 30 lines can be selected as a sample.

2. Select the Create a standard layer option and then click Yes.
The selected lines are copied to the Field Editor at the top of the window.
 Chapter 4: More Examples 33

With a multi-line entry in the Field Editor, one of the lines must be the Anchor Line. This is the
line that the trap will be defined on and is identified by the red arrow in the left margin. By
default the Anchor Line is the first line. If a common unique character or pattern for the trap
cannot be found in the first line, a different line can be selected as the Anchor Line. To
designate another line as the Anchor Line, click in the left margin of the required line. The red
arrow will then be displayed in the left margin of the selected line.

3. In the Anchor Editor, define a specific trap by entering the letters Acc in the appropriate spot.
4. Using the following image as a guide, define the required fields.

Notice the address information is selected as a multi-line field and is on a different line from the
Anchor Line (the line the trap is defined on). If required, the address information could also be
defined as separate fields, such as STREET, CITY, STATE, ZIP_CODE, and COUNTRY.

In this case, as the contact name field is defined on the Anchor Line, data for the contact name
field will always be selected from the lines in the report that match the Anchor Line, while the
address field will always be selected to start at a particular point on the line after the line
matching the Anchor Line in the report. However, as the address is a multi-line field, when
defining the information in the Attributes area in the Field Details window, you must specify
where the field should end.
34 Report Reader Version Nine Tutorial

Note: The maximum length of a field is 500 characters. If the number of characters in a field
exceeds 500, only the first 500 characters will be extracted.

The ADDRESS field is automatically identified as a multi-line field. With a multi-line field, you can
choose from among the following three options the one that will best determine where the
field should end: Blank line, Defined field, and Specify line length.

• Blank line: When a multi-line field varies in length, but is always followed by a blank line,
selecting this option will terminate the field when the blank line is reached. Report Reader
adds blank lines before and after a page break. In the case where a page break falls
between the lines of a multi-line field, define a Header Layer and a Footer Layer on the
blank lines around the page break so Report Reader will skip the page break and continue
collecting data for the field after the page break until it reaches a blank line as in the
image below.

• Defined field: With this option selected, a multi-line field will be terminated when it meets
the next defined field, as displayed in the image below.

• Specify line length: This option works well if the multi-line field is always a fixed length. The
multi-line field cannot be defined to be more than 200 lines long. If this option is chosen,
specify the line length by entering the correct value in the Line Length box in the
Attributes area of the Field Details window.

Pre-Append and Transactions or detail lines are defined in the Base Layer. Additional information is then defined in
Post-Append Layers Append Layers that are appended to the Base Layer. Details captured in a report that are above the
Base Layer, such as the account number and contact name in the previous example, are part of the
Pre-Append Layer. However, the report could have details related to the transactions that appear
after the transaction. The layer that captures the information that is below the Base Layer is the
Post-Append Layer.

Fields in a Pre-Append Layer or Post-Append Layer are appended to associated records in the Base
Layer. There is no difference between defining a Pre-Append Layer or Post-Append Layer, the
difference is in the output. If the default option Leave blank is selected for the Blank Cells attribute
in a Pre-Append Layer, the fields are appended to the first Base Layer record, whereas with the Post-
Append Layer, the fields are appended to the last Base Layer record it is associated with.
 Chapter 4: More Examples 35

Using the same monthly shipping report (Dave’sBookStore5.txt) as in the previous example, notice
in the image displayed below, the defined fields in the Base Layer (Date, Item, and Amount), Pre-
Append Layer (Account, Contact, Company, and Address) and in the Post-Append Layer (Area Code
and Telephone Number).

Based on the Base Layer and Append Layers that were defined, the report would look as follows as
an IDEA database.
36 Report Reader Version Nine Tutorial

While defining the field details for a field in a Pre-Append Layer, if the default option Leave blank is
selected, when the record is extracted into an IDEA database, the data in that field is output only
with the first Base Layer transaction (record) it is associated with. Whereas a field defined in a Post-
Append Layer is written out only to the last record in the Base Layer that it is associated with.

In order to include the data from the Append Layers with all the records in the Base Layer, select the
Use value from previous record option. The data from the Pre-Append Layer will be copied to all of
the associated Base Layer records below it, while the data from the Post-Append Layer will be
copied to the associated Base Layer records above it.

Using Floating Any trap defined in a Floating layer is a Floating trap. These traps are useful when extracting data
Traps from a file where the data is not laid out in columns, such as a web log. Unlike a trap in a Standard
layer that scans vertically for matching characters, a trap in a Floating layer scans lines in the report
horizontally. Furthermore, the position of the Floating trap also determines where a field can start or
terminate.

Here are a few rules to keep in mind when using a Floating trap.

• A space between trap characters makes them two separate traps. For example, “://” is a
single trap, whereas “: //” would be considered as two traps, each of which would be
searched for independently.

• A multi-line field cannot be defined in a Floating layer.

• The starting position of a field defined to the right of a Floating trap has a positional
relationship with the trap character; it is offset from the trap character. For example, if a
field is defined in the Field Editor to start two characters to the right of the trap character,
in the report the field will always start two characters away from the first occurrence of
the trap character.

• The starting position of a field defined to the left of a Floating trap is offset from the trap
character while the ending position is determined by the field’s actual width as defined in
the Field Details window or the occurrence of trap character, which ever comes first. The
first occurrence of the trap character in a line will terminate a field defined to the left of
the trap character. As the field width will not extend beyond the trap character, the field’s
actual width can be defined to be wide enough to capture all the data from the widest
field in the report without selecting unnecessary data in smaller fields.
 Chapter 4: More Examples 37

The image below displays a web log file (access.txt). Notice that the information is not laid out in
columns. It is a standard web log file containing client hostname, date, request type, requested
data, protocol, URL, and client browser information.

The following section uses the access.txt web log file to describe how to create a template using
Floating traps.
38 Report Reader Version Nine Tutorial

Create a Floating 1. Highlight the first line in the report.

Trap
2. Select the Create a floating layer option, and then click Yes.

The selected line is copied to the Field Editor. Notice that the Floating Trap button is activated
on the main toolbar.

The sample line contains the following data:

• Client Hostname: 195.110.142.84

• Date: 28/Oct/2002
• Time: 07:35:51
• Microseconds: 0700
• Request Type: GET / HTTP/1.1
• Number: 200 818
• URL: http://www.gallerianocode.it/
• Client Browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
Before you can define fields, you must anchor the required lines in the report. As every entry
has a client hostname and there is a space after the hostname, you can use a generic Space
trap. However, as the length of the client hostname varies, and the space does not appear in a
single column, you must make the Space trap a Floating trap. The Floating Space trap will scan
each line horizontally and when it finds a space anywhere in the line, it will anchor the line. As
you will see later on, the Floating trap will serve a dual purpose, not only to anchor lines but it
will also determine where a field will terminate.

3. Ensure that the Floating Trap button is selected on the toolbar.

 Chapter 4: More Examples 39

4. Place the cursor in the Anchor Editor just above the first space in the Field Editor, and then
click the Space Trap button on the toolbar to enter a Space trap.

Notice that all of the lines in the report are anchored (highlighted in blue), indicating that a
match was found on every line. As the trap is defined as a Floating trap, the position of the
space is irrelevant. If the line has a space, it will be anchored.
40 Report Reader Version Nine Tutorial

5. Scroll though the report to verify that the required lines are anchored. Once the lines are
anchored, the data to be extracted must be defined as fields.

Note: If a line is not anchored with a trap, data from that line will not be captured.

If a Space trap was selected without choosing to create a Floating layer, the following image
displays the lines that would be captured. Only the lines with a space in the same position as
the trap would be anchored

If this happens, delete the trap and click the Floating Trap button on the toolbar to define this
as a Floating layer. Now re-define the trap and it will anchor all of the lines.
 Chapter 4: More Examples 41

6. In the Field Editor, highlight the client hostname data in order to define it as the first field.

The client hostname is highlighted in orange in the Field Editor and blue in the report.

7. Scroll through the report to ensure that the required data has been captured.
42 Report Reader Version Nine Tutorial

8. Notice in the lower half of the report there is a section where the last digit of the client
hostname is not selected. A Floating trap not only anchors lines in a report, but also determines
where a field can begin or end. Increase the actual width in the Field Editor to include the
longest client hostname and the Floating trap will terminate the field width at the first
occurrence of a space.

This demonstrates the rule that fields defined to the left of a Floating trap terminate at the first
occurrence of the trap character. In this case because of the Floating Space trap, the field ends
when it finds a space.
 Chapter 4: More Examples 43

9. The next fields to be defined are the time and date information. Since the date and time
information on every line follows the same pattern of spaces and hyphens and has the same
width, you can select the date and time fields in relation to the position of the existing Space
trap. Selecting the date 28/Oct/2008, the time 07:35:51, and the microseconds 0700 as three
separate fields in the Field Editor will result in the fields being selected in the report as in the
following image.

This demonstrates the rule that fields defined to the right of a Floating trap are related to the
position of the trap character. In this case because the date field is defined to begin five
characters after the Floating Space trap and be 11 characters long on every line in the report,
the data that is positioned five characters after the first occurrence of a space is selected as the
date field.

Additional Floating traps will be defined in the report, but the purpose of these traps will not
be to anchor lines in the report, since the lines in the report have already been anchored. These
Floating traps will help in defining where a field can start and end.

10. The Field Anchor is highlighted in orange and the corresponding data in the report is
highlighted in blue. The highlighted data in the report is a visual indication of the data that is
being captured. Scroll through the report to verify that the required data is highlighted.
44 Report Reader Version Nine Tutorial

The rest of the information in the line, such as the request type, the information requested,
and the browser engine, are all enclosed in quotation marks (“). Using these quotation marks
as Floating traps, you can define where the field starts and ends.

11. With the Floating Trap button still selected, enter a quotation mark in the Anchor Editor over
the first quotation mark in the Field Editor.

12. Select the request type (GET / HTTP/1.1) as a field.

However, in the second line of the report, as the request type is much longer, you must extend
the field width where necessary.
 Chapter 4: More Examples 45

13. Using the second quotation mark as a Floating trap you can increase the field’s actual width to
include all of the characters in a longer field while excluding anything after the quotation mark
in shorter fields. The second quotation mark as a Floating trap terminates the field in the
report, even though the actual width defined in the Field Details window is larger.
46 Report Reader Version Nine Tutorial

14. Select the number 200 818 in the Field Editor.

Note that in the second and third line in the report, the quotation mark is being selected as
part of the field.
 Chapter 4: More Examples 47

15. This can be fixed by adding the third quotation mark as a Floating trap and increasing the
actual width. Notice that you will also hav e to increase the actual width of the previous field
as well. The results will be as displayed in the image below.

16. The last two fields to be defined are the URL and the client browser. Again, as this information
is within quotation marks, the quotation marks can be used as Floating traps and to anchor
fields.
48 Report Reader Version Nine Tutorial

Exclude Line In cases where a unique trap cannot be defined to select the required lines while excluding
unwanted lines, the Exclude Line task can be used to exclude unwanted data.

An Exclude Line cannot be applied in the following cases:

• Before the Base Layer has been defined.

• To a line that was selected as the sample line for the Base Layer.

• To a line in which an Append Layer field is defined. It can only be applied to lines without
defined fields or with fields that are part of the Base Layer. Check the Layer Information
area in the Field Details window to determine the layer the field belongs to.

• To a line that is already being excluded with a filter on a field.

Using the ABC Balance Sheet.txt file as an example, notice that you cannot define a trap that will
select all of the detail lines without selecting the ledger total line.
 Chapter 4: More Examples 49

In order to clear the ledger total data, an Exclude Line must be inserted. Note the Exclude Line mark
is in the left margin of the ledger total line and the data in this line is not selected.

Click in the left margin adjacent to the line to be excluded.

Alternatively, highlight the line(s) and then in the Report Reader dialog box, select the Exclude it
from the output option. A red X-mark will be displayed in the left margin adjacent to the line(s) and
any data selected in these lines will be cleared.

Previewing the report as an IDEA database will confirm that the data in the excluded line(s) is not
included in the data to be extracted.

Filters Unlike the Exclude Line task that must be applied manually to the line or lines that must be excluded,
a filter is added to a field and will impact the record that the field is part of. While Exclude Line
works at the level of the line, a filter works at the field level.

Filters can be of two types:

• Include/Exclude: Used to restrict records to be extracted, by inclusion or exclusion,

to those that meet specified criteria. The Include/Exclude filter is case sensitive and
looks for exact matches between text entered in the list and the data in the field to
determine whether a record should be included or not. It can be applied to any
data type.
• Range: Can only be applied to a field with a Numeric data type.

Note: A filter can only be added to fields defined in the Base Layer. Filters cannot be defined on
Append Layer fields.
50 Report Reader Version Nine Tutorial

Using the file that was used in the previous Exclude Line example (ABC Balance Sheet.txt), if the line
with the ledger total appeared in multiple locations in the file, it could be excluded by defining a
filter on the account name field to exclude records where the account name is Ledger Total.

1. Select the required field (in this case, the account name field).

2. In the Field Details window, in the Filters area, click the ellipsis button for the Include/Exclude
box.

3. Select the desired option and enter the text in the box in the same case as it appears in the
report. Click Add to add the text to the list.

4. Click OK.

The Include/Exclude property will be displayed as Filtered in the Field Details window.

The result of adding Ledger Total to the list with the Exclude record with the following data option
selected is that the record is not selected.
 Index

Index

Accessing Report Reader 13

Actual Width 21
Anchor Editor 12
Anchored lines 12
Append Layer 23
Creating 23
Pre-Append Layer 23

Creating a template 14
Basic steps 14
Examine the data 15
Select a sample 15

Examining the data 15

Example files 13
Exclude line 48

Field Anchors
Actual Width 21
Defining 18
Deleting 21
Editing 21
Offset 21
Field Details window 12, 19
Field Editor 12
Field order 25
Filters 49
Include/Exclude 49
Range 49
Floating layer 15, 36
Floating traps 36

Headers and footers 30

52 Report Reader Version Nine Tutorial

Layers
Floating 15, 36
Saving 21
Standard 15

Menu bar 12
Multi-line entries 32
Blank line 34
Defined field 34
Specify line length 34

Numeric trap 16

Offset 21

Post-Append layer 34
Pre-Append Layer 23
Pre-Append layer 34
Previewing the database 22
Print Report file 9

Report View 12

Sample line 15
Maximum number of lines 15
Scan for Errors 21
Selecting a sample 15
Space trap 16
Standard layer 15

Text trap 16
Toolbar 12
Traps
Defining 16
Floating 36
Non-Blank 16
Numeric 16
 Index: Index 53

Space 16
Text 16

Use value from previous record 24