Module 7: A MultiLoad Application

After completing this module, you will be able to: Describe the tables involved in a MultiLoad job. Set error limits as a time value or as a percentage of loaded rows. Specify a checkpoint interval. Redefine input record layout.

New Accounts Application Description

New Account Information Customer details New Existing Transaction(s) Applied to Balance Added to History

MULTILOAD

Accounts
A# UPI

Customer
C# UPI

Account_Customer
A# C# NUSI UPI

Trans_Hist
A# NUPI DATE

Each New Account requires an INSERT into the Accounts table and an INSERT
into Account_Customer.

An Account can be opened for new or pre-existing customer(s) UPSERT to

the Customer table.

Each New Account will probably require an opening transaction UPDATE to

Account (balance) and INSERT to Trans_History.

New Accounts Application Script (1 of 3)

.LOGTABLE Newacct_logtable_mld; .LOGON tdpid/username, password; .BEGIN IMPORT MLOAD TABLES Accounts, Account_Customer, Customer, Trans_Hist ; .LAYOUT New_Acc_Layout ; .FILLER in_Field_Indicator 1 CHAR(1) ; .FIELD in_Account_Number 2 INTEGER ; .FIELD in_Number * INTEGER ; .FIELD in_Street * CHAR(25) ; .FIELD in_City * CHAR(20) ; .FIELD in_State * CHAR(2) ; .FIELD in_Zip_Code * INTEGER ; .FIELD in_Balance_Forward * INTEGER ; .FIELD in_Balance_Current * INTEGER ;
.FIELD .FIELD .FIELD .FIELD .FIELD .FIELD .FIELD .FIELD .FIELD .FIELD in_Customer_Number in_Last_Name in_First_Name in_Social_Security in_AC_Account_Number in_AC_Customer_Number in_Trans_Number in_Trans_Account_Number in_Trans_ID in_Trans_Amount 2 * * * 2 * 2 * * * INTEGER CHAR(25) CHAR(20) INTEGER ; ; ; ;

INTEGER ; INTEGER ; INTEGER INTEGER INTEGER INTEGER ; ; ; ;

New Accounts Application Script (2 of 3)

.DML LABEL Label_A ; INSERT INTO Accounts VALUES ( :in_Account_Number, :in_Number, :in_Street, :in_City ,:in_State, :in_Zip_Code, :in_Balance_Forward, :in_Balance_Current ) ; .DML LABEL Label_B ; INSERT INTO Account_Customer VALUES ( :in_AC_Account_Number, :in_AC_Customer_Number); .DML LABEL Label_C MARK MISSING UPDATE ROWS DO INSERT FOR MISSING UPDATE ROWS ; UPDATE Customer SET Last_Name = :in_Last_Name WHERE Customer_Number = :in_Customer_Number ; INSERT INTO Customer VALUES ( :in_Customer_Number, :in_Last_Name, :in_First_Name, :in_Social_Security ) ; .DML LABEL Label_D ; INSERT INTO Trans_Hist VALUES ( :in_Trans_Number, DATE, :in_Trans_Account_Number, :in_Trans_ID, :in_Trans_Amount ) ; UPDATE Accounts SET Balance_Current =Balance_Current + :in_Trans_Amount WHERE Account_Number =:in_Trans_Account_Number ; .IMPORT INFILE datain4 LAYOUT New_Acc_Layout APPLY Label_A WHERE (in_Field_Indicator = 'A' ) APPLY Label_B WHERE (in_Field_Indicator = 'B' ) APPLY Label_C WHERE (in_Field_Indicator = 'C' ) APPLY Label_D WHERE (in_Field_Indicator = 'D' ) ; .END MLOAD ;

New Accounts Application Script (3 of 3)

.IF (&SYSINSCNT1 > 0 OR &SYSUPDCNT1 > 0) THEN; COLLECT STATISTICS ON Accounts; .ENDIF; .IF (&SYSINSCNT2 > 0) THEN; COLLECT STATISTICS ON Account_Customer; .ENDIF; .IF (&SYSINSCNT3 > 0 OR &SYSUPDCNT3 > 0) THEN; COLLECT STATISTICS ON Customer; .ENDIF; .IF (&SYSINSCNT4 > 0) THEN; COLLECT STATISTICS ON Trans_Hist; .ENDIF;

MultiLoad environment variables can be checked to optionally COLLECT STATISTICS as part of the job.
&SYSDELCNT_ &SYSINSCNT_ &SYSUPDCNT_ &SYSETCNT_ &SYSUVCNT_ where _ is 1 to 5 " " " "

.LOGOFF ;

Note:
.BEGIN IMPORT MLOAD TABLES Accounts, Account_Customer, Customer, Trans_Hist ; 1st table 2nd table 3rd table 4th table

.BEGIN IMPORT Task Commands

Specifies the tables and optionally the work and error tables used in this
MultiLoad job.

Also used to specify miscellaneous MultiLoad options such as checkpoint,

sessions, etc.
.BEGIN [IMPORT] MLOAD TABLES tname1, tname2, ... WORKTABLES wt_table1, wt_table2, ERRORTABLES et_ table1 uv_table1, et_ table2 uv_table2, ... ERRLIMIT errcount [errpercent] CHECKPOINT rate (Default 15 min.) SESSIONS limit (Default 1 per AMP + 2) TENACITY hours (Default 4 hours) SLEEP minutes (Default 6 minutes) AMPCHECK NONE | APPLY | ALL NOTIFY OFF | LOW | MEDIUM | HIGH . . . ; .END MLOAD ;

Work Tables
IMPORT Task: WORKTABLES wt_table1, wt_table2,
.BEGIN parameters

DELETE Task: WORKTABLES wt_table1

Default is in users default database and the work table is named WT_TableName. Alternative may be specified as DataBaseName.WorkTableName.

There must be one work table defined for each data table.
Example: .BEGIN [IMPORT] MLOAD TABLES Employee, PayCheck WORKTABLES util_db.WT_Emp ,util_db.WT_Pay ERRORTABLES util_db.ET_Emp util_db.UV_Emp ,util_db.ET_Pay util_db.UV_Pay ...;

The Error Tables are described on the next page.

Error Tables
ERRORTABLES et_tab1 uv_tab1, et_tab2 uv_tab2, ...
.BEGIN parameters

Error table 1 (et)

default is the users database and the table is named ET_Tablename. contains any errors that occur in the Acquisition Phase. contains primary index overflow errors that occur in the Application phase
default is the users database and the table is named UV_Tablename. contains Application Phase errors.

Error table 2 (uv)

Uniqueness violations Constraint errors Overflow errors on columns other than primary index .BEGIN [IMPORT] MLOAD TABLES Employee, PayCheck WORKTABLES util_db.WT_Emp ,util_db.WT_Pay ERRORTABLES util_db.ET_Emp util_db.UV_Emp ,util_db.ET_Pay util_db.UV_Pay ...;

Example:

ERRLIMIT
ERRLIMIT ErrCount Without ERRPERCENT:
.BEGIN parameters

Specifies approximate number of data errors permitted during Acquisition. Does not count Uniqueness violations.

ERRLIMIT ErrCount ErrPercent With ERRPERCENT:

Specifies a percentage of data errors after an approximate number of records has been transmitted.
Example:

ERRLIMIT 10000 5

In this example, after processing 10,000 input records, the system looks for an error rate of 5%.

CHECKPOINT

Rate may be specified in the Acquisition Phase of a complex IMPORT task as:
.BEGIN parameters

A number of incoming records (exact count; not less than 60) A time interval in minutes (approximate; less than 60)

If no CHECKPOINT value is specified MultiLoad will checkpoint every 15 minutes, and at the end of each Phase. The default is 15 minutes.

Example 1:

CHECKPOINT 30

In this example, a 30-minute time interval is specified.

Example 2:

CHECKPOINT 100000

In this example, a checkpoint after 100,000 input records is specified.

More .BEGIN Parameters

SESSIONS SESSIONS 64 48
.BEGIN parameters

Used to specify the maximum, and optionally, minimum sessions generated by MultiLoad.

TENACITY

TENACITY 10

Number of hours MultiLoad will try to establish a connection to the system. The default is 4 hours.

SLEEP

SLEEP 3

Number of minutes MultiLoad waits before retrying a logon; must be greater than 0. The default is 6 minutes.

NOTIFY

NOTIFY OFF

NOTIFY LOW NOTIFY MEDIUM for the most significant events. NOTIFY HIGH for every MultiLoad event that involves an operational decision point. NOTIFY OFF suppresses the notify option.

Note: The MultiLoad manual specifies in detail which events are associated with each level.

More .BEGIN Parameters: AMPCHECK

AMPCHECK NONE | APPLY | ALL
.BEGIN parameters

NONE MultiLoad will not perform an AMPCHECK. It will proceed if AMPs are offline, provided all target tables are
FALLBACK.

APPLY MultiLoad will continue in all phases except the Application phase with
AMPs offline, provided all target tables are FALLBACK. This is the default.

ALL MultiLoad will not proceed with down AMPs, regardless of the
protection-type of the target tables. Most conservative option.

DELETE Task Commands

Specifies the table and optionally the work and error tables used in this
MultiLoad Delete task.

Also used to specify miscellaneous MultiLoad options such as tenacity,

sleep, etc.
.BEGIN DELETE MLOAD TABLES tname1 WORKTABLES wt_table1 ERRORTABLES et_ table1 TENACITY hours SLEEP minutes NOTIFY OFF | LOW | MEDIUM | HIGH . . . ; .END MLOAD

(Default 4 hours) (Default 6 minutes)

.LAYOUT and .TABLE

.LAYOUT Must appear between the .BEGIN IMPORT MLOAD command and the applicable .IMPORT command. Must be immediately followed by .TABLE or .FIELD commands.

.TABLE The input fields are defined with the same name, data type, and order of an existing table.
Format: .LAYOUT layoutname CONTINUEIF position = variable INDICATORS tableref ; ;

.TABLE

.LAYOUT Parameters CONTINUEIF

CONTINUEIF

Record 1 and the immediately following Record 2 each represent a part of

the input record. Continuation Character Field is defined beginning in character position 1.
1 2 10 45 80

Y
Physical Input Records 1 2

F1
35

F2

F3
80

Record 1

N FILLER
MultiLoad .LAYOUT Definition

F5

Record 2

.LAYOUT Record_Layout CONTINUEIF 1= 'Y';

1 9 44 79 80 112 113 158

Logical Input Transaction

F1

F2

F3

FILLER

F5

Note: F1 begins in character position 1 and Filler immediately follows F3.

.LAYOUT Parameters INDICATORS

INDICATORS The INDICATORS contain bits that, when equal to 1, represent a null data field.
Physical Input Records Indicator Byte(s)

F1

F2

F3

MultiLoad .LAYOUT Definition

.LAYOUT Record_Layout INDICATORS;

.FIELD and .FILLER

.FIELD fieldname { startpos datadesc } || fieldexp [ NULLIF nullexpr ] [ DROP {LEADING / TRAILING } { BLANKS / NULLS } [ [ AND ] {TRAILING / LEADING } { NULLS / BLANKS } ] ] ; .FILLER [ fieldname ] startpos datadesc ; .FIELD

Input fields supporting redefinition and concatenation.

Startpos Fieldexpr identifies the start of a field relative to 1. specifies a concatenation of fields in the format:

fieldname1 || fieldname2 [ || fieldname3 ]

The option DROP LEADING / TRAILING BLANKS / NULLS is applicable only to character datatypes, and is sent as a VARCHAR with a 2-byte length field. .FILLER

Identifies data NOT to be sent to the Teradata database.

.LAYOUT Command Examples

Example 1: .LAYOUT transrecord CONTINUEIF 14 = 'ABC' INDICATORS; .FIELD .FILLER .FIELD .FIELD field1 field2 field3 field4 1 * * (char(5)) (char(1)) ; (char(3)) ; field2 | | '&' | | field3 ; NULLIF field1 = 'AAAAA' DROP LEADING BLANKS ;

Example 2: .LAYOUT emp_record; .TABLE Employee; Note: Employee is an existing table whose column names and data descriptions are assigned, in the order defined, to fields of the input data records.

Redefining the Input Example

A bank loads daily transactions sequentially on a tape for batch processing by MultiLoad. An input data record might be an Add, Update or Delete, each of which has a different length and contains different fields, as illustrated:
Add New Account

A U

PI PI

F1 F4 F5

F2

F3

Update Existing Account

Delete Inactive Account

PI
Record_Layout ; trans_type PI F1 F2 F3 F4 F5

.LAYOUT .FILLER .FIELD .FIELD .FIELD .FIELD .FIELD .FIELD

1 2 * * * 6 *

CHAR(1) ; INTEGER ; INTEGER ; CHAR(25) ; CHAR(20) ; CHAR(2) ; INTEGER ;

Note: FILLER data is not placed into ML work tables.

The .DML Command Options

Defines Labels along with Error Treatment conditions for one or more following INSERTs, UPDATEs or DELETEs to be applied under various conditions: .DML LABEL Labelname MARK or IGNORE
| UPDATE ] ROWS [INSERT UPDATE | DELETE Whether or not to record duplicate or missing INSERT, UPDATE, OR DELETE rows into the UV_error_table and continue processing.

DO INSERT FOR [MISSING UPDATE] ROWS Operation: INSERT (Duplicate violation) UPDATE (Duplicate violation) UPDATE (Fails - missing row) DELETE (Fails - missing row) UPSERT (If successful) UPSERT (Fails) Default:

Marked in UV_tablename Marked in UV_tablename Marked in UV_tablename Marked in UV_tablename Ignored Mark failure of INSERT in UV_tablename

Example of UPSERT failure: 1. PI value doesnt exist, so UPDATE cant occur. 2. INSERT fails because of check violation - e.g., cant put character data in a numeric field.

MARK | IGNORE MARK | IGNORE

DUPLICATE MISSING

The .DML Command Options (cont.)

.DML LABEL Labelname

DO INSERT FOR [MISSING UPDATE] ROWS DO INSERT FOR MISSING UPDATE

Key statement that indicates an UPSERT. An SQL UPDATE followed by an SQL INSERT is required.

Example 1:

.DML LABEL Action1

DO INSERT FOR MISSING UPDATE ROWS;

The default for an UPSERT operation is to not mark missing update rows.

Example 1:

.DML LABEL Action2

MARK MISSING UPDATE ROWS DO INSERT FOR MISSING UPDATE ROWS;

When the MARK MISSING UPDATE ROWS is used with an UPSERT, this will place (in the UV_table) data rows that cant be updated (no PI). If the insert also fails, the insert record is also marked in the UV_table.

MARK | IGNORE MARK | IGNORE

DUPLICATE MISSING

| UPDATE ] ROWS [INSERT UPDATE | DELETE

MultiLoad Statistics
At the end of the application phase, MultiLoad provides statistical information.
****06:06:41 UTY1803 Import Processing Statistics Total Thus far 200000 200000

Candidate Records considered . . . . Apply conditions satisfied . . . . .

Import 1 200000 200000

****06:18:34 UTY0818 Statistics for table ACCOUNTS: Inserts: 50000 Updates: 50000 Deletes: 0 ****06:18:35 UTY0818 Statistics for table ACCOUNT_CUSTOMER: Inserts: 50000 Updates: 0 Deletes: 0 ****06:18:35 UTY0818 Statistics for table CUSTOMER: Inserts: 25000 Updates: 25000 Deletes: 0 ****06:18:35 UTY0818 Statistics for table TRANS_HIST: Inserts: 50000 Updates: 0 Deletes: 0

INMODs
Data

MultiLoad
0

INMOD

Teradata

INMOD to MultiLoad return codes: After a READ call.

Valid data record in Buffer EOF not reached. Length field reflects correct length of output record. If an input record was supplied to the INMOD from MLOAD and is to be skipped, the length field should be set to zero. If no input record was supplied, setting the length to zero indicates EOF.

Non 0

INMOD indicates end-of-file condition.

Indicates a successful operation. Indicates a processing error; MLOAD will terminate Calling for the first time. INMOD should open files to read data. MLOAD expects a record. MultiLoad expects a record. MLOAD has been restarted. INMOD should position itself to the last checkpoint. There may be re-positioning information in the data buffer. Request for INMOD to take a Checkpoint. The INMOD should return any information (up to 100 bytes) needed to reposition itself. MLOAD saves this in the Restart Log Table. Request for INMOD to reposition itself at the last checkpoint. Repositioning information may be in the data buffer. Request for INMOD to wrap up at termination. Request for INMOD to initialize and receive record. Request for INMOD to receive next record.

INMOD to MultiLoad return codes: After a non-READ call.

0 1 0 1

MultiLoad to INMOD additional return codes.

2 3

Note: MultiLoad can also use the FastLoad INMOD return codes.

4 5 6 7

Summary

On the .BEGIN statement, optionally, the names of work and error tables can be specified. You can:

Specify error limits and checkpoints. Limit sessions. Designate time allowed for connection. Specify retry time limit. Designate the level of notification you prefer. Designate how MultiLoad will proceed if AMPs are offline.

.LAYOUT parameters define the data format.

.DML commands define Labels and Error Treatment conditions for one or more operations.
You can use FastLoad or MultiLoad INMODs.

Review Questions
1. Complete the BEGIN statement to accomplish the following:

Specify an error limit count of 200,000 and an error percentage of 5%. Specify a checkpoint at 50,000 records. Limit the sessions to 4. Set the number of hours to try to establish connection as 6.
.LOGTABLE RestartLog_mld; .LOGON ________________; .BEGIN [IMPORT] MLOAD TABLES Trans_Hist

ERRLIMIT 200000 5 CHECKPOINT 50000 SESSIONS 4

;

TENACITY 6

.END MLOAD ;

Review Questions
1. Complete the BEGIN statement to accomplish the following:

Specify an error limit count of 200,000 and an error percentage of 5%. Specify a checkpoint at 50,000 records. Limit the sessions to 4. Set the number of hours to try to establish connection as 6.
.LOGTABLE RestartLog_mld; .LOGON ________________; .BEGIN [IMPORT] MLOAD TABLES Trans_Hist

ERRLIMIT 200000 5 CHECKPOINT 50000

SESSIONS 4
TENACITY 6
; .END MLOAD ;

Lab Exercises
Lab Exercise 7-1
Purpose In this lab, you will use MultiLoad to insert the data rows you deleted in a lab from Module 6. The Accounts, Trans, and Customer tables each should have 100 rows in them (from Lab 6-1). What you need Populated tables and macro AU.Lab7_1. Tasks 1. Export data to a file called data7_1 by executing the macro AU.Lab7_1. Submit the following commands in BTEQ: .EXPORT DATA FILE = data7_1; EXEC AU.LAB7_1; .EXPORT RESET; Note: The Trans_Date is exported with an ANSI Date Format of 'YYYY-MM-DD' and a data type of CHAR(10). 2. Prepare a MultiLoad script which inserts rows into one of three different tables using the redefinition feature of MultiLoad. The input record contains a mixture of records which are to be inserted into the Accounts table if the first record byte is an A, the Customer table if the first byte is a C and the Trans table if the first byte is a T. 3. Check your tables for 200 rows per table. Your MultiLoad job should have a final return code of zero and should evidence 100 rows inserted into each of the three tables.

Lab Exercises
Lab Exercise 7-2
Purpose In this lab, you will prepare and execute a MultiLoad script which performs an 'UPSERT' operation (INSERT MISSING UPDATE) on your Accounts table as a single operation. What you need Populated tables and macro AU.Lab7_2. Tasks 1. Delete all rows from the Accounts Table and use the following INSERT/SELECT to create 100 rows of data in your table. INSERT INTO Accounts SELECT * FROM AU.Accounts WHERE Account_Number LT 20024101 ; 2. Export data to a file data7_2 using the macro AU.lab7_2. .EXPORT DATA FILE = data7_2 ; EXEC AU.Lab7_2 ; .EXPORT RESET ; 3. Prepare and execute a MultiLoad script which performs an 'UPSERT' operation (INSERT MISSING UPDATE) on your Accounts table as a single operation. Use the data from data7_2 as input to the MultiLoad 'UPSERT' script. If the row exists, UPDATE the Balance_Current with the appropriate incoming value. If not, INSERT a row into the Accounts table. 4. Validate your results. MultiLoad should have performed 100 UPDATES and 100 INSERTS with a final return code of zero.

Lab Solutions for Lab 7-1

cat lab713.mld .LOGTABLE Restartlog713_mld ; .LOGON u4455/tljc30,tljc30 ; .BEGIN MLOAD TABLES Accounts, Customer, Trans ; .LAYOUT Record_Layout _713; .FILLER in_code 1 CHAR(1); .FIELD in_accountno 2 INTEGER; .FIELD in_number * INTEGER; .FIELD in_street * CHAR(25); .FIELD in_city * CHAR(20); .FIELD in_state * CHAR(2); .FIELD in_zip_code * INTEGER; .FIELD in_balancefor * DECIMAL (10,2); .FIELD in_balancecur * DECIMAL (10,2); .FIELD in_custno 2 INTEGER; .FIELD in_lastname * CHAR(30); .FIELD in_firstname * CHAR(20); .FIELD in_socsec * INTEGER; .FIELD in_transno 2 INTEGER; .FIELD in_transdate * CHAR(10); .FIELD in_accid * INTEGER; .FIELD in_transid * CHAR(4); .FIELD in_transamount * DECIMAL(10,2); (continued) .DML LABEL Insert_Acct ; INSERT INTO Accounts VALUES (:in_accountno, :in_number, :in_street, :in_city, :in_state, :in_zip_code, :in_balancefor, :in_balancecur) ; .DML LABEL Insert_Cust ; INSERT INTO Customer VALUES ( :in_custno, :in_lastname, :in_firstname, :in_socsec ); .DML LABEL Insert_Trans ; INSERT INTO Trans VALUES ( :in_transno, :in_transdate, :in_accid, :in_transid, :in_transamount); .IMPORT INFILE data7_1 LAYOUT Record_Layout_713 APPLY Insert_Acct WHERE in_code = 'A' APPLY Insert_Cust WHERE in_code = 'C' APPLY Insert_Trans WHERE in_code = 'T'; .END MLOAD ; .LOGOFF ; mload < lab713.mld > lab713.out

Lab Solutions for Lab 7-2

cat lab723.mld .LOGTABLE Restartlog723_mld ; .LOGON u4455/tljc30,tljc30 ; .BEGIN IMPORT MLOAD TABLES Accounts ; .LAYOUT Record_Layout_723; .FIELD in_accountno 1 INTEGER; .FIELD in_number * INTEGER; .FIELD in_street * CHAR(25); .FIELD in_city * CHAR(20); .FIELD in_state * CHAR(2); .FIELD in_zip_code * INTEGER; .FIELD in_balancefor * DECIMAL (10,2); .FIELD in_balancecur * DECIMAL (10,2); .DML LABEL Fix_Account DO INSERT FOR MISSING UPDATE ROWS ; UPDATE Accounts SET WHERE Balance_Current = :in_balancecur Account_Number = :in_accountno ;

INSERT INTO Accounts VALUES (:in_accountno, :in_number, :in_street, :in_city, :in_state, :in_zip_code, :in_balancefor, :in_balancecur); .IMPORT INFILE data7_2 LAYOUT Record_Layout_723 APPLY Fix_Account; .END MLOAD; .LOGOFF; mload < lab723.mld > lab723.out