Expression Transformation

Passive and connected transformation.

Use the Expression transformation to calculate values in a single row before we write to the target. For example, we might need to adjust employee salaries, concatenate first and last names, or convert strings to numbers. Use the Expression transformation to perform any non-aggregate calculations. Example: Addition, Subtraction, Multiplication, Division, Concat, Uppercase conversion, lowercase conversion etc. We can also use the Expression transformation to test conditional statements before we output the results to target tables or other transformations. Example: IF, Then, Decode There are 3 types of ports in Expression Transformation:

Input Output Variable: Used to store any temporary calculation.

Calculating Values : To use the Expression transformation to calculate values for a single row, we must include the following ports:

Input or input/output ports for each value used in the calculation: For example: To calculate Total Salary, we need salary and commission. Output port for the expression: We enter one expression for each output port. The return value for the output port needs to match the return value of the expression.

We can enter multiple expressions in a single Expression transformation. We can create any number of output ports in the transformation. Example: Calculating Total Salary of an Employee

Import the source table EMP in Shared folder. If it is already there, then dont import. In shared folder, create the target table Emp_Total_SAL. Keep all ports as in EMP table except Sal and Comm in target table. Add Total_SAL port to store the calculation. Create the necessary shortcuts in the folder.

Creating Mapping: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. Open folder where we want to create the mapping. Click Tools -> Mapping Designer. Click Mapping -> Create -> Give mapping name. Ex: m_totalsal Drag EMP from source in mapping. Click Transformation -> Create -> Select Expression from list. Give name and click Create. Now click done. Link ports from SQ_EMP to Expression Transformation. Edit Expression Transformation. As we do not want Sal and Comm in target, remove check from output port for both columns. Now create a new port out_Total_SAL. Make it as output port only. Click the small button that appears in the Expression section of the dialog box and enter the expression in the Expression Editor. Enter expression SAL + COMM. You can select SAL and COMM from Ports tab in expression editor. Check the expression syntax by clicking Validate. Click OK -> Click Apply -> Click Ok. Now connect the ports from Expression to target table. Click Mapping -> Validate Repository -> Save

Create Session and Workflow as described earlier. Run the workflow and see the data in target table.

As COMM is null, Total_SAL will be null in most cases. Now open your mapping and expression transformation. Select COMM port, In Default Value give 0. Now apply changes. Validate Mapping and Save. Refresh the session and validate workflow again. Run the workflow and see the result again. Now use ERROR in Default value of COMM to skip rows where COMM is null. Syntax: ERROR(Any message here) Similarly, we can use ABORT function to abort the session if COMM is null. Syntax: ABORT(Any message here) Make sure to double click the session after doing any changes in mapping. It will prompt that mapping has changed. Click OK to refresh the mapping. Run workflow after validating and saving the workflow. Performance tuning : Expression transformation is used to perform simple calculations and also to do Source lookups. 1. Use operators instead of functions. 2. Minimize the usage of string functions. 3. If we use a complex expression multiple times in the expression transformer, then Make that expression as a variable. Then we need to use only this variable for all computations

Filter transformtioan

Active and connected transformation.

We can filter rows in a mapping with the Filter transformation. We pass all the rows from a source transformation through the Filter transformation, and then enter a Filter condition for the transformation. All ports in a Filter transformation are input/output and only rows that meet the condition pass through the Filter Transformation.

Example: to filter records where SAL>2000

Import the source table EMP in Shared folder. If it is already there, then dont Import. In shared folder, create the target table Filter_Example. Keep all fields as in EMP table. Create the necessary shortcuts in the folder.

Creating Mapping: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. Open folder where we want to create the mapping. Click Tools -> Mapping Designer. Click Mapping -> Create -> Give mapping name. Ex: m_filter_example Drag EMP from source in mapping. Click Transformation -> Create -> Select Filter from list. Give name and click Create. Now click done. Pass ports from SQ_EMP to Filter Transformation. Edit Filter Transformation. Go to Properties Tab Click the Value section of the Filter condition, and then click the Open button. The Expression Editor appears. Enter the filter condition you want to apply. Click Validate to check the syntax of the conditions you entered. Click OK -> Click Apply -> Click Ok. Now connect the ports from Filter to target table. Click Mapping -> Validate Repository -> Save

Create Session and Workflow as described earlier. Run the workflow and see the data in target table.

How to filter out rows with null values? To filter out rows containing null values or spaces, use the ISNULL and IS_SPACES Functions to test the value of the port. For example, if we want to filter out rows that Contain NULLs in the FIRST_NAME port, use the following condition: IIF (ISNULL (FIRST_NAME), FALSE, TRUE) This condition states that if the FIRST_NAME port is NULL, the return value is FALSE and the row should be discarded. Otherwise, the row passes through to the next Transformation. Performance tuning: Filter transformation is used to filter off unwanted fields based on conditions we Specify. 1. Use filter transformation as close to source as possible so that unwanted data gets Eliminated sooner. 2. If elimination of unwanted data can be done by source qualifier instead of filter,Then eliminate them at Source Qualifier itself. 3. Use conditional filters and keep the filter condition simple, involving TRUE/FALSE or 1/0 Source Qualifier:

Active and Connected Transformation. The Source Qualifier transformation represents the rows that the Power Center Server reads when it runs a session. It is only transformation that is not reusable. Default transformation except in case of XML or COBOL files.

Tasks performed by Source Qualifier:

Join data originating from the same source database: We can join two or more tables with primary key-foreign key relationships by linking the sources to one Source Qualifier transformation. Filter rows when the Power Center Server reads source data: If we Include a filter condition, the Power Center Server adds a WHERE clause to the Default query. Specify an outer join rather than the default inner join: If we include a User-defined join, the Power Center Server replaces the join information Specified by the metadata in the SQL query. Specify sorted ports: If we specify a number for sorted ports, the Power Center Server adds an ORDER BY clause to the default SQL query. Select only distinct values from the source: If we choose Select Distinct,the Power Center Server adds a SELECT DISTINCT statement to the default SQL query. Create a custom query to issue a special SELECT statement for the Power Center Server to read source data: For example, you might use a Custom query to perform aggregate calculations. The entire above are possible in Properties Tab of Source Qualifier t/f.

SAMPLE MAPPING TO BE MADE:

Source will be EMP and DEPT tables. Create target table as showed in Picture above. Create shortcuts in your folder as needed.

Creating Mapping: 1. 2. 3. 4. 5. 6. 7. Open folder where we want to create the mapping. Click Tools -> Mapping Designer. Click Mapping-> Create-> Give mapping name. Ex: m_SQ_example Drag EMP, DEPT, Target. Right Click SQ_EMP and Select Delete from the mapping. Right Click SQ_DEPT and Select Delete from the mapping. Click Transformation -> Create -> Select Source Qualifier from List -> Give Name -> Click Create 8. Select EMP and DEPT both. Click OK. 9. Link all as shown in above picture.

10. Edit SQ -> Properties Tab -> Open User defined Join -> Give Join condition EMP.DEPTNO=DEPT.DEPTNO. Click Apply -> OK 11. Mapping -> Validate 12. Repository -> Save

Create Session and Workflow as described earlier. Run the Workflow and see the data in target table. Make sure to give connection information for all tables.

SQ PROPERTIES TAB 1) SOURCE FILTER: We can enter a source filter to reduce the number of rows the Power Center Server queries. Note: When we enter a source filter in the session properties, we override the customized SQL query in the Source Qualifier transformation. Steps: 1. 2. 3. 4. 5. In the Mapping Designer, open a Source Qualifier transformation. Select the Properties tab. Click the Open button in the Source Filter field. In the SQL Editor Dialog box, enter the filter. Example: EMP.SAL)2000 Click OK.

Validate the mapping. Save it. Now refresh session and save the changes. Now run the workflow and see output. 2) NUMBER OF SORTED PORTS: When we use sorted ports, the Power Center Server adds the ports to the ORDER BY clause in the default query. By default it is 0. If we change it to 1, then the data will be sorted by column that is at the top in SQ. Example: DEPTNO in above figure.

Steps: 1. 2. 3. 4.

If we want to sort as per ENAME, move ENAME to top. If we change it to 2, then data will be sorted by top two columns.

In the Mapping Designer, open a Source Qualifier transformation. Select the Properties tab. Enter any number instead of zero for Number of Sorted ports. Click Apply -> Click OK.

Validate the mapping. Save it. Now refresh session and save the changes. Now run the workflow and see output. 3) SELECT DISTINCT: If we want the Power Center Server to select unique values from a source, we can use the Select Distinct option.

Just check the option in Properties tab to enable it.

4) PRE and POST SQL Commands

The Power Center Server runs pre-session SQL commands against the source database before it reads the source.

It runs post-session SQL commands against the source database after it writes to the target. Use a semi-colon (;) to separate multiple statements.

5) USER DEFINED JOINS Entering a user-defined join is similar to entering a custom SQL query. However, we only enter the contents of the WHERE clause, not the entire query.

Steps: 1. 2. 3. 4.

We can specify equi join, left outer join and right outer join only. We Cannot specify full outer join. To use full outer join, we need to write SQL Query.

Open the Source Qualifier transformation, and click the Properties tab. Click the Open button in the User Defined Join field. The SQL Editor Dialog Box appears. Enter the syntax for the join. Click OK -> Again Ok.

Validate the mapping. Save it. Now refresh session and save the changes. Now run the workflow and see output.

Join Type Equi Join Left Outer Join Right Outer Join

Syntax DEPT.DEPTNO=EMP.DEPTNO {EMP LEFT OUTER JOIN DEPT ON DEPT.DEPTNO=EMP.DEPTNO} {EMP RIGHT OUTER JOIN DEPT ON DEPT.DEPTNO=EMP.DEPTNO}

6) SQL QUERY For relational sources, the Power Center Server generates a query for each Source Qualifier transformation when it runs a session. The default query is a SELECT statement for each source column used in the mapping. In other words, the Power Center Server reads only the columns that are connected to another Transformation. In mapping above, we are passing only SAL and DEPTNO from SQ_EMP to Aggregator transformation. Default query generated will be:

SELECT EMP.SAL, EMP.DEPTNO FROM EMP

Viewing the Default Query 1. 2. 3. 4. Open the Source Qualifier transformation, and click the Properties tab. Open SQL Query. The SQL Editor displays. Click Generate SQL. The SQL Editor displays the default query the Power Center Server uses to Select source data. 5. Click Cancel to exit. Note: If we do not cancel the SQL query, the Power Center Server overrides the default query with the custom SQL query. We can enter an SQL statement supported by our source database. Before entering the query, connect all the input and output ports we want to use in the mapping. Example: As in our case, we cant use full outer join in user defined join, we can write SQL query for FULL OUTER JOIN:

SELECT DEPT.DEPTNO, DEPT.DNAME, DEPT.LOC, EMP.EMPNO, EMP.ENAME, EMP.JOB, EMP.SAL, EMP.COMM, EMP.DEPTNO FROM EMP FULL OUTER JOIN DEPT ON DEPT.DEPTNO=EMP.DEPTNO WHERE SAL>2000

We also added WHERE clause. We can enter more conditions and write More complex SQL.

We can write any query. We can join as many tables in one query as Required if all are in same database. It is very handy and used in most of the projects. Important Points:

When creating a custom SQL query, the SELECT statement must list the port names in the order in which they appear in the transformation.

Example: DEPTNO is top column; DNAME is second in our SQ mapping. So when we write SQL Query, SELECT statement have name DNAME first, DNAME second and so on. SELECT DEPT.DEPTNO, DEPT.DNAME

Once we have written a custom query like above, then this query will Always be used to fetch data from database. In our example, we used WHERE SAL>2000. Now if we use Source Filter and give condition SAL) 1000 or any other, then it will not work. Informatica will always use the custom query only. Make sure to test the query in database first before using it in SQL Query. If query is not running in database, then it wont work in Informatica too. Also always connect to the database and validate the SQL in SQL query editor.