12.2.5.

1 Engine deployment guide

Unless otherwise noted, all instructions refer to all target platforms: Windows, Linux and Solaris.
This release version (and following versions) is using the Oracle Advanced Queuing (AQ)
technology via the Notifications package to communicate between the Engine, Engine Manager
and Engine Starter, replacing previous DCOM technology in Windows and Application Server,
JNI, Tunneling technology used in UNIX-like OS. The end result is more consistent way of
deployment, communication, execution as well as providing enhanced security. Effectively, all
Engines related executables will run standalone from a system shell/command console via
provided scripts.
Use flow:
1. Engine starter is executed and registers itself in DB using a unique ID for that machine.
Different Engine Starters can run on several different machines. Engine starter then
listens for requests to start an engine process.
2. Engine Manager is executed in either Batch/Simulation mode, looks up the DB for
registered and active Engine Starters.
3. Engine Manager sends notification request to an active Engine Starter to start an Engine
Process.
4. Engine Starter receives the request and spawns a new Engine process, passing the
connection details or Oracle wallet name and context info as command line parameters.
5. Engine process starts and then communicates directly to Engine Manager via
notifications. Requesting new branch/simulation or status pings or completion
notification.

Terms:

Engine The Engine executable, either Engine on UNIX-like OS or

Engine2k_122.exe on Windows OS.
Engine modules Either the Engine Manager, Engine Starter or Engine
executables.
Host Any machine that will be used to run any of the Engine modules.
Engine host Any machine where Engine instances are intended to run and thus
Engine Starter must be running on that same host.
Engine Root directory This is the highest level directory where Engine files are
deployed. One level higher than the bin directory.
Script Either a batch .bat in Windows or shell script .sh on other platforms.
Any example or instructions referring to a script are thus meant for all
platforms despite referring to a specific form of a script. All script files are
located in the Engine root directory on Windows OS and under the <Engine Root
directory>/bin in all other platforms that can also be referred to $ENGINE_ROOT
after setup process has been completed (UNIX-like OS).

Prerequisites for Engine setup:

1. Selecting deployment type 64 bit or 32 bit.
Windows - By default the bin directory will contain 32 bit Engine binaries; in order
to use the 64 bit Engine, user must run the RegEng64.bat once. In order to revert back
to 32 bit Engine use: RegEng32.bat. The mentioned script files can be found under
the Engine Root directory.
UNIX-Like OS After performing a standard installation on a windows, the
following Unix deliverables will be under the LinuxDelivery\directory:
Oracle_Demantra_Linux_Engine.tar.gz
Oracle_Demantra_Linux64_Engine.tar.gz
Oracle_Demantra_Solaris_Engine.tar.gz
Oracle_Demantra_Solaris64_Engine.tar.gz
Select the one appropriate for ones system and deployment type and extract to the
chosen folder.
2. Oracle client a FULL latest 64/32 bit Oracle 11g (or higher) client (matching the
chosen Engine type) must be installed on each machine where either Engine Manager
or Engine is to be run and ORACLE_HOME set accordingly.
3. Oracle wallet - Oracle client deployment must include the tools (mkstore) to create
an Oracle wallet repository. The alternative is to provide full connection details to
Engine Manager, Engine Starter and Engine itself, including user and password in a
clear text form.

Ds.ini or any other forms of storing connection credentials except for Oracle
wallet are not supported.

Setup Process:
On each machine where either Engine Manager or Engines is to be run, the setup script
(Setup.bat/sh) must be run once to setup environment variables and/or Oracle wallet.
Setup process will generate a setenv script that contains the necessary environment
variables required for all engine modules to run, this script will be called from within the
other scripts used to run the Engine Manager, Engine Starter or Engine. Note that setup
process will overwrite any files produced by previous setup.
After setup completes successfully please review setenv script and correct any entries if
necessary.
ENG_CONNECTION This parameter defines the Oracle wallet connection name to be
used by Engine modules, modify only if you are using a different name than the default
DEM_CONN.
ENG_STARTER_ID This ID must be unique for Engine Starter per machine where
Engine Starter and Engines are to be run from. It defaults to the host name but in case
where host name is too long more than 8 characters), please choose shorter unique name
like: starter1, starter2 etc
ENG_NUM_CONCURRENT This parameter defines the number of concurrent Engine
processes allowed to run on the specific host. The default would be the number of
available CPU CORES. If this number is too large/small for your requirements please
modify it to the allowed number of Engines for that host. It replaces the existing
"ComputerNames" argument in Settings.xml, which is now deprecated.
TNS_ADMIN This parameter would point to the newly created Oracle wallet or
manually modified to point to your existing Oracle wallet.
UNIX-like OS only:
ORACLE_HOME Will point to included instant client binaries within Engine
deployment if ORACLE_HOME was found to be empty.
Example of setenv.bat:
@echo off
set ENG_CONNECTION=DEM_CONN
set ENG_STARTER_ID=%COMPUTERNAME%
for /f "skip=1 delims=" %%i in ('wmic cpu get numberofcores') do if not defined
NumberOfCores set NumberOfCores=%%i
SET ENG_NUM_CONCURRENT=%NumberOfCores%
set TNS_ADMIN=%~dp0TNS_ADMIN
set PATH=%~dp0\bin;%PATH%

Using a secure Oracle wallet (creating a new wallet repository):

1. Run Setup.sh script
2. When prompted provide the following details:
o DB Hostname The DB machine host name
o Port The DB server listener port
o Service Name The DB service/sid name
o Username The user name for your Demantra schema
o Password The Schema password that should be used for Engine run.
o Oracle wallet password Provide a password to protect the newly created
wallet repository, you will be asked to retype this password again for
verification and a third time to add the actual credentials into the wallet.
Password should at least be of 8 characters long, please memorize it in case
you want to manually change the existing wallet credentials.
3. After setup process completes successfully a default wallet connection name
DEM_CONN pointing to the specified DB schema user will be created and
stored in a new directory TNS_ADMIN that will also contain the required
tnsnames.ora and sqlnet.ora files.
4. Setup will create and automatically run the sql script UpdatEngPath.sql which
updates the new SYS_PARAMS parameter EngineBasePath which is used by
workflow to identify where Engine is deployed. This path will be accessible in
workflow via a new token #EngineBasePath# that can then be used to call any
of the Engine scripts.
5. System admin/DBA is responsible to modify the security privileges for the newly
created wallet files so only the intended user/s can have access to them. For
example: allow only read access to the user running the engine but read/write
access to admin.
Example of setup process:
>Setup.bat
Enter Hostname: my_host.il.oracle.com
Enter Port: 1521
Enter Service Name: my_service.il.oracle.com
Enter Username: my_schema_name
Enter Password: my_schema_pass
d:\Eng_Matlab_Test_64bit\engine2k\Source6.3DLL\install\TNS_ADMIN
Preparing DEM_CONN Credential. You will be asked to supply wallet password(8 symbols)
Oracle Secret Store Tool : Version 11.2.0.1.0 - Production
Copyright (c) 2004, 2009, Oracle and/or its affiliates. All rights reserved.
Enter password: mywalletpass1
Enter password again: mywalletpass1
Oracle Secret Store Tool : Version 11.2.0.1.0 - Production
Copyright (c) 2004, 2009, Oracle and/or its affiliates. All rights reserved.
Enter wallet password: mywalletpass1
Create credential oracle.security.client.connect_string1

Wallet Created DEM_CONN for my_schema_name

Using unsecure direct connection details (not recommended):

1. Run Setup.sh nowallet
2. You will not be prompted for any input as DB credentials will be provided
directly to Manager, Engine Starter and Engine.
3. Setup will create the sql script UpdatEngPath.sql which updates the new
SYS_PARAMS parameter EngineBasePath which is used by workflow to
identify where Engine is deployed. This path will be accessible in workflow via a
new token #EngineBasePath# that can then be used to call any of the Engine
scripts. You must run this script manually to update the parameter, so workflow
will be able to start Engine runs.

Using an existing secure wallet (Manual setup):

1. Run Setup.sh nowallet.
2. You will not be prompted for any input as DB credentials will be provided
directly to Manager, Engine Starter and Engine.
3. Setup will create the sql script UpdatEngPath.sql which updates the new
SYS_PARAMS parameter EngineBasePath which is used by workflow to
identify where Engine is deployed. This path will be accessible in workflow via a
new token #EngineBasePath# that can then be used to call any of the Engine
scripts. You must run this script manually to update the parameter, so workflow
will be able to start Engine runs.
4. Add a new DEM_CONN connection to your existing wallet repository,
pointing to the intended schema for Demand Engine. Your wallet must be
configured for auto login as otherwise engine modules will not be able to connect
without user intervention.
5. Review the generated setenv script, modify the TNS_ADMIN variable to point
to your existing wallet or alternately make sure it is defined at the system/user
level and remove it from the setenv script.

RAC advanced setup with Wallet:

1. Use one of the above wallet setup methods according to your preference.
2. Create a dedicated RAC service name for each of the RAC nodes.
3. To the created/existing wallet, add additional connections, one per each existing
RAC service pointing specifically to that service. While DEM_CONN should
point to the general scan address, the service specific connections

(DEM_CONN1, DEM_CONN2) should each point to a one specific RAC

service name.
4. In Settings.xml please modify the "RACServiceNames" argument value to list the
newly created wallet connection names. Example: argument="DEM_CONN1,
DEM_CONN2".
1.

On each Machine where Engines are to be run, Engine Starter must be running before an
actual run is started. It can be invoked either manually as background process or as automated
startup item. Engine Starter should be invoked via EngineStarter.bat/sh and not directly.

2.

Use either Start_Engine2K.bat/sh <Profile ID> and Start_Simulation2K.bat/sh to start an

Engine run. Internally these scripts/batch file will setup the environment variables as were
defined at setup stage (1) and will run Engine Manager to start the actual run.
* Currently, user is responsible to secure access to the Oracle wallet files by allowing read
access only the system user that created or requires use of wallet.

Note that currently RAC support for the engine requires an In-Memory Consumption
Driven Planning license.

Running the Engine:

As general rule one should always use the provided scripts to run the different Engine
modules either manually or via workflow, rather than running them directly.

Running with Oracle wallet defined:

1. Per Engine host machine run the EngineStarter script once either as background
process or in a different shell/command console. This must be repeated if
EngineStarter was killed, the shell from which it was run was aborted or the
host on which it was running was restarted.
Example: ./EngineStarter.sh &
2. Engine starter will register itself in DB as active and listen for requests.
3. Run the Start_Engine2K script with a profile ID as parameter to start a batch
run. Example: ./Start_Engine2K.sh <Profile ID>
4. When starting the Engine Manager will look up in DB for active Engine Starters
and use them to spawn Engine processes as needed.

Running without wallet defined:

1. Per Engine host machine run the EngineStarter script once either as background
process or in a different shell with the DB connection details. Example:
./EngineStarter.sh <DB host address>:<port>/<service name> <user>
<password> &
2. Engine Starter will register itself in DB as active and listen for requests.
3. Run the Start_Engine2K script with a profile ID as parameter to start a batch
run. Example: ./Start_Engine2K.sh <Profile Id> <DB host
address>:<port>/<service name> <user> <password>

Running Simulation Engine:

1. Make sure that Engine Starter/s are running or run them as per previous sections.
2. Run the Start_Simulation2K script to start the simulation Engine either with or
without DB connection details depending on the setup method.
Example 1: ./Start_Simulation2K.sh
Example 2: ./Start_Simulation2K.sh <DB host address>:<port>/<service
name> <user> <password>

Stopping an Engine Run:

1. Run the KillEngine script to stop an existing run, for example when Engine is
running in simulation mode and user wants to start a batch run. Engine Starter
will not be killed.

Running Engine from workflow

1. EngineStarter must be running as background process before the Demantra
application and workflow are started. This can be done either manually from a
console or automated via startup scripts of the host OS.
2. Engine Manager must be physically located on the same host as the Demantra
application/workflow.
3. Make sure the SYS_PARAMS.EngineBasePath parameter is set correctly,
pointing to the Engine Root Directory where all scripts are located.
4. Only Engine scripts must be used to either start/stop an Engine run. Either
Start_Simulation2K, Start_Engine2K or KillEngine should be used from
within workflow steps. The appropriate script extension and foreword/backward
slash must be used depending on the host OS. Usually the parent workflow step
implements a condition step to determine the type of OS and branch to the
appropriate execution step. Review existing steps for more details.
5. Any custom/existing executable step calling one of the Engine scripts must be
preceded with the new path token, pointing to where Engine scripts are located,
rather than a hardcoded path or the old token pointing to the Demantra application
base path.
Example: #EngineBasePath#\Start_Simulation2K.bat

Troubleshooting:
Wallet Setup
Once the setup process have completed successfully. Verify that the TNS_ADMIN directory
contains both tnsnames.ora and sqlnet.ora then follow the steps below to validate the DB
connection setup.
Verifying the wallet connection: (Windows)
1. After completing the setup process successfully CD into the Engine Root directory.
2. Run the setenv.bat batch that was generated by setup process.
3. Run tnsping %ENG_CONNECTION% Tnsping should complete successfully.
4. Run sqlplus /@%ENG_CONNECTION% Sqlplus should connect successfully and
allow you to run queries against the configured schema.
Verifying the wallet connection: (UNIX-like OS)
1. After completing the setup process successfully CD into the Engine Root directory.
2. Run the setenv.sh script that was generated by setup process.
3. Run tnsping $ENG_CONNECTION Tnsping should complete successfully.
4. Run sqlplus /@$ENG_CONNECTION Sqlplus should connect successfully and
allow you to run queries against the configured schema.
Possible issues:
1. Either tnsping or sqlplus are not found.
Solution: Make sure you have the ORACLE_HOME environment set up correctly and
that you have the ORACLE_HOME\bin directory in your PATH variable.
2. tnsping fails.
Solution: Verify that contents of tns_names.ora are correct as quoted below.
3. sqlplus fails to connect.
Solution: Verify that contents of sqlnet.ora are correct as quoted below. If they are and
the reported error is about incorrect username/password, please redo the setup or recreate
the wallet with the correct user/password credentials.

TNS_NAMES.ORA will contain a connection entry similar to the following:

------------------------------------------- tnsnames.ora example ---------------------------------------------DEM_CONN = (DESCRIPTION= (ADDRESS =
(PROTOCOL = tcp)
(HOST = myserver.mydomain.com)
(PORT = 1521))
(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = myservice)))

--------------------------------------------------------------------------------------------------------------------Note that all the values marked in RED are the ones configured via the setup process.
SQLNET.ORA will contain the following settings:

----------------------------------------------- sqlnet.ora example ---------------------------------------------SQLNET.AUTHENTICATION_SERVICES = (NTS)

NAMES.DIRECTORY_PATH= (TNSNAMES,EZCONNECT)
WALLET_LOCATION = (SOURCE = (METHOD = FILE) (METHOD_DATA = (DIRECTORY =
%TNS_ADMIN%) ) )
SQLNET.WALLET_OVERRIDE = TRUE
SSL_CLIENT_AUTHENTICATION = FALSE
SSL_VERSION = 0

--------------------------------------------------------------------------------------------------------------------Note the directory for wallet is pointing to the same directory as set for the environment variable
TNS_ADMIN and contains the wallet files (cwallet.soo and ewallet.p12).

Engine Starter
Engine Starter should be executed once on each one of engine hosts.
Engine Starter script can handle 0, 3, 4 or 5 command line parameters. Any other number will
display the error and usage string.
So if you run with one argument for example you will get the usage details:
EngineStarter.bat <any parameter> - Will display the Usage.
General Usage:
EngineStarter.bat [ mandatory CONNSTR ] [ mandatory USERNAME ] [ mandatory
PASSWORD ] [ optional STARTER_ID ] [ optional ENGINES_NUM ]
Example Usage:
EngineStarter.bat
EngineStarter.bat <DB host address>:<port>/<service name> <user>

<password>
EngineStarter.bat <DB host address>:<port>/<service name> <user>

<password> <Starter ID> <Num Engines>

EngineStarter.bat <DB host address>:<port>/<service name> <user>
<password> <Starter ID>
Possible failures:
1. Cant connect to DB.
Solution: Check Oracle wallet setup or provided connection details.
2. Failed to create a queue for the chosen starter ID.
Solution: Check that chosen ID is not too long as it is appended/prepended with some
additional chars and should eventually be a valid table name. Check that no invalid
characters are part of the chosen ID. Try manually setting the starter ID in setenv script
to something short and simple like starter1 or directly pass this ID via command line.

Engine Manager
Engine Manager will first look for registered and active Engine Starters. If any registered Engine
Starter is found to be inactive (non responsive) it will be removed from the list of registered
Engine Starters and thus need to be restarted again.
User can run Engine Manager by calling Start_Engine2K.bat/sh for Batch or
Start_Simulation2K.bat/sh for simulation.
Engine Manager scripts can handle 0, 1,or 4 command line parameters. Any other number of
arguments will display error and usage information. Start_Engine2K for and Start_Simulation2K
scripts are similar except that profile ID is in fact required for a batch run and not necessary for
simulation run as the actual profile ID is passed as part of the specific simulation request. Profile
ID 1 will be assumed for batch if not provided.
General Usage:
Start_Engine2K.bat [ mandatory CONNSTR ] [ mandatory USERNAME ] [ mandatory
PASSWORD ] [ optional PROFILE_ID ]
Usage Examples:
Start_Engine2K.bat
Start_Engine2K.bat <Profile ID>
Start_Engine2K.bat <DB host address>:<port>/<service name> <user> <password>
Start_Engine2K.bat <DB host address>:<port>/<service name> <user> <password>
<Profile ID>
Start_Simulation2K.bat
Start_Simulation2K.bat <DB host address>:<port>/<service name> <user>

<password>
Possible failures:

1. Cant connect to DB.

Solution: Check Oracle wallet setup or provided connection details.
2. Failed to locate any Engine Starters.
Solution: Engine Starter either wasnt started at all or failed to start. Make sure
you have at least one successfully running Engine Starter.
3. A registered Engine Starter was found but not detected as active and removed
from list.
Solution: Engine Starter partially failed, DB is too slow to respond and
notifications are lost or delayed.
4. Failed to create Manager Notification queue.
Solution: May be a DB/ Notification Package issue. Retry the run / check DB
configuration.
5. Run started but stuck as Engines are not created.
Solution: Check screen output of Starter. Could be that Engine executable have
no run permissions or queue messages are being lost/delayed too long due to
slow/erratic DB performance. Try to restart Engine Starter and Manager.