Lecture 5 Two-way FSI Solving and Post Processing 14.

5 Release

Solving FSI Applications Using ANSYS Mechanical and ANSYS CFX

1 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and backup points. Making physics changes to the CFD and Mechanical setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX simulations.

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Starting the Run

Write the Definition file and proceed to the CFX Solver Manager WB writes the def file in the background Run Definition panel: Use Double Precision for FSI runs, since
the mesh deformations are often very small compared to the size of the domain

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Starting the Run

MultiField panel: MFX Run Mode:
Start ANSYS and CFX Both solvers started automatically Start ANSYS only / Start CFX only If running on different machine see parallel discussion later Process Input File only Produces a .mf file Used for physics changes; discussed later

Mechanical Input File

As set in CFX-Pre, but can be changed here

Process ANSYS Input File

Uncheck if the Mechanical Input File is a .mf file (i.e. has already been processed) See restarts discussion later
4 2011 ANSYS, Inc. July 26, 2013

then Start Run

Release 14.5

Running in Parallel
CFX can use all the usual parallel methods

Set up as for any non-FSI run

MAPDL solver is limited to local parallel To run MAPDL in local parallel specify, for example, -np 4 in the Additional Arguments field

Will run on the local machine MAPDL can run on two cores without any
additional licenses

From the Command Line:

> cfx5solve -def CFX_Setup.def -double -mfx-run-mode "Start ANSYS and CFX" -ansys-input ds.dat -ansys-arguments "-np 2" Start both solvers using the files specified, run CFX in double precision, run MAPDL solver on two partitions. Note that on some systems spaces in the argument strings will need to be escaped, e.g. Start\ ANSYS\ and\ CFX and -np\ 2
5 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Running on Two Different Hosts

To run on different machines:

Start ANSYS first on machine A

MFX Run Mode = Start ANSYS only CFX out file will show a port and hostname that ANSYS is listening on

... ... +--------------------------------------+ | | | Starting ANSYS Solver | | | +--------------------------------------+ ANSYS solver listening on 1971@MACHINE1

Start CFX on machine B

Other machines can be used as slaves Provide the port and hostname output by ANSYS

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Running on Two Different Hosts

From the command line:
> cfx5solve -def CFX_Setup.def -mfx-run-mode Start ANSYS only" -ansys-input ds.dat

Starts the ANSYS solver using the input file specified

> cfx5solve -def CFX_Setup.def -double -mfx-run-mode "Start CFX only" -cplg-host 2249@MACHINE1

Starts the CFX solver using the specified def file in double precision,
connects to the ANSYS solver already running on MACHINE1 using the specified port

Run cfx5solve help or see doc for all command options

7 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Running in Parallel
Slave processes/threads must be on different cores Otherwise things will run VERY slow The codes are (usually) running sequentially, it would seem logical
that they can share cores However, a slave process pulls 100% of the available CPU while its in waiting mode The other code is starved of CPU OK for CFX and MAPDL to share a master processor

Example: single quad-core machine

CFX: 4 partitions, MAPDL serial

OK only the master is shared OK when MAPDL is waiting 1 core will be in use, leaving 3 for CFX
July 26, 2013

CFX: 3 partitions, MAPDL 2 partitions (cores)

8 2011 ANSYS, Inc.

Release 14.5

Another Look at the Workflow

MAPDL is the master process CFX-Pre Mechanical

MAPDL controls the transient

timestep, interface mapping, interface convergence checks, load transfer, etc

Definition File
Contains CCL definition of fluid model and interface / coupling settings

Input File
Contains APDL definition of structural model

But all these settings are defined in CFX-Pre We have an input file from Mechanical that contains the structural model with the FSI surfaces tagged

Process Input File

CFX adds APDL commands for interface / coupling settings

Workbench writes this input file

into the project directory

CFX Solver
Settings from the Definition file

MAPDL Solver
Input File + appended APDL commands from CFX

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Another Look at the Workflow

CFX-Pre writes a Definition File (.def) containing the CFD model and the interface / MFX controls When the run is started, CFX creates APDL commands for the interface / coupling settings and combines them with the contents of the input file from Mechanical CFX-Pre Mechanical

Definition File
Contains CCL definition of fluid model and interface / coupling settings

Input File
Contains APDL definition of structural model

Process Input File

CFX adds APDL commands for interface / coupling settings

Important to understand whats

happening here when performing restarts See Appendix A for restart details

CFX Solver
Settings from the Definition file

MAPDL Solver
Input File + appended APDL commands from CFX

10

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

MAPDL Commands Inserted by CFX

You can view the commands generated by CFX in the MAPDL Solver output, following the end of the Input File
End of Input File from Mechanical

CFX generated these commands

11

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Process Input File

If you disable Process ANSYS Input File then CFX does not generate any APDL commands CFX-Pre Mechanical

Therefore the Input File for

MAPDL must already contain the interface and coupling settings
Occurs in some restart workflow

Definition File
Contains CCL definition of fluid model

Input File
Contains APDL definition of structural model and interface / coupling settings

Also see CFX documentation:

ANSYS CFX Solver Manager Users

Guide >> Working with Solver Manager >> ANSYS Multi-field Run >> Processing the ANSYS Input File

CFX Solver
Settings from the Definition file

ANSYS Solver
Settings from Input File

12

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Solver Files Generated

CFX

The working directory will contain the CFX Output

and Results files as usual The transient (.trn) and backup (.bak) results files are stored in the <name>_001 directory

MAPDL Solver

The subdirectory <name>_001.ansys is created and

contains all the MAPDL Solver files All results are stored in the .rst file The auto-generated ANSYS.mf file can be viewed in this subdirectory ANSYS.stdout is the ANSYS Solver output as displayed in the CFX Solver Manager

Workbench

The working directory is:

/<ProjectName>_files/dp0/CFX/CFX
13 2011 ANSYS, Inc. July 26, 2013

Release 14.5

CFX Solver Output

Start of Time Step First Coupling Iteration Solve Mesh Displacement - Should be fully converged CFX Coefficient Iterations
- Should be converging

Second Coupling Iteration Solve Mesh Displacement - Should be fully converged

CFX Coefficient Iterations

- Should be converged by the end of the last Coupling Iteration

14

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

CFX Solver Output

ANSYS Field Solver and Interface Loads plot also shown

X-axis values do not correspond to each other or to CFX Cumulative ANSYS Iterations L2 normalized values should be less than corresponding convergence criteria (CRIT) Cumulative Coupling Iteration Interface loads should be negative at the end of each timestep

15

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

MAPDL Output Check Interface Mapping

In the MAPDL output, ALWAYS check the interface mapping Scroll to the top of the output file (ANSYS.stdout)

Right-click in the window and select Find

Search for the following make sure Match case is enabled:

This section provides information on the mapping between the CFX and ANSYS meshes. If the Non-Matching Area Fraction or the No. of UnMapped Nodes is non-zero, then some parts of the FSI interface will not be passing loads between the field solvers. In this case check that the two side physically match and completely overlap.
16 2011 ANSYS, Inc. July 26, 2013

Release 14.5

MAPDL Solver Output

Time Step and Coupling Iteration # Start and End Time for this Time Step Actual Force Loads Received (and/or Thermal Loads) Start and End Time for this Load Step 1:1 correspondence to Time Step ANSYS Field Solver convergence and iterations Check if CFX & ANSYS Field Solvers are Converged Check Interface Load Convergence
17 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Summary
The CFX Solver Manager or CFX Solver command line is used to launch 2-way FSI simulations

Mapping is performed by the MAPDL solver. Mapping diagnostics are provided in the MAPDL solver output file and should always be checked.
FSI settings defined in CFX-Pre are passed to MAPDL at runtime by generating APDL commands Understanding this workflow is important for restarts Restarts can be made from completed runs, interrupted runs and from backup points. Physics changes can be made when restarting. See Appendix A.

18

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and backup points. Making physics changes to the CFD and Mechanical setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX simulations.

19

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
Editing the CFX Results cell for a 2-way FSI case will automatically load both the CFX and structural results into CFD-Post Can also use File > Load Results and browse for .rst (structural) or .rth (thermal) results Enable Keep Current Case Loaded to post-process multiple sets of results

Disable Open in new view to have CFX and Mechanical results in a single window

20

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
In CFD-Post CFX and structural results entries and regions are shown in the outline tree When creating objects they can be scoped to all or selected domains

21

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing
Use the Timestep Selector to load different time steps as usual

Tab panels show the available results from each solver

Sync Cases allows both sets of data to be updated when a time step from one set is selected

Animations can show both fluid and structural results simultaneously

22

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing Animations

Transient blood flow through a
three-leaf valve Symmetry used with 1/6th of fluid domain Non-Newtonian fluid (i.e. blood) defined in Fluent Non-Linear contact in Transient Structural to cater for closure of valve leaflets Blood vessel wall has combination of isotropic and anisotropic (non-linear) materials

23

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Combined Post-Processing Animations

Transient free surface flow in
a liquid storage tank with baffles Tank is excited by a time varying gravitational load for a duration of 10s Baffles are non-metallic and fixed to the tank with an adhesive Forces are transferred (oneway only) via co-simulation to determine the stresses acting on the baffles Determine the integrity and viability of the adhesive bonding
24 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Combined Post-Processing
When selecting Variables to plot, the variables list shows all variables from CFD and structural results
If a variable is not available (e.g. Pressure on a slice plane in a structural region) is it plotted in the specified Undefined Color

Some variables apply to both fluid and structural regions, e.g.:

Total Mesh Displacement (X/Y/Z) Velocity (u/v/w) Mesh metrics (Area, Edge Length Ratio, X, Y, Z, etc)
July 26, 2013

25

2011 ANSYS, Inc.

Release 14.5

Combined Post-Processing
CFD-Post read components from a .cm file written out by the MAPDL solver
This is created by default for FSI cases using the command CMWRITE,file,cm Youll see a Mesh Regions branch in CFD-Post containing the components

When load structural results containing shell elements, CFD-Post reads and displays data from the top side of the shells
To read and display results from the bottom side set the system environment variable CFDPOST_RST_READ_BOTTOM_SHELL_DATA to any value
July 26, 2013

26

2011 ANSYS, Inc.

Release 14.5

Post-Processing in Mechanical
Structural results are associated with the CFX Solution and Results cells

To post-process in Mechanical:
Edit the Mechanical Solution cell Select the Solution entry from the Outline tree

Select Tools > Read Results Files and browse for the .rst/.rth file associated with the CFX FSI system

27

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and backup points. Making physics changes to the CFD and Mechanical setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX simulations.

28

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts from a Completed Run

This includes runs that were manually stopped before reaching the End Time On the CFX side restart as usual

If manually specifying a previous res file,

Continue History From must be enabled for FSI restarts

MAPDL always resumes from a previous database

Select the db file from the previous *.ansys

run directory

The Mechanical Input File must be

specified but is not used in this case

Enable Process ANSYS Input File:

APDL commands are created again from the CFX Solver Input File and a RESUME command is also inserted
29 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Restarts from a Completed Run

Example of the commands generated by CFX shown Note that theres no reference to the Mechanical Input File
Typical Restart Commands Generated by CFX
resume,ANSYS,db /SOLU MFAN,ON MFCLEAR,MFLC MFRS,-1,SING MFTI,1 MFDT,1,1,1,OFF MFLC,SURF,ANSYS,1, MFLC,SURF,CFX,'Wall', MFIT,20,1,20 MFCO,ALL,1e-3 MFRE,ALL,0.75,RELX MFPS,group1,ANSYS MFPS,group2,CFX MFSO,group2,group1 CMWRITE,ANSYS,cm KBC,1 /GST,ON,ON SOLVE SAVE CMWRITE,ANSYS,cm FINISH /EXIT
Release 14.5

Any changes made there are not used

The MF commands are recreated from the CFX Solver Input File

The MF commands stored in the db file are not

used

If Process ANSYS Input File is disabled, the Mechanical Input File is used and is run as is

The Restart ANSYS Run toggle is ignored because

CFX doesnt generate a RESUME command No MF commands generated
30 2011 ANSYS, Inc. July 26, 2013

Restarts from a Backup Point

A failed run can be restarted using a CFX backup file and MAPDL files produced using the MFRC command

Must be from a consistent time point

Pick the CFX .def file and use the CFX backup file as the Initial Values File, with Continue History From enabled Same MultiField settings as restarting from a completed run, except the rdb (restart database) file is used

This file is created by the MFRC command

Again, the Mechanical Input File is not used and CFX re-creates the MF commands
31 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Restarts from a Backup Point

Only the first FSI run will contain a rdb file. For multiple restarts copy the rdb file to the most recent run directory

Any previous changes you have manually made in a .mf file must be re-applied to every restart from a rdb file, since the rdb file represents the structural setup from the first FSI run
The correct nodal displacements, velocities, etc are still used, since the rdb file will read the .r00# file corresponding to the restart time

32

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts with Physics Changes

Changes to CFX physics can be made as usual In Workbench edit the Setup cell Or create a new .def file and use the previous res file as the Initial Values
File, with Continue History From enabled

Changes to coupling, interface or time settings specified in CFX can use the same procedure as a CFX physics change Since CFX re-creates the MF commands each time, these changes will be
passed to MAPDL Example: Extending a run by increasing the Coupling Time Duration

Changes to Mechanical settings are less straight-forward since the Mechanical Input File is not used when restarting Where should be changes be made?
33 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Restarts with Physics Changes

First, create the .mf file

This is what CFX is doing in the background when Process ANSYS Input File is checked Use Process Input File only option Select the Mechanical Input File and the previous db file Provide the CFX def file and Start Run

Run will complete in a few seconds Inside the *.ansys/ run directory an ANSYS.mf file is created
/RunName_002.ansys/ANSYS.mf

34

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Restarts with Physics Changes

Manually edit the ANSYS.mf file, inserting APDL commands to make any changes Start the run using the edited .mf file as the Mechanical Input File Disable Process ANSYS Input File

Select the previous db file

Can also use this approach to change interface / coupling settings (MF commands) instead of making a new .def file in CFX-Pre You need to use this approach to restart a steady-state 2-way FSI case as transient

35

Add TIMINT,ON to the .mf file to switch Mechanical from steady-state to transient
July 26, 2013

2011 ANSYS, Inc.

Release 14.5

Restarts - General
MAPDL results are appended to the rst file Old rst is copied into new run directory
Can be a very large file!

You can move the old rst file to a new directory before restarting to
prevent it being copied
A new rst file will be created for the new run

For all restarts also see the CFX documentation: ANSYS CFX Solver Manager Users Guide >> Working with Solver
Manager >> ANSYS Multi-field Run >> Restart Procedure for ANSYS Multi-field Run

36

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Outline
Solution Process
Here we discuss starting and monitoring the FSI solution in serial and parallel. The various log and output files are also discussed.

Post Processing
This section will discuss common post-processing in CFD-Post and how to post-process FSI results in the Mechanical post-processor.

Appendix A: Restarting FSI Simulations

This section covers restarting FSI simulations from completed runs and backup points. Making physics changes to the CFD and Mechanical setups when restarting is also discussed.

Appendix B: Multifield APDL Commands

This appendix provides some useful APDL commands relevant to MFX simulations.

37

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Multi-Field Commands - Background

Note: the Command Reference section in the Mechanical APDL documentation provide details on all MAPDL commands MF commands are APDL commands that control Multifield parameters They are created by CFX-Pre and appended to the MAPDL input at run-time

38

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Basic MF Commands
MFRS

Multifield restart time (for restarts only) Usually has a value of -1, meaning restart from end of previous solution
MFAN

Turns on Multifield solver

MFTI

End time for the Multifield analysis

MFIT

Controls the number of Coupling Iterations

MFDT

Control the transient time step size

39

2011 ANSYS, Inc.

July 26, 2013

Release 14.5

Basic MF Commands
MFOU

Controls how often results are stored

MFRC

Control how often restart files are written

MFCO

Controls Field Convergence Criteria

MFRE

Field Relaxation Factors

MFLC,SURF,CFX,'Interface','Total Force', ANSYS, 1,FORC,CPP

Load transfer command

MFPS and MFSO

Which field goes first, sequential or parallel

40 2011 ANSYS, Inc. July 26, 2013

Release 14.5

Other Relevant APDL Commands

BCSOPTION,,INCORE

For the sparse solver tries to retain the entire factorized matrix in memory Runs faster, but significantly increases memory usage
KBC,1

Sets stepped loads - CFX will always append this command to the ANSYS input If using Prep7 make sure you set it manually
OUTRES

Minimum data you can write and still restart Note that you may need more data for post-processing (e.g Stress data)
OUTRES, ERASE OUTRES, ALL, NONE OUTRES, NSOL, LAST OR OUTRES, ERASE OUTRES, ALL, NONE OUTRES, NSOL, LAST OUTRES, ESOL, LAST

Structural Solution Only

41 2011 ANSYS, Inc. July 26, 2013

Structural + Thermal or Thermal Solution

Release 14.5

Other Relevant APDL Commands

NEQIT

Sets the maximum number of equilibrium iterations

NCNV

The ANSYS solver will continue when convergence has not been achieved
Note that MAPDL runs until convergence or max. number of equilibrium iterations, but the calculation will stop if max. number of equilibrium iterations is reached. You can set NCNV so that MAPDL will continue if convergence is not achieved and this can be combined with NEQIT to limit the number of equilibrium iterations. However, you need to be careful when doing this you must make sure MAPDL is converged by the end of the last Coupling Iteration. If you are reaching the maximum number of Coupling Iterations then something hasnt converged CFX, interface loads, or MAPDL this needs to be corrected. If one of the field solvers is very poorly converged, it may pass spurious loads to the other which could cause failure.

42

2011 ANSYS, Inc.

July 26, 2013

Release 14.5