ANSYS High P f Performance Computing

User Group 2010

2010 CAE Associates

Parallel Processing in ANSYS

ANSYS offers two parallel processing methods: Shared-memory ANSYS: Shared-memory ANSYS uses the sharedmemory architecture in ANSYS, meaning using multiple processors on a single machine machine. Most Most, but not all all, of the solution phase runs in parallel when using the shared-memory architecture. Many solvers in ANSYS can use the shared-memory architecture. In addition, pre- and postprocessing can make use of the multiple processors, including graphics operations, processing of large CDB files, and other data and compute intensive operations

Parallel Processing in ANSYS

Distributed ANSYS: Distributed ANSYS can run over a cluster of machines or use multiple processors on a single machine and works by splitting the model into different parts and distributing those parts to each machine/processor. By solving only a portion of the entire model on each machine/processor, the processing time and memory requirements can be reduced.

With Distributed ANSYS, , the entire solution phase p runs in parallel, p , including g the stiffness matrix generation, linear equation solving, and results calculations. If you are running Distributed ANSYS on a single machine with multiprocessors, lti th then th the non-solution l ti phases h (f (for example, l pre- and d postprocessing) will run in shared-memory parallel mode, making use of the multiprocessors for graphics and other data and compute intensive operations, y ANSYS. as is done in shared-memory

FEA Benchmark Problem

Bolted Flange with O-Ring Nonlinear material properties (H perelastic O-Ring) (Hyperelastic O Ring) Large Deformation Nonlinear Contact 1 Million Degrees of Freedom ANSYS 12.1 12 1

Datahal DHCAD5650 High End Workstation

SuperMicro 4U Tower Black SATA 5.25 5 25 Bays 8 Hotswap EATX3 800W RPS Dual Hex Core (12 cores total)

Intel XEON 5650 2.66GHz 2 66GHz Processors (4) 6GB (3 x 2GB) 1333MHz DDR3/PC3 10600 Non-ECC DDR3/PC3-10600 Non ECC DDR3 SDRAM 240-Pin DIMM

24 GB RAM

Four 300GB Toshiba SAS 15,000 RPM 16MB 3.5IN 3 5IN drives in RAID 0 0. One 500GB SATA Hard Drive for the Operating System Nvidia Quadro FX 1800 Video Card LG DVD/RW

FEA Benchmark Performance

SingleMachine SMPSparsevs vs.MPPDSPARSE
7

Sol lverSpeedU Up

SPARSE DSPARSE

0 0 2 4 6 8 10 12 14

#Cores
6

CFX Benchmark Problem

Flow around an airfoil 1 Million Elements 3-D, Steady State, Compressible Flow k- Turbulence Model

CFX Benchmark Performance

CFXParallelPerformance
7

Sol lverSppedU Up

CFX

0 0 2 4 6 8 10 12 14

#Cores
8

Disk Drive Speed

The bolted flange analysis was run on the two different drives of our high end workstation to compare disk speed influence on solution time. The RAID array arra completed the sol solution tion almost t twice ice as fast as the SATA drive:

Run #1: PCG Solver, 12 CPU, In-Core, RAID Array. Wall time = 8754 sec. R #2 Run #2: PCG S Solver, l 12 CPU CPU, I In-Core, C SATA Drive. Di Wall W ll time ti = 16822 sec.

Hyperthreading

Hyperthreading allows one physical processor to appear as two logical processors to the operating system. This allows the operating system to perform two different processes simultaneously. It does not, however, allow the processor to do two of the same type of operation simultaneously (i.e. floating point operations). This form of parallel processing is only effective when a system has many lightweight tasks.

10

Hyperthreading and ANSYS

The bolted flange analysis was run with Hyperthreading on and then again with it off to determine its influence.

Run #1: PCG Solver, 12 CPUs, Hyperthreading Off. Wall time = 8754 sec. Run #2: PCG Solver Solver, 24 CPUs CPU s, Hyperthreading On. On Wall time = 8766 sec sec.

An LS-Dyna analysis was also run in the same manner as above with the following results. results

Run #1: 12 CPUs, Hyperthreading Off. Wall time = 19560 sec. Run #2: 24 CPUs, Hyperthreading On. Wall time = 32918 sec.

11

High Performance p g Computing (HPC) g Product Configurations

Presented by: Tony Solazzo

HPC Changes

ANSYS HPC solutions now support multiphysics

Single solution enables parallel processing for all physics and level of fidelity fluids, structures, thermal, and electromagnetics
Mechanical products : ANSYS Multiphysics, ANSYS Mechanical/Emag, ANSYS Mechanical/CFD-Flo, ANSYS Mechanical, ANSYS Structural, ANSYS Professional NLS, ANSYS Professional NLT, ANSYS AUTODYN, ANSYS AUTODYN Single Task, ANSYS Emag Fluids products : ANSYS CFD CFD, ANSYS FLUENT FLUENT, ANSYS CFX, CFX ANSYS CFD-Flo CFD-Flo, ANSYS Icepak, ANSYS POLYFLOW Solver products : ANSYS Multiphysics Solver, ANSYS Mechanical Solver, ANSYS Structural Solver, ANSYS Emag Solver, ANSYS CFD Solver, ANSYS FLUENT Solver ANSYS CFX Solver Solver, Solver, ANSYS CFD CFD-Flo Flo Solver

Eliminates the need to separately acquire and deploy parallel processing for separate simulation domains Increases the value from your overall investment in high performance computing and ANSYS multiphysics solutions

13

HPC Changes

How are they packaged in Version 12?

HPC Configurations
ANSYS HPC - Individual processor based ANSYS HPC Pack - Sold in groups of 8 processors
Each simulation consumes one or more packs

ANSYS HPC Workgroup - provides parallel capacity for multiple users and multiple simulations

14

Overview ANSYS HPC Packs

ANSYS HPC Packs enable high high-fidelity fidelity insight Each simulation consumes one or more packs Parallel enabled increases quickly with added packs
Parallel Enabled (Cores)

3 2

51 12 2 8

2048

1 2 3 4 5 Packs per Simulation

15

Enabling Insight ANSYS HPC Packs

Example Customer Owns (5) HPC Packs

Can run (5) 8-processor based projects

Parallel a a e Enabled (Cores)

8 8
Solver Jobs 2 3 8 8 4 5 1

3 2

51 12 2 8

2048

1 2 3 4 5 Packs per Simulation

16

Enabling Insight ANSYS HPC Packs

Example Customer Owns (5) HPC Packs

Can run (3) 8-processor based projects Can run (1) 32-processor based project

Parallel a a e Enabled (Cores)

8
Solver Jobs 2 3 8 4 1

3 2

51 12 2 8

2048

8 3 2

1 2 3 4 5 Packs per Simulation

17

Enabling Insight ANSYS HPC Packs

Example Customer Owns (5) HPC Packs

Can run (1) 32-processor based project Can run (1) 128-processor based project

Parallel a a e Enabled (Cores)

8
Solver Jobs 2 1 1

3 2
2

51 12 2 8

2048

3 2 12 8
18

Enabling Insight ANSYS HPC Packs

Example Customer Owns (5) HPC Packs

Can run (1) 2048-processor based project

Parallel a a e Enabled (Cores)

8
Solver Jobs 1 1

3 2
2

51 12 2 8

2048

20 48
19

Enabling Productivity - ANSYS HPC Workgroup Solution ANSYS HPC Workgroup provides parallel capacity for multiple users and multiple simulations

Volume access to parallel processes Available in blocks from 128 to 2048 processes Shared across any number of simulation tasks on single server

20

Enabling Productivity - ANSYS HPC Workgroup Solution

ANSYS HPC Workgroup

ANSYS HPC Workgroup 128 ANSYS HPC Workgroup 256 ANSYS HPC Workgroup 512 ANSYS HPC Workgroup 1024 ANSYS HPC Workgroup 2048

HPC Server ANSYS HPC Workgroup Parallel Block

21