Now Viewing: EXN/Aero User Manual

Jump to Theory Manual
  1. Introduction
    1. Our Goal
      1. Why Try Something New?
      2. How we Get There
    2. EXN Concept
      1. Cell - Interface Design
      2. Cell-Based Mapping Module
      3. Flexible Manycore Computing
    3. Solver Description
      1. Solver Features & Models
  2. Getting Started
    1. EXN/Aero Walkthrough
      1. Connecting to EXN/Aero on the NIMBIX cloud
      2. A first look at EXN/View
      3. Preparations
        1. Download a Test Mesh
        2. Create a Project
        3. Import a mesh into EXN/View
      4. Models, Properties & Initial Conditions
        1. A bit about Cells & Cell Families
        2. Editing Cell Families
        3. Editing BC Families
        4. Editing Solver Control Settings
          1. Assigning Resources
        5. EXN/View Live
      5. Running a simulation
      6. Project file structure
      7. Finding solution files
    2. Meshing Best Practices
      1. Mesh Requirements
      2. Labeling
      3. Family Names
      4. Subdomains
      5. Building a FAST mesh
        1. Use Structured Blocks Where Possible
        2. Be Mindful of Interface Count
        3. Minimize Double Precision Regions
        4. The Right Blend of CPU & GPU
  3. EXN/View User Interface
    1. Menus
      1. File
      2. Mesh
      3. Edit
      4. View
      5. Run
      6. Settings
      7. Help
    2. Tree View
    3. Render Window
    4. Toolbar
    5. Tabs
    6. Status Bar
  4. EXN/Aero Settings
    1. Cell Families
      1. RANS Models
      2. LES / DES Models
      3. Initial Conditions
        1. RANS / LES / DES ICs
        2. Subdomain ICs
      4. Properties
        1. Fluid Prop.
        2. Subdomain Prop.
      5. Cell Statistics
      6. Subdomains
        1. Porous Media
        2. General Scalar
    2. BC Families
      1. RANS / LES / DES BCs
      2. BC Statistics
      3. Subdomain BCs
    3. Solver Control
      1. Simulation
        1. Simulation Type
        2. Misc Solver Settings
        3. Solution Timing
        4. Restarting
        5. Solution Output
        6. Task Execution
        7. Convergence Criteria
      2. Exporting Boundaries Only
      3. Compute Resources
      4. Monitor Points
    4. Database
    5. Variables
    6. Monitor
      1. Graphing
  5. File Management
    1. Network Storage
      1. Zipping CGNS & VTU files
      2. Transfer files by SFTP
      3. Nimbix File Manager
    2. Parent Directory
    3. CVC Directories
      1. XML Structure Files
    4. Results Directory
    5. VTK Directory
  6. Post Processing With Paraview
    1. Launching on the Cloud
    2. Paraview Anatomy
    3. Common Operations
  7. Troubleshooting
    1. Solver Errors

Introduction #

Our Goal #

For EXN/Aero to be the number one CFD software choice for engineers in small and medium firms.

 

Why Try Something New? #

The biggest supercomputing facilities are continually pushing boundaries and adopting new technologies as they approach the challenging goal of exa-scale computing.

The adoption and rapid increase in multi-CPU cluster size arose because individual processor clock-speeds stopped getting faster in the early 2000s.

By 2010 in order to develop petascale supercomputers, and to minimize power consumption, muti-CPU clusters  began  to adopt energy-efficient manycore co-processing technologies such as GPUs.

The innovations that drive the world’s fastest supercomputers inevitably impact the work of everyday scientists and engineers, and we see over time that updates in workstation-grade technology tracks closely the architectural evolution of the large scale supercomputing systems.

General-purpose CFD now has an excellent opportunity to capitalize on the manycore trend to increase simulation performance and reduce costs.

There is also an entrenched licensing model that limits what engineers can do with CFD. With a transformation in the available computing power comes the ability to try new things around licensing and offer an alternative high-performance CFD pricing model for the first time in decades.

 

How we Get There #

Our unique parallel computing design enables engineers to operate economically on the cloud and to build inexpensive, powerful systems that boost CFD solution speed by an order of magnitude.

EXN/Aero is built from the ground up to take full advantage of hybrid parallel computing architectures like multi-core CPU and massively-parallel GPU. It exposes every aspect of the simulation to parallelism, removing serial tasks and maximizing speed-up.

We’re also innovating on the distribution model, enabling engineers to access a large amount of compute power on-demand and with minimal risk to their operations.

 

EXN Concept #

The EXN manycore computing platform was built to do engineering calculations quickly and economically on massively parallel processors. This approach is the future for engineering simulation; as industry demands greater fidelity, speed, and complexity from its simulation tools, software designers need to optimize — and parallelize — to keep up.

 

Cell - Interface Desgn #

Our patented design subdivides simulation data and associated computing tasks into two types of objects: cells and interfaces.

Cell Objects contain CFD mesh and represent the bulk of the computational load. Cells might contain structured or unstructured mesh, and use single or double precision arithemetics.

Interface Objects are multi-function data transformation utilities with many uses, including boundary condition input, communication between cells, reduction of output data, addition of source terms and interpretation of sensor inputs.

In the schematic below, cells ( grey mesh blocks) are connected to each other and the outside world via interfaces (red lines).

cell-interface

The number and location of Cell and Interface objects are largely aligned with the block/interface specification at the meshing stage. Cells and interfaces are tagged with attributes such as data structure and precision arithmetics that help to identify the type of computation required.

 

Cell-Based Mapping Module #

Distribution and assignment of Cell and Interface objects to compute resources depends on a parallel load balancing module we call the CBMM.

Cell and Interface attributes are used to automate cell splitting and automatic interface generation as part of off-line load-balancing operations. In practical terms it means some Cell and Interface compute tasks are assigned to GPU resources, while others are sent to the CPU resources, all with goal of maximizing parallelism and minimizing computing time.

The CBMM effectively separates the engineering problem from the underlying compute hardware. It therefore plays an important role in enabling EXN-based software to adapt quickly to new architectures in an environment where technology is changing very rapidly.

 

Flexible Manycore Computing #

The Cell and Interface design, working in concert with the CBMM allows for greater creativity and flexibility in the way a CFD run is parallelized. Optimal use of manycore compute resources requires that:

  1. Cell and interface objects are subdivided by task and data type attributes
  2. The computing environment is automatically interrogated to determine its layout
  3. Tasks can be dynamically assigned to resources depending on predicted load

These requirements exist because the manycore computing environment consists of diverse compute architectures, and not all architectures are equally good at every computing task.

Expensive Cell tasks are computed on GPU devices much of the time, where the thousands of simple processors work in concert to accelerate physics calculations

cell2gpu

Interface tasks are computed based on the location of the Cell objects in order to minimize communication overhead.  For example if two Cell objects that communicate both reside on GPUs then the interface data and tasks will also reside on GPUs.  However, if one of the Cells is located on a host CPU, and its tasks are executed there, then the connecting Interface will reside there also.

intf2cpu

EXN efficiently load balances a single simulation over manycore computing resources, but co-processors such as GPUs also allow multiple simulations to access the same device. This provides further opportunities for speeding up the engineering workflow by allowing multiple simulations to run concurrently on the same shared resource.

EXN simulations are maintained in a higher level object called a Propagator.  To take advantage of the large amounts of parallelism becoming available on manycore systems, it is possible to solve problems involving multiple Propagators working together in a coupled manner. We are demonstrating this capability today with our prototype multi-propagator parallel-in-time (aka para-real) solution technique. Para-real is designed to accelerate large steady-state runs, and also to speed up the start-up phase for transient simulations.

parallel-process

Solver Description #

EXN/Aero is a pressure-based, finite volume, fully implicit CFD solver based on the SIMPLEC method with Algebraic Multigrid (AMG).  

The EXN/Aero solver utilizes a hybrid multiblock approach where structured and unstructured mesh regions are concurrently solved in their native data format, preserving the inherent benefits of each type.  This allows EXN/Aero to better exploit CPU and GPU resources, playing to the strengths of a heterogeneous compute architecture layout.

EXN/Aero’s true multiblock approach also allows the user to specify the computational precision of each block. In this way, double precision can be applied only where it is needed (e.g. boundary layers), while other areas of the mesh operate in single precision (e.g. far-field mesh). This approach minimizes the memory footprint and increases performance relative to an all-or-nothing double precision approach. 

Solver Features & Models #

The following solver modes are available in release version 1.0x

  • Steady State & Transient solution modes
  • Structured & unstructured data support
  • Mixed precision by mesh block

The following engineering models are available in release version 1.0x

  • RANS – SST & k- \omega models
  • LES – Smagorinsky-Lily
  • Detached eddy – SST formulation
  • Synthetic turbulence generation for LES/DES
  • Dispersed phase with massive particles
  • Ideal gas model
  • Heat transfer
  • Pressure-based compressible flow
  • Buoyancy-driven flows

 

 

Getting Started #

EXN/Aero Walkthrough #

Welcome to the EXN/Aero walkthrough, and thanks for deciding to evaluate our product.

This tutorial is designed to get you up and running quickly and will help you understand the EXN/Aero workflow. We’ll cover:

  1. Preparations
  2. Importing a mesh
  3. Setting up physics
  4. Running the simulation
  5. Monitoring the simulation in progress
  6. Post analysis
  7. Summary of the project file structure

The walkthrough takes an hour or so and you’ll probably notice some similarities to the live walkthrough we did with you during your trial.

Connecting to EXN/Aero on the NIMBIX cloud #

This part of the walkthrough instructs you how to set up an account on our cloud providers.

PLEASE READ CAREFULLY:

Computing time on cloud resources is billed at an hourly rate and instances do not shut down automatically.

If you want to avoid unnecessary charges for compute hours make sure your job is shut down at the end of the compute task. 


1. Log in to the Jarvice platform once your account is created: https://platform.jarvice.com/login-page

2016-10-27-15_55_05-jarvice


2. From the Jarvice dashboard page, select the ‘Compute’ tab and select EXN/Aero from the list. A link will pop up below that says EXN/Aero Suite: Run the EXN/Aero suite directly in your browser. Click this link.  

2016-10-27-16_03_40-jarvice


3. Select the resource allocation. You can choose one of four compute node hardware layouts:

  • 4 CPU + 1 Nvidia K40
  • 4CPU + 1 Nvidia K80
  • 16 CPU + 2 Nvidia K40
  • 16 CPU + 2 Nvidia K80

Roughly speaking, two K40 cards are about equal to 1 K80 card in terms of memory and parallel core.

GPU computations tend to be memory-limited; for an incompressible isothermal flow, you can expect to fit about 50 million control volumes on a K80 card.

2016-10-27-16_06_12-jarvice


4. After selecting resources, click ‘Continue’. Use default options under ‘Select Vault. This will give a succession of two pop-up screens that contain launching scripts. Click to continue through these.

The ‘Dashboard’ tab will show that an EXN/Aero job has started. This display updates automatically. The image takes 30 – 60 seconds to boot up. You will know it is ready when a ‘connect’ button appears beneath the job information. 

The connect button opens a browser-based remote connection to the EXN/Aero desktop.

You can also connect with TigerVNC client, which has less display latency. The ‘help’ button to the right of the ‘connect’ button provides information on how to run through TigerVNC.

 

2016-10-27-16_13_46-jarvice


5. Click the ‘Connect’ link and a new browser tab will connect you to the EXN/Aero desktop. There are two icons — one for the file manager, and one for EXN/View.

EXN/View and EXN/Vizer take 20-30 seconds to launch. Double click and wait patiently please!

Your account-bound persistent storage directory is /data. This is the only directory that persists when the server is shut down. You must store all simulation files and results files in ‘/data’ if you wish to keep them. 

2016-10-27-16_14_42


6. When the run is complete and data is downloaded or safely stored in ‘/data’, remember to shut down the server.

Closing the browser-based remote desktop does not kill the session. You must click ‘Shutdown’ on the dashboard and confirm shutdown to end a session otherwise compute hours will continue counting and Nimbix will bill your account for them. 

Please re-read the above and make it part of standard procedures…we don’t want our users to be billed for hours they do not use.

2016-10-27-16_21_51-settings


7. To double check that the job was terminated, the job should appear in the ‘Recent Jobs’ tab on the dashboard. You will also receive a ‘job shutdown’ email from Nimbix.

If you encounter an error shutting down a job, please reach out to Nimbix directly. They are very responsive and can make sure you are not accidentally billed for a job that was left running because of the error

2016-10-27-16_26_28-settings

A first look at EXN/View #

EXN/View is our lightweight UI designed to get you up and running quickly.

In addition to the menu & toolbar, it has two main areas:

  1.  Tree viewer: This is the area where you specify physics, model settings, boundary conditions and solver settings. Menu selections are dynamic and become available only when the relevant model is activated. For example, the turbulent kinetic energy and turbulent dissipation variables are only available when a turbulence model is active.
  2. Mesh window: This area displays a wireframe of the model, output from the solver and indicators to help identify the mesh axes and scale. Left click to rotate, right click to pan, scroll to zoom. Boundaries and blocks in the mesh are highlighted when they are selected
    in the tree viewer.

Preparations #

Download a Test Mesh #

You can use your own mesh, or one of our other example meshes if you like, but for this tutorial we will assume you are studying a cylinder in cross-flow.

Open a Firefox browser by hitting the Envenio logo in the bottom left corner of the the EXN/Aero desktop. Navigate to and launch Firefox.

Go to our public FTP site: ftp://acenetm01.cs.unb.ca and navigate to

/nimbix_mesh/2Dcylinder/Cylinder.proj/

Download Cylinder.cgns.

Alternatively, you can double click on the Demo Cases folder on the the desktop and connect to our public FTP site directly. There you can find many meshes to work with.

Create a Project #

Click File to open the File Menu, and then select Create New Project.
This will create a new project folder with the name you assign. If you are on cloud resources, make sure that the save location is in the persistent storage directory (/data).
For nimbix users, the persistent storage directory is /data. Everything not saved in the data directory will be deleted when you terminate the compute job. 
If successfully created, the Envenio logo disappears from the mesh outline view and the project tree area populates with some default entries.

Import a mesh into EXN/View #

On the menu bar, click Mesh and select Import CGNS File. This will open another file dialog, where you need to navigate to your CGNS file.
Behind the scenes, EXN/View makes a copy of the CGNS file to your project directory and will give it the same name as the project, plus the .cgns extension. For example, the project name ‘test.proj’ will copy and rename the CGNS file to ‘test.cgns’. EXN/View also creates a .json file, which contains data to generate the wireframe object in the mesh outline view.
A new tab will appear in the render view, and shortly after, the mesh geometry will appear. This may take a few seconds or several minutes, depending on the size and complexity of the mesh. 
An output ticker in the bottom left of the tree view displays that the mesh has successfully loaded

Models, Properties & Initial Conditions #

Within the Cell_families branch of the tree view, you can specify:

  • Model settings
  • Fluid properties
  • Initial conditions for the fluid volume
  • Set sub-domains
  • Source terms
  • Running statistics

A bit about Cells & Cell Families #

 

When we say ‘Cell families’ we do not refer to individual control volumes.

Cells are mesh blocks and Cell families are groups of mesh blocks within the mesh. In EXN/Aero, blocks and families are given attributes, and they do not need to be consistent throughout the mesh.

Some attributes are inherited and cannot be modified. For example, unstructured and structured mesh data are baked into the mesh blocks by the meshing tool. Mesh generators can also enforce single precision.

Other attributes are malleable. If the mesh is exported as double precision, EXN/View can set individual cells as single or double precision in a zonal fashion. Cells can also be designated as sub-domains that will inherit special source terms or models (e.g. porous media).

Editing Cell Families #

Rather than using static images, a dynamic version of the EXN/View UI is available here: EXN/View Live section. This shows the complete and correct problem set up ready for execution.

If you are following this guide as part of a trial, you can refer to the example UI to get the exact values needed to reproduce our results.

This will give you a general idea about the set up procedure, but during your trial you can select any settings you like.


1. Create a fluid domain

Right click on the Cell Families and select ‘Add Cell Family’, then select ‘FluidDomain’. A new fluid domain will appear beneath Cell Families.

Next, select Cell Blocks and open the drop-down menu to add blocks to the fluid domain. If your mesher gave the blocks a family name (‘unspecified’ is the default here) then select the family. If not, then select multiple mesh blocks by holding the CTRL key or CTRL+SHIFT and click on the blocks you wish, just like you would in a file manager.

2. Turn on turbulence and energy models

Expand Model Parameters and then Turbulence. Set Turbulent Flow to on. Selection drop-downs appear that allow you to select from multiple LES and RANS models. For this example, select RANS and SST.

Expand Energy and set Energy Equation to on.

3. Set Initial Conditions and Properties

Expand both of these branches and copy the values over to your trial run.

4. From the File menu, select ‘Save Project’.

Editing BC Families #

Rather than using static images, a dynamic version of the EXN/View UI is available here: EXN/View Live section. This shows the complete and correct problem set up ready for execution.

If you are following this guide as part of a trial, you can refer to the example UI to get the exact values needed to reproduce our results.
This will give you a general idea about the set up procedure, but during your trial you can select any settings you like.


1. Insert an inlet condition

Right click BC Families and select Add BC Family…, then select InletBC.

Next, select BC Location and open the drop-down menu to add interfaces to the inlet condition. If you mesher gave the interface a family name (‘inlet’ is used here) then select that family. If not, then select multiple mesh blocks by holding the CTRL key or CTRL+SHIFT and click on the blocks you wish, just like you would in a file manager.

Copy inlet settings form the EXN/View live section. In your trial, you can modify these settings any way you like to generate different results

2. Insert an outlet

Same procedure as for the inlet. Select the ‘outlet’ family at the BC Location.

3. Insert wall boundaries

Same procedure as for the inlet. Select the ‘wall’ family at the BC Location.

4. Insert symmetry planes

Same procedure as for the inlet. Select both the ‘symmetry’ and ‘sides’ families at the BC Location using the CTRL key.

5. From the File menu, select ‘Save Project’.

Editing Solver Control Settings #

In the Solver Control branch the settings for the flow solver are set. There are a lot of settings under this branch, but for the purposes of the present tutorial, the more important ones will be shown. Refer to the EXN/Live section to see options under Solver Control.

Under the Simulation branch:

Simulation Type sets the solver for an unsteady or steady-state simulation.

Total Run Time sets the total physical time for an unsteady simulation. When the simulation time reaches this value, EXN/Aero writes output files as per save/output settings, writes a restart solution to the CGNS file and exits.

Delta Time sets the time step size.

Initial time from Solution has two options: On and Off. Upon restart, setting this variable to ‘On’ causes EXN/Aero to use the final simulation time from the previous run as the start time of the current run. If set to ‘Off’ then the initial time is set to zero.

Initial Values from Solution enables the user to pick which solution data arrays are initialized from a previous solution in a restart run. If an array is required but not included in this list on restart, EXN/Aero defaults to the initial value settings in Cell Families.

Transient Output Frequency sets the interval in time steps at which solution output files (in VTK format) are written to the solution directory. A value of ‘1’ outputs every time step, while a value of ‘0’ disables VTK output entirely. Writing VTK files to the hard drive is very time consuming and can slow the solution time significantly if the output frequency is small.

Solution Backup Frequency sets the interval in time steps at which EXN/Aero writes a restart-able solution to the CGNS file. A value of ‘1’ writes a solution restart at every time step, while a value of ‘0’ disables backups entirely. Writing solution arrays to the CGNS file is very time consuming and can slow the solution time significantly if backup frequency is small.

Transient Output Variables controls which solution variables are included in the VTK output files for fluid volumes. Some output variables are vectors (e.g. Velocity) while others are scalars or individual components of vectors (e.g. Temperature, Velocity_X). Picking a vector in this field is sufficient to include its components — for example, picking Velocity means the user does not also need to select ‘Velocity_X’, ‘Velocity_Y’ and ‘Velocity Z’.

Transient Output Variables BC controls which solution variables at the domain boundaries are included in the VTK output files, simultaneously with the VTK output for fluid volumes. Vector & scalar picking rules are the same as for Transient Output Variables.

Assigning Resources #

Resource assignment works differently from CPU-only parallel computing. CPU/GPU resources are not reserved for the task and parallelization is handled automatically through a combination of OpenMP and CUDA. There are 3 important settings for resources set at the Resources branch:

CPU Threads sets the number of  CPUs allocated to the EXN/Aero launch. This number cannot be greater than the available cores on the user’s system.  The bare minimum is to assign one CPU for every GPU device. It is usually advisable to set CPU count to the number of GPU devices plus one (e.g. 2 GPU + 3CPU).  More CPUs should be allocated if the multi-block mesh has a large number of internal block-to-block interfaces.

GPU Devices sets the number of GPU cards allocated to the EXN/Aero launch. This number cannot be greater than the available GPU cards installed on the user’s system. Keep in mind that GPUs do not exhibit strong scaling in all cases. Small runs are often faster on a single GPU rather than multiple GPUs.

Memory Size (in the Advanced Mode) is a multiplier for the default memory allocation for data arrays. Sometimes, very large models require this limit to be increased. If it is too large, the system will report that it has insufficient memory at run time.

EXN/View Live #

 

Running a simulation #

 

Run EXN/Aero launches EXN/Aero on the specified compute resources. After running, the status bar (lower left corner at the GUI) will indicate the process ID number of the session. Shortly thereafter, an output window should appear, detailing the solver output during solution.

When an EXN/Aero session is active, it will appear in the Run in the menu bar with a three digit suffix, indicating the name of the output directory in the project directory, along with a completion percentage. Hovering over this option will display a sub menu with options specific to that run session.

Show displays data from the EXN/Aero session in an output tab.

Backup causes EXN/Aero to write a solution file to a new CGNS file having the same name as the original CGNS file, with a BAK suffix and sequential number. EXN/Aero continues to run after a backup.

Update causes EXN/Aero to re-read selected fields in the CGNS file (e.g. time step) at the end of the next time step. No solutions are written and EXN/Aero continues to run.

Stop causes EXN/Aero to complete the current iteration, write a solution file for restart to the CGNS file, and then exit.

Kill stops the process without writing solution files.

Properties opens a small dialog window with information about the EXN/Aero session, including the duration of the current session.

Project file structure #

The project directory contains all data related to a simulation in EXN/Aero, including setup, solver control and results files.  For the sample file structure shown here, the simulation has already been set-up and several simulations have been performed. Some files may not be present in the user’s directory at early stages of the set up process. 

When the user creates a new project in EXN/View, the project parent folder is created. The image below shows the contents of the Example.proj folder.


exampleprojectdirectory


Parent Directory

CGNS file (<projectname>.cgns) contains mesh information and is created by EXN/View when the user imports a mesh. Upon import, the CGNS file contains only mesh and block connectivity information, but eventually will contain custom folders after the problem is set up and ready to run on EXN/Aero.

JSON file (<projectname>.json) is a reduced version of the mesh information in the CGNS file. It is used by the render view in EXN/View to visualize the outline of the mesh blocks.

CVC directories and Results Directories are explained below.


CVC directories (<projectname>_cvc#.##)

 XML structure files are housed in the CVC directories, where #.## represents the version number. CVC stands for ‘CGNS Version Control”. These are updated to the latest version each time a project is saved in EXN/View. Old CVC files are kept for reference, but can be deleted once the user is satisfied that they are no longer needed.

An example of a CVC directory contents is shown below:


examplecvc


XML structure files contain all project settings. These are created by EXN/View when the user creates a new project. They can also be imported by other projects that share similar settings. The formatted text contained in XML files drives the appearance of the tree viewer in EXN/View, and can be edited manually in any text editor if the user so desires.

CVC.log is a record of version changes.

Info.txt provides some basic information for manual editing of XML files and cvc.log. This might be of interest for advanced users wishing to automate XML structures using scripts for running large parameter sets


Results directories (<projectname>-###)

Simulation output data files are contained in the results directories. These are created each time EXN/Aero is launched, and are given a unique name, where ### is the sequential number of the run.

 An example of the results directory contents is shown below


simulationdirectory


output_stream_prop-0.txt contains condensed simulation health data such as residuals and global mass imbalance. This is the file read by the output view in EXN/View.

SQLite Output (<projectname>-it_1-ts_1.sqldb) is a SQLite database file that contains an array of solver outputs. By default it contains equation residuals, CPU timing information. Monitor points and integrated quantities also end up in the SQLite file as additional tables.

The SQLite database can be appended and queried in real time, and it is the primary input to the simulation monitoring tool EXN/Vizer. There are several open-source tools available for viewing, editing and exporting data from SQL databases.

XML directory contains an exact copy of the structures used at run time. These are useful if a user wishes to re-create the exact conditions of a run, and can be imported directly into EXN/View.

Memory Map  directory contains memory usage summaries. This data is used mostly for debugging and will not be of interest to the average user.

Parareal Profile Status (parareal_profile_win-1.txt) contains diagnostic information from Envenio’s parallel-in-time solver. This capability is in alpha and the contents of this directory will not be of interest to the average user.


VTK sub-directory

This contains all field data outputs from the simulation, saved in VTK database format. This format is native to Paraview but is supported by several other visualization tools.

Drilling down through this file structure to the Iteration-1 directory, and then a step further into the Example_1 directory shows the following:


vtu_directory


vtu_files


VTK Multiblock file (<projectname>.vtm) contains information on how the various data blocks in the results folder are connected. Its name will always be identical to its corresponding results folder.

VTK blocks folder (<projectname>-###) contains the data blocks that are connected to the multi-block file. In the above image there are four blocks. One each for wall boundaries, outlet plane, inlet plane, and the mesh block.

Finding solution files #

The project file structure article talks about the various files within the project directory in detail. For the purposes of the walkthrough, it suffices to know that files are written in 3 main areas:

Restart solutions are written directly to the CGNS file in the main project directory. The CGNS file contains only one solution file at a time. Backup solutions are copies of the main CGNS file written within a solution directory.

Monitor & probe data are written to a SQL database that can be viewed from the EXN/View UI. If a simulation is in progress, they can be displayed and updated as new data are appended to the SQL file by the solver. If a simulation is complete, the SQL file is static.

VTK files contain complete solutions, including cell volumes and boundaries, and are written to the solution directory. Each one of them represents an instant in time. If the solution is transient, the files are written out with sequential numbers equal to the time step count at the time of writing. If the solution is steady-state, there is one solution written at the end of the run.

Meshing Best Practices #

This section explains how to label blocks and boundaries during the meshing process to optimize the workflow and take advantage of subdomain designation in EXN/View.

We provide examples from the Pointwise meshing tool, but equivalent operations are available in other meshing tools. 

A simple hybrid structured / unstructured mesh of a square channel flow shown below is used to explain the concept.


blocks

Mesh Requirements #

Please adhere to these requirements to ensure a fault-free run on the EXN/Aero solver.

  • Export in CGNS format.
  • Export mesh in a single file.
  • Subject to the single file requirement, the mesh can be structured, unstructured, or hybrid.
  • Minimum included angle throughout the mesh should be greater than 15 degrees.
  • Aspect ratios less than 1000 in single precision, or else plan to use double precision.
  • Minimize skewness.
  • Check for negative mesh volumes.
  • Label all boundary conditions.
  • As much as possible, group blocks into block families and boundary patches into BC families.

Labeling #

The figure below shows a section of the control panel in Pointwise that is used to alter boundary (Domains) and volume (Blocks) names. In the figure below, three blocks: blk-1, blk-2 and a third labeled “blk-porous_media” are shown, along with a list of domains.


dom_and_blk_names


In EXN/View, the same block names are listed one-for-one under the Cell Families branch, where they can be selected and assigned to a FluidDomain.  The same convention holds for BC Families.


cell_blocks


Notice that the first two entries in the drop-down list under Cell Blocks are not present in the block list from the meshing tool. In many meshing tools, blocks and block boundaries can be grouped using ‘family names’. Selecting blocks by family name automatically includes all mesh blocks bearing that name. In a mesh with a continuous fluid domain for example, it is sufficient to label all blocks as the same fluid family name and select only the family name in EXN/View to include them in the same fluid domain specification. The same holds true for boundaries.

Family Names #

The figure below shows another Pointwise example of family labeling during the meshing process in Pointwise. Here, we see nine boundary domains combined to one “symmetry” boundary family (yellow), one ‘outlet’ boundary family (orange), and two blocks are combined to one “Fluid_str” cell family. Family names can be chosen arbitrarily by the user.


families

 

Subdomains #

If required, the user may identify certain blocks as subdomains within a cell family. For example, to treat the blue block as porous media, the corresponding cell is selected under the Cell Blocks field of the sub-domain.


sub_domain_gui

Building a FAST mesh #

Manycore compute resources (CPU & GPU)  dramatically improve speed and reduce costs of running a CFD solver. On GPU resources especially, parallel comptuations on EXN/Aero are handled automatically with minimal user-intervention.   

There are a few caveats when operating in a manycore CPU/GPU environment. To realize their maximum potential however, it’s helpful to understand how the devices work, and to be aware of their general strengths and weaknesses.

The following is a list of optimizations that users can apply in most situations. 

 

Use Structured Blocks Where Possible #

EXN/Aero supports hybrid multiblock meshing and computes both structured and unstructured blocks (Cells) in their native format. EXN/Aero does this to improve performance, in particular with GPU processing — GPUs have a very high compute-to-memory bandwidth ratio and structured data is extremely cache-efficient due to its aligned arrangement. In testing, developers at Envenio see about 2.4x speedup operating on structured data vs. the exact same mesh written as unstructured hex data.

Be Mindful of Interface Count #

Multiblock meshing allows a lot of flexibility, but block-to-block interfaces introduce computational overhead. EXN/Aero mitigates this overhead by processing many of the interface operations on CPUs, while mesh blocks are computed concurrently on GPUs.  An ideal mesh is broken into enough blocks, which can be easily spread out across compute resources, but without many interfaces, as they become a bottleneck in the computation.

It is good practice to avoid using very small blocks.  GPU resources are very good at handling large amounts of data, but if a block becomes too small the performance gains of a GPU are reduced. If a user is generating large numbers of small structured blocks solely to avoid unstructured mesh it is often better to go with the larger unstructured block.  Like many things, balance is key and Envenio support engineers can help users strike that balance.

 

Minimize Double Precision Regions #

High Reynolds number flows, and high aspect ratio meshes often require double precision calculations near wall regions, but not in far-field areas. EXN/Aero’s multiblock approach enables users to specify single or double precision arithmetic block-wise.

In this way, the speed penalty and increased memory footprint associated with double precision calculations can be minimized. This is especially important on GPU resources where the double-precision penalties are greater.

 

The Right Blend of CPU & GPU #

Users can specify the number of CPU cores and GPUs to use during the simulation.  EXN/Aero distributes the workload across the resources to optimize performance.  The most efficient resource selection depends on the layout of a user’s system, but a few general rules apply:

  1. The number of CPU threads (cores) should always be at least one greater than the number of GPUs.
  2. Increase the number of CPU threads if using a mesh with a large number of block-to-block interfaces.
  3. Two GPUs is not always faster. Often, it’s better to ‘fill’ a single card and use a smaller cloud resource to minimize the cost of the simulation.

EXN/View User Interface #

EXN/View is a lightweight simulation set up utility intended to get you up and running quickly with your CGNS formatted mesh.

In EXN/View you can:

  • Load and render a wireframe mesh.
  • Set up physical models for the fluid domains.
  • Set up boundary and initial conditions.
  • Edit solver control variables.
  • Launch EXN/Aero runs

Menus #

The following outline the options available in the menu bar of the EXN/Aero user interface.

File #

Create New Project opens a file dialog and asks the user to designate a project directory. This creates a new directory with a .proj extension. A sub-directory structure is also created, containing default XML structures and the current CVC version number.

Open Existing Project opens a file dialog where you can navigate to your project and open it. Single-click on the ‘.proj’ directory and then click open.

Open Recent Projects displays a list of recently-opened files. Selecting one will open that project.

Open Running Projects displays a list of projects attached to currently running EXN/Aero sessions. Selecting one will open that project.

Save Project commits any changes made in the tree viewer to the XML structure files in the CVC folder for that project.

Save Project As combines ‘create new project’ and ‘save project’ simultaneously. It opens a file dialog and asks for a new project name & location. It saves changes without affecting the original project.

Import Structures (advanced mode) enables you populate the CVC folder in a project with pre-existing XML structures from a previous version of the project, or from another project altogether. Multiple XML files can be selected simultaneously and imported at once.

Update Structures (advanced mode) updates XML structures from an older version to match the current EXN/Aero installation. Versions must match for the Run EXN/Aero command to be successful. Update structures is applied automatically in basic mode.

Revert Project reloads the current project from the last saved version.

Close Project closes the current project, after a save prompt.

Exit closes the program, after a save prompt.

Mesh #

Import CGNS file opens a file dialog where the user can select a CGNS mesh file to associate with the project. During the import, EXN/View copies the CGNS file to the project directory and renames it to match the project name. CGNS files created by some meshers are also ‘converted’ so that labeling of nodes within the CGNS file match the expected standard used by EXN/Aero.

A 3D wireframe of the mesh will appear* in the render view when conversion is complete. During the conversion process, an output will display the conversion progress.

*There may be a delay for larger CGNS meshes that can last several minutes.

Edit #

Undo will revert reverts the last change. (not yet implemented)

Redo recovers the last change. (not yet implemented)

Insert Group inserts a group, such as a Cell Family, into the Tree, based on the currently selected branch. Some family groups have multiple types, which are displayed in a submenu.

Copy Group copies the currently selected group. The name of the new group is the same as the selected group with a suffix of ‘_copy’.

Delete Group deletes the currently selected group, after a confirmation.

View #

View Toolbar toggles the toolbar.

Use Large/Small Toolbar Icons toggles the size of the icons in the toolbar between 16 and 24 pixels.

View Coordinate Axes toggles the coordinate axes in the mesh window.

View Mesh Info toggles the information box in the mesh window.

Reorient to +/- X/Y/Z axis rotates the mesh to a particular axial orientation.

Zoom In/Out zooms the mesh in or out by approximately 5%.

Reset Zoom resets the zoom on the mesh to a an auto-scaled value.

Run #

Update CGNS (advanced mode) updates the project’s CGNS file with the data structures shown in the tree viewer. In basic mode, this process is done automatically when the run command is selected.

Run EXN/Aero launches EXN/Aero on the specified compute resources. Once the run begins, a new option appears beneath the run command showing the current solution directory associated with the run and a % completion indicator.

Show displays solver output in a new tab.

Backup causes EXN/Aero to write a solution file to a new CGNS file having the same name as the original CGNS file, with a BAK suffix and sequential numerical designation. EXN/Aero continues to run after a backup.

Update causes EXN/Aero to re-read select fields in the CGNS file (e.g. time step) at the end of the next time step. No solutions are written and EXN/Aero continues to run.

Stop causes EXN/Aero to complete the current iteration, write a solution file for restart to the CGNS file, and then exit

Kill is the same as issuing the kill command at the command prompt. It terminates the EXN/Aero process and no solution files are written.

Properties opens a small dialog window with information about the EXN/Aero session, including the duration of the current session.

Settings #

Switch to Advanced/Basic Mode toggles the interface between basic and advanced mode.  Advanced mode gives users additional options and forces some processes to be done manually. 

View Alpha/Beta features makes new and in-development engineering models & solver features available to the user. 

Alpha features generally have not been thoroughly tested or validated, but are known not to affect the integrity of release models. Alpha features might produce spurious results.

Beta features have at least one validation case in our automated test repository and are known not to affect the integrity of release models. Beta models have been tested internally and usually will be included in the next major release.  

Help #

Knowlege Base & Report a Bug links to the knowledge base section of www.envenio.ca. The knowledge base contains helpful articles and links to our zendesk bug reporting system.

User Manual links to the user manual section of www.envenio.ca (this page!)

Tree View #

The left hand pane of the EXN/Aero UI enables the user to set up physics, initial conditions, boundary conditions and solver settings.

Practically, this information is stored in the project CVC directory in the form of formatted text files (XML structures).

Plus and minus icons next to the branch names expand and collapse branches of the tree. This can also be done by right clicking and selecting expand or collapse from the context menu.

Node Insertion is available by right clicking the Cell Families and BC Families node and selecting the desired branch type from the context menu. Places where a branch can be inserted are designated by the insert icon .

Copy & Delete are available for nodes that were inserted by the user. Deletion and copy operations are available on the right-click context menu. Branches that can be copied and deleted are indicated by the copy-delete icon.

Formatted Text Links can be inserted anywhere a folder icon openfolder is present to the right of a node value. Formatted text is used primarily for monitor point databases and user-defined fields. See the section on formatted text for more information.

Database Connections are available anywhere the ‘hamburger’ database icon database is present. Clicking the icon brings up a list of available databases. Upon selecting a database, a sub-node will appear beneath the connected node.

Render Window #

The render view displays a wireframe mesh constructed from the project’s CGNS file, and highlights blocks when they are selected.

Left-clicking and dragging the cursor rotates the mesh. The mesh can also be rotated to a specific axial orientation using options in the View Menu and the mesh’s toolbar.

Right-clicking and dragging the cursir translates the mesh. The mesh can also be moved by using the arrow keys.

Scrolling the mouse wheel zooms the display. The zoom can also be adjusted using the ‘+’ and ‘-‘ keys, as well as through options in the View Menu.

Orientation Axes aligned with the mesh axes are displayed for reference. The axes can be toggled on and off using the option in the View Menu or the mesh’s toolbar.

Mesh Orientation controls are displayed when the user clicks the plus sign + in the lower left corner of the render view. These controls are the same as those shown in the View menu.

Mesh Size information is displayed at bottom right in the form of max & min coordinates for reference. These coordinates are multiplied by the ‘mesh scale’ variable (default mesh scale = 1.0). This information can be toggled on and off using the option in the View Menu or the mesh’s toolbar.

Toolbar #

Located below the menu, this option bar holds icons for the more commonly used features, such as Create New Project, Open Existing Project, and Save Project.

The toolbar can be toggled on and off in the View menu

Tabs #

Located above the render window, tabs allow the user to switch between render view and solver text output.

The small ‘x’ next to the tab name closes the tab. If an output tab is closed, it can be reopened through the run menu.

If a mesh window’s tab is closed, the project closes with a save-before-quit prompt.

Status Bar #

The status bar is located beneath the tree view. It confirms actions the user has taken and occasionally displays warnings in red text.

EXN/Aero Settings #

For a successful launch, a user must:

  1. Define all mesh cells, including physical models, properties and initial conditions.
  2. Define all boundaries.
  3. Choose appropriate solver settings based on the problem being studied.
  4. Indicate the quantity and frequency of output from the solver.

Cell Families #

Cell Families contain settings for cell families and zones in the mesh; for new projects this branch is empty.

To add a new family, right-click on the Cell Families branch and move to ‘Insert’. This produces a sub-menu which contains all of the available Cell Families, such as FluidDomains and SolidDomains. Once a Cell Family has been added, options are available to copy and delete it.

Clicking on the cell family branch highlights the regions of the mesh associated with this cell family in the render view. Right-clicking on the cell family produces context menu with a ‘Keep Visible’ option, which can be toggled on and off if the user wishes the highlighting to persist in the render view.

The Cell Blocks node opens a drop-down menu listing available mesh blocks. These can be selected in a group using the block family name, or they can be selected individually using shift+click  or ctrl+click.

RANS Models #

RANS turbulence model options are used in both unsteady and steady-state solver modes.

Cell families contains the main toggle and selector for turbulence models. Keep in mind that turbulence models require boundary conditions as well.

Turbulent Flow is the master switch for all RANS models, whether in steady state, URANS, RANS or LES/DES mode.

LES Spatial Filter must be set to RANS. Other options in this node are LES/DES options.

RANS_Option can be set to either SST (Menter) or k- \omega (Wilcox). Other common RANS models such as k- \epsilon and RSM are not supported.

 

LES / DES Models #

Several Large-Eddy and Detached Eddy simulation models are available, including:

  • Smagorinsky-Lily: a zero-equation model that scales SGS turbulent viscosity on the strain rate tensor of the mean flow —  cfd-online.com summary
  • Canopy K-equation: a one-equation sub-grid stress model that uses the transported SGS turbulent kinetic energy ‘k’ and an assumed length scale to compute SGS turbulent viscosity. Extra source terms are active in the k-equation, and in the momentum equations to model the dissipative, momentum absorbing effect of a vegetative canopy.
  • WALE: also known as Wall Adapting Local Eddy-Viscosity model. This zero-equation model scales SGS turbulent viscosity on both the strain rate tensor and the vorticity tensor, and is generally known for better stress prediction near walls — cfd-online.com summary.
  • SST DES: is a detached eddy formulation where the two-equation SST model serves as the universal turbulence model in the boundary layer, and as the SGS model in unconstrained regions of the flow — Strelets, M. (2001), “Detached Eddy Simulation of Massively Separated Flows”, AIAA 2001-0879.

Turbulent flow  is a master switch for turbulence models. To run in LES / DES mode, this must be set to ‘on’

LES spatial filter enables the user to pick which LES / DES model that they wish to use. Choices are shown in the ‘models’ section below.

LES Length Scale branch controls the method by which subgrid length scale is computed.

Max Edge Length computes the LES length scale as the maximum edge length of a control volume when set to ‘on’. When set to ‘off’ it computes length scale as the cube root of the element volume.

Cube Root Volume Weight allows the user to specify multipliers on the x, y and z dimensions of a mesh element when computing its volume. Usually this is set to [1 1 1] and is only active when the cube root spatial filter is selected.

Initial Conditions #

This branch sets the initial conditions for the various transport equations for all mesh blocks that are part of a given cell family. 

RANS / LES / DES ICs #

Guidelines for setting free-stream conditions for turbulence models are found here, although some experimentation may be required to find a value that works for a user’s particular case.

Turbulence Option gives the user the option to set turbulence initial conditions either as k and epsilon, or as an intensity and length scale. Practically, these values are interchangeable and the option is given for convenience.

Turbulent Kinetic Energy sets the TKE in the domain

Turbulent Dissipation is related to the turbulence length scale and sets the dissipation variable in the domain.

Turbulence Intensity sets the magnitude of the varying component (ie kinetic energy) and is parameterized as a fraction of the mean flow velocity. 

Turbulence Length Scale sets the size of the energy-containing eddies (which is related to the dissipation). 

 

 

Subdomain ICs #

Scalar 1 sets the initial value for that scalar throughout the domain.  

Scalar 2…N sets the initial value for any additional scalars. One initial value for each scalar equation inserted in subdomains. 

Properties #

This branch sets fluid properties and user-editable coefficients for various transport equations. 

Fluid Prop. #

Many of the fluid properties a user needs are available from online sources and reference tables. All units are SI. 


Density sets a constant density when solving incompressible flows. When compressible flow option is activated, the density is variable and is calculated by the solver using the ideal gas equation.

Gas Constant is available when the thermodynamic property model is set to Ideal gas. This sets the ideal gas constant for the fluid. 

Thermal Conductivity is a constant value for the fluid

Compressible Opion toggles between compressible flow and incompressible flow. Generally, compressible flow is only required when mach numbers exceed 0.5. 

Thermodynamic Property Model is available when the energy equation is activated. The drop-down menu gives options for different properties and specific heat models. When compressible flow is activated, the property model is locked to ideal gas and polynomial Cp. 

Dynamic Viscosity sets a constant viscosity for the fluid. 

Specific Heat is a constant value when the thermodynamic property model is set to constant density and Cp. Otherwise, the specific heat is defined as a 4th order polynomial, as C1, C2, C3, C4, C5, where C1 is a constand ant C5 is the coefficient of the x^4 term. 

Specific heat temp range defines the absolute bounds bounds on the temperature used in the ideal gas equation.  

 

Subdomain Prop. #

General Scalar Difffusion is available when scalar transport models are active. This sets the diffusion coefficient

Cell Statistics #

The statistics calculator computes a cumulative mean & variance of selected variables in a transient solution. The Welford method is exact in the mean and converges for variance on a longer time frame than the mean. 

When a vector is selected (e.g. Velocity) the mean field will have 3 components and the variance is computed as a symmetric covariance tensor having 6 unique elements.


Collect Statistics toggles the statistics calculator on and off

Continue Stats from Solution gives the user the option to include the previous solution’s cumulative average and standard in the re-started solution

Collect Statistics Time Delay gives the user the option to set a time-based delay on computation of mean and variance. This can be used to exclude the ‘start up’ phase of a transient solution.

Variables for Statistics allows the user to choose variables that included in the computation. The mean fields are output as separate fields in the VTU output file.

Subdomains #

EXN/Aero uses subdomains where a special model or source term is applied to a limited region of the domain. 

If a simulation is to use a subdomain, then labeling & mesh subdivision must be applied at the meshing stage to accomodate it.

For example, a pipe mesh with a sponge media near the exit would be generated in 2 blocks — one for the free fluid flow, and one for the porous region. Both blocks can have their properties set by the same Cell Family, but the subdomain relies on there being a distinct mesh block to define the its boundaries. 

Porous Media #

Porous media models add a drag source term to the momentum equations that approximates the average effect of the media on the flow. 

Porous regions are defined as subdomains. To designate a mesh block as a porous media subdomain, right click on subdomains and select ‘porous media. 

Media Property can be set to homogeneous or ansiotropic.

Permeability and Inertial Resistance Factor  are scalars that control the total resistance of the porous media to fluid flow. Permeability is often termed the ‘Darcy’ coefficient. For laminar flows, the inertial component is ignored. 

Viscous Resistance Coefficient and Inertial Resistance Coefficient are specified as symmetric tensors when the anisotropic media option is selected.

EXN/Aero requires the 6 unique components of the symmetric tensor, D1, D2, D3, D4, D5, D6, corresponding to the following positions in the tensor:

2016-10-28-16_03_07-porous-media-model-docx-read-only-word

General Scalar #

Scalar transport models activate an extra (or several extra) transport equations that track the movement of a passive, dilute, non-reacting species. 


General Scalar Transport toggles the general scalar transport model on and off.

Schmidt Number is the ratio between momentum diffusivity and mass diffusivity.

Scalar Variables becomes a parent node when the user right clicks and inserts scalar variables. Once inserted, a corresponding initial condition and diffusion coefficient will appear for each scalar.

BC Families #

BC Families contains settings for boundary conditions and surfaces in the mesh; for new projects this branch is empty

To add a new family, right-click on the ‘BC_Families’ branch and move to ‘Insert’. This will produce a sub-menu which contains all of the available BC Families, such as ‘InletBC’ and ‘WallsBC’. Once a BC Family has been added, options are available to copy and delete it.

Clicking on the BC family branch highlights the regions of the mesh associated with this BC family in the render view. Right-clicking on the BC family produces context menu with a ‘Keep Visible’ option, which can be toggled on and off if the user wishes the highlighting to persist in the render view.

BC_Location node opens a drop-down menu listing available BCs. These can be selected in a group using the interface family name, or they can be selected individually using shift+click  or ctrl+click.

RANS / LES / DES BCs #

Guidelines for setting boundary conditions for turbulence models are found here, although some experimentation may be required to find a value that works for a user’s particular case.

Turbulence Option gives the user the option to set turbulence initial conditions either as k and epsilon, or as an intensity and length scale. Practically, these values are interchangeable and the option is given for convenience.

Turbulent Kinetic Energy sets the TKE in the domain

Turbulent Dissipation is related to the turbulence length scale and sets the dissipation variable in the domain.

Turbulence Intensity sets the magnitude of the varying component (ie kinetic energy) and is parameterized as a fraction of the mean flow velocity. 

Turbulence Length Scale sets the size of the energy-containing eddies (which is related to the dissipation). 

 

 

BC Statistics #

Collect Statistics currently gives the option to calculate only total body force and moment, both instantaneous and cumulative mean, about the mesh origin for a wall boundary group.

BC statistics are written to the SQL database rather than the VTU file at every simulation time step. 

Subdomain BCs #

Scalar equations can be specified at inlets & walls, or else set to zero-flux. 


Inlet Scalar Option, or  Wall_Scalar Option can be set to ‘specified value’, which is a constant, or ‘zero flux’ which blocks all movement of the scalar through the boundary.

Scalar Variable indicates which scalar is associated with a particular boundary condition

Specified Value if selected above sets a boundary value. This node is not available when the ‘zero flux’ option is selected.

Solver Control #

Solver Control contains options related to the EXN/Aero solver including time step, total run time, compute resource allocation, output options and coefficient loop limiters (among other things). These settings are applied globally when the EXN/Aero run is launched.

In Solver_Control, you can:

  • Modify global solver settings
  • Change simulation timing
  • Toggle restart, and restart arrays
  • Set relaxation parameters
  • Set simulation output frequency and manage output arrays
  • Assign compute resources
  • Add monitor point databases
  • Set Iteration limiters and multigrid parameters

Simulation #

Branches under simulation affect the solution as a whole, and the way that the solver outputs data. 

Simulation Type #

Steady State solution methods converge quickly at the expense of temporal resolution for processes that are statistically stationary in time. These solutions satisfy the unsteady partial differential equations in the limit of very large time steps. EXN/Aero employs a false transient solution method combined with implicit (explicit for pressure only) relaxation schemes of the underlying correction equations.

Steady state mode is controlled primarily by the time step and the relaxation parameter. Time step is found in the main Solver Control  branch. Relaxation parameters and other convergence settings are found in the Convergence Criteria branch.

Unsteady solution methods converge the entire flow field at each real time step, and typically run much more slowly than steady state solutions. Unsteady methods enable temporal resolution of the flow, and are useful for situations where the process is not statistically stationary in time, or where a steady solution is inaccurate (e.g. vortex shedding from a cylinder).

Unsteady mode employs both inner and outer coefficient loops controlled by a series of convergence criteria at both the global and the individual equation level. Settings for unsteady convergence are found in the Convergence Criteria branch

Misc Solver Settings #

Task Scheduling changes whether physics set up computations are done in serial or parallel. Typically, this should remain on the default ‘concurrent’ setting.

Element Quality Threshold is used on unstructured grids. Element quality is measured by ‘flatness’, ‘skewness’ and ‘aspect ratio’ by the EXN/Aero solver on start up. Any elements that exceed the threshold (big numbers are bad) will have 2nd order treatments turned off. This can create solution inaccuracy, but can also help stabilize solutions on grids with poor element quality. Setting this below 0.8 is not advised.

Transient Scheme Order toggles between 1st and 2nd order transient scheme.

Reference Pressure Level sets the reference for all fluctuating pressures (e.g. BCs and initial conditions) in the simulation. Pressure at any point in the solution (absolute) is the local pressure plus this reference pressure.

Reference Temperature sets the reference for all fluctuating temperatures (e.g. BCs and initial conditions) in the simulation. Temperature at any point in the solution (absolute) is the local temperature plus this reference temperature.

Point Specified Pressure is used when the user wishes a location within the solution to be at a specific pressure. Usually the automatic setting is used and location is specified in XYZ coordinates.

 

Solution Timing #

Total Run Time sets the internal time-based stop condition for the solver. When the simulation time reaches this value, EXN/Aero writes output files per transient output settings, writes a restart-able solution to the CGNS file and then exits.

Unsteady Time Step is the amount of simulated time that elapses during the outer coefficient loop.

Steady State Time Step should be comparable in magnitude to the characteristic time scales of the problem. It is good practice to use a small time step at the beginning of the simulation, and to increase it gradually. Increasing the time step rapidly can trigger numerical instability, and so relaxation of the transport equations is needed to aid convergence in many cases.

Restarting #

When a simulation completes, a single restart-able solution is written to the main CGNS file for the project. Likewise, when ‘backup’ commands (automated or manual) are issued, a single restart-able solution is written to a backup CGNS file. 

To restart a completed simulation or to restart from a backup, ensure that the solution is in the main project CGNS file. For completed solutions, it will already be there. For backup solutions, it is necessary to import the mesh from the backup location using the Mesh menu.


Initial time from solution has two options. Upon restart, setting this variable to ‘on’ causes EXN/Aero to use the final simulation time from the previous run as the start time of the current run. If set to ‘off’ then the initial time is set to zero.

Initial Values from Solution enables the user to pick which data arrays are included in a restart. If an array is required but not included in this list on restart, EXN/Aero defaults to the initial value settings in Cell_Families.

Use random seed from solution is used in the synthetic turbulence models. If a user wishes to reuse the random number generator seed from a previous solution in the current solution, this should be set to ‘on’.

 

 

Solution Output #

Solution files for post-processing are written in VTK format. Practically, the VTK for a single solution consists of an index (.vtm) file, and a series of associated solution files (.vtu) in a directory of the same name. There is one VTU file for each BC family and each Cell family set in the tree viewer. 


Transient Output Frequency controls the interval in time steps at which solution output files (VTK format) are written to the solution directory. A setting of ‘1’ outputs every time step, while a setting of ‘0’ disables VTK output entirely. Keep in mind that writing VTK files to the hard drive is very time consuming and can slow a simulation significantly if interval is small.

Solution Backup Frequency controls the interval in time steps at which EXN/Aero writes a restart-able solution to a backup CGNS file in the solution directory. A setting of ‘1’ outputs every time step, while a setting of ‘0’ disables backups entirely. Keep in mind that writing solution arrays to the CGNS file is very time consuming and can slow a simulation significantly if interval is small.

Transient Output Variables controls which solution variables are included in VTK output for fluid volumes. Some output variables are multi-component vectors (e.g. Velocity) while others are scalars or individual components of vectors (e.g. Temperature, Velocity_X). Picking a vector in this field is sufficient to include its components — for example, picking Velocity means the user does not also need to select ‘Velocity_X’, ‘Velocity_Y’ and ‘Velocity Z’ also.

Transient Output Variables BC controls which solution variables are included in VTK output, simultaneously with the VTK output for fluid volumes. Vector & scalar picking rules are the same as for Transient Ouput Variables above.

 

Task Execution #

Inner Loop Equations sets which transport equations are computed as part of the main coefficient loop. Any equations not included in this list are by default calculated outside the coefficient loop.

Deactivate Equations lets users selectively deactivate transport equations. For runs, this is usually set to ‘none’ but might be modified as part of diagnostic checks. Deactivated fields are frozen from the point of view of the solver.

Convergence Criteria #

Unsteady Convergence Criteria tell the solver how hard to work to achieve convergence of a solution.

  • Global Settings control the outer coefficient loop and set residual targets. These settings can be modified at the global level, which affects all transport equations, or at the local level on a per-equation basis.
  • Individual Equation Settings (e.g. pressure, momentum, etc….) control the inner loop for each equation and can override global residual targets.

Some global settings have a different meaning for steady-state solutions; these are noted below.

All relaxation parameters can be adjusted mid-solution by changing the value in EXN/View and issuing an ‘update’ command from the Run Menu.


Unsteady Global Settings

Outer loop Exit Condition: determines the exit condition for the outer loop.

  • Static: every outer coefficient loop runs to the maximum number of iterations. Mass imbalance and residual targets are not checked
  • Dynamic: the outer loop will perform iterations between the minimum and maximum number settings. Mass imbalance and residual targets checked, and the coefficient loop stops if these thresholds are met.

Minimum number of iterations: sets the minimum iteration number for Dynamic outer loops. This is not used for static outer loops.

Maximum number of iterations: sets the maximum iteration number for dynamic and static outer loops outer loops, in all conditions.

Residual Option: sets the normalization equation for residuals.

  • Default: residuals are un-normalized and have units according to the equation they are associated with.
  • Normalize Globally: The residual for each equation is divided by its a_P coefficient and by the range of that variable in the domain. Globally-normalized residuals are dimensionless.
  • Normalize Locally: The residual for each equation is divided by its a_P coefficient and by the local variable in a control volume. Locally-normalized residuals are dimensionless

Mass Imbalance Target: Sets a threshold for mass imbalance below which a dynamic coefficient loop will exit. This target is ignored for static loop exit conditions.

Residual RMS Target Default: Sets the RMS residual threshold below which the dynamic coefficient loop will exit. If an individual equation has a lower threshold, then the solver will use that as an exit criterion instead. This target is ignored for static loop exit conditions.

Residual MAX Target Default: Sets the maximum residual threshold below which the dynamic coefficient loop will exit. If the RMS threshold is met, but a local maximum residual exceeds this threshold, the loop will not exit. This target is ignored for static loop exit conditions.


Steady State Global Settings

Outer loop Exit Condition: No outer loops are performed; each false time step is iterated upon a single time.

  • Static: The simulation runs to the maximum number of iterations and no convergence targets are checked.
  • Dynamic: the simulation iterates a number of times between the minimum and maximum number settings. Mass imbalance and residual targets checked, and the run stops if these thresholds are met.

Minimum number of iterations: the minimum number of false transient time steps taken to reach a steady-state solution.

Maximum number of iterations: the maximum number of false transient time steps taken to reach a steady-state solution.


Unsteady Individual Equation Settings

Residual RMS Target: Same concept as the global setting, except it applies only for a single equation and it overrides the default target in global convergence settings if  is given a lower value. This target is ignored for static loop exit conditions.

Residual MAX Target: Same concept as the global setting, except it applies only for a single equation and it overrides the default target in global convergence settings if it is given a lower value. This target is ignored for static loop exit conditions.

Residual Reduction Level: Sets the target residual reduction threshold for an equation, defined as the ratio of the initial residual to the final residual in any inner loop. If the target is met, then the inner coefficient loop exits. This threshold is dynamically increased to drive a solution toward convergence, pursuant to the Iteration Threshold setting.

MG Residual Reduction Rate: Sets the target residual reduction threshold for an equation, defined as the ratio of the initial residual to the final residual at any multigrid level. If the target is met, then the multigrid solver will not initiate additional multigrid levels.

Max Iterations: The maximum number of iterations at the finest multigrid level in the linear solver for an equation.

Forced Iterations: Forces a user-defined number of iterations at the finest multigrid level in the linear solver for an equation. This setting overrides all residual target thresholds at the individual equation and global level.

MG Forced Iterations: Forces a minimum number of multigrid levels in the linear solver for an equation.

Iteration Threshold: Below this iteration threshold, the Residual Reduction Level is not dynamically modified. Above this number of iterations, the Residual Reduction level is dynamically modeified. If set to a number greater than Max Iterations, this feature is disabled.


Steady Individual Equation Settings

Relaxation Factors: must have values between 0 and 1. A typical pressure relaxation is 0.3 and a typical momentum relaxation is 0.7. Other model equations use relaxation factors similar to that of momentum. A general rule of thumb for optimal convergence is to choose pressure and momentum relaxation factors such that their sum equals 1.0.

At early time steps, the pressure correction loop is working harder than at later time steps. It is often necessary to use a smaller pressure relaxation early on and to then increase it as the simulation progresses. To drive final convergence, relaxations should be increased to a value close to 1 as the simulation nears completion.

Exporting Boundaries Only #

The export boundaries branch controls which solution variables are included in VTK output for selected boundaries. This capability is useful where the user desires to generate a boundary VTK file at a different frequency from the regular transient output controlled above in the main solver control branch. To generate a boundary export group, right click Export Boundaries. Multiple groups can be created, each with unique settings.


BC Locations controls which boundaries or boundary families inherit these output settings.

Variable_List enables the user to pick which variables are included in the VTK output for this group. Picking rules are the same as for solution variables above.

Output_Frequency controls the interval in time steps at which EXN/Aero writes a boundary VTK file to the hard drive. This interval can be a different from the solution output frequency.

 

Compute Resources #

CPU Threads sets the number of  CPUs allocated to the EXN/Aero launch. This number cannot be greater than the available cores on the user’s system. A good rule of thumb is to have at least one CPU allocated for every GPU device plus one. More CPUs should be allocated if the multi-block mesh has a large number of internal block-to-block interfaces.

GPU Devices sets the number of GPU cards allocated to the EXN/Aero launch. This number cannot be greater than the available GPU cards installed on the user’s system. Keep in mind that GPUs do not exhibit strong scaling in all cases. Small runs are often faster on a single GPU rather than multiple GPUs.

Memory Size is a multiplier on the default memory allocation for data arrays. Very large runs sometimes require that this limit be increased. If it is too large, the system will report that it has insufficient memory.

 

Monitor Points #

Monitor points, otherwise known as probes, are points in the simulation domain that gather data (via distance-weighted interpolation) and output to an SQL database. Monitor points are a low-overhead means of writing transient data at points of interest for the complete time series.

Cartesian Coordinate Locations are the XYZ coordinates, relative to the domain origin, of monitor point positions. These can be entered manually as position vectors separated by commas, or, more often, they are read from a formatted text file by clicking the folder icon (see exn/view tree-view section).

Variable List controls which variables are reported at each monitor point. By and large, these are the same variables available in the Solution Output Variables node above.


Format for monitor points:

  • comma separated
  • X coordinate, Y coordinate, Z coordinate, in that order
  • new line for each monitor point location

Multiple monitor point sets can be loaded with different names. Naming in the SQL database is:

<Monitor point group name>_CX

where ‘monitor point group name’ equals the name set by the user in the tree view, and X corresponds to the row number for the monitor point in the formatted text file.

Database #

The Database branch holds custom databases, such as scripted changes to time steps, user-defined boundary fields and user-defined initial conditions.

Variables #

This branch lists all of the variables available as outputs in EXN/Aero. The two main categories are Cell (mesh block) and Interface (boundary) variables.

Variable visibility depends on model selections made in BC_Families and in Cell_Families — variables associated with inactive models are not shown by default.

Monitor #

The graphical monitoring tool enables users to view certain solver outputs in a 2D graphical format. These outputs include:

  • Convergence information (residuals)
  • Monitor point data
  • Mean and variance for boundaries, as selected in the statistics calculator.

Data displayed by the monitor tool is contained in the solution directory within an SQL database file — see the results directory section for more information.

Graphing #

After about 10 time steps, data begins writing from the solution buffer to the SQL database and the monitor branch appears. Clicking on this branch causes the render view to display available solution files in a project.



To select a run, drag and drop the name of the results file (on the right) over to the”Results” branch of the tree. This opens a new branch with the name of the results file beneath.



Clicking on the results file name in the tree (at left) shows tables in that SQL database (at right). In this example, ‘Convergence’, is the only choice. Monitor points and boundary statistics are shown if these options are activated for a run.



Dragging and dropping the ‘Convergence’ table under the results file name in the tree selects that table; multiple tables may be selected if they are available.

Clicking the table in the tree shows the available variables at right.



Right-clicking “Charts” gives the option to create a new chart.

Once a new chart has been created, variables are added by dragging and dropping them onto the X and Y axis.

Multiplot (Y-axis only) is available by dragging and dropping multiple variables onto the Y axis branch. Multi-selection of variables is available using CTRL+click.



Clicking on the chart causes the render view to display a 2D plot of the selected variables. The legend displays the variables by name and colour, and displays the value of that variable based upon the position of the mouse.

Variables can be removed by clicking the small red X icon, or hidden by un-checking the box next to that variable.

Left clicking and dragging the mouse enables the user to zoom in on sub-regions of the chart. Chart range can be edited directly on the axes in variables outlined by a rectangular border.

Right-clicking gives several additional options for the chart.



Disable Mouse Tracking turns on the live display of variables based on the mouse position

View Legend toggles the legend on and off

Save Chart as PNG opens a file dialogue where the user can save an screen capture of the plot

Save Chart as CSV opens a file dialogue where the user can save the chart data as a comma separated text file of columnar data, containing variable names in the header.

Reset Zoom Sets both X and Y axis ranges to that of the visible variable having the largest range.

Dynamic Updating activates an automated read of the SQL database at the update interval. Automated updates rescale the x-axis to include new data appended to the database.

Update Interval is the interval in real-world seconds at which the SQL database is re-read. When the SQL file is very large (e.g. long unsteady runs, or runs with many monitor points) it is best to set a long update interval, as re-reading the tables can take awhile

File Management #

The project directory contains all data related to a simulation in EXN/Aero, including setup, solver control and results files.  This section walks through the project directory, explaining the purpose of each component. File structure shown here simulation has already been set-up and several simulations have been performed; some files may not be present in the user’s directory at early stages of the set up process. 

When the user creates a new project in EXN/View, the project parent folder is created. The image below shows the contents of the Example.proj folder.


exampleprojectdirectory

Network Storage #

The default storage area /data in the Nimbix cloud environment. Nimbix provides users with 1 terabyte (TB) of free persistent data storage for compute jobs in a service called drop.jarvice.com.

Data can be uploaded and downloade at any time whether jobs are running or not. The best tools to use are Secure FTP protocol (SFTP) or the Nimbix File Manager over HTTPS.

Please note, it is best to use SFTP for large file transfers, i.e., any file over 100 GB.

Zipping CGNS & VTU files #

File compression utilities are unusually effective at shrinking the file size for both CGNS and VTU files — 50% or greater compression is the norm.

Nimbix does not provide a zip or unzip desktop utility, but these commands can easily be issued from the command line.

As an example:

zip at the command line:

zip <zipfilename>.zip /nimbix/data/<project_folder>/<cgnsfile>.zip

unzip at the command line:

unzip <zipfilename>.zip

Whether storing files for later use, or compressing before transferring a file, zip is an effective way to reduce your memory footprint on the cloud.

Transfer files by SFTP #

SFTP is a good choice for moving large files reliably.  Access to cloud storage requires a nimbix username username and the account’s API key.

For users that are behind a corporate firewall, it might be necessary to request OUTBOUND access to port 22 (TCP) from IT management. 

To connect, follow these steps:

  1. Download an SFTP client, such as FileZilla or SCP and install
  2. enter server hostname drop.jarvice.com
  3. port 22
  4. username user’s account name
  5. password user’s API key

For help finding the API Key, see related article Where do I find my JARVICE API Key?

Once connected, transfer files to and from the nimbix storage drive. It is not necessary to have an active compute session to do this. 

In the next nimbix compute session, files will be available in the /data directory.

Nimbix File Manager #

The nimbix file manager is a browser-based utility offered by nimbix that makes file transfer simple

  1. Login to the nimbix dashboard
  2. Click “Compute” in the top navigation bar and select Nimbix File Manager. Launch a session using default settings. 
  3. The browser now returns to the dashboard and shows a running file manager job.
  4. Click the blue file manager link in the ‘active jobs’ area to show the session details. Connect to this using the green ‘connect’ tab that appears about 30 seconds after the job is launched.
  5. A new browser tab appears and it shows the contents of the account’s /data folder. Use the links to download and upload files. 

When data transfer is complete, return to the dashboard and shut down the file manager task to avoid unnecessary compute charges. 

Parent Directory #

CGNS files (<projectname>.cgns) contain mesh information and are created by EXN/View with the user imports a mesh. Upon import, the CGNS file contains only mesh and block connectivity information, but eventually will contain custom folders after the problem is set up and ready to run on EXN/Aero.

JSON files (<projectname>.json) are a reduced version of the mesh information in the CGNS file. These are used by the render view in EXN/View to visualize the mesh.

CVC directories and Results Directories.

CVC Directories #

XML structure files are housed in the CVC directories, where #.## represents the version number. CVC stands for ‘CGNS Version Control”.

An example CVC directory contents is shown below:


examplecvc


XML structure files contain all project settings and information displayed in the tree viewer. These are created by EXN/View when the user saves a new project. They can also be imported by other projects that share similar settings. The formatted text contained in XML files drives the appearance of the tree viewer in EXN/View, and can be edited manually in any text editor if the user so desires.

CVC.log is a record of version changes.

Info.txt provides some basic information for manual editing of XML files and cvc.log. This might be of interest for advanced users wishing to automate XML structures using scripts for running large parameter sets

XML Structure Files #

When a user makes changes to the default layout of the tree, they are modifying a default XML file. These custom files are specific to a project, and they are saved in the CVC directory. At launch time, the XML structures are used to write the necessary information to the CGNS file which is then read by the solver. 

The physical model settings, solver control configuration settings and the position of these settings in the tree view change incrementally from time to time as EXN/Aero is updated.

Every update to EXN/Aero contains a template set XML structure files that are compatible with that release. 

When an older project is loaded in a new update of EXN/Aero, the structures are automatically updated to the newest template, preserving any data and settings from the older files. 

Upon updates, older XML structure files are preserved in their original CVC directory and a new CVC directory is created with the up-to-date versions. 


 

 

XML structures can be edited by the user in any text editor, or edited automatically using a 3rd party script or programming language (e.g. matlab). This can be useful if users are generating a set of permutations on a single mesh.

Formatting of XML structure files is specific to EXN/Aero and must be correct in order to run without errors. Envenio engineers can help explain the formatting if a user wishes to make edits directly. 

Results Directory #

Simulation output data files are contained in the results directories. These are created each time EXN/Aero is launched, and are given a unique name, where ### is the sequential number of the run.

 An example of the results directory contents is shown below


simulationdirectory


output_stream_prop-0.txt contains condensed simulation health data such as residuals and global mass imbalance. This is the file read by the output stream in EXN/View.

SQLite Output (<projectname>-it_1-ts_1.sqldb) is a SQLite database file that contains an array of solver outputs. By default it contains equation residuals, CPU timing information. Monitor points and integrated quantities also end up in the SQLite file as additional tables.

The SQLite database can be appended and queried in real time, and it is the primary input to the simulation monitoring function within EXN/View. There are several open-source tools available for viewing, editing, appending and exporting data from SQL databases. Some of these (e.g. CSV export of chart data) are available in EXN/View

XML directory contains an exact copy of the structures used at runtime. These are useful if a user wishes to re-create the exact conditions of a run, and can be imported directly into EXN/View.

Memory Map  directory contains memory usage summaries. This data is used mostly for debugging and will not be of interest to the average user.

Parareal Profile Status (parareal_profile_win-1.txt) contains diagnostic information from Envenio’s parallel-in-time solver. This capabilty is in alpha and the contents of this directory will not be of interest to the average user.

VTK Directory #

VTK subdirectory

This contains all field data outputs from the simulation, saved in VTK database format. This format is native to Paraview but is supported by several other visualization tools.

Drilling down through this file structure to the Iteration-1 directory, and then a step further into the Example_1 directory shows the following:


vtu_directory


vtu_files


VTK Multiblock file (<projectname>.vtm) contains information on how the various data blocks in the results folder are connected. Its name will always be identical to its corresponding results folder

VTK blocks folder (<projectname>-###) contains the data blocks that are connected to the multiblock file. In the above image there are four blocks. One each for wall boundaries, outlet plane, inlet plane, and the mesh block.

Post Processing With Paraview #

Paraview is general-purpose open-source plotting software maintained by Kitware. It can be used:

  1. on a user’s local machine
  2. on a internal server via remote connection
  3. on cloud resources such as Nimbix, which enables users to access their cloud storage drive directly 

Paraview installers & source code can be accessed at:

http://www.paraview.org/

The complete user manual is available here:

http://www.paraview.org/paraview-guide/

We encourage users to learn more from the official Paraview guide. This section is intended to give some basic information to get up and running quickly with the Paraview tool. 

Launching on the Cloud #

Running paraview on cloud resources is done in the exact same way as EXN/Aero. Simply find Paraview on the Compute tab of the nimbix dashboard. There is only one selection available in the drop-down list for machine type ($2.50/h).

Connect to the Paraview desktop either through the browser remote connection or through a VNC connection. 

When using Paraview on the cloud, users have access to the same /data directory as EXN/Aero sessions. This is advantageous for situations where results files are too large to download efficiently. 

Paraview Anatomy #

Paraview uses hierarchical data filtration to produce useful data sets. 

  • At the top of the hierarchy is the root data file(s). For EXN/Aero output, these output files are found in the VTK directory (see the file management section of this manual). Either .vtu or .vtm output files can be opened in Paraview.
  • ‘Filters’ are applied to reduce the complete data set into useful displays, contours and plots. 
  • Filters can be applied cell-wide, or on interfaces, using any combination of variables included in solver transient output (see solver control for output settings).
  • Filters, among other things, can project variables, perform arithmetic, plot over lines, create contours, produce streamlines, integrate in 2D and 3D, and analyze time-resolved data sets. 

The default paraview display has 3 main areas. 

  1. File and filter list displays the filter hierarchy and provides some controls for modifying its branches. 
  2. File and filter controls are displayed for the filter or data set selected in the list above
  3. The render view displays data from filters in a variety of formats. 3D render is shown in the image above, but 2D charts, spreadsheets and other views are available. 

Common Operations #

Examining a cylinder in cross-flow, we added 2 filters to the root data set cylinder0M_6001.vtm

  1. Cell data to point data converts the raw ‘cell’ data output by EXN/Aero to Paraview’s  ‘point’ data format and removes jaggedness from the data in subordinate filters.
  2.  Clip slices the data set along a plane and keeps data on one side. 

Notice that the render view automatically turns off visibility for higher filters and keeps visibility for the lowest filter. Visibility in the render view can be changed anytime. 



Next, we plotted streamwise velocity along a line using a new filter

  1. Plot over Line gives the user control over two endpoints of the line and displays results in a split render view (2D), and allows the user to choose which plots are displayed. 

Here, rather than applying the filter to the clipped data set, we drew off the same root set (cell data to point data) as the clip filter. The plot over line could easily be applied to the clipped set, but we would need to ensure that the line endpoints lie within the clipped set. 



To compute the vortex shedding frequency, we computed lift and drag on the upper surface of the cylinder. To accomplish this we used 3 additional filters:

  1. Extract Block enables the user to turn off certain vtu files from the root data set. In this case, we turned off all but the cylinder surface
  2. Clip was explained above; it is used here to remove the lower half of the cylinder surface. 
  3. Integrate Variables performs a surface integral for all variables in the data set. We are concerned primarily with pressure in this case, since its surface integral equals force. 

In the figure below, we apply extract block to the cell data to point data branch. If we applied this filter to the first clipped block (described above) we would see a truncated cylinder. 

An additional filter can be used here for an unsteady simulation. Plot over time gives the time series of the set to which it is applied. When using any ‘over time’ filter in Paraview, it must open each individual VTU file and perform the exact same computation. When the data set is large or the file size is large (or both) this can be a slow process. 



We plan to grow this section of the manual with feedback from our user base. If you have comments or suggestions for paraview tutorials (in an EXN/Aero context) please send us a message!

Troubleshooting #

This section is intended to help you overcome some of the common startup issues and warnings that you might encounter.

The troubleshooting section is divided into two sections:

  1. Solver Errors which users encounter only after launching EXN/Aero.
  2. UI errors which users might encounter anytime when working with the user interface.

If your error is not shown here or if you are stuck, please submit a support ticket and Envenio engineers will respond as soon as possible

Solver Errors #

cg_goto error

This solver error occurs when EXN/Aero tries to access a setting in the CGNS file but fails to find it.

This error is most often triggered when a boundary has not been properly defined, or else when the XML file is corrupted.

To resolve the error, ensure that XML files are updated to the latest version (automatic in basic mode), that all boundaries are defined. If this does not help, try importing XML files from a previous solution. Envenio technical support can assist in finding the exact source of the error.


infinite loop NLNLNLN…

This solver error means that the CGNS file is corrupted. This can occur occasionally during file transfer, or by mishandling the file (e.g. copying or moving) when it is being written to by EXN/Aero.

To resolve this issue, it is best to re-import a CGNS file from another source. These sources can include:

  • The original CGNS file exported from the user’s meshing tool, or a fresh export from the mesher
  • A backup file, either kept manually by the user, or else backed up automatically using settings in ‘solver control >> simulation’

NOTE: please keep a copy of the corrupted file. Corruption in CGNS files tends to be localized to one or a few nodes. Even though EXN/Aero solver cannot launch a corrupted file, other generic and 3rd party CGNS editing tools can open the file directly and recover solution directories and settings. Please contact Envenio technical support for assistance.


Error: EXN-Aero cvc version( 1.380) and input CGNS file cvc version( 1.370) are incompatible!

This solver error should only occur in advanced mode if the user neglects to update XML structure files to the latest version.

Periodically, XML structure templates are updated to include new features. The solver expects a certain CGNS Version Control (CVC) number and will not attempt to launch a CGNS file that has not been updated

Updating happens can be triggered by the user, or when in basic mode will happen automatically. Updating causes no loss of data


Device real memory exceeded.../coef/mip0 185018483 > 183187236

In the example above, the solver indicates real memory, but may also indicate integer memory, double precision, or character.

This can be resolved in advanced mode by increasing the default memory allocation in the tree view under solver control >> resources.

This error can also be resolved by increasing the number of GPU devices or CPU core.


Error in creation of element neighbour list ... 16005, 15000

The CGNS format is an open source standard but different meshing tools do not apply the standard in an identical manner.

Envenio supports compatibility with CGNS export from Pointwise, ICEM, AnsysMesh and GridPRO.

This solver error occurs when a compatibility issue arises. Please contact technical support about your CGNS file.


Insufficient host memory. 18321465 bytes requested, 1655200 bytes free

This solver error occurs when the memory required for the current run is greater than the available memory. Install additional host RAM, reduce the size of the mesh or deactivate memory-intensive settings (e.g. double precision & second order transient)


Cannot have different quadrature levels in a multifluid DQMOM simulation....

The number of quadrature levels is not consistently defined across all Cell_Families. Set quadrature levels equal to clear this error


Cannot have mixed buoyant flow cells in simulation....
Cannot have mixed compressible/incompressible flow cells in simulation....
Cannot have mixed turbulent/laminar cells in simulation....
Cannot have mixed non-isothermal/isothermal cells in simulation....

The models referred to in the error messages above must be activated consistently across all cell families.

To resolve this solver error, activate models consistently to clear this error.


Stop, ill posed fine mesh size

EXNAero only allows maximum of 1024 points in each ijk direction for a structured block. To avoid this solver error, subdivide a mesh into a multi-block mesh so that number of points in any one block is less than 1024.


Not found for cell coordinate in calc_database_bc...

When reading inlet conditions from a 2D database, this solver error means some elements in this boundary cannot find the closest point in the database ‘inletDB’.

To clear this error, check that the database points (e.g. Mesh_X,Mesh_Y and Mesh_Z), are large enough to cover all elements at that boundary, or else increase the Bounding_Box_Extension to cover all elements.

Suggest Edit