The CFD General Notation System transition to HDF5

71 %
29 %
Information about The CFD General Notation System transition to HDF5

Published on February 17, 2014

Author: HDFEOS



The specific purpose of the CFD General Notation System (CGNS) project is to provide a standard for recording and recovering computer data associated with the numerical solution of the equations of fluid dynamics. The intent is to facilitate the exchange of Computational Fluid Dynamics (CFD) data between sites, between applications codes, and across computing platforms, and to stabilize the archiving of CFD data. The CGNS system consists of two parts: (1) the Standard Interface Data Structures, SIDS and (2) the ADF library. The "Standard Interface Data Structures" specification constitutes the essence of the CGNS system. While the other elements of the system deal with software implementation issues, the SIDS specification concerns itself with defining the substance of CGNS. It precisely defines the intellectual content of CFD-related data, including the organizational structure supporting such data and the conventions adopted to standardize the data exchange process. The SIDS are designed to support all types of information involved in CFD analysis. While the initial target was to establish a standard for 3D structured multi-block compressible Navier-Stokes analysis, the SIDS extensible framework now includes unstructured analysis, configurations, hybrid topology and geometry-to-mesh association. Although the SIDS specification is independent of the physical file formats, its design was targeted towards implementation using the ADF Core library. The "Advanced Data Format" (ADF) is a concept defining how the data is organized in the storage media. CGNS is currently transitioning to use HDF5 as the low level storage format. In this presentation I will give an overview of CGNS, SIDS and why the CGNS standards committee decided to switch to HDF5. Obstacles to the transition and a performance comparison will also be presented.

The CFD General Notation System transition to HDF5 Thomas Hauser Director Center for High Performance Computing Chair, CGNS steering committee

Outline • CGNS – What is CGNS – Structure • CGNS tools • Switching to HDF5 • Benchmarks – Current release 2.6 • ADF • HDF5 – Alpha release 3.0 • ADF • HDF5 2

What is CGNS ? • CFD General Notation System – Principal target is the data normally associated with compressible viscous flow (i.e. Navier-Stokes) – Applicable to computational field physics in general with augmentation of the data definitions and storage conventions • Objectives – Provide a general, portable and extensible standard for the storing and retrieval of CFD analysis data – Offer seamless communication of CFD analysis data between sites, applications and system architectures – Eliminate the overhead costs due to file translation and multiplicity of data sets in various formats – Provide free, open software – GNU Lesser General Public License 3

What is CGNS ? • Standard Interface Data Structures (SIDS) – Collection of conventions and definitions that defines the intellectual content of CFD-related data. – Independent of the physical file format • SIDS to ADF Mapping – Advanced Data Format • SIDS to HDF5 Mapping – Defines how the SIDS is represented in HDF5 • CGNS Mid-Level Library (MLL) – High level Application Programming Interface (API) which conforms closely to the SIDS – Built on top of ADF/HDF5 and does not perform any direct I/O operation 4

CGNS Steering Committee • Public forum made up of international representatives from government, industry and academia • Responsibilities – Maintain the software, documentation and CGNS web site – Ensure a free distribution of the software and documentation – Promote the acceptance of the CGNS standard • Organization – Meets at a minimum of once a year – Represented by an elected Chair person • currently Thomas Hauser, Utah State University – Governs by consensus – Welcomes participation of all parties, members or not 5

CGNS Steering Committee • Membership – 20 organizations – NASA Ames – US Air Force / AEDC – NASA Langley – CD ADAPCO – NASA Glenn – Intelligent Light – Boeing Commercial – Pointwise – Boeing – Rocketdyne – Aerospatiale Matra Airbus – Boeing Integrated Defense Systems – NUMECA – Pratt & Whitney – ONERA – ICEM CFD Engineering – Stanford University – Fluent, Inc. – Utah State University – Rolls-Royce Allison – ANSYS CFX 6

User Base • Registered Users – 591 users from more than 25 countries • CGNStalk (as of May 2003) – 153 participants from 20 different countries and at least 63 different organizations • SourceForge (last 2 years) – Average of 20 page views and 7.5 downloads per day • Known implementations – 13 commercial, 9 government, 5 industry, 3 academia 7

CGNS Main Features • Hierarchical data structure: quickly traversed and sorted, no need to process irrelevant data • Complete and explicit problem description • Standardized naming conventions • Unlimited internal documentation, and application specific data • Layered so that much of the data structures are optional • Database: universal and self describing • Based on a single data structure - node • The data may encompass several files through the use of links • Portable ANSI C software, with complete Fortran and C interfaces • Complete and architecture independent API 8


Top Level Structure Thomas Hauser

CGNS – Grid information • Grid coordinates and elements – – – – 1D, 2D and 3D support (physical and cell dimensions) Any number of structured and/or unstructured zones Cartesian, cylindrical and spherical coordinates systems Linear and higher-order elements (22 predefined element types) – 2D axisymmetry • Grid connectivities – 1-to-1 abutting, mismatched abutting, and overset (chimera) – Connectivity properties (average and periodic interfaces) 11

CGNS - Boundaries • Boundary conditions – Simple or complex boundary conditions with predefined identifiers – Any number of Dirichlet or Neumann conditions may be specified globally or locally on a boundary condition patch – Boundary patch normals, area and wall function properties 12

CGNS - Solutions • Governing flow equations – General class of flow equations – Gas, viscosity, thermal conductivity, thermal relaxation, chemistry, turbulence, and turbulence closure models • Solutions – – – – Vertex, cell, face or edge centered with rind (ghost points/cells) Any number of solution variables Predefined identifiers for solution variables Generic discrete data (not typically part of the solution) • Time-dependent flows – Time-accurate or non-time-accurate – Rotating, rigid motion or arbitrary motion grids – Storage of base and/or zone iterative data 13

CGNS - Data • Physical data – Data class: dimensional, normalized, or non-dimensional – Data conversion factors – Dimensional units: mass, length, time, temperature, angle, electric current, amount of a substance, and luminous intensity – Dimensional exponents: powers of base units • Auxiliary data – – – – – Global and/or local convergence history Reference state variables Gravity and global integral data Arbitrary user-defined data Textual data for documentation and annotations 14

Zone_t Node Thomas Hauser

GridCoordinates_t Node 16 Thomas Hauser

Standardized Names for Flow Solution 17

CGNS • Partial read and write – Partial read and write for grid coordinates, elements and solution variable • Families – Provides a level of indirection to allow mesh to geometry associations – Boundary conditions may be applied on families – Links mesh surfaces to one or more CAD entities – Rotating coordinates and complex boundary conditions 18

CGNS • Electromagnetics – Electric field, magnetic field and electrical conductivity models added to the governing flow equations – Voltage, electric and magnetic field strengths, current density, electrical conductivity, Lorenz force and Joule heating added to list of solution identifiers 19

CGNS Tools • ADFviewer – Views and/or edits ADF/CGNS files. – May create, delete or modify nodes – Nodes are displayed in a Windows-like collapsible tree – Additional utilities may be accessed from the menus – Configurable menus – Written in Tcl/Tk Thomas Hauser

CGNS Tools • CGNSplot – Viewer for CGNS files – Displays zones, element sets, connectivities, and boundary conditions – Written in Tcl/Tk with OpenGL – Runs standalone, or may be called from ADFviewer Thomas Hauser

CGNS Tools • File conversion – Convert Patran, PLOT3D and Tecplot files to CGNS – Convert CGNS files to PLOT3D and Tecplot • CGNS file manipulation – Data conversion utilities for modifying the solution location (vertex or cell-center), solution variables (primitive or conservative), and data class (dimensional or normalized) – Subset extraction and interpolation • CGNS bindings – – – – Tcl/Tk interface to ADF and MLL PyCGNS: Python interface to ADF and MLL ADFM: in memory representation of ADF trees CGNS++: C++ interface to ADF and MLL 22

CGNS Tools • Other utilities – CGNScheck: checks CGNS files for valid data and conformance to the SIDS – ADFlist: lists ADF/CGNS file tree structure and node data – ADF_Edit: command-line based interactive browser/editor for ADF/CGNS files – CGNS_readhist: reads a CGNS file and writes convergence history to a formatted file. – FTU (File Transfer Utility): converts to and from PLOT3D, and has a text-based menu allowing the manipulation of a CGNS base – CGNS Viewer: ADF/CGNS file editor/viewer with a GUI using the GTK+ library 23

HDF5 Interface • Implementation – no change to MLL • Advantages – HDF5 used in many other applications and tools – Ability to use HDF tools with CGNS – Parallel I/O using MPI • Disadvantages – File sizes are 2 to 3 times larger for large number of zones – I/O times are generally 2 to 3 times slower, but may be up to a order of magnitude for a large number of nodes 24

HDF5 interface • Current Status – HDF5 Task Force set up to further evaluate implementation – Added as option to CGNS with conversion routines • Current implementation – HDF 1.8 fixed the link problem – ADF/HDF5 transparent to the user • Problem – ADF stores data on disk with column major order – HDF5 stores data on disk in C storage order • User confusion


Serial performance - CGNS 2.5 • • • • ADF Better performance for large number of zones 27 HDF5 Performance penalty for large number of zones

Serial performance - CGNS 3.0 • • • • ADF Better performance for large number of zones 28 HDF5 Performance penalty for large number of zones

Serial performance - 1000 zones • • • Performance comparison XML better for small data sizes ADF and HDF5 much better for larger sizes 29

Serial performance - 1000 zones • • Size comparison ADF and HDF5 fairly similar especially for large problem sizes 30

Conclusion • CGNS – Standard for storing CFD data – Standardization of names – Storage layers independent of standard • ADF • HDF5 • Currently switching to HDF5 as main storage format – Transparent to users – Investigating performance problems – Parallel I/O next 31

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

PPT - The CFD General Notation System transition to HDF5 ...

The CFD General Notation System transition to HDF5. Thomas Hauser Director Center for High Performance Computing Chair, CGNS steering committee thomas ...
Read more

⚡Presentation "The CFD General Notation System transition ...

The CFD General Notation System transition to HDF5 Thomas Hauser Director Center for High Performance Computing Chair, CGNS steering committee thomas ...
Read more

The CFD General Notation System transition to HDF5

Home; Documents; The CFD General Notation System transition to HDF5; of 31 ×
Read more

Switch to HDF5 - CFD General Notation System

CFD General Notation System Switch to HDF5 : CGNS site pages ... The CGNS implementation of ... This format was based on a common file format system ...
Read more

Recent Updates to the CFD General Notation System (CGNS)

Recent Updates to the CFD General Notation System ... (for HDF5 files only), ... CGNS CFD General Notation System,
Read more

CFD General Notation System

... January 5-6, 2008: "Effective Use of the CFD General Notation System (CGNS) ... (Revision 2) of the CGNS library released. The HDF5 interface ...
Read more

CFD General Notation System cgns Bruce Wedan ANSYS/ICEM ...

CFD General Notation System Bruce Wedan ANSYS/ICEM CFD. Presentation Overview. What is CGNS ? History of CGNS CGNS Steering ...
Read more


... (CFD) General Notation System ... your HDF5? Subscribe to the HDF Blog via Email. Enter your email address to receive notifications of new HDF Blog ...
Read more

Parallel and Large-scale Simulation Enhancements to CGNS ...

Parallel and Large-scale Simulation Enhancements to CGNS. ... (CFD) General Notation System ... The new HDF5 APIs have the benefit of packing multiple ...
Read more