Published on February 21, 2014
HDF-EOS Development Current Status and Schedule Larry Klein, Alex Muslimov & David Wynne Emergent Information Technologies, Inc. February 27, 2002 larry @eos.hitc.com, email@example.com 1
HDF-EOS 2 • HDF4 based, storage format for EOS standard products. • Used operationally by MODIS, MISR, ASTER, Landsat, other EOS instruments • Will be used by EOS Aqua instruments. • Support for Grid/Point/Swath structures • HDF4, HDF-EOS 2 will be supported by NCSA/NASA for the fore-seeable future. 2
Current Archive Holdings • GSFC DAAC: MODIS L1, L2 Atmos./Ocean – 416 TB 5,182,000 granules (EOS Terra) • EDC DAAC: Landsat/ASTER/MODIS Land – 323 TB 3,551,000 granules • Langely DAAC: MISR – 141 TB 1,266,000 granules • NSIDC DAAC: MODIS L2, L3 – 10 TB 887,000 granules 3
HDF-EOS 5 • Based on HDF5, a complete rewrite of HDF4 with a different interface. – First released in 2000. • Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. – – – – Support same data structures Added prefix ‘HE5_’ to HDF-EOS 2 functions. Doesn’t preclude HDF5 functionality. Data Type changes, e.g. INT64 -> 4 H5T_NATIVE_LONG
HDF-EOS 2 -> HDF-EOS 5 •HDF-EOS 2 SWdefdatafield(swathID, fieldname, dimlist, numtype, merge) •HDF-EOS 5 HE5_SWdefdatafield(swathID, fieldname, dimlist, maxdimlist, numtype, merge, Maxdimlist*) *New HDF5 functionality passed through. This allows the user to set an upper limit to the size of the dataset. 5
HDF-EOS 5 • Provide a conversion tool: heconvert – HDF-EOS 2 -> HDF-EOS 5 – compliments HDF4 -> HDF5 conversion tool • HDF-EOS 5 will be used by EOS Aura – Standard format for profile data developed • Assume that HDF-EOS 2 producers will convert when PI’s determine that the time is right. 6
HDF-EOS 5 Functionality • • • • • • • • Basic File I/O Fill Values Compression Chunking/Tiling Swath Interface Grid Interface Point Interface Profile Interface • Global (File), Group & Local Attributes • External Data Files • Subsetting • Unix/Linux Support • Threadsafe Version • FORTRAN, C, C++ • General Table Interface (proposed) 7
Top Level of HDF-EOS 5 File Root -- “/” HDFEOS INFORMATION HDFEOS STRUC. METADATA ADDITIONAL SWATH GRID POINT Global (File) Attributes The new ADDITIONAL Group has global (file) attributes The new functionality is added to the EH(utility) interface. 8
Swath Structure Global Attribute <SwathName>:<AttrName> Group Attribute <DataFields>:<AttrName> Local Attribute <FieldName>:<AttrName> SwathName Data Fields Data Data Field.1 Field.n Profile Fields Profile Field.1 Profile Field.n Geolocation Fields Longitude Latitude Time Each Data Field can have Attributes and/or Dimension Scales Group CoLatitude Shaded Objects are implemented in a fixed way so the user doesn’t have direct access via the interface Attribute Data Set 9
ECS support of HDF5 • ECS toolkit V 5.2.8 supports HDF4 and HDF5 - based applications (eg. Metadata access – HDF-EOS 2.7 (HDF4.1r5) – HDF-EOS 5.1 (hdf5-1.4.1) • HDF5 and HDF4 must be compiled – HDF4 users not affected – HDF5 users must use PGS_MET_SDstart() and PGS_MET_SDend() 10
ECS support of HDF5 • Both flavors of HDF-EOS (HDF) part of ECS baseline. • ECS will not crack HDF5 - based files for near future. (contents transparent to archive) • Size limit is 2 GBytes 11
Applications • HE5View (HDF-EOS 5 browser) • Java Earth Browser (HDF-EOS 2 and 5 access) • HDF-EOS to GeoTIFF converter (HDF4 based only) • heconvert (convert HDF-EOS 2 Grid/Point/Swath to HDF-EOS 5 equivalents) • Access libraries and applications at: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit .html 12
Issues • Compound data types in FORTRAN77 • Chunking with variable length and compound data types • Only deflation compression method implemented. • File size limit in ECS - Is two Gbytes enough? 13
HDF and HDF-EOS Workshop V. February 26 ... HDF-EOS Development - Current Status and Schedule: n/a: ... HDF-EOS Data Applications at the Goddard Space ...
What is HDF-EOS?. An extension to HDF ... HDF-EOS Overview and Status . ... Development overview . Maintain current HDF-EOS interface to maximum extent .
HDF-EOS 2.6 Version Description Document ... 1.4 Document Status and Schedule ... HDF-EOS Development Configuration
1.4 Document Status and Schedule ... The current version contains ... These test programs are provided to aid the user in the development of software using ...
HDF-EOS Development Current Status and Schedule - Global Attribute SwathName : ... "HDF-EOS to GeoTIFF Conversion Tool" is the property of its rightful owner.
HDF-EOS Status, Related Tools and Issues - HDFEOS Status, ... HDF-EOS Development Current Status and Schedule - Global Attribute SwathName : ...
... which are implemented in the current version of ... 1.4 Status and Schedule ... to aid the user in the development of software using the HDF-EOS ...
This document is a Users Guide for HDF-EOS ... which are implemented in the current version of the library. ... Development Contract, March, ...