The Homogenization and Reporting of Groundbased Atmospheric Datasets for the Validation of Earth Observing Satellite Instruments - the Aura and Envisat validation approach using HDF

0 %
100 %
Information about The Homogenization and Reporting of Groundbased Atmospheric Datasets for...
Technology

Published on February 18, 2014

Author: HDFEOS

Source: slideshare.net

Description

The geophysical validation of satellite borne atmospheric chemistry instruments requires a large number of independent observations by a variety of in-situ, remote-sensing and satellite instruments. In the case of the Aura and Envisat missions more than 300 instruments have been formally included in the validation efforts.

In order to extract maximum information from the "ground-truth" measurements, these independent validation datasets must be readily accessible to all investigators in an simple format with standard metadata, variable naming and data structures.

Beginning in 2000, NASA and ESA have been collaborating under the auspices of the CEOS Working Group Calibration/Validation (WGCV) Atmospheric Chemistry Sub Group (ACSG) on the file format definition, metadata formulations, and variable definitions for validation datasets.

This standard, known as the AVDC/Envisat HDF format, is the basis of the Aura Validation Data Center (AVDC) and the Envisat Cal/ Val Data Center implementations. Recently, the AVDC/Envisat HDF formulation has been accepted as the new reporting standard of the Network for the Detection of Atmospheric Composition Change (NDACC), an international network of high quality remote sensing stations, and for the next ESA GMES Earth Observing missions. The concept behind the AVDC/Envisat HDF reporting standard, its implementation and future applications will to be presented in detail.

The Homogenization and Reporting of Groundbased Atmospheric Datasets for the Validation of Earth Observing Satellite Instruments - the Aura and Envisat validation approach using HDF B. R. Bojkov, NASA-GSFC (USA) R. M. Koopman, ESA-ESRIN (I) M. De Maziere, BIRA-IASB (B) I. Boyd, NIWA (NZ)

Outline • Context (and some history) • Implementation for Envisat and Aura • Future activities

• Why did we do this for atmospheric sciences? (from the perspective of validation)

Near-real-time ozonesondes • Ensuing from European Arctic campaigns in the 1990’s • ~35 stations reporting » operational NRT and scientific • All stations uses NASA-Ames » “simple” ASCII • More than 18 file variations » nomenclature and formulations • Clear need of homogenization

The problem is accentuated: • Different measurement networks » LIDAR, MWR, FTIRs, UV-Vis instruments, etc. • Different platforms » Groundbased, aircraft, balloon, ship, in-situ, etc. • Different agencies » “Dialog” across timezones, reporting facilities, etc.

So, in 1996/1997… • Near impossible to use effectively for research or tools/RDB/web implementations • Need to resolve this “file reporting ” problem by investigating/understanding the state of affairs in the field • First objective: satellite validation of atmospheric chemistry instruments » up and coming ESA and NASA missions

COSE • COSE - Compilation of atmospheric Observations in support. of Satellite measurements over Europe • European Commission (EC) funded project • Consortium of 25 groundbased and satellite teams • Timeline: 1998-2000

COSE (2) • Approach: » Investigated existing file formats, usage, community needs in consultation with the different stake-holders • Recommendations: » Defined fixed file formulation: HDF4 (SDS) » Rigid metadata, including strict guidelines for attributes requirements, variable naming, etc. » Make files truly homogeneous and self explanatory » Final document by Bojkov et al., 2002 (available through AVDC)

Basic HDF4 SDS file layout • Global attributes: 31 attributes » Data source / contact information » Dataset contents / location / … » File generation date / version / caveats / … • Attributes for each SDS: » » » » 19 attributes Variable description / notes / caveats Dependencies / dimensions Units / SI conversion factor / Valid min. & max. / Fill values Display attributes (label, axis, …)

G lo ba la ttr ib ut es Va r ia bl e at tr ib ut es

Variable naming convention • Variable name construction: 3 part construction VARIABLE_NAME + VARIABLE_MODE (+ VARIABLE_DESCRIPTOR) • Example: » The variable name for O3 measured by an O3sonde and a Brewer spectrometer are: O3_CONCENTRATION O3_VERTICAL.SOLAR O3_VERTICAL.ZENITH » for the ozonesonde for the Brewer in direct sun mode for the Brewer in Umkehr mode The RMS for a FTIR ozone measurement can be expressed as: O3_VERTICAL.ZENITH_UNCERTAINTY.RMS

Similar conventions for • Attribute entries • Instrument naming • File names • …

First Implementation: ESA • Spring 2000: ESA Envisat mission was is it’s final validation planning: » 10 instruments, focus on land, atmosphere and oceans. • ESA required: » Fully relational database for calibration and validation » File “QA/QC” on all incoming files » Specifically COSE concept, including HDF 4, to be implemented for atmosphere and ocean instrument validation

Envisat Cal/Val • Implemented in October 2000 » Meets all ESA requirements, including QA and the RDB » 200+ users, intercontinental participation » Backbone of the coordinated validation efforts of 3 atmospheric and 2 ocean instruments » Data sources collected from ship-based to satellites » Extended to other Envisat/ESA validation activities (2004) • One issue: heterogeneous reporting - the ozonesonde problem was back!

2004-present • EOS-Aura validation activities • The Network for Detection of Atmospheric Composition Change (NDACC - former NDSC)

Aura Validation Data Center • Support the NASA EOS-Aura mission » 4 atmospheric instruments » troposphere to mesosphere measurements » Share validation communities with Envisat • AVDC uses the same concept/approach as Envisat » Maintain Envisat files compatibility • Operational since October 2004 » ~300 users - many new participants » ~2TB validation data

AVDC modifications • Refined file and metadata requirements through thorough analysis of Envisat Cal/Val in early 2004 » Eliminate common user errors and misconceptions » Added HDF5 implementation (user request due to HDF5 satellite data) • Implemented rigid measurement reporting templates » Collaborative effort with NDACC » Measurement specific and defined by expert community » Results in truly homogeneous files • Document describing changes by Bojkov et al., 2006 available through AVDC web site

Future of the AVDC/Envisat implementation • AVDC refinements are being implemented into Envisat Cal/Val • AVDC/Envisat format being phased into NDACC network » LIDAR, MWR, FTIR completed • AVDC concept to be extended to NASA A-train (and most probably NPP wrt validation data) • Format is ESA requirement for ESA/EC GMES program » To be extended to land and radar altimetry • NASA and ESA to remain synchronized

Closing remarks • This type of work takes time » Began in 1998, and will be ongoing • Requires proactive involvement by all parties » Validation teams, instrument PIs, satellite teams, … • Requires rigid guidelines - down to the measurement level • Once implemented, it is extremely useful for all involved as experienced for the Envisat and Aura missions when validation relies on very different data sources

bojan.bojkov@gsfc.nasa.gov http://avdc.gsfc.nasa.gov/

Add a comment

Related presentations

Related pages

PPT - B. R. Bojkov, NASA-GSFC (USA) R. M. Koopman, ESA ...

The Homogenization and Reporting of Groundbased Atmospheric Datasets ... Earth Observing Satellite Instruments ... Envisat validation approach using HDF ...
Read more

HDF and HDFEOS Workshop - PowerPoint PPT Presentation

The Homogenization and Reporting of Groundbased ... UV-Vis instruments, etc ... The Homogenization and Reporting of Groundbased Atmospheric ...
Read more

BEAT - The Basic ENVISAT Atmospheric Toolbox - Documents

MIPAS Quality assessment of ENVISAT Atmospheric mission: implication for the scientific user community mipas@dpqc.org The monthly reports for each instrument.
Read more

Agenda - HDF and HDF-EOS Workshop X

... Reporting of Groundbased Atmospheric Datasets for the Validation of Earth Observing Satellite Instruments ... Aura and Envisat validation approach ...
Read more

Status of the Atmospheric Chemistry Instruments onboard ...

... Reporting of Groundbased Atmospheric Datasets for the Validation of Earth Observing Satellite Instruments ... Aura and Envisat validation approach ...
Read more

1 ENVISAT Validation using the DLR Falcon 20 Research ...

Home; Documents; 1 ENVISAT Validation using the DLR Falcon 20 Research Aircraft Overview over the Validation Activities Prepared by Andreas Fix, 04/11/02.
Read more

Envisat | LinkedIn

View 1159 Envisat posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn. LinkedIn Home What is LinkedIn? Join Today
Read more

Page 1 Envisat Validation Workshop, ACVT-GBMCD, MIPAS O 3 ...

Page 1 Envisat Validation Workshop, ACVT-GBMCD, MIPAS O 3 12/12/2002 Envisat Validation Workshop Atmospheric Chemistry Validation Team Ground-Based ...
Read more

Page 1 Envisat Validation Workshop, ACVT-GBMCD, SCIAMACHY ...

... ACVT-GBMCD, SCIAMACHY scient. prod. 12/12/2002 Envisat Validation Workshop Atmospheric Chemistry Validation Team Ground-Based. ...
Read more