Dataset Independent Subsetting

75 %
25 %
Information about Dataset Independent Subsetting
Technology

Published on February 24, 2014

Author: HDFEOS

Source: slideshare.net

Description

Source: http://hdfeos.org/workshops/ws01/presentations/UAH/matt.ppt

Dataset Independent Subsetting A Dataset Independent Subsetting Prototype http://minnie.cs.uah.edu/ Matthew R. Smith - matt.smith@msfc.nasa.gov Bruce Beaumont Dr. Sara J. Graves The University of Alabama in Huntsville Information Technology & Systems Laboratory UAH The University of Alabama in Huntsville 8-10 September 1997

Outline Context Purpose Design Functionality Web pages Future Summary UAH The University of Alabama in Huntsville 8-10 September 1997

Context NASA’s Mission to Planet Earth (MTPE) Earth Observing System (EOS) Data and Information System (DIS) EOSDIS Core System (ECS) Contractor: Hughes Information Technology Systems Design and Implement a prototype datasetindependent subsetter UAH The University of Alabama in Huntsville 8-10 September 1997

Subsetting? l Goal: to provide a science data user with only the data they request as quickly as possible. l Benefits science data users and data centers: - reduces analysis time by reducing amount of data - reduces time for data delivery - reduces resources (network, personnel, media, etc.) l Steps: - locate spatial, temporal, and spectral area of interest - extract data - re-assemble for distribution UAH The University of Alabama in Huntsville 8-10 September 1997

Design Web-based Dataset - independent HDF-EOS formatted data HDF-EOS software library Data types Swath Grid UAH The University of Alabama in Huntsville 8-10 September 1997

Functionality Front-end ( user interface ) Forms-based Web application - obtains subsetting selection criteria criteria file (ODL) Back-end ( subsetter ) C software using HDF-EOS and HDF libraries executed in batch mode UAH The University of Alabama in Huntsville 8-10 September 1997

User Interface File selection Parameters/channels Geographic bounding box Time range Subsampling stride Non-geolocated objects UAH The University of Alabama in Huntsville 8-10 September 1997

Summary of Current Functionality Subsetter Functionality Can subset grid and swath data Files may contain multiple grids and/or swaths; user may select any or all for subsetting Subset swath data on latitude/longitude and/or time Subset grid data on latitude/longitude Non-geolocated data may be included or excluded Output is HDF-EOS file using same data types “Back-end” runs as a batch job at archive center User may check status of job and/or cancel it E-mail sent to user when complete Data retrieved via FTP UAH The University of Alabama in Huntsville 8-10 September 1997

Restrictions Number of subsettable datasets limited by HDF-EOS library subsetting functions: Latitude must be “Latitude” or “Colatitude” Longitude must be “Longitude” Latitude and longitude must be FLOAT32 or FLOAT64 Latitude and longitude must be 1- or 2-dimensional Latitude and longitude must have identical dimensions Time must be “Time” Time must be FLOAT64 in TAI93 format Time must be 1- or 2-dimensional “Track” must be slowest varying dimension in geo fields Grid data must be in one of six supported projections UAH The University of Alabama in Huntsville 8-10 September 1997

Future Plans Relax requirements for latitude/longitude and time in swath datasets Provide Java-based GUI for area-of-interest selection Allow user to apply one subset specification to multiple input files Study integrating subsetter with a data visualization tool Study separating structural metadata from data UAH The University of Alabama in Huntsville 8-10 September 1997

What is Needed More test datasets in HDF-EOS format Additional support for modifications to HDF-EOS calls Accurate HDF-EOS documentation (internal and external) Functional Java map applet Resolution of metadata issues Publication of official metadata standards Name, content, and format of granule metadata UAH The University of Alabama in Huntsville 8-10 September 1997

Risks HDF-EOS not currently in widespread use HDF-EOS requirements for dataset-independent subsetting not widely known to data producers Legacy datasets are not in HDF-EOS format Converting to HDF-EOS may increase storage requirements Many datasets are on non-volatile media UAH The University of Alabama in Huntsville 8-10 September 1997

Summary A prototype Web-based dataset-independent subsetter has been developed by UAH. Allows spatial, temporal, and spectral subsetting and subsampling of HDF-EOS datasets Benefits science data users and data centers Great potential. but limited current use UAH The University of Alabama in Huntsville 8-10 September 1997

Add a comment

Related presentations

Related pages

Dataset Independent Subsetting - HubSlide

Dataset Independent Subsetting A Dataset Independent Subsetting Prototype http://minnie.cs.uah.edu/ Matthew R. Smith - matt.smith@msfc.nasa.gov Bruce ...
Read more

A Dataset-Independent Subsetting Prototype | ITSC

Title: A Dataset-Independent Subsetting Prototype: Publication Type: Conference Paper: Year of Publication: 1997: Authors: Smith, M, Beaumont, B, Graves, SJ
Read more

R subset by date - Stack Overflow

R subset by date. up vote 11 down vote favorite. 3. I have a dataset called EPL2011_12. I would like to make new a dataset by subsetting the original by date.
Read more

HEW: A Dataset-Independent Subsetter for HDF-EOS Files

HEW: A Dataset-Independent Subsetter ... Dataset parameter, e.g ... HEW uses a user-friendly web-based front-end to gather the user's subsetting ...
Read more

Introduction to the data.table package in R

Introduction to the data.table package in R Revised: September ... 3Subsetting a keyed data.tableby a n-column is consistent with subsetting a n-dimension ...
Read more

Brief SPSS Tutorial - DePaul University

Brief SPSS Tutorial Table of ... Subsetting a Dataset Part G: ... Move the desired y-variable to the Dependent box and the x-variable to the Independent box.
Read more

Subsetting data - Environmental Computing

Basics of subsetting in R. Check the structure after you have loaded the data with the str function. Now that we have some idea of the dataset we are ...
Read more

Matt Smith | ITSC

A Dataset-Independent Subsetting Prototype. Distributed Technologies in a Data Pool. Earth Science Markup Language. Earth Science Markup Language: ...
Read more

Subset.org - projects.itsc.uah.edu

University of Alabama in Huntsville releases HDF-EOS Subsetting Engine for HDF-EOS 5. The ... provides dataset-independent subsetting for data in HDF ...
Read more