Published on February 18, 2014
nub - npoess user block tool Richard Ullman NASA GSFC
nub • nub (nŭb) n. – 1. A protuberance or knob. – 2. A small lump. – 3. The essence; the core: the nub of a story. – [Variant of knub, probably from Low German knubbe, from Middle Low German, variant of knobbe.] • (selected from http://www.answers.com/topic/nub) 2
NPOESS user block • For NPOESS files, the user block is ASCII text written in XML that provides a quick-look into the metadata attributes contained in the HDF5 file. – The content of the user block includes metadata that is attached to the NPOESS: • / (root) group • <collection_shortname> group, and • <collection_shortname>_Aggr group. 3
nub_Motivation • When evaluating NPOESS sample data last year, I wished to examine the user block, as a stand-alone item, but I found that the h5unjam tool to be expensive – h5unjam reads an HDF5 file and writes two output files • 1. a file containing the content of the user block • 2. a file containing the H5 data without the user block. – The second part was unnecessary for my purposes, but because the NPOESS files are large, it is expensive. • So .. I wrote nub … 4
nub_Design • Design criteria: – “c” program (that’s all I know how to write) – Efficient - do no ‘EXTRA’ work. – Command line operation, with Unix-like options. – Pipe-able. (stdin/stdout) – Assume user block is text. – Work only on the front user block. – Complete set of functions to manipulate the user block: • List, Create, Overwrite, Append, Delete, Get size • Requirements creep … – Complete replacement for h5jam/h5unjam/tellub – No reliance on h5 library (no need). • Availability: – Applied for NASA Open Source release. All paperwork completed. Waiting for final approval. – Evaluation copy available to anyone interested in NPOESS. 5
nub operates on the front user block ONLY signature: “211HDFrn32n” user block user block HDF5 HDF5 sizeof( user_block) = 2^n, where n>8, (or n=0) 6
nub file.h5 signature: “211HDFrn32n” content,content/0/0/0/0/0/0/0/0 content,content/0/0/0/0/0/0/0/0 --tell -b HDF5 HDF5 --tell -s --tell -a • Tell: name comes from HDF5 internal tool “tellub” • This is the nub default operation. – Count the size of the user block content [-b], the size of the user block [-s], and the size of the complete H5 file [-a] in bytes. – No options given is equivalent to: nub --tell -bsa file.h5 – Output is similar to wc 7
nub -l file.h5 signature: “211HDFrn32n” file.txt file.txt content,content/0/0/0/0/0/0/0/0 content,content/0/0/0/0/0/0/0/0 HDF5 HDF5 • List: list the content of the user block write to stdout, or if -f write to the given file 8
nub -w file.h5 content,content/0/0/0/0/0/0/0/0 content,content/0/0/0/0/0/0/0/0 signature: “211HDFrn32n” user_file.txt user_file.txt HDF5 HDF5 • Write: write text content to the user block – append by default, or clobber (replace) – fit into existing space (by overwriting only null chars) by default, or “jam” (extend the block) – “kram” option shrinks user block into smallest that will fit the content – read from stdin, or use the file specified on the -u switch 9
nub_ compatibility • nub will respond to the same command line options that are used with h5jam and h5unjam. • “nub -J” is an explicit substitute for h5jam • “nub -U” is an explicit substitute for h5unjam • “nub -T” is an explicit substitute for tellub • Or, use just “nub” as a substitute for either and the code will decide which of the two is most appropriate based on the flags provided. 10
nub_test • Results of nub as a substitute for h5jam, h5unjam in the standard hdf5 tools regression tests … • 68 PASSED • 6 FAILED – fails are due to difference in concept of concatenation. – nub appends to the end of the content of an existing user block. – h5jam appends to the end of the existing user block. – User blocks are of size=2^n, where size > 256 and is big enough to fit the content. So, if user block content is 513 bytes, the size of the user block is 1024 bytes, including 511 bytes of NULL padding. – nub will overwrite the NULL padding, h5jam leaves the padding in place. 11
npoessuser block tool nub npoessuser block tool Richard Ullman NASA GSFC Richard Ullman NASA GSFC smalllump. knub,probably from Low German knubbe, from ...
nub - NPOESS User Block Tool: ... information is stored in a section of the HDF5 file called the "user block".This command line tool is designed to ...
The document was removed. Please view another documents! 11 × Close Share nub - npoess user block tool
NUB: NPOESS User Block Tool† by Richard Ullman (Code 423) Safe to Mate† by Moses McCall (Code 580) Issued Patents: 1 . U.S. Patent No . 7,060,968 ...
The technologies made available in FY06 are listed here. These technologies can provide our partners with ... NUB: NPOESS User Block Tool
U.S. Government Purpose Release nu-Anomica ... http://ti.arc.nasa. gov/opensource/projects/nu-anomica/ Open Source nub-NPOESS User Block Tool GSC ...