Darwin Core Archive (DwC-A) validation: A New Collaborative Effort

56 %
44 %
Information about Darwin Core Archive (DwC-A) validation: A New Collaborative Effort
Software

Published on November 2, 2014

Author: kristgen

Source: slideshare.net

1. Darwin Core Archive (DwC-A) validation: A New Collaborative Effort Christian Gendreau, Université de Montréal / Canadensys David P. Shorthouse, Université de Montréal / Canadensys Marie-Élise Lecoq, GBIF France Tim Robertson, GBIF

2. Darwin Core Archive (DwC-A) DarwinCore standard does not impose strong rules on the content associated with any DarwinCore terms.

3. Current GBIF DwC-A Validator Original goal “… test Darwin Core Archives as specified in the Darwin Core Text Guide.” http://tools.gbif.org/dwca-validator/

4. Current GBIF DwC-A Validator Original target DwC-A are simple and can be created using simple custom scripts. “… make sure GBIF and others can read the information as expected.”

5. Current GBIF DwC-A Validator • Validates archive structure • Offer web presence – Report viewer – API

6. Next GBIF DwC-A Validator? New goal Extends validation to the content of the archive https://github.com/gbif/dwca-validator

7. Current content validators • Atlas of Living Australia sandbox • VertNet – Spatial quality • GBIF Spain – Darwin Test • Encyclopedia of Life – dwc-validator • Scratchpads – dwca-validator • GlobalNames – dwc-archive ruby gem • … much more See Appendix 1 for links

8. What we need? • Accommodate different scopes • Configuration/customizations – Use more knowledge when available • Web access (page and API)

9. Scopes • Data entry • Desktop software – Scientific Work Flow – Statistical software • Integrated Publishing Toolkit (IPT) • National nodes • Aggregators

10. Configuration/Customization • Where the validator will be used? • Can we provide more information? – e.g. I know all the dates in my file should be ISO

11. Components • Library • Web • Extension Support

12. Library • Define structure for validation process • Provide a validation framework enabling sharing • Close to DarwinCore specification

13. Web • Web page to submit archive or URL • Report viewer • API

14. Extension Support • Include domain knowledge • Propose interpreted data

15. Internals • Validation types – Structure • Metadata – Records : Rows • Fields data (e.g. date, coordinates) – Records : Columns • ID uniqueness

16. Internals – Record level • Validation chain – Composed by chain elements – Possible parallelism

17. Internals – Record level • Immutable Chain element – Self contained • Never relies on another chain element – Ordering independent • Same behaviour wherever the element is used in the chain But what if I need really ordering?

18. Internals - Composition • Composed chain element • Exposed as one chain element

19. Composition example • Mandatory Latitude/Longitude – Check record completion on lat/long – Check decimal lat/long value

20. Configuration example • Select mandatory DarwinCore terms – scientificName must be provided • Restrict bounding box – decimalLatitude and decimalLongitude must be between

21. Customization example • Apply your own controlled vocabulary – Use your own dictionary for a term – ControlledVocabularyEvaluationRule

22. Extension Example • Suggester, link to narhwal-processor – Suède –> ISO 3166-2:SE – URI –> http://sws.geonames.org/2661886

23. Collaborative • Share configuration • Share customization (dictionary) • Implement new reusable component – e.g. validation on specific Dwc-A extension

24. Collaboration • Where to go? – https://github.com/gbif/dwca-validator • Who can contribute? – Everyone • What is needed? – Ideas, constructive comments – Code review, feedback

25. Project status • Not yet released • Command line interface available Follow the project on GitHub

26. Acknowledgments

27. Special thanks • SiB Colombia • SiB Brazil • Peter Desmet • John Wieczorek • Dag Endresen • …

28. Appendix 1 DwC Content validators Atlas of Living Australia sandbox http://sandbox.ala.org.au/datacheck/ VertNet – Spatial quality Displayed on occurrence pages at http://portal.vertnet.org/search GBIF Spain – Darwin Test http://www.gbif.es/darwin_test/Darwin_Test_in.php Encyclopedia of Life – dwc-validator http://services.eol.org/dwc_validator/

29. Appendix 1 - continue Scratchpads – dwca-validator https://github.com/edwbaker/dwca_validator/ GlobalNames – dwc-archive ruby gem https://github.com/GlobalNamesArchitecture/d wc-archive

Add a comment

Related presentations

Speaker: Matt Stine Developing for the Cloud Track Marc Andressen has famou...

This presentation explains how to develop a Web API in Java using (JAX-RS or Restl...

1 App,

1 App,

November 10, 2014

How to bring innovation to your organization by streamlining the deployment proces...

Cisco Call-control solutions can handle voice, video and data

Nathan Sharp of Siemens Energy recently spoke at the SAP Project Management in Atl...

Related pages

Darwin Core Archive (DwC-A) validation: A New ...

Darwin Core Archive (DwC-A) validation: A New Collaborative Effort Christian Gendreau, Université de Montréal / Canadensys David P. Shorthouse, Université.
Read more

Darwin Core Archive (DwC-A) validation: A New ...

Darwin Core Archive (DwC-A) validation: A New Collaborative Effort Darwin Core Archive (DwC-A) validation: A New Collaborative Effort Christian Gendreau ...
Read more

Rules & Ontology for Validation Effort - Documents

Rules & Ontology for Validation Effort. by les. on ...
Read more

A Collaborative Effort | LinkedIn

View 19688 A Collaborative Effort posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn. ... Looking for a new ...
Read more

Collaborative Effort - Documents

Collaborative Effort; Download. of 31
Read more

1 FaCES Clinic and Evaluation A Collaborative Effort ...

1 FaCES Clinic and Evaluation A Collaborative Effort…. Slide 2. 2 Agenda The Need & Recommendations FaCES (Foster Care Evaluation Services) ...
Read more

Christian Gendreau (@christgendreau) | Twitter

... do you have a R package for reading DarwinCore Archive? ... Darwin Core Archive (DwC-A) validation: A New Collaborative Effort #tdwg14 ... New @PLOSONE ...
Read more

Dwc | LinkedIn

Excerpts from the Hot New DWC Ethics Report. ... Why is DWC a strategic location? ... Darwin Core Archive (DwC-A) validation: A New Collaborative Effort.
Read more