The Fourth Paradigm - Deltares Data Science Day, November 2014

48 %
52 %
Information about The Fourth Paradigm - Deltares Data Science Day, November 2014

Published on November 10, 2014

Author: Azure4Research



In this talk we describe how the Fourth Paradigm for Data-Intensive Research is providing a framework for us to develop tools, technologies and platforms to support actionable science. We discuss applications that take advantage of cloud computing, particularly Microsoft Azure, to realise the potential for turning data into decisions, knowledge and understanding. and

1. 휌 퐷푣 퐷푡 =−훻푝+훻∙휯+풇

2. Data Acquisition & modelling Collaboration and visualisation Analysis & data mining Dissemination & sharing Archiving and preserving Data-intensive Research

3. X-Info • Data ingest • Managing a petabyte • Common schema • How to organize it • How to reorganize it • How to share with others • Query and Vis tools • Building and executing models • Integrating data and Literature • Documenting experiments • Curation and long-term preservation The Generic Problems Experiments & Instruments Simulations Literature Other Archives facts facts facts facts Questions Answers

4. All Scientific Data Online •Many disciplines overlap and use data from other sciences. •Internet can unify all literature and data •Go from literature to computation to data back to literature. •Information at your fingertips – For everyone, everywhere •Increase Scientific Information Velocity •Huge increase in Science Productivity (From Jim Gray’s last talk) Literature Derived and recombined data Raw data

5. Gartner:

6. Manual Measurement Automated Measurement Sample Collection Historical Photographs Counting Ubiquitous Motes Aircraft Surveys Model Output Typing

7. Monitoring Collation Quality assurance Aggregation Analysis Reporting Forecasting Distribution Done poorly, but a few notablecounter-examples Done poorly to moderately, not easy to find Sometimes done well, generally discoverable and available, but could be improved Integration (I. Zaslavsky& CSIRO, BOM, WMO)

8. Web search: “open weather data azure”

9. Water depth map of London(~130km2). Storm eventof 60 minutes and 100 years return period


11. Parker MacCready: Univ. of Washington Rob Fatland:, Wenming Ye, NelsOscar, Microsoft Research

12. Numerical model of 3-D ocean currents and water properties •salinity, •temperature, •biogeochemistry Relies on external data sources: •Bathymetry •Wind and heating •Open Ocean BC’s •Tides •Rivers

13. Model Validation Comparisons are done to an extensive suite of in-situ observations •sea surface height 12 NOAA tide gauges •salinity and temperature over 2000 CTD casts from ECOHAB, RISE, DOE, NANOOS, Hood Canal, IOS, King County, and NOAA •velocity and moored S,T 7 coastal ADCP / CTD moorings from the ECOHAB and RISE projects, 2 moorings from IOS

14. Interactive 3-D Model Visualization using WorldWideTelescope, Narwhal and Layerscape www.layerscape.

15. EH4 32 m Figure from SA Siedlecki, UW/JISAO; Observations from Connolly et al., 2010 Validation: Dissolved Oxygen & Temperature

16. LiveOcean: System Architecture HPC linux150 cores Forecast NetCDFfiles LiveOcean Server •Post Processing •Pre-make .png“views” •Archive NetCDFfiles •API for web sites •Admin.js •Client.js Blob Storage: Forecast Copy Science User python Azure Table: Log Info Admin Website Client Website http://mappable.azurewebsites. net/liveocean/ Rivers USGS Atmosphere UW WRF Ocean HYCOM


18. Cloud Big data Aggregation Machine Learning Analytics

19. The Cloud democratizes access to scale & economies of scale

20. Commodity at Scale



23. Research Cloud Ecosystem


25. Use laptops & desktop computers Overwhelmed by data Finding analysis ever more difficult; sharing even harder

Add a comment


Karmen | 09/03/15
This is just the pefcret answer for all of us
Ellie Grace | 22/01/16
This blog (Y)

Related presentations

Habitos minerales

Habitos minerales

November 11, 2014

habitos minereles muy buenos

How organisms adapt and survive in different environment.

Aplicación de ANOVA de una vía, modelo efectos fijos, en el problema de una empres...

Teori pemetaan

Teori pemetaan

November 10, 2014

learning how to mapping

Libros: Dra. Elisa Bertha Velázquez Rodríguez

Materi pelatihan gis

Materi pelatihan gis

November 10, 2014

learning GIS

Related pages

The Fourth Paradigm - Deltares Data Science Day, 31 ...

Download The Fourth Paradigm - Deltares Data Science Day, 31 October 2014
Read more

Dsd int 2014 - data science symposium - 4th paradigm - a ...

Dsd int 2014 - data science symposium - 4th paradigm a research perspective, prof. arnold bregt, ...
Read more

Free Information Technology (IT) Books - Part 2 : PDF Download

Download free Information Technology eBooks ... This book will show you that project management isn't rocket science. ... “The Fourth Paradigm: Data ...
Read more

The fourth paradigm of scientific knowledge generation ...

The fourth paradigm of scientific knowledge generation ... Hey and Szalay have proposed a fourth – data-intensive science. ... (compared with 13 days on ...
Read more

Big Data, new epistemologies and paradigm shifts | Big ...

Abstract. This article examines how the availability of Big Data, coupled with new data analytics, challenges established epistemologies across the ...
Read more

Sentiment Analysis and The Fourth Paradigm MSE 2400 ...

Share Sentiment Analysis and The Fourth Paradigm MSE 2400 EaLiCaRA Spring 2014 Dr. Tom Way. Embed ...
Read more

November | 2011 | eScience @ Microsoft

... November 2011 WorldWide Telescope ... of days at Supercomputer 2011 we’ve been demoing ... talk for the release of The Fourth Paradigm ...
Read more

2014 afpclt paradigm - Documents -

System is processing data Please download to view 1 × Close Share 2014 afpclt paradigm.
Read more