Published on December 8, 2016
1. Calibration and scenario generation of Hydrodynamics and Water Quality models over a Cloud Computing environment: INDIGO-DataCloud RIA-653549 Presented by Fernando Aguilar (IFCA-CSIC) firstname.lastname@example.org INDIGO-DataCloud WP2 Delft3D Days – 1st November 2016
2. Introduction • Framework: Collaboration with SME Ecohydros. Last year ROEM+. • Reservoir Hydrodinamic and Water Quality modelling. Cuerda del Pozo: water supply, water activities. • Previous work • Platform takes data from water: physical, chemical, biological, etc. Allows to know water status (data taken since 2010 aprox.) • Data visualization tool. Aims to alert authorities when the water quality is under the limits. • High Resolution Models require a powerful and scalable computing environment. November 2016 2
3. Case Study: Algae Bloom in a Water Reservoir • Research Community: LifeWatch (ESFRI) • Topic/Area: Biodiversity & Ecosystem research • Objective of the Case Study: • Monitor the evolution of the potential eutrophication of a Water Reservoir. • Two main objectives: • Scenarios: What if? Water flow, nutrients, etc. Range. • Calibration: Algae Parameters like Mortality, Respiration, etc. • Both requires a number of simulations. • Scalability and flexibility needed. • User Friendly Management: Input, Configuration, N outputs. • Cloud Computing is the answer! • INDIGO-DataCloud Project is developing all the solution needed. 3
4. The INDIGO-DataCloud Project • INDIGO = INtegrating Distributed data Infrastructures for Global ExplOitation • H2020 project, Apr 2015 – Oct 2017, 11M EURO • 26 partners in 11 European countries • Objectives: Develop a data/computing platform, targeting various scientific communities and deployable on hybrid (private or public) Cloud infrastructures • Website: https://www.indigo‐datacloud.eu INDIGO-DataCloud RIA-653549 4
5. INDIGO-DataCloud RIA-653549 INDIGO Architecture 5 • Enhanced features in • IaaS • PaaS • SaaS • 1st release MidnightBlue • Data Center Solutions • Data Solutions • Automated Solutions • High-level User Oriented Solutions
6. INDIGO Communities November 2016 6 Life Sciences: ELIXIR, INSTRUCT/WeNMR, EuroBioImaging Physical Sciences & Astronomy: CTA, LBT, WLCG Social Sciences & Humanities: DARIAH, DCH-RP Environmental Science: LifeWatch, EMSO, ENES Agile Methodology to satisfy User Requirements!
7. Solution Developed 7 Local OneClient
8. Solution Developed 8 • Access to all the services • Roles/Groups Definition • Federated
9. Solution Developed 9 • Distributed Storage Solution • Accessible by users and Machines • DropBox-like, but online • WebClient, POSIX, etc. • Sharable Environment • Input/Output
10. Solution Developed 10 • GitHub: Software Repository. Script for running the “job”. • DockerHub: SO, Delft3D Software and all dependencies ready to be deployed. • Ansible: Recipes for installing requirements (OneData).
11. Solution Developed 11 • User Interface • Generates TOSCA template from user configuration, including model params. • Sends json to Orchestrator. • Everything transparent for the user.
12. Solution Developed 12 • PaaS Orchestrator: Deploys the docker and manage the execution. • Deployment over many different IaaS Services. • Using Mesos/Chronos. • Running instances mount OneData, get the input and write the output. • Output available for the user.
13. Solution Developed 13 Local OneClient
14. Demo description • Testbed resources that will be used: Bari TestBed • Teams involved: INFN/Bari, PSNC, IFCA/CSIC • Prerequisites: application in Docker • Sequence of actions • Connect to IAM to access OneData. Input data upload. • Access to the Graphical User Interface (FutureGateway). Fill the form (OneData, Access, Sweep Parameter values). Submit. • The TOSCA template edited and sent to the orchestrator. • Check Deployment status. • After finishing, output accessible via OneData. Comparing models using Delft3D tools. • NOW WE SWITCH TO THE DEMO SCREEN… 14
15. INDIGO added value • Scalable (storage and computing) resources in the cloud to perform o(100-10000) tests… • …and share directly within the community • User Friendly interface to use cloud resources: • Final users only need to fill a form to submit a new simulation, avoiding the script edition or direct contact with the infrastructure (Supercomputer, Grid, Cloud) (very helpful for non IT experts). • First time we use a flexible and “universal” user authentication (quite relevant to collaborate with SMEs also) • Transparent access to shared large storage (OneData) 15
16. Conclusions • Cloud Computing environment is a recommendable framework to run a number of Delft3D simulations. • INDIGO-DataCloud solutions integration offers a user- oriented alternative: powerful and easy-to-use. • IT skills are not needed to use Computing Resources. • Distributed and Sharable Storage system available: no need to copy files and scientists can collaborate easily. • AAI Solution for accessing. • INDIGO-DataCloud solutions available for using (Open Source). 16
17. Thank you https://www.indigo-datacloud.eu Better Software for Better Science. 17 Fernando Aguilar (IFCA-CSIC) email@example.com