Xsede at orion_workshop_feb2014

50 %
50 %
Information about Xsede at orion_workshop_feb2014

Published on March 3, 2014

Author: jtownsil



Title: XSEDE and Building a National/International Cyberinfrastructure

In this talk I will present a high level overview of XSEDE from the point of view of building a national/international scale cyberinfrastructure and the associated collaborations necessary. If nothing else, XSEDE has been an adventure in social engineering and I will comment on some of the aspects of these highly distributed and complex interactions. Along the way I will share some of the pitfalls, lessons learned and continuing challenges.

March 3, 2014 XSEDE and Building a National/International Cyberinfrastructure John Towns PI and Project Director, XSEDE Director, Collaborative Cyberinfrastructure Programs, NCSA

Who is this guy? • John Towns – PI and Project Director of XSEDE – Director, Collaborative eScience Program Office at NCSA – Co-Founder, Illinois Campus Cluster Program • Background – just a mid-western boy from Missouri • I don’t see a lot of folks from Missouri in the circles I am in – failed physicist • general relativity – failed computational scientist • numerical simulation of black hole spacetimes – failed networked applications guy • NLANR-DAST: National Laboratory for Applied Network ResearchDistributed Applications Support Team – but, a budding social engineer • Wearing my XSEDE hat for this discussion 2

Who is this guy, part deux? full disclosure/caveats/disclaimers • To be clear – I am a member of the Compute Canada Board of Directors – I am not here in that capacity • I am not in a position to speak for the CC Board – I may make comments regarding the CC Board if appropriate – I am open to comments from the community that I can carry back to the CC Board • My capacity here – PI and Project director of XSEDE – someone with a lot of experience in building institutional/regional/national/international cyberinfrastructure 3

Step-by-Step-Instructions • How to create an open cyberinfrastructure ecosystem in 1,200 easy steps .... 4

Boundary Conditions and Principles for XSEDE • XSEDE inherited TeraGrid environment • XSEDE inherited TG community and their expectations • Point of view has changed – not an HPC/CS/tech play – about productivity and creating the environment necessary to be productive • focus on the success of researchers! • Finally figured out that the project must define a solution that is designed to evolve! – technologically and organizationally! • Identify the greatest needs and start there – Don’t forget what you have learned – both good and bad! • Oh yea… the researchers don’t care about your existence (per se) – they care about access to resources, services and support 5

XSEDE – accelerating scientific discovery • XSEDE’s Vision: a world of digitally enabled researchers, engineers, and scholars participating in multidisciplinary collaborations to tackle society’s grand challenges • XSEDE’s Mission: to substantially enhance the productivity of a growing community of researchers, engineers, and scholars through access to advanced digital services that support open research 6

XSEDE’s Strategic Goals • Deepen and extend the use of the advanced digital research services ecosystem – deepen use by existing researchers, engineers, and scholars – extend use to new communities – prepare the current and next generation via education, training, and outreach – raise the general awareness of the value of advanced digital services • Advance the advanced digital research services ecosystem – create an open and evolving e-infrastructure – enhance the array of technical expertise and support services offered • Sustain the advanced digital research services ecosystem – assure and maintain a reliable and secure infrastructure – provide excellent user support services – operate an effective and innovative virtual organization 7

What is XSEDE? • An ecosystem of advanced digital services accelerating scientific discovery – support a growing portfolio of resources and services • advanced computing, high-end visualization, data analysis, and other resources and services • interoperability with other infrastructures • A virtual organization (partnership!) providing – dynamic distributed infrastructure – support services, and technical expertise to enable researchers engineers and scholars • addressing the most important and challenging problems facing the nation and world • A project funded by the National Science Foundation 8

XSEDE Factoids: high order bits • 5 year, US$121M project – plus US$9M, 5 year Technology Investigation Service • separate award from NSF – option for additional 5 years of funding upon major review after PY3 • No funding for major hardware – coordination, support and creating a national/international cyberinfrastructure – coordinate allocations, support, training and documentation for >$100M of concurrent project awards from NSF • ~140 FTE /~250 individuals funded across 20 partner institutions – this requires solid partnering! 9

Convenience requirements will always increase Each generation of users requires more convenience than the former: thus we must always be adding new layers of software while maintaining and extending existing reliability and capability. Change is the only Constant – Heraclitis 535BC-475BC No, his mind is not for rent To any god or government. Always hopeful, yet discontent, He knows changes aren't permanent, But change is. – Rush - Tom Sawyer 10

Some Important Lessons Learned • TeraGrid Project Final Report – – a “lessons learned” document from the TeraGrid experience • Some highlights that speak to partnership needs – “The work of managing such a large project needs to be carefully distributed to make sure all the work gets done, but also needs to be well coordinated, to make sure there is a consistent message.” – “TeraGrid lacked a structure to make contentious decisions and the authority to enforce those decisions even when some participants disagreed.” – “The heterogeneity of the user environments on the various computing systems was a difficult challenge for all stakeholders.” – “TeraGrid was able to sustain and scale-up projects through highly leveraged funding from other grants raised by the RPs, particularly in the education realm.” 11

Some Unexpected Challenges: XSEDE is a socio-technical ecosystem • Highly distributed organization – challenges in managing a project that involves staff at 20 partner institutions • A completely virtual organization – breaking new ground from an organizational structure and management point of view • Highly distributed engineering project – developing new methodologies to adapt traditional practices to the unusual context of XSEDE 12

Some Musings on Cyberinfrastructure Partnerships Work • Partnerships can be a Good Thing – leverage of resources, services are personnel – alignment of multiple efforts and funding steams to accomplish larger goals – ultimately facilitate greater productivity of researchers! • But there are practical pitfalls – socio-political issues often obscure the real goal – strong individual personalities can upset the applecart in a variety of ways – the importance of the mechanics of managing the partnership and maintaining accountability among the partners can never be underestimated 13

Strategies for Success in (Cyberinfrastructure) Partnerships • Keep your eye on the ball! – vision, mission, goals, KPIs, metrics – know why the partnership exists and how you will assess your collective success – all partners must support the vision • Know your stakeholders and their needs/interests/priorities – all partners are also stakeholders, not just the research community being supported • KISS principle – hierarchies of partnerships seriously complicate the management of the whole enterprise • coordination of efforts, reporting, accountability, flowing money, … – institutions must have the freedom to partner locally, regionally, nationally and internationally in order to pursue their objectives and priorities • Make sure you can make decisions! – decision-making processes that do not require 100% consensus but are also inclusive of the stakeholders are key 14


Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...