grid 06talk

40 %
60 %
Information about grid 06talk
Product-Training-Manuals

Published on June 19, 2007

Author: Mahugani

Source: authorstream.com

Slide1:  'How are Real Grids Used?' The Analysis of Four Grid Traces and Its Implications IEEE Grid 2006 Alexandru Iosup, Catalin Dumitrescu, and Dick H.J. Epema PDS Group, EEMCS, TU Delft Hui Li and Lex Wolters LIACS, U. Leiden The Analysis of Four Grid Traces and Its Implications 'How are Real Grids Used?' Outline:  Outline Why 'How are Real Grids Used?'? What do we expect from Grids? Grid Computing works! Our approach Grid Traces Summary Towards 'How are Real Grids Used?'? System-wide characteristics VO, group, and user characteristics Evolution of grid environments over time Grid systems performance The Answer to 'How are Real Grids Used?' Applications, Future Work, Conclusion Outline:  Outline What do we expect from Grids? The Analysis Answers for 'How are Real Grids Used?' Final Words Slide4:  What do we expect from Grids? What do we expect from grids? (1/3):  What do we expect from grids? (1/3) Goal: Grid systems will be dynamic, heterogeneous, very large-scale (world), and have no central administration The all users are interested assumption: Grids support many VOs, many users, and they all use the grid intensively The 0-day learning curve assumption: a grid must support many different projects in a way that facilitates an immediate jump to productivity The theory sounds good, but how about the practice? Let’s look at Real Grid Traces! What do we expect from grids? (2/3)Grids vs. Parallel Production Envs.:  What do we expect from grids? (2/3) Grids vs. Parallel Production Envs. The HPC successor assumption: the Grid’s typical user comes from the HPC world, and the typical workload can be described with well-known models of parallel production environments workloads We will have from the beginning lots of parallel jobs We will have from the beginning lots of power-of-2 sized jobs We will handle from the beginning tightly-coupled parallel jobs The infinite power assumption: the Grid can deliver any computing power demand, and in particular will offer at least computing power comparable to traditional Parallel Production Environments Promising in theory, but how about in practice? Let’s look at Real Grid Traces! What do we expect from grids? (3/3)Grid Computing Works!:  What do we expect from grids? (3/3) Grid Computing Works! The Power of Sums is Greater Than the Sum of Powers assumption: by coupling individual environments the Grid will provide better performance than by using the individual environments alone Higher throughput Shorter wait times Shorter response times Same or lower jobs slowdown There is a need for the Power of Sums assumption: institutions have the will and the need to couple their environments into a larger, grid-based, environment I’m sure it’s true, but what happens in practice? Let’s look at Real Grid Traces! Slide8:  The Analysis The Grid Traces: LCG, Grid3, TeraGrid, the DAS:  The Grid Traces: LCG, Grid3, TeraGrid, the DAS Production Grids: LCG, Grid3, TeraGrid Academic Grid: the DAS (two overlapping traces: local, and shared as Grid platform) Features: long traces (6+ months), active environments (500+K jobs per trace, 100s of users) Trace Analysis:System-Wide Characteristics:  System utilization is on average 60-80% for production Grids, and andlt;20% for academic Grids Average job size is 1 (that is, there are no [!] tightly-coupled, only conveniently parallel jobs) Trace Analysis: System-Wide Characteristics Trace Analysis:VO, Group, and User Characteristics:  Trace Analysis: VO, Group, and User Characteristics Top 2-5 groups/users dominate the workload Top groups/users are constant submitters The week’s top group/user is not always the same Trace Analysis:Evolution of Grid Environments:  Trace Analysis: Evolution of Grid Environments Everything evolves! The infrastructure, the projects, and the users Similar submission patterns Stages learning automated tool production phase cool-down phase Trace Analysis:The Performance of Real Grid Systems:  Trace Analysis: The Performance of Real Grid Systems With the notable exceptions of average waiting time, and of the occurrence of errors, the studied grid systems perform in normal ranges predicted by simulation work [Shan et al.’03, Ernemann et al.’04]. The AWT is much higher in real grids than expected (see paper) The occurrence of errors in real grids seem much higher than expected (future work to prove) Running jobs Waiting jobs Trace Analysis: Grids vs. Parallel Production Systems:  Trace Analysis: Grids vs. Parallel Production Systems From A. Iosup, D.H.J. Epema, C. Franke, A. Papaspyrou, L. Schley, B. Song, R. Yahyapour, On Grid Performance Evaluation using Synthetic Workloads, JSSPP’06: Similar CPUseconds/Year, Higher job arrivals spikes (5x) Grid Systems Slide15:  Answers for 'How are Real Grids Used?' What did we expect from grids? (1/3):  What did we expect from grids? (1/3) The all users are interested assumption: Grids support many VOs, many users, and they all use the grid intensively The 0-day learning curve assumption: a grid must support many different projects in a way that facilitates an immediate jump to productivity What did we expect from grids? (1/3) … and what have we observed:  What did we expect from grids? (1/3) … and what have we observed The all users are interested assumption: Grids support many VOs, many users, and they all use the grid intensively (top 2-5 groups/users dominate the workload) The 0-day learning curve assumption: a grid must support many different projects in a way that facilitates an immediate jump to productivity What did we expect from grids? (1/3) … and what have we observed:  What did we expect from grids? (1/3) … and what have we observed The all users are interested assumption: Grids support many VOs, many users, and they all use the grid intensively (top 2-5 groups/users dominate the workload) The 0-day learning curve assumption: a grid must support many different projects in a way that facilitates an immediate jump to productivity (learning curves of up to 60 days, 100 days for an automated tool development) What did we expect from grids? (2/3)Grids vs. Parallel Production Envs.:  What did we expect from grids? (2/3) Grids vs. Parallel Production Envs. The HPC successor assumption: the Grid’s typical user comes from the HPC world, and the typical workload can be described with well-known models of parallel production environments workloads We will have from the beginning lots of parallel jobs We will have from the beginning lots of power-of-2 sized jobs We will handle from the beginning tightly-coupled parallel jobs The infinite power assumption: the Grid can deliver any computing power demand, and in particular will offer at least computing power comparable to traditional Parallel Production Environments Grids vs. Parallel Production Envs. … and what have we observed:  Grids vs. Parallel Production Envs. … and what have we observed The HPC successor assumption: the Grid’s typical user comes from the HPC world, and the typical workload can be described with well-known models of parallel production environments workloads No parallel jobs No power-of-2 sized jobs No tightly-coupled parallel jobs The infinite power assumption: the Grid can deliver any computing power demand, and in particular will offer at least computing power comparable to traditional Parallel Production Environments Grids vs. Parallel Production Envs. … and what have we observed:  Grids vs. Parallel Production Envs. … and what have we observed The HPC successor assumption: the Grid’s typical user comes from the HPC world, and the typical workload can be described with well-known models of parallel production environments workloads No parallel jobs No power-of-2 sized jobs No tightly-coupled parallel jobs The infinite power assumption: the Grid can deliver any computing power demand, and in particular offers at least computing power comparable to traditional Parallel Production Environments Grid Computing Works! … and what have we observed:  Grid Computing Works! … and what have we observed The Power of Sums is Greater Than the Sum of Powers assumption: by coupling individual environments the Grid will provide better performance than by using the individual environments alone Higher throughput Shorter wait times Shorter response times Same or lower jobs slowdown There is a need for the Power of Sums assumption: institutions have the will and the need to couple their environments into a larger, grid-based, environment Grid Computing Works! … and what have we observed:  Grid Computing Works! … and what have we observed The Power of Sums is Greater Than the Sum of Powers assumption: by coupling individual environments the Grid will provide better performance than by using the individual environments alone Higher throughput (as predicted) Shorter wait times (NOT as predicted) Shorter response times Same or lower jobs slowdown (as predicted) There is a need for the Power of Sums assumption: institutions have the will and the need to couple their environments into a larger, grid-based, environment Grid Computing Works! … and what have we observed:  Grid Computing Works! … and what have we observed The Power of Sums is Greater Than the Sum of Powers assumption: by coupling individual environments the Grid will provide better performance than by using the individual environments alone Higher throughput (as predicted) Shorter wait times (NOT as predicted) Shorter response times Same or lower jobs slowdown (as predicted) There is a need for the Power of Sums assumption: (there already is a large community using them!) Slide25:  Final Words How does this affect your work?:  How does this affect your work? We identify or quantitatively assess several challenges for Grid scheduling, monitoring, and benchmarking (in the paper) More realistic simulation assumptions, based on these findings Currently building a Grid Workloads Archive for the benefit of the whole community. We need your help! Take home message:  - NO tightly-coupled parallel jobs, - top groups/users dominate the workload, etc. Immediate impact - Community goals: try to improve on learning curve, waiting time, … - Simulation work: try to use more realistic setups Take home message 'How are real grids used?' Not like we used to believe! Grid Workloads Archive We need your help! Community effort Thank you! Questions? Remarks? Observations?:  Thank you! Questions? Remarks? Observations? Help building our community’s Grid Workloads Archive Contact: ] Alexandru Iosup Dick H.J. Epema ] A.Iosup@tudelft.nl D.H.J.Epema@tudelft.nl

Add a comment

Related presentations

Related pages

“How are Real Grids Used?” The Analysis of Four Grid ...

September 20, 2006 1 “How are Real Grids Used?” The Analysis of Four Grid Traces and Its Implications IEEE Grid 2006 Alexandru Iosup, Catalin ...
Read more

Talk About Off Grid & The Ole Saturday Homesteading ...

Print this Post. Talk About Off Grid & The Ole Saturday Homesteading Trading Post Ed. No. 29
Read more

Log Cabin Trim Tool by Creative Grids - YouTube

Join Cathy Culp of Ruth's Stitchery in Colorado Springs as she demonstrates the Log Cabin Trim Tool by Jean Ann Wright of Creative Grids.
Read more

BaeckereiPadeffke - YouTube

Rezept: Kokosmakronen backen mit Dominic Padeffke von der Bäckerei-Konditorei Padeffke aus Mössingen - Duration: 9 minutes, 23 seconds. 18,331 views; 2 ...
Read more

How beginners - European Globus Community Forum (EGCF)

How beginners experience grid computing Michael Krieger RISC Software GmbH Softwarepark 35, 4232 Hagenberg, Austria . RISC Software GmbH – Johannes ...
Read more

Essen Motorshow Grid Girls Galerien - pagenstecher.de ...

Essen Motorshow Grid Girls Galerien; Top Tuning Auto. Renault Twingo 1.6 RS. Tuning Statistiken. 36141 Tuner Neu Anmifox. 83913 Tuning Themen 1065600 ...
Read more

speech grid sum06 - De Anza College

Heistein ‘06 Talk #_____ Class_____ Speech Evaluation Grid Give this copy to me before you give your talk. Staple your outline ...
Read more

Using Active Scanning to Identify Wireless NICs

Using Active Scanning to Identify Wireless NICs Raheem Beyah Communications Assurance and Performance Group Georgia State University (with Cherita Corbett ...
Read more

Asymptotic analysis of lattice Boltzmann methods

Asymptotic analysis of lattice Boltzmann methods Martin Rheinländer ICMMES 2006 Applications of asymptotic analysis Arbeitsgruppe Numerik (LS Junk)
Read more