What is [Open] MPI?

50 %
50 %
Information about What is [Open] MPI?
Technology

Published on January 1, 2009

Author: jsquyres

Source: slideshare.net

Description

Description of the Message Passing Interface (MPI) and a brief overview of of the Open MPI open source software project.

Slidecast: What is [Open] MPI? Jeff Squyres May 2008

What is MPI? Message Passing Interface “De facto” standard Not an “official” standard (IEEE, IETF, …) Written and ratified by the MPI Forum Body of academic, research, and industry representatives MPI is two spec documents: MPI-1 and MPI-2 Specified interfaces in C, C++, Fortran 77/90

Message Passing Interface

“De facto” standard

Not an “official” standard (IEEE, IETF, …)

Written and ratified by the MPI Forum

Body of academic, research, and industry representatives

MPI is two spec documents:

MPI-1 and MPI-2

Specified interfaces in C, C++, Fortran 77/90

MPI Forum Published MPI-1 spec in 1994 Published MPI-2 spec in 1996 Additions to MPI-1 Recently reconvened (Jan 2008) Working on MPI-2.1 (small bug fixes) Will issue a single document for MPI 1+2 Also working on MPI-2.2 (bigger bug fixes) Also working on MPI-3 (entirely new stuff)

Published MPI-1 spec in 1994

Published MPI-2 spec in 1996

Additions to MPI-1

Recently reconvened (Jan 2008)

Working on MPI-2.1 (small bug fixes)

Will issue a single document for MPI 1+2

Also working on MPI-2.2 (bigger bug fixes)

Also working on MPI-3 (entirely new stuff)

What is MPI? Software implementations of spec Mostly host-side software “ Middleware” Sits between the application and network Simplifies network activity to the application Source code portability Run apps on commodity clusters and “big iron” supercomputers

Software implementations of spec

Mostly host-side software

“ Middleware”

Sits between the application and network

Simplifies network activity to the application

Source code portability

Run apps on commodity clusters and “big iron” supercomputers

MPI High-Level View User application MPI API Operating System

What is MPI? Intended to deliver very high performance Low latency, high bandwidth Examples 2 servers + switch, user-level processes DDR InfiniBand ~1-2  s half-round trip 0-byte ping pong ~14Gbps bandwidth for large messages 10Gbps Ethernet ~5-7  s half-round trip 0-byte ping pong ~10Gbps bandwidth for large messages

Intended to deliver very high performance

Low latency, high bandwidth

Examples

2 servers + switch, user-level processes

DDR InfiniBand

~1-2  s half-round trip 0-byte ping pong

~14Gbps bandwidth for large messages

10Gbps Ethernet

~5-7  s half-round trip 0-byte ping pong

~10Gbps bandwidth for large messages

MPI Implementations Many exist / are available for customers Vendors: HP MPI, Intel MPI, Scali MPI Have their own support channels Open source: Open MPI, MPICH[2], … Rely on open source community for support But also have some vendor support Various research-quality implementations Proof-of-concept Not usually intended for production usage

Many exist / are available for customers

Vendors: HP MPI, Intel MPI, Scali MPI

Have their own support channels

Open source: Open MPI, MPICH[2], …

Rely on open source community for support

But also have some vendor support

Various research-quality implementations

Proof-of-concept

Not usually intended for production usage

Why So Many MPI’s? A complicated question… Some aim to make money (closed source) Some targeted at specific platforms Others aimed at research (open source) History and politics also involved (yuck) Open MPI is a fascinating blend of research and industry

A complicated question…

Some aim to make money (closed source)

Some targeted at specific platforms

Others aimed at research (open source)

History and politics also involved (yuck)

Open MPI is a fascinating blend of research and industry

Target Audience Scientists and engineers Don’t know or care how network works Not computer scientists Sometimes not even [very good] programmers Parallel computing Using tens, hundreds, or thousands of servers in a single computational program Intended for high-performance computing

Scientists and engineers

Don’t know or care how network works

Not computer scientists

Sometimes not even [very good] programmers

Parallel computing

Using tens, hundreds, or thousands of servers in a single computational program

Intended for high-performance computing

Parallel Computing Use 10’s, 100’s, 1000’s of processors When the computation is too big for one server Spread the job across multiple servers Individual user processes running in concert Acting together as a single application More RAM More processing power Divide and conquer

Use 10’s, 100’s, 1000’s of processors

When the computation is too big for one server

Spread the job across multiple servers

Individual user processes running in concert

Acting together as a single application

More RAM

More processing power

Divide and conquer

MPI Abstracts the Network Sockets? Shared memory? Ethernet? InfiniBand? …something else? Doesn’t matter Application calls MPI_SEND / MPI_RECV The Right magic happens Connections are made automatically Sockets (IP address/port) Shared memory (e.g., mmap file) InfiniBand (queue pair setup)

Sockets? Shared memory? Ethernet? InfiniBand? …something else?

Doesn’t matter

Application calls MPI_SEND / MPI_RECV

The Right magic happens

Connections are made automatically

Sockets (IP address/port)

Shared memory (e.g., mmap file)

InfiniBand (queue pair setup)

MPI High-Level View User application MPI API Operating System

Example: 4 Servers Server User application MPI API Operating System Server User application MPI API Operating System Server User application MPI API Operating System Server User application MPI API Operating System Network

Example: 2 Servers Server User application MPI API Operating System User application MPI API Server User application MPI API User application MPI API Operating System Network

Example: 1 Server Server User application MPI API Operating System User application MPI API User application MPI API User application MPI API

Runtime MPI implementations also include a runtime environment Need to start processes on multiple servers simultaneously Typically requires some user-level setup Common source of errors

MPI implementations also include a runtime environment

Need to start processes on multiple servers simultaneously

Typically requires some user-level setup

Common source of errors

Trivial MPI Application int rank, size, message = -1, tag = 11111; MPI_Init(NULL, NULL); /* Startup */ MPI_Comm_rank(…, &rank); /* Who am I? */ MPI_Comm_size(…, &size); /* How many peers do I have? */ to = (rank + 1) % size; from = (rank + size - 1) % size; /* Send a trivial message around in a ring */ if (0 == rank) { message = 42; MPI_Send(&message, 1, MPI_INT, to, tag, …); MPI_Recv(&message, 1, MPI_INT, from, tag, …); } else { MPI_Recv(&message, 1, MPI_INT, from, tag, …); MPI_Send(&message, 1, MPI_INT, to, tag, …); } MPI_Finalize();

int rank, size, message = -1, tag = 11111;

MPI_Init(NULL, NULL); /* Startup */

MPI_Comm_rank(…, &rank); /* Who am I? */

MPI_Comm_size(…, &size); /* How many peers do I have? */

to = (rank + 1) % size;

from = (rank + size - 1) % size;

/* Send a trivial message around in a ring */

if (0 == rank) {

message = 42;

MPI_Send(&message, 1, MPI_INT, to, tag, …);

MPI_Recv(&message, 1, MPI_INT, from, tag, …);

} else {

MPI_Recv(&message, 1, MPI_INT, from, tag, …);

MPI_Send(&message, 1, MPI_INT, to, tag, …);

}

MPI_Finalize();

Trivial MPI Application Process 0 to=1 from=3 Process 2 to=3 from=1 Process 1 to=2 from=0 Process 3 to=0 from=2 MPI_Send(…) MPI_Send(…) MPI_Send(…) MPI_Send(…) MPI_Recv(…) MPI_Recv(…) MPI_Recv(…) MPI_Recv(…)

Open MPI YAMPI (yet another MPI) … but not really Replaces several prior MPI’s Collaborate = great MPI implementation What a concept! Lots of “MPI-smart” people out there Open source project Influenced by both research and industry Research / academia Industry

YAMPI (yet another MPI)

… but not really

Replaces several prior MPI’s

Collaborate = great MPI implementation

What a concept!

Lots of “MPI-smart” people out there

Open source project

Influenced by both research and industry

Open MPI It’s two words! Open MPI NOT “OpenMPI” Frequently abbreviated “OMPI” Pronounced “oom-pee”

It’s two words!

Open MPI

NOT “OpenMPI”

Frequently abbreviated “OMPI”

Pronounced “oom-pee”

Open MPI Fundamentally based on plugins A.k.a. “components” or “modules” Plugins for everything Back-end resource manager Back-end network Back-end checkpointer … etc. Currently ~30 types of plugins in Open MPI Recurring theme: run-time decisions

Fundamentally based on plugins

A.k.a. “components” or “modules”

Plugins for everything

Back-end resource manager

Back-end network

Back-end checkpointer

… etc.

Currently ~30 types of plugins in Open MPI

Recurring theme: run-time decisions

Plugin High-Level View User application MPI API Modular Component Architecture (MCA) Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … …

Resources MPI Forum http://www.mpi-forum.org/ Open MPI General web site: http://www.open-mpi.org/ FAQ: http://www.open-mpi.org/faq/ Magazine columns about MPI http://cw.squyres.com/

MPI Forum

http://www.mpi-forum.org/

Open MPI

General web site: http://www.open-mpi.org/

FAQ: http://www.open-mpi.org/faq/

Magazine columns about MPI

http://cw.squyres.com/

 

Add a comment

Related presentations

Related pages

Open MPI: Open Source High Performance Computing

Open MPI is developed in a true open source fashion by a consortium of research, academic, and industry partners. The Open MPI Team page has a ...
Read more

Open MPI: Version 2.0.1

Open MPI User Docs Open Tool for Parameter Optimization PMIx: ... Enter your e-mail address to be notified when new releases of Open MPI are released
Read more

Open MPI - Wikipedia

Open MPI is a Message Passing Interface (MPI) library project combining technologies and resources from several other projects (FT-MPI, LA-MPI, LAM/MPI ...
Read more

The Open MPI Development Team

The Open MPI project has many members, contributors, and partners. "Contributors" provide resources to the project such as code (which requires having a ...
Read more

Open MPI - HLRS Platforms

Open MPI is an Message Passing Interface (MPI) library project combining technologies and resources from several other projects (FT-MPI, LA-MPI, LAM/MPI ...
Read more

Open MPI · GitHub

Personal Open source Business Explore Sign up Sign in. ... Scripts for handling the Open MPI SVN-to-git-and-Github conversion. Updated Dec 16, 2015.
Read more

MOOCs - start enjoy learning now | openHPI

open online education. It is difficult to imagine life without the Internet or our smartphones, which rarely leave our hands. Sometimes we wonder how all ...
Read more

Courses | openHPI

Open Source; Blog; Feeds; What is a MOOC? ... On openHPI you take part in a worldwide social learning network based on interactive online courses covering ...
Read more

Open MPI | heise Download

Open MPI. Kategorien; Blog; Themenspecials; Deal des Tages; Open MPI The Open MPI Project {{commentsTotalLength}} Kommentar Kommentare. Zum Download. 0
Read more

Message Passing Interface – Wikipedia

Message Passing Interface (MPI) ist ein Standard, der den Nachrichtenaustausch bei parallelen Berechnungen auf verteilten Computersystemen beschreibt.
Read more