MPI - From serial to parallel: Hello world!¶

Objectives

Know how a serial program looks like and how to compile and run it
Know how to compile and run an MPI program (from the outside)
Understand what makes an MPI program (how to activate MPI inside of the program)

Instructor note

15 min teaching
10 min exercises

Note

MPI Standard: MPI: A Message-Passing Interface Standard Version 5.0 (PDF)

Serial “Hello world!” program¶

To warm up, let’s start with a serial “Hello world!” program, no changes/editing needed:

#include <stdio.h>

int main(int argc, char *argv[])
{
   printf ("Hello world!\n");
}

program hello

  implicit none

  write(*,*) 'Hello world!'

end program

print("Hello world!")

Compile the serial program:

gcc hello_serial.c -o hello_serial

gfortran hello_serial.f90 -o hello_serial

Run the serial program (serial run on 1 core):

./hello_serial

python3 ./hello_serial.py

Expected output:

Hello world!

MPI process model¶

Note

An MPI program must be linked with an MPI library (compiler wrapper) –> mpicc, …
An MPI program must be started with an MPI startup tool –> mpirun, mpiexec, srun, …
- Usage: mpirun -np # ./a.out (# … number of MPI processes)
An MPI program must use the include file of this MPI library –> #include <mpi.h>
MPI must be initialized and finalized –> MPI_Init and MPI_Finalize
- Usage: MPI_Init(&argc, &argv); OR MPI_Init(NULL, NULL);
- Usage: MPI_Finalize();
- Note: This applies for the World Model and for starting a single-threaded MPI application. MPI offers different thread-level support, but that’s an advanced topic to be covered later.

Exercise

Write a minimal MPI program that prints “Hello world!” by each MPI process:

What happens if you do NOT modify the code below? Try it out!

You can compile and run without modifying the serial code below.
Give it a try before you actually modify.
What happens here? Why is this possible at all?
Of course, before you can proceed to the next step (2), you have to modify the code below.

#include <stdio.h>

int main(int argc, char *argv[])
{
   printf ("Hello world!\n");
}

program hello

  implicit none

  write(*,*) 'Hello world!'

end program

print("Hello world!")

In C and Fortran compile it as an MPI program (you need to change the code below):

_____ hello.c -o hello

_____ hello.f90 -o hello

Run it in parallel with several (e.g., 4) MPI processes (you need to change the code below):

__________ ./hello

____________________ ./hello.py

Expected output with 4 MPI processes:

Hello world!
Hello world!
Hello world!
Hello world!

Got stuck? Checkout the solution:

The correct MPI program:

Solution

#include <stdio.h>                                        // Add just 3 lines of code:
#include <mpi.h>                                          // #include <mpi.h>

int main(int argc, char *argv[])
{
   MPI_Init(&argc, &argv);  // or  MPI_Init(NULL, NULL);  // MPI_Init(&argc, &argv);

   printf ("Hello world!\n");

   MPI_Finalize();                                        // MPI_Finalize();
}

program hello                       ! Add just 3 lines of code:

  use mpi_f08                        ! use mpi_f08
  implicit none

  call MPI_Init()                    ! MPI_Init

  write(*,*) 'Hello world!'

  call MPI_Finalize()                ! MPI_Finalize
   
end program

from mpi4py import MPI             # add just one line of code

print("Hello world!")

How to compile an MPI programm in C and Fortran:

Solution

mpicc hello.c -o hello

mpif90 hello.f90 -o hello

How to run an MPI programm:

Solution

mpirun -np 4 ./hello

mpirun -np 4 python3 ./hello.py

Exercise

Play around with different numbers of MPI processes Run with different numbers of MPI processes:
E.g., in the VSC JupyterHub, there are 4 physical cores available for the MPI course.
If you want to run with more MPI processes you have to add the option: –oversubscribe
Note: For performance runs you want to avoid oversubscribing, but it’s okay for our little examples.
Note: You can not do oversubscribing and pinning (see 4. Demo) at the same time.

Solution

mpirun -np 8 --oversubscribe ./hello

mpirun -np 8 --oversubscribe python3 ./hello.py

Keypoints

Know how to compile and run an MPI program
Understand what makes an MPI program