Table of Contents

Modifying the Program Running this Legion-MPI Program Limitations of this "fewest changes" approach Modifying the program for the Typed Binary Interface

Click on the to move to the selected text.

  Other relevant on-line documents:

Logging in to a running Legion system Legion graphical user interface Introduction to Legion context space Context-related commands How to start remote programs in Legion Sample makefile for remote programs Object permissions Legion tty objects Running a PVM code in Legion

Running an MPI code in Legion Quick list of all Legion commands Usage of all Legion commands Starting and shutting down Legion 1.5 Using Legion security features Legion host and vault objects Adding host and vault objects The list of all on-line 1.5 tutorials

Depending on how your system is set up, you may need to set up your access to your system before you can run Legion commands. This will probably involve running a command such as this:

$ . ~legion/setup.sh

or

$ source ~legion/setup.csh

Modifying the Program

Let's say that you have the following MPI program:

      program foo

      implicit none
      include 'mpif.h'
      integer ierr, nodenum, numprocs
      integer i, j, k

      call mpi_init( ierr )
      call mpi_comm_rank( mpi_comm_world, nodenum, ierr )
      call mpi_comm_size( mpi_comm_world, numprocs, ierr )

      if (nodenum .eq. 0) then
         open (10, file = 'input', status = 'old')
         read (10,*) i, j, k
         close (10)
      endif

      call do_work (i, j, k)

      if (nodenum .eq. 0) then
         open (11, file = 'output', status = 'new' )
         write (11, *) i, j, k
         close (11)
      endif

      call mpi_finalize(ierr)      
      stop
      end

In order to run this program over Legion and get the benefit of remote I/O, it is necessary to insert extra I/O calls. The MPI calls do not change. Here is the result, in which the changes are in all upper-case:

      program foo

      implicit none
      include 'mpif.h'
      integer ierr, nodenum, numprocs
      integer i, j, k
      CHARACTER*256 INPUT, OUTPUT

      call mpi_init( ierr )
      call mpi_comm_rank( mpi_comm_world, nodenum, ierr )
      call mpi_comm_size( mpi_comm_world, numprocs, ierr )

      if (nodenum .eq. 0) then
         call LIOF_LEGION_TO_TEMPFILE ('input', INPUT, ierr)
         open (10, file = INPUT, status = 'old')
         read (10,*) i, j, k
         close (10)
      endif

      call do_work (i, j, k)

      if (nodenum .eq. 0) then
         call LIOF_CREATE_TEMPFILE (OUTPUT, IERR)
         open (11, file = OUTPUT, status = 'new' )
         write (11, *) i, j, k
         close (11)
         call LIOF_TEMPFILE_TO_LEGION (OUTPUT, 'output', IERR)
      endif

      call mpi_finalize(ierr)      
      stop
      end

         CHARACTER*256 INPUT, OUTPUT

         call LIOF_LEGION_TO_TEMPFILE ('input', INPUT, ierr)

         open (10, file = INPUT, status = 'old')

         call LIOF_CREATE_TEMPFILE (OUTPUT, IERR)

         open (11, file = OUTPUT, status = 'new' )

         call LIOF_TEMPFILE_TO_LEGION (OUTPUT, 'output', IERR)

Running this Legion-MPI Program

In order to run this program under Legion MPI, we need to:

• compile and link the program
• set up access to the Legion system
• tell Legion where the executable and input file are
• run the program
• copy the output file back to our workstation

1. Compiling and linking

   Assuming that our program is in the file example.f, we would compile and link using:

   % f77 -c example.f -I$LEGION/include/MPI
   % legion_link -Fortran -mpi -o example example.o
   

2. Setting up access to the Legion system

   Before you issue Legion commands, you may need to execute an extra command:

   % . /home/appnet/setup.sh

   or

   % source /home/appnet/setup.csh

   Also, before you run a Legion program, it is useful to have a Legion tty, which is an object which allows you to observe the output of your program.

   % legion_tty my_tty

   This creates a tty with the name my_tty in Legion space, and send the output from this tty object to your current terminal.

3. Tell Legion where the executable and input file are:

   • executable

        % legion_mpi_register example ./example $LEGION_ARCH

     This command registers the unix file ./example as the MPI Class example. Legion will create a class object, named /mpi/programs/example, in Legion space.

   • input file

        % legion_cp -localsource ./input input

     This command creates a Legion FileObject with the name input, and copies the Unix file with the name ./input into it.

4. Run the program

   Now you may run the program:

   % legion_mpi_run -n 4 /mpi/programs/example

   This command will run your program on four nodes somewhere in the Legion system. Any output sent to the screen will be sent to your tty object.

5. Copy the output file back to our workstation

   Finally, we might want to copy the output file named output in Legion space back to our local workstation.

   legion_cp -localdest output ./output

Limitations of this "fewest changes" approach

While this approach allows you to run MPI programs and transparently read and write files remotely, it does have one limitation: it does not support heterogeneous conversions of data. If you run this program on several machines which have different formats for an integer, such as Intel PC's (little-endian) and IBM RS/6000's (big-endian), the result of using unformatted I/O will be surprising. If you want to use such a heterogeneous system, you will have to either use formatted I/O (all files are text) or use the "typed binary" I/O calls instead of Fortran READ and WRITE statements. These "typed binary" I/O calls are discussed in "Buffered I/O Library, low impact interface," in the Legion Developer Manual.

Modifying the program for the Typed Binary Interface

Here's how the program would change if we used the typed binary interface:

      program foo

      implicit none
      include 'mpif.h'
      integer ierr, nodenum, numprocs
      integer i, j, k
      integer fd, ierr

      call mpi_init( ierr )
      call mpi_comm_rank( mpi_comm_world, nodenum, ierr )
      call mpi_comm_size( mpi_comm_world, numprocs, ierr )

      if (nodenum .eq. 0) then
         call LIOF_OPEN ('input', 0, FD)
         call LIOF_READ_INTS (FD, I, 1, IERR)
         call LIOF_READ_INTS (FD, J, 1, IERR)
         call LIOF_READ_INTS (FD, K, 1, IERR)
         call LIOF_CLOSE (FD, IERR)
      endif

      call do_work (i, j, k)

      if (nodenum .eq. 0) then
         call LIOF_OPEN ('output', 0, FD)
         call LIOF_WRITE_INTS (FD, I, 1, IERR)
         call LIOF_WRITE_INTS (FD, J, 1, IERR)
         call LIOF_WRITE_INTS (FD, K, 1, IERR)
         call LIOF_CLOSE (FD, IERR)
      endif

      call mpi_finalize(ierr)      
      stop
      end

Running the program would be the same.

Last modified: Tue Jun 15 10:18:17 1999

[Home] [General] [Documentation] [Software] [Testbeds] [Et Cetera] [Map/Search] legion@Virginia.edu http://legion.virginia.edu/