BasicFiles_read(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)
The Legion file interface library is a set of C functions that allow Legion file objects to be manipulated from user programs with a familiar interface similar to that provided by the Unix file system. To use the library, programmers should include the header file Legion_libBasicFiles.h and link their programs against the library libBasicFiles.[a|so] (located in the same directory as the basic Legion libraries). Linkage against the basic Legion libraries is also required.
A sample Legion file library makefile is below.
|
Figure 14: Sample Legion File Library makefile | |
|
This directory contains the Legion basic file objects and libraries for using these objects. Legion basic file objects are quite simple at their core, representing a random-access linear array of bytes. This linear byte array is indexed starting at zero and preceding to (fileSize - s1). Files grow to accommodate the data that are written into them. The random access nature of the basic file object requires clients of the file to maintain their own notion of a file pointer if sequential access is desired. While automatic maintenance of file pointers is not performed by file objects, utility libraries to provide this service are available and are described in section 6.2.
There is a set of public methods exported by Legion basic files (see section 6.1), but the public interface is somewhat inconvenient in cases where sequential access, and/or frequent, fine-grained access is desirable. To address these issues, a set of library functions is provided to support convenient access to file objects. The interface functions provided with the Legion file library fall into several main categories:
The raw I/O library is supported for programs written in either C or C++. Use of the library requires including the header file legion/Legion_libBasicFiles.h and linking against the library libBasicFiles.
Using the raw I/O library can be costly if you perform frequent, fine-grained file access. Each read or write requires a remote method invocation. A buffered version of the file library is therefore available in order to provide a more efficient interface for fine-grained access.
The buffered interface is callable from and well-matched to use in C and C++. However, this library is neither conveniently callable nor well-suited for use in Fortran. To better support Legion file access from Fortran programs a Fortran buffered file interface is provided.
The object of this interface is to give the user the option of making the smallest number of changes to their program and still use Legion file objects.
If you have an input file, one lio_legion_to_tempfile call is needed. For an output file, the name must first be created using lio_create_tempfile. Then, after the file is written, a single lio_tempfile_to_legion call will write the file to Legion space.
This interface does not allow conversion of the data between different data formats. If you were to write integer values on a RS/6000 and read them on a DEC Alpha, for example, the bytes would be in the wrong order and the DEC would be unable to read them. In order to get full access to Legion's type conversion facilities, the typed binary interface routines such as lio_write_ints must be used.
The C and C++ interfaces described in 6.2.1 and 6.2.2 are similar to typical system-call level and C standard I/O level file interfaces. However, a potentially more convenient and useful interface is available for C++ programs--the LegionBuffer interface. The LegionBuffer is the basic data container supported by the Legion library. The standard Legion library supports LegionBuffer implementations for in-memory buffers, and Unix file buffers. The Legion basic file libraries add on a new LegionBuffer implementation that can be used to access Legion file objects.
This interface is designed to provide more efficient file access for programs that use large two-dimensional arrays of data. To use this library, link against the lib2DFiles.a library and include either header file legion/lib_TwoDFileObject.h (for C++) or legion/libc_TwoDFileObject.h (for C). The TwoDFileObject object manages several BasicFileObjects that block partition the array. This interface is available for any C++ object that is derived from the LegionPackable class and all the basic types. The C and FORTRAN interfaces support integer, real/float, and double basic types. Below are methods that are supported by the TwoDFileObject object. Note that in the C interface <type> is one of int, float, or double while in the FORTRAN interface <type> is one of integers, reals, or doubles.
The TwoDFileObject asynchronous interface is designed to provide non-blocking access to the TwoDFileObject. To use this library, programmers should link against the lib2DFiles.a library and should include legion/lib_TwoDFileObject.h for C++ or legion/libc_TwoDFileObject.h for C. This interface is available for any C++ object that is derived from the LegionPackable class and all the basic types. The C and FORTRAN interfaces support the integer, real/float, and double basic types. Here are methods that are supported by the TwoDFileObject for the C interface <type> is one of int, float, or double, for the FORTRAN interface <type> is one of integers, reals, or doubles.
long
write(long startAtByte, BasicFileTransferBlock writeData)
The write method copies a linear array of bytes into the file starting at the location specified by the startAtByte parameter. The data to copy in is transferred in the form of a BasicFileTransferBlock, which contains an integer size field followed by the corresponding number of bytes of data. write returns the number of bytes actually copied in (for example, a smaller number than requested might be written in case of file size limitations). writes that are performed at starting locations beyond the offset (fileSize - 1) may be performed, and cause the bytes in the file ranging (inclusive) from offset fileSize through offset (startAtByte - 1) to be set to zero.
BasicFileTransferBlock
read(long startAtByte, long numBytesToRead)
The read method returns a copy of the specified linear sub-array of the file starting at the position indicated by the startAtByte parameter and proceeding for numBytesToRead bytes. The read data is returned in the form of a BasicFileTransferBlock (see write above)
The stat method returns a BasicFileStatBuffer structure containing meta-information about the file object. Currently this structure contains only the file size, but is intended for future expansion.
The trunc method truncates the file size to zero bytes, discarding all data contained in the file. Future writes to the file may once again expand its size, but immediately after the trunc call the file will contain no data. This method returns a 1 upon success.
long
append(BasicFileTransferBlock writeData)
Whereas the write method copies a linear array of bytes to a specified offset within the file, the append method appends a specified array of bytes on to the end of the file, regardless of what offset that may reside at. This method is convenient for use when multiple writers are inserting data into a file object. Whereas use of the write method would require the multiple writers to coordinate a consistent file pointer, the append method allows the multiple writers to write non-overlapping data regions without coordinated file pointers. This method returns the resulting file size (which might be considered the resulting file pointer by the caller).
long
truncAppend(BasicFileTransferBlock writeData)
The truncAppend method atomically truncates the file object to zero bytes, and then appends a linear array of bytes to the file. This method can be used for atomically replacing the entire contents of a file. This action is convenient when a writer and reader are coordinating through the use of a file, and the reader should never be allowed to see an incomplete version of the file. For example, trying to achieve the same effect using the trunc and append methods separately might cause a reader to see an empty file if a read were serviced between the trunc and append. This method returns the number of bytes written (which is equivalent to the resulting file size, and might be considered the resulting file pointer by the caller).
BasicFiles_init initializes the calling object's Legion library state. Note that this function only needs to be called in programs that do not initialize their Legion state in some other way. For example, a Legion-MPI program will have its Legion state initialized by the MPI library, and thus BasicFiles_init need not be called.
BasicFiles_done disconnects the calling program from Legion. As with BasicFiles_init, this function only needs to be called by programs that do not detach from Legion in some other way (e.g., MPI and PVM programs need not call this function).
int
BasicFiles_exists(char *path)
This function returns a non-zero result if the specified path refers to a valid file object in Legion context space.
BasicFiles_FileDescriptor
BasicFiles_creat(char *path)
BasicFiles_creat creates a new file object, assigning the name specified by path to the new object in context space. Upon success a valid file descriptor is returned. Upon failure the value BasicFiles_BadFd is returned.
BasicFiles_FileDescriptor
BasicFiles_open(char *path, int flags)
BasicFiles_open opens the file object named by path for reading and writing. On success, a valid file descriptor is returned. On failure, the value BasicFiles_BadFd is returned. After an open, the implicit file pointer (i.e. next position for read or write operations on the file) is set to BasicFiles_seek can be used to set the file pointer to alternate locations.
The flags parameter to BasicFiles_open can contain the bitwise inclusive OR of the following values:
BASIC_FILE_O_CREAT Create the file if it does not exist (default action is to return failure).
BASIC_FILE_O_APPEND Specify that all writes be performed as append operations.
BASIC_FILE_O_TRUNC Truncate the file to zero bytes on successful open.
int
BasicFiles_close(BasicFiles_FileDescriptor fd)
BasicFiles_close closes the file connection associated with fd. Returns -1 upon failure, a non-negative integer upon success.
void
BasicFiles_delete(char *path)
BasicFiles_delete destroys the file object referred to by path. Note that unlike Unix file systems (whose unlink system calls do not interrupt the file service to processes that still have the deleted file open) BasicFiles_delete causes all future file operations performed by other clients to fail.
BasicFiles_FilesOffset
BasicFiles_read(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)
BasicFiles_read reads up to len bytes of data from the file associated with fd, starting at the current file pointer associated with fd. If the file pointer is at or beyond the end of the file, zero bytes are read. BasicFiles_read returns the actual number of bytes read and copied into data. Upon failure -1 is returned.
BasicFiles_FileOffset
BasicFiles_write(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)
BasicFiles_write writes up to len bytes of data contained in the buffer data to the file referred to by fd, starting at the current file pointer associated with fd. The actual number of bytes written is returned.
BasicFiles_FileOffset
BasicFiles_seek(BasicFiles_FileDescriptor fd,
BasicFiles_Whence whence,
BasicFiles_FileOffset offset)
BasicFiles_seek sets the value of the file pointer associated with fd. The new file pointer location is computed depending on the value of whence:
BASIC_FILE_SEEK_BEGINNING The new file pointer is set to offset.
BASIC_FILE_SEEK_CURRENT The new file pointer is set to the old file pointer plus offset.
BASIC_FILE_SEEK_END The new file pointer is the size of the file referred to by fd plus offset.
Note that offset may be negative. Also note that, except in the case of BASIC_FILE_SEEK_END, BasicFiles_seek does not perform remote method invocations (a remote method is required by BASIC_FILE_SEEK_END to compute the file size).
BasicFiles_seek returns the new value of the file pointer associated with fd, or -1 on failure.
BasicFiles_FileOffset
BasicFiles_tell(BasicFiles_FileDescriptor fd)
BasicFiles_FileOffset
BasicFiles_size(BasicFiles_FileDescriptor fd)
BasicFiles_FileOffset
BasicFiles_trunc(BasicFiles_FileDescriptor fd)
Truncates the file object associated with fd to zero bytes and resets the file pointer associated with fd to zero.
BasicFiles_FileOffset
BasicFiles_append(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)
BasicFiles_append appends up to len bytes of data contained in the buffer data to the end of the file object referred to by fd. The resulting file size is returned on success, -1 upon failure.
BasicFiles_FileOffset
BasicFiles_truncAppend(BasicFiles_FileDescriptor fd,
char *data, BasicFiles_FileOffset len)
Atomically truncates the file referred to by fd to zero bytes, then appends up to len bytes of data contained in the buffer data to that file. The resulting file size is returned on success, -1 upon failure.
BasicFILE *
BasicFiles_fopen(char *path, char *flags)
Opens the file object names by path for reading and writing. If the file does not exist it is created. Currently, flags is ignored but is reserved for future use--a flags value of NULL will retain the default semantics described here in future versions. On open, the implicit file pointer (location for the next read or write operation) is set to zero. Upon success BasicFiles_fopen returns a non-NULL BasicFILE pointer. Upon failure NULL is returned.
int
BasicFiles_setbufsize(BasicFILE *file, int sz)
Sets the memory buffer size associated with the file referred to by file to sz. The buffer size specifies the maximum amount of data that may be read from or written to the file before an additional remote method invocation is performed. In general, larger buffers can result in better performance but consume more memory. The default buffer size is 8 KB. BasicFiles_setbufsize returns the resulting buffer size upon success, and -1 upon failure.
int
BasicFiles_fclose(BasicFILE *file)
Closes the file associated with file, flushing any pending write operations on the file. Returns a non-negative integer upon success, -1 upon failure.
int
BasicFiles_fflush(BasicFILE *file)
Flushes any pending write operations on the file (i.e., if any data written to the file has been buffered, this function performs the necessary remote method invocations to transfer the data to the actual file object). Returns a non-negative integer upon success, -1 upon failure.
BasicFiles_FileOffset
BasicFiles_fseek(BasicFILE *file, BasicFiles_FileOffset offset,
BasicFiles_Whence whence)
BasicFiles_seek sets the value of the file pointer associated with file. The new file pointer location is computed depending on the value of whence:
BASIC_FILE_SEEK_BEGINNING The new file pointer is set to offset.
BASIC_FILE_SEEK_CURRENT The new file pointer is set to the old file pointer plus offset.
BASIC_FILE_SEEK_END The new file pointer is the size of the file referred to by fd plus offset.
BasicFiles_fseek returns the new value of the file pointer associated with file, or -1 on failure.
BasicFiles_FileOffset
BasicFiles_ftell(BasicFILE *file)
int
BasicFiles_fputc(int c, BasicFILE *file)
BasicFiles_fputc writes the character c, cast to an unsigned char, to the file referred to by file. Returns the character written as an unsigned char cast to an int upon success, or EOF upon failure.
int
BasicFiles_fputs(char *str, BasicFILE *file)
BasicFiles_fputs writes the null-terminated string str to the file referred to by file at the current file pointer location associated with file. The trailing \0 character is not written (i.e., the call writes strlen(str) bytes). Returns a non-negative integer upon success, -1 upon failure.
int
BasicFiles_fwrite(void *ptr, int size, int nmemb, BasicFILE *file)
BasicFiles_fwrite writes nmemb elements of data contained in the buffer ptr, each size bytes long, to the file referred to by file starting at the current file pointer location associated with file. Returns the number of items (not the number of bytes) successfully written.
int
BasicFiles_fgetc(BasicFILE *file)
BasicFiles_fgetc reads the next character from the file referred to by file and returns it as an unsigned char cast to an int. EOF is returned upon failure.
char *
BasicFiles_fgets(char *str, int sz, BasicFILE *file)
BasicFiles_fgets reads in at most one less than sz characters from the file referred to by file and copies them into the buffer str. Reading stops when the end of file or a newline character is encountered (the terminating newline is copied into str). After either sz-1 characters are read or EOF or newline is encountered, str is terminated with a \0 character. BasicFiles_fgets returns str upon success and NULL upon failure.
int
BasicFiles_fread(void *ptr, int size, int nmemb, BasicFILE *file)
BasicFiles_fread reads nmemb elements of data, each of size bytes, from the file referred to by file, copying the read items into the buffer ptr. BasicFiles_fread returns the number of items (not the number of bytes) successfully read.
The fprintf function is not supported in Legion: fprintf uses varargs, which vary according to architecture and therefore cannot be used in Legion. However, you can use BasicFiles_fputs to get the same effect as a BasicFiles_fprintf function:
char buffer[1024]; sprintf(buffer,"control string",args...); BasicFiles_fputs(buffer,fp);
Please note that the buffer size is large enough: 1024 is simply an example.
Initializes the calling object's Legion library. This function should not be called if the object initializes the library through other means (such as mpi_init).
Disconnects the calling program from Legion. As with liof_init, it only needs to be called by programs that do not detach from Legion in some other way (MPI and PVM programs, for example, do not need to call this function). This function will cause Legion to tell your program to exit so it should be invoked just prior to program termination.
subroutine liof_open(path, flags, fd)
character path(*)
integer flags, fd
Opens the file referred to by the string path for reading and writing. If the file does not exist, it is created. An integer file descriptor is returned in fd, and the file pointer associated with fd is set to zero. Currently flags is ignored but reserved for future use. However, in the future a flags value of 0 will retain the default semantics described here. Upon successful return fd will contain a non-negative integer. Upon failure fd will contain -1.
subroutine liof_close(fd, ret)
integer flags, ret
Closes the file associated with the file descriptor fd. Upon successful return fd will contain 0. Upon failure fd will contain -1.
subroutine liof_write(fd, buf, sz, ret)
integer fd
character buf(*)
integer sz, ret
Writes up to sz bytes of data from the buffer buf to the file referred to by fd starting at the current file pointer associated with fd. Upon successful return ret will contain the number of bytes actually written. Upon failure ret will contain -1.
subroutine liof_write_line(fd, line, sz, ret)
integer fd
character line(*)
integer sz, ret
Writes a formatted line of output contained in the buffer line and spanning no more than sz bytes to the file referred to by fd. The expected use of this routine is for writing a buffer of data created using FORMAT and WRITE Fortran statements to a Legion file. For example:
character msg*256 10 format(I8,' .... ',I5) write(msg,10) X, Y call lio_write_line(fd, msg, 256, ret)
subroutine liof_flush(fd, ret)
integer fd, ret
Flushes any pending data writes to the file referred to by fd. Upon successful return ret will contain a non-negative integer. Upon failure ret will contain -1.
subroutine liof_read(fd, buf, sz, ret)
integer fd
character buf(*)
integer sz, ret
Reads up to sz bytes of data from the file referred to by fd into the buffer buf. Upon successful return ret will contain the number of bytes read (zero indicates end of file). Upon failure ret will contain -1.
subroutine liof_read_line(fd, line, sz, ret)
integer fd
character line(*)
integer sz, ret
Reads a line of characters from the file referred to by fd into the buffer line. Up to sz bytes will be read. The resulting data is formatted as acceptable input to the Fortran FORMAT and READ statements. For example:
character msg*256 10 format(I8,' .... ',I5) call lio_read_line(fd, msg, 256, ret) read(msg,10) X, Y
Upon successful return, ret will contain a non-negative integer. Upon failure ret will contain -1.
subroutine liof_rewind(fd, ret)
integer fd, ret
Resets the file pointer associated with fd to zero. Upon successful return ret will contain a non-negative integer. Upon failure, ret will contain -1.
subroutine liof_write_ints(fd, x, num, ret)
integer fd
integer x(*)
integer num, ret
Writes num integers from the vector x to the file associated with fd in binary form. Upon return, the number of integer values actually written is returned in ret. Upon failure ret will contain -1.
subroutine liof_read_ints(fd, x, num, ret)
integer fd
integer x(*)
integer num, ret
Reads num integers stored in binary form into the vector x from the file associated with fd. Upon return, the number of integer values actually read is returned in ret. Upon failure, ret will contain -1. Note that platform-dependent data representation issues are masked from the caller (E.g., a caller on a big endian host can read values written by a program on a little endian host).
subroutine liof_write_reals(fd, x, num, ret)
integer fd
real x(*)
integer num, ret
Writes num reals from the vector x to the file associated with fd in binary form. Upon return, the number of real values actually written is returned in ret. Upon failure, ret will contain -1.
subroutine liof_read_reals(fd, x, num, ret)
integer fd
real x(*)
integer num, ret
Reads num reals stored in binary form into the vector x from the file associated with fd. Upon return, the number of real values actually read is returned in ret. Upon failure, ret will contain -1. Note that platform-dependent data representation issues are masked from the caller.
subroutine liof_write_doubles(fd, x, num, ret)
integer fd
double precision x(*)
integer num, ret
Writes num double precision values from the vector x to the file associated with fd in binary form. Upon return, the number of double precision values actually written is returned in ret. Upon failure, ret will contain -1.
subroutine liof_read_doubles(fd, x, num, ret)
integer fd
double precision x(*)
integer num, ret
Reads num double precision values stored in binary form into the vector x from the file associated with fd. Upon return, the number of double precision values actually read is returned in ret. Upon failure, ret will contain -1. Note, platform-dependent data representation issues are masked from the caller.
char *
lio_legion_to_tempfile(char* legion_name)
Copies a Legion file object to the local disk. The name of the temporary file created on local disk is a return value and cannot be specified by the application.
Returns the name of a temporary file. To be used with the lio_tempfile_to_legion call (below).
int
lio_tempfile_to_legion(char* tempfile, char* legion_name)
Writes a local file into Legion space. The tempfile name is created via the previous function.
Using the new LegionBuffer type is simple:
LRef<LegionBuffer> BasicFiles_openBuffer(char *path);
This function returns a LegionBuffer that can be used to access the file object referred to by path. If no such file currently exists, it is created. The standard LegionBuffer interface is described in section 7.2, "Basic module and data container: the Legion buffer."
Parallel applications often encounter a bottleneck while trying to perform file operations [27] [28]. Legion offers a two-dimensional file object, which attempts to address this issue in two ways: it block partitions the file into several subfiles [18] and it allows the application to specify how it wants to access the file so that only needed data are transferred to each of the application's nodes. This makes it easier for the programmer to perform non-sequential I/O operations. And, since the file's decomposition is specified at creation time, the file can be broken up in such a manner as to minimize both disk and network contention within the application during I/O operations. The TwoDFileObject reduces the amount of application code that is required by the standard file interface to perform the same operations, since multiple reads or writes can be bundled into a single request without the need for seeks in-between.
The two-dimensional file object provides users with the following features:
The two-dimensional file object is layered on top of Legion. This means that it can be run on multiple hosts and architectures without any modifications to the underlying file systems. This is particularly useful in cluster computing environments where large files typically come from an NFS file server. Two-dimensional file objects take advantage of Legion's data-coercion mechanism to provide heterogeneous access to the file. Thus the subfiles and the nodes of the application can all be on different architectures but still generate a file with a single data format.
The two-dimensional file object provides five access patterns for both reading and writing. These patterns are access by:
Row and column access patterns allow the program to access several rows or columns of the file at a time. The rows and columns can be separated by a fixed stride. Thus with a single read request the program could gather five columns of the file where each column is separated by ten unneeded columns. The block access pattern allows the program to access a rectangular subsection of the file. This should be particularly useful for stencil problems, which tend to decompose the problem space into regular subsections. The element access pattern allows programs to access a regular pattern of elements within some subsection of the file. This pattern can also be used to access subrows or subcolumns of the file. The sequential access mode allows the file to be viewed as if it were a sequential file. This is intended for pre-existing filters that will access the entire file either before or after the application has run (such as visualization tools). For a description of the calling conventions and usage of these access patterns see sections 6.2.6.2 (synchronous interface) and 6.2.6.3 (asynchronous interface).
Applications use the lib2Dfiles library to access a TwoDFileObject. The two-dimensional file is implemented as a collection of BasicFileObjects (sequential files in Legion) which are controlled by a single master object. To prevent bottlenecks, a library running on the client gets relevant file information from the TwoDFileObject and then communicates directly with the BasicFileObjects. This allows several nodes to access different parts of the file concurrently without contention.
When the library receives a request from an application it iterates over the subfiles and determines if it needs to communicate with that particular subfile. It then builds a list of offsets and lengths within that subfile that are required to satisfy the read request. Next, it sends the list of offsets to the subfile and determines what is required from the remaining subfiles. Once all the requests have been sent to the appropriate subfiles the library iterates over them a second time to gather the results. When the library gets back the results it coerces the data into the correct representation and then places it at the appropriate locations in the result array. Note that only one message is sent to each subfile involved in the request and that the results come back in a single message. This uses the minimum possible amount of communication that can still satisfy the request.
This section contains four sample two-dimensional file object programs. The first three show C++ applications that use various calls to create and use blocks, columns, and rows. The last shows a Fortran application.
The first, Example A, shows a C application that creates a 11x9 2D file consisting of six subfiles. The biggest and smallest subfiles do not differ by more than one row and one column, so that the biggest subfile is 4x5 and the smallest is 3x4. After the 2D file is created it is filled in with a writeBlock call. Finally, the program reads back a sub-block of the file with a readBlock call.
#include<stdio.h>
#include<legion/libc_TwoDFileObject.h>
int main()
{
int tst;
int i, j, rv;
int foo[9][11];
int bar[6][7];
/* initialize the legion libraries */
BasicFiles_init();
fprintf(stderr, "calling create\n");
/* create the TwoDFileObject and all of its subfiles */
rv = lio_2d_create_int(&tst, "TwoDFileTestFile", 11,
9, 3, 2, NULL);
fprintf(stderr, "back from create\n");
/* check return code */
if(rv < 0)
{
fprintf(stderr, "unable to create TwoDFileObject\n");
return 1;
}
/* initialize the foo array */
for(i = 0; i < 9; i++)
{
for(j = 0; j < 11; j++)
{
foo[i][j] = (i * 11) + j;
}
}
fprintf(stderr, "calling writeRows\n");
/* populate the TwoDFileObject */
rv = lio_2d_writeBlock_int(tst, &foo[0][0], 0, 0, 9,
11, 11, 9, 0, 0);
fprintf(stderr, "back from writeRows\n");
/* check return code */
if(rv < 0)
{
fprintf(stderr, "unable to write to TwoDFileObject\n");
return 1;
}
printf(stderr, "calling readRows\n");
/* read back a sub-section of the TwoDFileObject */
rv = lio_2d_readBlock_int(tst, &bar[0][0], 0, 0, 6,
7, 7, 6, 0, 0);
fprintf(stderr, "back from readRows\n");
/* check return code */
if(rv < 0)
{
fprintf(stderr, "unable to read from TwoDFileObject\n");
return 1;
}
/* print out the results of the read */
for(i = 0; i < 6; i++)
{
for(j = 0; j < 7; j++)
{
fprintf(stderr, "%d, ", bar[i][j]);
}
fprintf(stderr, "\n");
}
return 0;
}
Example B shows a C++ application that reads a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so that the biggest subfile is 4x5 and the smallest is 3x4. After reading the two-dimensional file with readCol, the program deletes the file object.
#include<stdio.h>
#include<legion/lib_TwoDFileObject.h>
main()
{
int c;
int i, j, rv;
// initialize the legion libraries
BasicFiles_init();
TwoDFileObject<int> tst;
LegionArray<int> *bar;
fprintf(stderr, "calling open\n");
// open the TwoDFileObject
rv = tst.open("TwoDFileTestFile");
fprintf(stderr, "back from open\n");
// check return value
if(rv < 0)
{
fprintf(stderr, "unable to open TwoDFileObject\n");
return 1;
}
fprintf(stderr, "calling readCols\n");
// read three columns, skipping over two columns
// between each column read
bar = tst.readCols(1, 3, 3);
fprintf(stderr, "back from readCols\n");
// check return value
if(bar == NULL)
{
fprintf(stderr, "unable to read from TwoDFileObject\n");
return 1;
}
// print results of read
for(i = 0; i < 9; i++)
{
for(j = 0; j < 3; j++)
{
c = bar->GetElement(i, j);
fprintf(stderr, "%d, ", c);
}
fprintf(stderr, "\n");
}
fprintf(stderr, "calling destroy\n");
// delete the TwoDFileObject
rv = tst.destroy();
fprintf(stderr, "back from destroy\n");
return rv;
}
Example C shows a C++ application that creates a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so the biggest subfile is 4x5 and the smallest is 3x4. After creating a two-dimensional file, the program fills it in with a writeBlock call. Finally, the program reads back a sub-block of the file with readRow calls.
#include<stdio.h>
#include<legion/lib_TwoDFileObject.h>
main()
{
int c;
int i, j, rv;
// initialize the legion libraries
BasicFiles_init();
TwoDFileObject<int> tst;
LegionArray<int> foo(9, 11);
LegionArray<int> *bar;
fprintf(stderr, "calling create\n");
// create the TwoDFileObject and all of its subfiles
rv = tst.create("TwoDFileTestFile", 11, 9, 3, 2);
fprintf(stderr, "back from create\n");
// check return value
if(rv < 0)
{
fprintf(stderr, "unable to create TwoDFileObject\n");
return 1;
}
// initialize the array
for(i = 0; i < 9; i++)
{
for(j = 0; j < 11; j++)
{
c = (i * 11) + j;
foo.SetElement(i, j, c);
}
}
fprintf(stderr, "calling writeRows\n");
// populate the TwoDFileObject
rv = tst.writeRows(&foo, 0, 1, 9);
fprintf(stderr, "back from writeRows\n");
// check return value
if(rv < 0)
{
fprintf(stderr, "unable to write to TwoDFileObject\n");
return 1;
}
fprintf(stderr, "calling readRows\n");
// read back every other row of the file
bar = tst.readRows(0, 2, 5);
fprintf(stderr, "back from readRows\n");
// check return value
if(bar == NULL)
{
fprintf(stderr, "unable to read from TwoDFileObject\n");
return 1;
}
// print out the results of the read
for(i = 0; i < 5; i++)
{
for(j = 0; j < 11; j++)
{
c = bar->GetElement(i, j);
fprintf(stderr, "%d, ", c);
}
fprintf(stderr, "\n");
}
return 0;
}
Example D shows a Fortran application that creates a 11x9 two-dimensional file consisting of six subfiles. The biggest and smallest subfile do not differ by more than one row and one column, so the biggest subfile is 4x5 and the smallest is 3x4. After creating the file, the program fills it in with a writeBlock call. Finally, the program reads back a subsection of the file with the readElements call.
program main implicit none integer c, i, j, rv, fd, ierr integer foo(9,11), bar(9,11) c initialize the legion libraries call liof_init(ierr) write (6,*) 'calling create' c create the TwoDFileObject and all of its subfiles call liof_2d_create_integers(fd, +'TwoDFileTestFile', 11, 9, 3, 2, ' ', ierr) write (6,*) 'back from create' c check the return value if(ierr.lt.0) then write (6,*) 'unable to create TwoDFileObject' stop endif c initialize the array do 100 j=1,11 do 200 i=1,9 foo(i, j) = (i - 1) * 11 + (j - 1) 200 continue 100 continue write (6,*) 'calling writeRows' c populate the TwoDFileObject call liof_2d_writeElements_integers(fd, foo, 1, +1, 1, 1, 9, 11,11, 9, 1, 1, ierr) write (6,*) 'back from writeRows' c check the return value if(ierr.lt.0) then write (6,*) 'unable to write to TwoDFileObject' stop endif write (6,*) 'calling readRows' c read back a sub section of the file call liof_2d_readElements_integers(fd, bar, 1, 1, +2, 2, 5, 6,11, 9, 1, 1, ierr) write (6,*) 'back from readRows' c check the return value if(ierr.lt.0) then write (6,*) 'unable to read from TwoDFileObject' stop endif c print out the results do 300 j = 1,6 do 400 i=1,5 write (6,*) bar(i,j) 400 continue 300 continue stop end
C++:
int TwoDFileObject<TYPE>::create(char *name,
long long rowSize,
long long colSize,
long long numSubRows,
long long numSubCols,
char *vaults_context)
C:
int lio_2d_create_<TYPE>(int *fd, char *name,
long long rowSize,
long long colSize,
long long numSubRows,
long long numSubCols,
char *vaultsContext)
FORTRAN:
subroutine liof_2d_create_<TYPE>(fd, name, rowSize, colSize,
numSubRows, numSubCols, vaultsContext, ierr)
integer fd
character name(*)
integer rowSize, colSize, numSubRows, numSubCols
character vaultsContext
integer ierr
Creates a TwoDFileObject with the given name. The file will be able to contain rowSize by colSize elements of the specified type. The file will be partitioned into numSubRows by numSubCols blocks each of which contains rowSize/numSubRows by colSize/numSubCols elements. Use the vaultsContext variable to specify a context path (full or relative) that contains a list of vaults where the subfile objects can be placed. In the C and FORTRAN interfaces a handle to the file is returned in fd. The function returns a negative number if it is unable to create the file.
C++:
int TwoDFileObject<TYPE>::open(char *name)
C:
int lio_2d_open_<TYPE>(int *fd, char *name)
FORTRAN:
subroutine liof_2d_open_<TYPE>(fd, name, ierr)
integer fd
character name(*)
integer ierr
Opens an existing TwoDFileObject with the given name. For the C and FORTRAN interfaces a handle to the file is returned in fd the function returns a negative number if an error occurs.
C++:
int TwoDFileObject<TYPE>::destroy()
C:
int lio_2d_destroy_<TYPE>(int fd)
FORTRAN:
subroutine liof_2d_destroy_<TYPE>(fd, ierr)
integer fd, ierr
This function deletes the TwoDFileObject and all its subfiles. Note that unlike the Unix file system destroy causes all future file operations performed by other clients to fail.
C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readRows(
long long startRow,
long long stride,
long long num)
C:
int lio_2d_readRows_<TYPE>(int fd, <type> *data,
long long startRow,
num, int data_width,
int data_height, int data_x, int data_y)
FORTRAN:
subroutine liof_2d_readrows_<TYPE>(fd, data, startRow,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, ierr
Read num rows of the file starting at startRow. The read request will advance by stride rows until it has fetched all num rows. For the C++ interface, a LegionArray consisting of rowsize by num elements will be returned. If an error occurs while reading, a NULL pointer is returned instead. For the C and FORTRAN interfaces the results are returned in the data buffer and a negative number is returned if a read error occurred. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
int TwoDFileObject<TYPE>::writeRows(
LegionArray<TYPE> *data,
long long startRow,
long long stride,
long long num)
C:
int lio_2d_writeRows_<TYPE>(int fd, <type> *data,
long long startRow,
long long stride,
long long num,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_writerows_<TYPE>(fd, data, startRow,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, ierr
Write num rows to the file starting at startRow. The write request will advance by stride rows until it has written all num rows. A zero is returned on a successful write or a negative number if there was an error. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readCols(
long long startCol,
long long stride,
long long num)
C:
int lio_2d_readCols_<TYPE>(int fd, <type> *data,
long long startCol,
long long stride,
long long num,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_readcols_<TYPE>(fd, data, startCol,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, ierr
Read num columns of the file starting at startCol. The read request will advance by stride columns until it has fetched all num columns. For the C++ interface a LegionArray consisting of num by colsize elements will be returned. If an error occurs while reading, a NULL pointer is returned instead. For the C and FORTRAN interfaces the results are returned in the data buffer and a negative number is returned if a read error occurred. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
int TwoDFileObject<TYPE>::writeCols(
LegionArray<TYPE> *data,
long long startCol,
long long stride,
long long num)
C:
int lio_2d_writeCols_<TYPE>(int fd, <type> *data,
long long startCol,
long long stride,
long long num,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_writecols_<TYPE>(fd, data, startCol,
stride, num, data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, ierr
Write num columns to the file starting at startCol. The write request will advance by stride columns until it has write all num columns. A zero is returned on a successful write or a negative number if there was an error. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readBlock(
long long startRow,
long long startCol,
long long numRows,
long long numCols)
C:
int lio_2d_readBlock_<TYPE>(int fd, <type> *data,
long long startRow,
long long startCol,
long long numRows,
long long numCols,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_readblock_<TYPE>(fd, data,
startRow, startCol, numRows, numCols,
data_width, data_height, data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, ierr
Read a block of the file that contains all elements in the rectangle defined by the points (startRow, startCol) and (startRow + numRows, startCol + numCols). For the C++ interface a LegionArray consisting of numRows by numCols elements will be returned. If an error occurs while reading, a NULL pointer is returned instead. For the C and FORTRAN interfaces the results are returned in the data buffer and a negative number is returned if a read error occurred. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
int TwoDFileObject<TYPE>::writeBlock(
LegionArray<TYPE> *data,
long long startRow,
long long startCol,
long long numRows,
long long numCols)
C:
int lio_2d_writeBlock_<TYPE>(int fd, <type> *data,
long long startRow,
long long startCol,
long long numRows,
long long numCols,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_writeblock_<TYPE>(fd, data,
startRow, startCol, numRows, numCols,
data_width, data_height, data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, ierr
Write a block to the file that covers all elements in the rectangle defined by the points (startRow, startCol) and (startRow + numRows, startCol + numCols). A zero is returned on a successful write or a negative number if there was an error. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readElements(
long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long numCol)
C:
int lio_2d_readElements_<TYPE>(int fd, <type> *data,
long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long numCol,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_readelements_<TYPE>(fd, data,
startRow, startCol,
strideRow, strideCol,
numRow, numCol,
data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, strideRow, strideCol
integer numRow, numCol, data_width, data_height, data_x
integer data_y, ierr
Read a block of the file that contains elements in the rectangle defined by the points (startRow, startCol) and (startRow + strideRow * numRows, startCol + strideCol * numCols). The read will advance by strideRow and strideCol within the defined block. For the C++ interface a LegionArray consisting of numRows by numCols elements will be returned. If an error occurs while reading, a NULL pointer is returned instead. For the C and FORTRAN interfaces the results are returned in the data buffer and a negative number is returned if a read error occurred. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
int TwoDFileObject<TYPE>::writeElements(
LegionArray<TYPE> *data,
long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long numCol)
C:
int lio_2d_writeElements_<TYPE>(int fd, <type> *data, long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long numCol,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_writeelements_<TYPE>(fd, data,
startRow, startCol,
strideRow, strideCol,
numRow, numCol,
data_width, data_height,
data_x, data_y, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, strideRow, strideCol
integer numRow, numCol, data_width, data_height, data_x
integer data_y, ierr
Write a block to the file that covers elements in the rectangle defined by the points (startRow, startCol) and (startRow + strideRow * numRows, startCol + strideCol * numCols). The write will advance by strideRow and strideCol within the defined block. A zero is returned on a successful write or a negative number if there was an error. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
LegionArray<TYPE> *TwoDFileObject<TYPE>::readSequential(
long long startElement,
long long num)
C:
int lio_2d_readSequential_<TYPE>(int fd,
<type> *data long long startElement,
long long num,
int data_leng, int data_off)
FORTRAN:
subroutine liof_2d_readsequential_<TYPE>(fd,
data startElement, num,
data_leng, data_off, ierr)
integer fd
<TYPE> data(*)
integer startElement, num, data_leng, data_off, ierr
Read the file as if it were a sequential file, starting at startElement and reading num elements. For the C++ interface a LegionArray consisting of one by num elements will be returned. If an error occurs while reading, a NULL pointer is returned instead. For the C and FORTRAN interfaces the results are returned in the data buffer and a negative number is returned if a read error occurred. The C and FORTRAN interfaces also have a data_leng parameter, which is used to specify the length of the data buffer. The data_off parameter is used to specify the starting offset in the data buffer.
C++:
int TwoDFileObject<TYPE>::writeSequential(
LegionArray<TYPE> *data,
long long startElement,
long long num)
C:
int lio_2d_writeSequential_<TYPE>(int fd, <type> *data,
long long startElement,
long long num,
int data_leng, int data_off)
FORTRAN:
subroutine liof_2d_writesequential_<TYPE>(fd,
data, startElement,
num, data_leng, data_off, ierr)
integer fd
<TYPE> data(*)
integer startElement, num, data_leng, data_off, ierr
Write to the file as if it were a sequential file, starting at startElement and writing num elements. If an error occurs while writing, a negative number is returned. The C and FORTRAN interfaces also have a data_leng parameter, which is used to specify the length of the data buffer. The data_off parameter is used to specify the starting offset in the data buffer.
C++:
long long TwoDFileObject<TYPE>::numRows()
FORTRAN:
subroutine liof_2d_numrows(fd, numRow, ierr)
integer fd, numRow, ierr
Returns the number of rows contained in the TwoDFileObject.
C++:
long long TwoDFileObject<TYPE>::numCols()
FORTRAN:
subroutine liof_2d_numcols(fd, numRow, ierr)
integer fd, numRow, ierr
Returns the number of columns contained in the TwoDFileObject.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadRows(
long long startRow,
long long stride,
long long num)
C:
void *lio_2d_nbReadRows_<TYPE>(int fd,
long long startRow,
long long stride, long long num)
FORTRAN:
subroutine liof_2d_nb_readrows_<TYPE>(fd, startRow,
stride, num, req, ierr)
integer fd, startRow, stride, num, req, ierr
Read num rows of the file starting at startRow. The read request will advance by stride rows until it has fetched all num rows. For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteRows(
LegionArray<TYPE> *data,
long long startRow,
long long stride,
long long num)
C:
void *lio_2d_nbWriteRows_<TYPE>(int fd, <TYPE> *data,
long long startRow,
long long stride,
long long num,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_nb_writerows_<TYPE>(fd, data,
startRow, stride, num,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, stride, num, data_width, data_height
integer data_x, data_y, req, ierr
Write num rows to the file starting at startRow. The write request will advance by stride rows until it has written all num rows. For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadCols(
long long startCol, long long stride,
long long num)
C:
void *lio_2d_nbReadCols_<TYPE>(int fd,
long long startCol,
long long stride,
long long num)
FORTRAN:
subroutine liof_2d_nb_readcols_<TYPE>(fd, startCol,
stride, num, req, ierr)
integer fd, startCol, stride, num, req, ierr
Read num columns of the file starting at startCol. The read request will advance by stride columns until it has fetched all num columns. For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteCols(
LegionArray<TYPE> *data,
long long startCol
long long stride,
long long num)
C:
void *lio_2d_nbWriteCols_<TYPE>(int fd, <TYPE> *data,
long long startCol,
long long stride,
long long num,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_nb_writecols_<TYPE>(fd, data,
startCol, stride, num,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startCol, stride, num, data_width, data_height
integer data_x, data_y, req, ierr
Write num columns to the file starting at startCol. The write request will advance by stride columns until it has written all num columns. For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadBlock(
long long startRow,
long long startCol,
long long numRows,
long long numCols)
C:
void *lio_2d_nbReadBlock_<TYPE>(int fd,
long long startRow,
long long startCol,
long long numRows,
long long)
FORTRAN:
subroutine liof_2d_nb_readblock_<TYPE>(fd,
startRow, startCol,
numRows, numCols,
req, ierr)
integer fd, startRow, startCol, numRows, numCols, integer req, ierr
Read a block of the file that contains all elements in the rectangle defined by the points (startRow, startCol) and (startRow + numRows, startCol + numCols). For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteBlock(
LegionArray<TYPE> *data,
long long startRow,
long long startCol,
long long numRows,
long long numCols)
C:
void *lio_2d_nbWriteBlock_<TYPE>(int fd, <TYPE> *data,
long long startRow,
long long startCol,
long long numRows,
long long numCols,
int data_width, int data_height,
int data_x, int data_y)
FORTRAN:
subroutine liof_2d_nb_writeblock_<TYPE>(fd, data,
startRow, startCol,
numRows, numCols,
data_width, data_height,
data_x, data_y, req, ierr)
integer fd
<TYPE> data(*,*)
integer startRow, startCol, numRows, numCols
integer data_width, data_height, data_x, data_y, integer req, ierr
Write a block to the file that covers all elements in the rectangle defined by the points (startRow, startCol) and (startRow + numRows, startCol + numCols). For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable. The C and FORTRAN interfaces also have data_width and data_height parameters, which are used to specify the dimensions of the data buffer. The data_x and data_y parameters are used to specify the starting offset in the data buffer.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbReadElements(
long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long numCol)
C:
void *lio_2d_nbReadElements_<TYPE>(int fd,
long long startRow,
long long startCol,
long long strideRow,
long long strideCol,
long long numRow,
long long)
FORTRAN:
subroutine liof_2d_nb_readelements_<TYPE>(fd,
startRow, startCol,
strideRow, strideCol,
numRow, numCol, req, ierr)
integer fd, startRow, startCol, strideRow, strideCol
integer numRow, numCol, req, ierr
Read a block of the file that contains elements in the rectangle defined by the points (startRow, startCol) and (startRow + strideRow * numRows, startCol + strideCol * numCols). The read will advance by strideRow and strideCol within the defined block. For the C++ and C interfaces a pointer to the request is returned, or a NULL pointer if an error occurs. For the FORTRAN interface the request is returned through the req variable.
C++:
TwoDFileRequest *TwoDFileObject<TYPE>::nbWriteElements(
LegionArray<TYPE> *data,
long long startRow,
long long startCol,