The ARCO Parallel Seismic Environment
                     
Charles Mosher and Siamak Hassanzadeh


Major Application Field:
 Seismic Data Processing
Application Subfield(s):
 Parallel I/O, signal processing, solution of PDE's


Background

The application began as a prototype system for seismic data processing
on parallel computing architectures.  The prototype was used to design
and implement production seismic processing on ARCO's Intel iPSC/860, where
it is used today.

Like other companies, ARCO continues to upgrade our HPC facilities.  We found
that we were spending a large amount of time on benchmarking, as were other
companies in the oil industry.  We decided to place our system in the public
domain as a benchmark suite, in the hopes that the benchmarking effort could
be spread across many participants. In addition, we hope to use the system
as a mechanism for code development and sharing between academia, national
labs, and industry.

Our first attempt was to work with the Perfect Benchmark Club at the
University of Illinois Center for Supercomputing Research and Development.
Many members of that group provided valuable input that significantly
improved the structure and content of the suite.  Special thanks to 
David Schneider for his work on organizing and managing the Perfect effort.

The Perfect effort was merged with the SPEC (Standard Performance Evaluation
Corporation)/HPG(High Performance Group) effort to establish, maintain, and 
endorse a suite of benchmarks, representative of real world high-performance 
computing applications, for standardized, cross-platform performance comparison
on a level playing field.  

In order to ensure relevance of the codes in the suite, the authors plan
to retain control of the source and algorithms contained therein, and request
that suggestions for changes and updates be directed to the authors only.


The length in bytes of integers and floating-point numbers used in this application:

        Integers:   4 bytes
        Floats  :   4 bytes


Documentation describing the implementation of the application:

High level: ARCO Parallel Seismic Environment Users's Guide
Low  level: source comments


Research papers describing sequential code and/or algorithms:

Yilmaz, Ozdogan, 1990, Seismic Data Processing: Investigations in Geophysics
    vol. 2, Society of Exploration Geophysicists, P.O. Box 702740,
    Tulsa, Oklahoma, 74170


Research papers describing parallel code and/or algorithms:

Mosher, C., Hassanzadeh, S., and Schneider, D., 1992, A Benchmark Suite 
    for Parallel Seismic Processing, Supercomputing 1992 proceedings.


Application available in the following languages (give message passing
system used, if applicable, and machines application runs on):

Language:
    Fortran 77, C

Message Passing:
    Yet Another Message Passing Layer (YAMPL)
	Sample implementations for PVM, Intel NX, TCGMSG

Machines Supported:
	Workstation clusters and multiprocessors (i.e. Sun, Dec, HP, IBM, SGI)
	Cray YMP 
	Intel iPSC/860
        IBM SP2


Total number of lines in source code:
 ~ 20000
Number of lines excluding comments:
 ~ 15000
Size in bytes of source code:
 ~ 1 MByte 

List input files (filename, number of lines, size in bytes, and if formatted):

ASCII parameter files, 10-100 lines


List output files (filename, number of lines, size in bytes, and if formatted):

Binary seismic data files, 100 MByte (small),  1 GByte (medium), 
                          10 Gbyte (large), 100 Gbyte (huge)



Brief, high-level description of what application does:

Synthetic seismic data for small, medium and large test cases are generated
in the native format of the target machine.  The test data are read and
processed in parallel, and the output is written to disk.  Simple checksum
and timing tables are printed to standard output.


Main algorithms used: 

Signal processing (FFT's, Toepplitz equation solvers, interpolation)
Seismic Imaging (Fourier domain, Kirchhoff integral, finite difference algorithms)


Skeleton sketch of application:

Processing modules are applied in a pipeline fashion to 2D arrays of seismic
data read from disk.  Processing flows are of the form READ-FLTR-MIGR-WRIT.
The same flow is executed on all processors.  Individual modules communicate
via message passing to implement parallel algorithms.  Nearly all message
passing is hidden via transpose operations that change the parallel data 
distribution as appropriate for each algorithm.


Brief description of I/O behavior:

2D arrays are read/written from HDF style files on disk.  Parallel I/O is
supported for both a single large file read by multiple processors, and a
a separate file read by each processor.  A significant part of the seismic
processing flow requires data to be read in transposed fashion across all
processors.


Brief description of load balance behavior:

Assumes a homogeneous array of processors with similar capabilities.
Load balance is rudimentary, with an attempt to distribute equal-sized
'workstation' chunks of work.


Describe the data distribution (if appropriate):

Seismic data is inherently parallel, with large data sets that offer mutliple
opportunities for parallel operation.  Typically, the data is treated as a
collection of 2D arrays, with each processor owning a 'slab' of data.


Parameters of the data distribution (if appropriate):

The data is defined as a 4-dimensional array with Fortran dimensions
(sample, trace, frame, volume).  The third dimension (frame) is typically
spread across the processors.


Parameters that determine the problem size:

The ASCII parameter files define the data set size in terms of the number
of samples per seismic traces, the number of traces per shot, the number
of shooting lines, and the number of 3D volumes.


Memory as function of problem size:

Requires enough memory to hold 2 frames on each node, and a 3D volume
spread across the node.


Number of floating-point operations as function of problem size:

Reported by code as appropriate. On a Cray YMP, medium sized problems with
750 MB of output run at 30-100 Mflops for about an hour.


Communication overhead as function of problem size and data distribution:

On an Intel iPSC/860, there are parts of the suite that have comp/comm
ratios ranging from near infinite to 1/10.


Problem sizes: small, medium, large, huge, and ultra (estimated sizes of I/O
files and number of floating point operations):

Trace Data sizes:(MB)
                       small  medium  large   huge   ultra  test
  velocity               2      27     164     750    8470    1
  input                 96    1536   18432   90000 3840000   16
  stack                  5      70     420    1875   16940    2
  time mig               5      70     420    1875   16940    2
  depth mig              2      27     164     750    8470    1
  Est. CPU time        0.5       8      48     240    2400  0.2
  (hours @ 100 Mflops)



 

The following application characteristics are described:

Type of Problem

Lines of code

 
Fortran   12950
C          5934
---------------
Total     18884
 

 
Fortran     229
C           119
---------------
Total       348