Vol. 27 · No. 2

The Interactive European Grid (i2g) project has set up an advanced e-Infrastructure in the European Research Area specifically oriented to support the friendly execution of demanding interactive applications. While interoperable with existing large e-Infrastructures like EGEE, i2g software supports execution of parallel applications in interactive mode including powerful visualization and application steering. This article describes the strategy followed, the key technical achievements, examples of applications that benefit from this infrastructure and the sustainable model proposed for the future.

The int.eu.grid project aims at providing a production quality grid computing infrastructure for e-Science supporting parallel and interactive applications. The infrastructure capacity is presently about 750 cpu cores distributed over twelve sites in seven countries. These resources have to be tightly coordinated to match the requirements of parallel computing. Such an infrastructure implies high availability, performance and robustness resulting in a much larger management effort than in traditional grid environments which are usually targeted to run sequential non-interactive applications. To achieve these goals the int.eu.grid project offers advanced brokering mechanisms and user friendly graphical interfaces supporting application steering. The int.eu.grid environment is deployed on top of the gLite middleware enabling full interoperability with existing gLite based infrastructures.

Execution of parallel and interactive applications on a Grid environment is a challenging problem that requires the cooperation of several middleware tools and services. In this paper, we present our experiences in the development of CrossBroker, a job management service that provides transparent and reliable support for such types of applications. We outline the main components of CrossBroker and how they interact with other middleware services. We also describe specific features of the scheduler used to guarantee resource co-allocation for running MPI jobs remotely over multiple machines spread across several Grid sites or to start interactive applications as fast as possible. These features include a simple time-sharing mechanism that allows fast execution of interactive applications even under heavy occupancy of Grid resources.

In Grid computing, the dominating paradigm is batch processing. Grid middleware ships with batch-job support only, while lacking support for interactive applications. The reason is that grid middleware was developed for compute intensive jobs, which may run for a long time before a result becomes available. This leads to a ``post-mortem'' approach of analysing the output, possibly resulting in a waste of computing and research time. Adding the possibility to observe and steer the job during execution enables the researcher to modify job-parameters without restarting the entire job. In this paper, several interactivity support techniques are explored, followed by several examples proving their usefulness.

Grids as infrastructures offer access to computing, storage and other resources in a transparent way. The user does not have to be aware where and how the job is being executed. Grid clusters in particular are an interesting target for running computation-intensive calculations. Running MPI-parallel applications on such clusters is a logical approach that is of interest to both computer scientists and to engineers. This paper gives an overview of the issues connected to running MPI applications on a heterogenous Grid consisting of different clusters located at different sites within the Int.EU.Grid project. The role of a workload management system (WMS) for such a scenario, as well as important modifications that need to be made to a WMS oriented towards sequential batch jobs for better support of MPI applications and tools are discussed. In order to facilitate the adoption of MPI-parallel applications on heterogeneous Grids, the application developer should be made aware of performance problems, as well as MPI-standard issues within its code. Therefore tools for these issues are also supported within Int.EU.Grid. Also, the special case of running MPI applications on different clusters simultaneously as a more Grid-oriented computational approach is described.

Considering how complex Grid technology is, it is essential to provide adequate support on users' level in order to simplify interaction and attract users. Intuitive and easy-to-use graphical user interfaces could be critical for the usefulness of the whole infrastructure and may play a significant role within successful stories. We would like to present the int.eu.grid approach to a Migrating Desktop product that brings the grid interactive services to a user’s desktop. The Migrating Desktop as the framework for grid applications aims at providing a unified and intuitive graphical work environment that allows users to control sophisticated interactive services, access Grid resources, run sequential and parallel jobs – using both batch and interactive paradigms. The Migrating Desktop provides a front-end for embedding some of the application mechanisms and interfaces, and it allows the user to have virtual access to Grid resources. Generic API for applications based on the OSGi specification provides mechanisms for the job submission phase, interaction with an application and remote visualization of the results which assemble the powerful platform for the Grid environment.

General purpose Problem Solving Environments (PSEs) like Matlab are widely used in the fields of science for development of new algorithms. If a lot of computing power is required to run these algorithms, today's PSEs lack support for accessing the distributed infrastructures of the organisation (i.e. grids), which limits the size of the problems that can be solved. This contribution shows a new approach to utilize the grid from within PSEs without major adjustments by the user. The primary tools are GridSolve and and the grid-middleware gLite. The applicability is illustrated by an exemplary algorithm (Mandelbrot calculations).

Modern open source high-level languages such as R and Python are increasingly playing an important role in increasing programmer productivity when programming high-performance computers. In this article, we describe Python Star-P, a high-level interactive parallel programming environment in Python. We discuss the architecture of the environment and the programming model along with a number of examples. We also describe the performance of the examples on a cluster of multi-core machines. Finally, we compare our environment with that of other existing parallel computing tools for Python and describe the advantages of our model over others.

Within the framework of the int.eu.grid project, IVISDEP, which stands for Interactive Visualizer of an Integrator of Stochastic Differential Equations for Plasmas, has been successfully ported and executed. It is an example of integration of many of the innovative features that this grid infrastructure offers. The application itself simulates and visualizes the evolution of the plasma inside a stellarator fusion device. The followed approach implies the simulation of a great number of trajectories of particles, whose positions are calculated among a large number of nodes in the Grid. The graphical interface allows the user to see and modify the physical and computational parameters of the simulation interactively, making this application a valuable tool for researchers in plasma physics.

This paper describes two different approaches to exploiting interactivity and MPI support available in the Interactive European Grid project. The first application is an air pollution simulation using Lagrangian trajectory model to simulate the spread of pollutant particles released into the atmosphere. The performance of the sequential implementation of the application was not satisfactory, therefore a parallelization was planned. The MPI programming model was used because of some previous experience with it and its support in the grid infrastructure to be used. Then the interactivity enabling the user to receive visualizations of simulation steps and to exercise control over the application running in the grid was added. The user interface for interacting with the application was implemented as an plug-in into the Migrating Desktop user interface client platform. The other application is an interactive workflow management system, which is a modification of a previously developed system for management of applications composed of web and grid services. It allows users to manage more complex jobs, composed of several program executions, in an interactive and comfortable manner. The system uses the interactive channel of the project to forward commands from a GUI to the on-site workflow manager, and to control the job during execution. This tool is able to visualize the inner workflow of the application. User has complete in-execution control over the job, can see its partial results, and can even alter it while it is running. This allows not only to accommodate the job workflow to the data it produces, extend or shorten it, but also to interactively debug and tune the job.

This work proposes a joint implementation of spatially distributed runoff and soil erosion analysis in watersheds allowing subsequent modelization of nutrients transport processes originating from distributed sources. Implemented relying on the open source GRASS (Geographic Resources Analysis Support System) GIS (Geographical Information System), a new design for the raster operation routines is specially created to take advantage of the MPI possibilities and available GRID resources.