AstroGrid

Collaborative, grid-based research environment for astronom

AstroGrid-D is a pioneering D-Grid project. The community grid was established by five research institutions in the fields of astronomy and astrophysics in conjunction with grid expert groups from computer science institutions and super computer centres. Its objective is to create a grid-based, collaborative research environment for the community.

The field of astronomy has had considerable impetus in establishing the Grid and handling astronomical data has been one of the driving forces behind the development of e-Science in the past few years. Astronomy-based initiatives number among the founders of the range of national grid initiatives throughout the world. Although the role of astronomical research is similar to that played by particle physics in preparing the LHC experiment at CERN, it is more acutely focused towards data aspects. An example would be the LOFAR radio telescope which in operation can generate up to 150 GBit/s of raw correlation data and an annual raw data volume of many petabytes. Grid-based methods make it possible to store, process and scientifically evaluate data volumes of this size. AstroGrid-D has created a grid based on Globus middleware in the field of astronomy that integrates the available resources of the participating institutes in a grid infrastructure.

These resources are available to the members of the "AstroGrid" virtual organisation (VO). Furthermore, resources from the D-Grid special investments are integrated via the middleware and are available to all D-Grid VOs.

A well-developed, on-site support structure exists in almost all astronomy institutions for supporting members of virtual organisations in all issues pertaining to certification through registration authorities (RAs). A VOMRS (Virtual Organisation Membership Registration System) concept developed by AstroGrid-D is used in combination with Java HTTP servlets and local user administration as a cross-middleware system in the D-Grid.

Gaps in the information provided by the middleware systems about the Grid status, the various pieces of information from the different middleware services that can only be combined with difficulty and the embryonic display options, all contributed to the utilisation of Stellaris on a much larger scale as the central information service in AstroGrid-D. On this basis, a number of tools were realised (including monitoring tools such as a timeline and a grid map) that can display information about the resource consumption of a grid job for each user in a browser.

The standard components of the Globus middleware were enhanced with the GridWay Metascheduler. This broker handles the distribution of grid jobs to the available resources. The usefulness of this broker has proved itself especially in situations where special resources such as so-coalled GRAPE boards (a special purpose hardware for calculating gravitational interactions) were in use.

Data management in AstroGrid-D encompasses the processing of data in various formats, ranging from files and databases  to data streams. File staging and replication management form the basis for the distributed file management. Two standard software tools from Globus Toolkit 4 are used: GridFTP for the file transfer and the Replica Location Service (RLS) for replication management. For example, the ADM (AstroGrid-D Data Management System) was implemented for the use of the GridWay broker, which facilitates the use of the RLS.  

The grid middleware OGSA-DAI allows to integrate databases. AstroGrid-D was involved in the further development of this component as a beta tester. AstroGrid-D users are authenticated using their grid certificates. This guarantees access to the data resources without the need for the administrative effort associated with database accounts. Data stream management is also available in all resources in AstroGrid-D. Scientists can encapsulate typical processing steps, describe them in Stellaris and store them. This allows cooperating researchers to identify helpful operations and to use them repeatedly. In particular, astronomers can use the grid certificates to verify that the operation originated from a trustworthy source.

Further enhancements to the middleware that were used or developed further in AstroGrid-D are GridSphere, a browser-based interface for grid middleware and GAT (the Grid Application Toolkit), which provides programmers with uniform access to a variety of middleware.

The integration of robotic telescopes in the grid was of special interest to AstroGrid-D. A software package was implemented that realises this task by using the RTML standards of the community, Globus middleware and Stellaris.

With over 20 use cases, AstroGrid-D has included many of the community"s main applications and ported some of them to the grid. Some of these use cases, such as GEO600, have reached the production stage and are currently consuming around 60 000CPUh per day in the D-Grid. The experience gained has been documented in the "App2Grid-Guide" and has made it significantly easier to port similar applications to the grid. Comprehensive instructions and notes also exist for the contribution of resources to AstroGrid-D.

The International Virtual Observatory Alliance (IVOA) develops worldwide standards and tools for accessing data in astronomical archives. GAVO represents Germany in the IVOA. AstroGrid-D cooperated with GAVO to connect the grid"s existing infrastructure with the tools of the Virtual Observatory. An example of the results of this cooperation is the TheoSSA service, which creates synthetic stellar spectra from an IVAO-compatible web service on grid resources.