META-BROKER FOR FUTURE GENERATION GRIDS: A NEW APPROACH FOR A HIGH-LEVEL INTEROPERABLE RESOURCE MANAGEMENT

Attila Kert sz e
Institute of Informatics, University of Szeged H-6701 Szeged, P.O. Box 652, Hungary MTA SZTAKI Computer and Automation Research Institute H-1518 Budapest, P. O. Box 63, Hungary CoreGRID Institute on Resource Management and Scheduling
keratt@inf.u-szeged.hu

P ter Kacsuk e
MTA SZTAKI Computer and Automation Research Institute H-1518 Budapest, P. O. Box 63, Hungary CoreGRID Institute on Resource Management and Scheduling
kacsuk@sztaki.hu

Abstract

As grid technology matures the number of production grids dynamically increases. The management and the optimal utilization of these grid resources cannot be handled by the users themselves. One very important aspect of utilizing multiple Grids within one complex scientic workow is to interact with multiple and potentially different grid brokers supported by different Grids. To ease the utilization of different resource brokers, we have introduced a meta-brokering concept to solve grid interoperability. With this new grid middleware component users and portals can utilize a growing number of Grids simultaneously for highly computation intensive applications in the future. Grid, Grid interoperability, Grid resource management, Grid broker, Grid scheduler

Keywords:

This research work is carried out under the FP6 Network of Excellence CoreGRID funded by the European Commission (Contract IST-2002-004265)

54

GRID MIDDLEWARE AND SERVICES: CHALLENGES AND SOLUTIONS

1.

Introduction

The Grid was originally proposed as a global computational infrastructure to solve grand-challenge, computational intensive problems [1]. The rst decade of grid research aimed at creating relatively reliable infrastructures to serve researchers and attract users. The present grid systems are mature enough to be used in production, therefore current research efforts are focusing on user requirements and interoperability. As grid technology matures the number of production grids dynamically increases, therefore end-users typically access grid resources through resource management systems or grid portals that serve as both application developer and executor environments. Unfortunately, these tools are typically tightly coupled to one specic grid environment and do not provide multi-grid support. Even if a tool is connected to multiple grids, applications that utilize services from these grids simultaneously are not supported. To enhance the manageability of grid resources and users, Virtual Organizations (VO) were founded. This kind of grouping has started an isolation process in grid development, too. Interoperability among these islands will play an important role in grid research and usage. There have been several attempts to make existing production Grids and grid services interoperable. Grid researchers seem to follow two different ways in the area of resource management: The rst one is to extend existing resource brokers with multiple grid middleware support. The Gridbus Grid Service Broker [7] is designed for computational and data-grid applications and supports all Globus [2] [3] middleware and Unicore [4] in experimental phase. Gridway [8] has been developed in a Globus incubation project, therefore it supports all Globus versions, and it also supports EGEE [5] [6] utilization. The JSS [9] is a decentralized resource broker that is able to utilize both GT4 [3] and NorduGrid [10] resources. GTbroker [11] is a lightweight Globus-based grid broker that can simultaneously utilize Globus and LCG-2 [5] resources. The GRIP [12] broker is the one that tries to support interoperability with a semantic matching of the resource descriptions enabling job submissions to Globus and Unicore sites. This summary shows that these tools are forming separate user groups, again. They use different job descriptions and do not communicate with each other: stopping this separation process would need high efforts by all parties. The second approach is to provide a higher level tool that supports different middleware services, including job submission, brokering or storage access. Possible instances of this approach are grid portals. The well known ones are Pegasus [13], GridFlow [14], K-Wf [15] grid portal, SPA portal of the HPC-Europa Project [16] and the P-GRADE portal [17]. Though the rst 3 examples provide high-level access to grid services, they usually operate only on one middleware. The SPA (Single Point of Access) is a portal component

Meta-Broker for Future Generation Grids

55

that enables brokers to be utilized through plug-in interfaces. These interface methods need to be used by all brokers, providing the same abstract functionality. The P-GRADE Portal is a workow-oriented multi-grid portal with the main goal to support all stages of grid workow development and execution processes. It supports the execution of these workows in multiple Globus-, and EGEE-based computational grids relying on user credentials. This portal is interfacing several grid brokers to reach the resources of different grids in an automated way. Another instance of this approach is a meta-scheduler, which coordinates some communication process between existing schedulers. A part of the SPA operates in a similar way, but it does not take into account the different broker properties, it rather acts as a job submitter. The GSA-RG of OGF [18] is currently working on a project enabling grid scheduler interaction. They try to dene a common interface among schedulers enhancing interoperability. The greatest problem is that the existing schedulers/brokers need to support this interface, so they need to be modied. We have already introduced the meta-brokering approach in [23], where we have stated the basic requirements that are necessary to create a higher level resource management system. In the following sections of this paper we show how the implementation of this system can be realized (which is an instance of the second, above mentioned approach).

Figure 1.

The Meta-Broker Architecture

56

GRID MIDDLEWARE AND SERVICES: CHALLENGES AND SOLUTIONS

2.

Meta-Brokering Approach for high-level resource management

Multiple utilization of the existing, widely used and reliable resource brokers and solving grid interoperability issues through them are still open issues in resource management. Users usually have more certicates to access different VOs. When a user wants to run an application on the grid, he/she needs to answer the following question: which VO, which broker to choose for my specic application? Just like users needed Resource Brokers to choose proper resources within a VO, now they need a meta-brokering service to decide, which broker (or VO) is the best for them and also to hide the differences of utilizing them. The solution of this problem is particularly important when a workow is executed as a parameter sweep application. Without a meta-broker the user (portal) is not able to dynamically and evenly distribute the various workow runs among the connected VOs [19]. Figure 1. shows the revised architecture of a Meta-Broker that enables the users to access resources of different grids through their own brokers.

Figure 2.

Languages of the Meta-Broker: JSDL and BPDL

2.1

Languages for understanding each-other

Heterogeneity appeared not only in the fabric layer of Grids, but also in the middleware. Even components and services of the same middleware may support different ways for accessing them. After a point this variety makes the users and developers life miserable. Languages are one of the most important factors of communication. Different resource management systems use different resource specication languages like RSL, JDL, etc. These documents need to be written by the users to specify all kinds of job-related requirements and

Meta-Broker for Future Generation Grids

57

data. The GGF has already started to take several steps towards interoperability among these coordinating components, and developed a standard resource specication language called JSDL [20]. As the JSDL is general enough to describe jobs of different grids and brokers, we have chosen this to be the job description language of the Meta-Broker. Besides describing user jobs, we also need to describe resource brokers in order to make difference among them. These brokers have various features for supporting different user needs. These needs should be able to be expressed in the users JSDL, and identied by the Meta-Broker for each corresponding broker. Therefore we proposed a Broker Property Description Language (BPDL) [22] similarly to the JSDL , to store metadata about brokers. These two kinds of languages are used by the Meta-Broker to communicate with the inner and outer world (Figure 2).
Table 1. RSL (GTbroker) (*sched=random*) (*sched=CPU/Memory/Disk*) A subset of special job description language attributes. xRSL (NorduGrid) (*sched=random*) (*sched=CPU/ Memory/Disk*) JDL (EGEE) FuzzyRank=true; rank=other.GlueHostProcessorClockSpeed/ GlueHostMainMemoryRAMSize/ GlueSAStateAvailableSpace; Requirements: (GlueHostMainMemoryRAMSize>int); anyMatch(other.storage.CloseSEs,target. GlueSAStateAvailableSpace>int); /*skiptime=int*/ RetryCount=max.10; JSDL extension extension

(*minMemory=int*), (*mindisk=int*)

(memory=int), (disk=int)

<resources> <jsdl:IndividualDiskSpace> <jsdl:IndividualPhysicalMemory> ... </resources> extension extension

(*skiptime=int*) rescheduling by default

(*skiptime=int*) (rerun=max.5)

2.1.1 JSDL for job requirements. The Translator components of the Meta-Broker are responsible for translating the resource specication language dened by the user to the language of the appropriate resource broker that the Meta-Broker selects to use for a given job. Once a utilized broker is capable of supporting the JSDL standard, the corresponding Translator component could be removed from the Meta-Broker (since the input job description is written in JSDL). From all these job specication languages a subset of basic job attributes can be chosen, which can be denoted relatively in the same way in

58

GRID MIDDLEWARE AND SERVICES: CHALLENGES AND SOLUTIONS

each document. The translation of these parts is almost trivial. The rest of these attributes describe special job handling, various scheduling features and remote storage access. Generally these cases can hardly be matched among the different systems, because only few of them support the same solution. We gathered these special attributes of the different job description languages. A sample collection of a minimal set of languages can be seen in Table 1. If an attribute of a job description language cannot be expressed in JSDL, we specify it as an extension. These attributes are collected and specied in a proposed JSDL extension called jsdl-metabroker (the overview of the schema can be seen in Figure 3). Regarding other languages we express the missing attribute in comments in order to keep the translations consistent.

Figure 3.

General XML schema of the jsdl-metabroker JSDL extension

2.1.2 BPDL for broker properties. For describing broker capabilities we proposed a language similar to JSDL. We have created a taxonomy of the recent Resource Brokers [21] from this survey we could easily identify the relevant properties. This information was used to create the attributes of this XML-based language called BPDL [22]. The common subset of the individual broker properties are the basic properties: the supported middleware types, job types, certicates, job descriptions, interfaces. There are also special ones, such

Meta-Broker for Future Generation Grids

59

as remote le handling, fault tolerant features, agreement support and various scheduling policies. The overview of the schema can be seen in Figure 4. The union of these properties forms a complete broker description document that can be lled out and regularly updated for each utilized resource broker.

2.2

Information repository and matchmaking

The Information Collector stores the data of the reachable brokers and historical data of the previous submissions. This information shows whether the chosen broker is available, or how reliable its services are. The previously introduced languages are used for matching the user requests to the description of the interconnected brokers, which is the role of the Matchmaker component. During broker utilization the successful submissions and failures are tracked, and regarding these events a rank is modied for each special attribute in the BPDL of the appropriate broker. The JSDL contains the user request this supposed to be an exact specication of the users job, using the extended attributes. The load of the resources behind the brokers is also taken into account to help the Matchmaker to select the proper environment for the actual job. When too many similar jobs are needed to be handled by the Meta-Broker the so-called best effort matchmaking may ood a broker and its grid. That is the main reason, why load balancing is an important issue. In order to cope with this problem, there are IS Agents in the Information Collector, which regularly check the load of the underlying grids of each connected resource broker, and store this data. With this additional information the matchmaking process can adapt to the load of the utilized grids. During matchmaking the following steps are taken to nd the ttest broker: 1 First the Matchmaker compares the JSDL of the actual job to the BPDL of the registered resource brokers. First the basic attributes are matched against the basic properties: this selection determines a group of brokers that are able to submit the job. 2 In the second phase those brokers are kept, which are able to fulll the special requirement attributes of the job (these capabilities are looked up from the BPDL). 3 Finally a priority list of the remaining brokers is created taking into account the ranks (stored for the requested features) and the load of the underlying grid of each broker. The rst resource broker is chosen from the list.

2.3

How to reach the Resource Brokers

Mainly two different scenarios can be done with our proposed Meta-Broker. The rst one allows the users or portals to invoke and track the brokers them-

60

GRID MIDDLEWARE AND SERVICES: CHALLENGES AND SOLUTIONS

Figure 4.

General XML schema of the BPDL

selves. In this case only the JSDL document of a job needs to be provided for the Meta-Broker, and it responds with the name of the matched broker and its

Meta-Broker for Future Generation Grids

61

own job description (or with a message that none of the registered brokers is able to fulll the specied job requirements). Finally the user/portal needs to provide the result of the submission to the Meta-Broker (to modify the broker property ranks). This limited operation is useful for systems that already have reliable connections to resource brokers and would like to use this service as broker-ltering and inter-grid load balancing. Currently these issues are not taken into account in grid portals. Even multi-grid access is rarely supported, where the users need to choose from a list of resource brokers. Furthermore this utilization can be achieved with minimal adaptation efforts and requires less data transfers. (See the direct user-broker communication in Figure 1.) In the second scenario the utilization of the resource brokers is done by the Meta-Broker. The Invokers are broker-specic components: they communicate with the interconnected brokers, invoking them with job requests and collecting the results. Data handling is also an important task of this component. After the user uploaded the job, proxy and input les to the Meta-Broker, the Matchmaker component tries to nd a proper broker for the request. If no good broker was found, the request is rejected, otherwise the JSDL is translated to the language of the selected broker. The responsible Invoker takes care of transferring the necessary les to the selected grid environment. After job submission it stages back the output les, and upgrades the historical data stored in the Information Collector with the log of the utilized broker. The core component of the MetaBroker is responsible for managing the communication (information and data exchange) among the other components. The communication to the outer world is also done by this part through its web-service interface. The users job description is independent from the execution environment, and the Meta-Broker does not need to know how to access resources of different grids. It is the task of the interconnected brokers to perform the actual job submissions and to nd the best resource within their scopes (the VOs they have access to). The Meta-Broker only needs to communicate with the users (via its web-service interface) and the brokers (via the Invokers). In this sense meta-brokering stands for brokering over resource brokers instead of resources. Notice that the same way as the resource specication language was standardized, a communication protocol between the Meta-Broker and various brokers would be worth standardizing. Once it is done and brokers implement that standard the Invoker components of the Meta-Broker Architecture can be removed. In a well standardized multi-grid environment the Meta-Broker Architecture will contain only three components: Meta-Broker, Matchmaker and Information Collector. The P-GRADE Portal already has interfaces to several grid brokers providing multi-grid usage. Our future work aims at extending the portal with the proposed meta-brokering service to ease the addition of further resource brokers and to make a better utilization of its multi-grid environment (Figure 5).

62

GRID MIDDLEWARE AND SERVICES: CHALLENGES AND SOLUTIONS

Figure 5.

Switching from Multiple Brokering to Meta-Broker Support

3.

Conclusions

The introduced meta-brokering approach opens a new way for interoperability support. The Meta-Broker itself is a standalone Web-Service that can serve both users and portals. We showed in the paper how such a service can be realized based on the latest OGF standards. The design and the architecture of the Grid Meta-Broker enable a higher level resource management by utilizing resource brokers of different grid middleware systems. This service can act as a bridge among the separated islands of the current production Grids and user groups, therefore it enables more benecial resource utilization and collaboration. Recently P-GRADE portal was extended with parameter sweep execution management of workows [19]. This new and eagerly waited feature raised the need for dynamic, well-balanced distribution of workows and workow components among several grids. In the future we will use the Meta-Broker to perform multi-broker utilization.

References
[1] I. Foster, C. Kesselman, Computational Grids, The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, 1998. pp. 15-52. [2] I. Foster C. Kesselman, The Globus project: A status report, in Proc. of the Heterogeneous Computing Workshop, IEEE Computer Society Press, 1998, pp. 4-18.

Meta-Broker for Future Generation Grids

63

[3] Globus Team, Globus Toolkit 4.0 Release Manuals, http://www.globus.org/toolkit/ docs/4.0/ [4] D. W. Erwin and D. F. Snelling., UNICORE: A Grid Computing Environment, In Lecture Notes in Computer Science, volume 2150, Springer, 2001, pp. 825-834. [5] LCG-2 User Guide, 4 August, 2005: https://edms.cern.ch/le/454439/2/LCG-2-UserGuide.html [6] The gLite website, http://glite.web.cern.ch/glite/ [7] S. Venugopal, R. Buyya, L. Winton, A Grid Service Broker for Scheduling e-Science Applications on Global Data Grids, Concurrency and Computation: Practice and Experience, Volume 18, Issue 6, 2006, pp. 685-699. [8] E. Huedo, R. S. Montero, I. M. Llorente, A framework for adaptive execution in grids, Software: Practice and Experience. vol. 34, 7, 2004, pp. 631-651. [9] E. Elmroth and J. Tordsson, An Interoperable Standards-based Grid Resource Broker and Job Submission Service, First IEEE Conference on e-Science and Grid Computing, IEEE Computer Society Press, 2005, pp. 212-220. [10] GridLab Grid Resource Management System (GRMS), http://www.gridlab.org/WorkPackages/wp-9/res/docs/GRMS 1.9.3 UserGuide.pdf [11] A. Kertesz, G. Sipos and P. Kacsuk, Multi-Grid Broker Utilization with the P-GRADE Portal, Post-Proceedings of the Austrian Grid Symposium 2006, OCG Verlag, Austria, 2007. [12] The GRIP project web site, http://www.grid-interoperability.org/grip-workpackages.htm [13] G. Singh et al, The Pegasus Portal: Web Based Grid Computing In Proc. of 20th Annual ACM Symposium on Applied Computing, Santa Fe, New Mexico, 2005. [14] J. Cao, S. A. Jarvis, S. Saini, and G. R. Nudd, GridFlow: WorkFlow Management for Grid Computing, In Proc. of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID03), 2003, pp. 198-205. [15] F. Neubauer, A. Hoheisel and J. Geiler, Workow-based Grid applications, Future Generation Computer Systems, Volume 22, Issues 1-2, January 2006, pp. 6-15. [16] The HPC-Europa Project website, http://www.hpc-europa.org [17] P. Kacsuk and G. Sipos, Multi-Grid, Multi-User Workows in the P-GRADE Grid Portal, Journal of Grid Computing, Vol. 3, No 3-4, 2005, pp. 221-238. [18] OGF Grid Scheduling Architecture Research Group: https://forge.gridforum.org/sf/projects/gsa-rg [19] P. Kacsuk, G. Sipos, A. Toth, Z. Farkas, G. Kecskemeti and G. Hermann, Dening and Running Parametric Study Workow Applications by the P-GRADE Portal, Krakow Grid Workshop, 2006 [20] Job Submission Description Language: http://www.ggf.org/documents/GFD.56.pdf [21] A. Kertesz, P. Kacsuk, A Taxonomy of Grid Resource Brokers, 6th Austrian-Hungarian Workshop on Distributed and Parallel Systems (DAPSYS), Innsbruck, Austria, 2006. [22] A. Kertesz, I. Rodero and F. Guim, BPDL: A Data Model for Grid Resource Broker Capabilities, Technical report, TR-0074, Institute on Resource Management and Scheduling, CoreGRID - Network of Excellence, March 2007 [23] A. Kertesz, P. Kacsuk, Grid Meta-Broker Architecture: Towards an Interoperable Grid Resource Brokering Service, CoreGRID Workshop on Grid Middleware in conjunction with Euro-Par 2006, pp. 112-116, Dresden, Germany, August 28-29, 2006