INDIAN INSTITUTE OF TECHNOLOGY

ROORKEE

Grid Computing
&
Applications
 Networking and Computing
Technologies Advancements

* HTC
Technologies Introduced
COMPUTING

* P2P
* PDAs
* Mainframes * Minicomputers * PCs * Workstations
* Grids
* PC Clusters
* Crays * MPPs
* WS Clusters
NETWORKING

* W3C
* TCP/IP
* Ethernet * HTML * Mosaic * Web Services
* Email
* Internet Era * WWW Era * XML
* ARPANET

1960 1970 1975 1980 1985 1990 1995 2000

 Internet and WWW Growth

10,000,000

1,000,000

100,000 Internet Hosts

Number in millions

10,000

1,000
WWW Servers
100

10
4
1
1969 1970 1975 1980 1985 1990 1995 2000
Why Grids ? Large Scale Exploration
needs them—Killer Applications.
 Solvinggrand challenge applications using
computer modeling, simulation and analysis

Aerospace
Internet &
Life Sciences
Ecommerce

CAD/CAM Digital Biology Military Applications

Cluster of Clusters - Hyperclusters

Cluster 1
Scheduler

Master
Daemon

LAN/WAN
Submit
Graphical Cluster 3
Control Execution
Daemon Scheduler
Clients
Master
Daemon

Cluster 2
Scheduler Submit
Graphical
Master Control Execution
Daemon Daemon
Clients

Submit
Graphical
Control Execution
Daemon
Clients
Grid: Towards Internet Computing
for (Coordinated) Resource Sharing

Grid enables:
❃Resource Sharing
❃Selection

❃Aggregation

- Unification of geographically distributed resources

What is Grid ?
A paradigm/ infrastructure that enables the sharing,
selection, & aggregation of geographically distributed
resources like:
 Computers – PCs, workstations, clusters, supercomputers, laptops,
notebooks, mobile devices, PDA, etc;
 Software – e.g., ASPs renting expensive special purpose applications on
demand;
 Catalogued data and databases – e.g. transparent access to human
genome database;
 Special devices/instruments – e.g., radio telescope – SETI@Home
searching for life in galaxy.
 People/collaborators.
[depending on their availability, capability, cost, and
user QoS requirements] for solving large-scale
problems / applications.
 A Typical Grid Computing
Environment
IIT Roorkee (GRB)

Grid Information Service

Grid Resource Broker

R2 Application
database
R3 R7 R4

R5 RN
Grid Resource Broker
R6
R1
Resource Broker

Grid Information Service

Building and Using Grids requires...
 Services that make our systems Grid Ready!
 Security mechanisms that permit resources to be
accessed only by authorized users.
 New Programming tools that make our applications
Grid Ready!.
 Tools that can translate the requirements of an
application into requirements for computers, networks,
and storage.
 Tools that perform resource discovery, trading,
composition, scheduling and distribution of jobs and
collects results.
Grid@IITR
Grid@IITR using Alchemi
 What is Alchemi?
 Enterprise grid framework and runtime machinery to
create a high-throughput computing environment by
harnessing distributed resources
 .NET-based (Windows)
 Voluntary execution (cycle stealing) or
 Dedicated execution
 LAN or Internet
 Programming environment
 Independent grid threads (.NET API)
 File-based jobs (input, executable, output)
 Web service for interoperability with other grid
middleware
 File-based jobs
 Monitoring, administration tools
Why .NET based ?
 Why .NET/Windows?
 More than 90% of machines worldwide run variants of
Microsoft Windows operating system. Hence designing
for Windows is seen as key factor in industry adoption
of grid computing technology
 Many features of the new .NET platform can be
leveraged
 Support multiple languages - write API/libraries once in
any .NET language and make use from any other .NET
supported language.
 Alchemi Architecture
e-Science Application e-Business Application e-Engineering e-Commerce Application
Application

Precompiled executables

Alchemi .NET API Alchemi Jobs

(Object-Oriented Grid Programming (XML representation)
Environment)

Alchemi Alchemi Cross-

Console Platform
Interface Manager

Grid Threads (.NET objects)

Alchemi Manager

Alchemi Executor Alchemi Executor Alchemi Executor

Windows-based machines with .NET Framework

 Role-based
Security
• Authentication:
Simple username /
password

• Authorization:
Role-based
permissions

• Auditing:
All jobs/threads
executed are
recorded in a
database and linked
to user account used
for Authentication
 Multi-Level Grid

Managers

connected
oU
hierarchically



1+ Executor
1+ User
m oU
 Intermediate
Managers are e e e m
viewed as Executors

e e e e
As a Node for Global Grids

 Cross-Platform
Manager – Web grid broker
Services Interface
 Alchemi grids as
nodes (classical grid grid node x m
model) (Globus-based)

e e e e
Grid Components:
Alchemi
 Grid application Custom Grid
Middleware
 Consists of
independent grid
threads U X
U User Node

 Manager – central M Manager Node

controller E Executor Node

 Discovery, scheduling, M X
Cross Platform
Manager Node
dispatching, Legend
monitoring
 Cross Platform
Manager E E E E E
 Web service interface
 Executor – worker
agent
 User
 Runs grid applications
 Monitoring
 Execution Model

 Dedicated Executor
 2-way communication between Executor
and Manager
 Voluntary Executor
 1-way communication between Executor
and manager (Executor works from behind
firewalls)
 Dual benefit
 Flexible resource management
 Flexible deployment
Performance Evaluation: Standalone
Node (High Precision Pi Calculation)

450
400
Execution Time (seconds)

350
1 Executor
300
2 Executors
250 3 Executors
200 4 Executors
5 Executors
150
6 Executors
100
50
0
1000 1200 1400 1600 1800 2000 2200
Thread Size (no. of digits of Pi)
Performance Monitor: High Precision Pi
Calculation

Grid - Single Executor

Performance Monitor: High Precision Pi
Calculation

Grid - Eight Executor

Performance Result: Pi Calculation – Single
Node
Digits to Calculate: 1000 Total Time Taken: 00:03:39.8125000
Performance Result: Pi Calculation – Three
Nodes
Digits to Calculate: 1000 Total Time Taken: 00:00:11.3437500
Grid Database
@IITR
 Grid@IITR

Applications
Types of Grid Applications

Sequential – dusty deck codes.

Data Parallel:
 Synchronous – tightly coupled;
 Loosely synchronous.
Asynchronous:
 Irregular in time and space;
 Difficult to parallelise to exploit the massive
parallelism.
Embarrassingly Parallel.
P2P/ Grid Applications
 Distributed HPC (Supercomputing):
 Computational Science.
 High-Capacity/Throughput Computing:
 Large scale simulation/chip design & parameter studies.
 Content Sharing (free or paid):
 Sharing digital contents among peers (e.g., Napster)
 Remote software access/renting services:
 Application service provides (ASPs) & Web services.
 Data-intensive computing:
 Drug Design, Particle Physics, Stock Prediction...
 On-demand, real-time computing:
 Medical instrumentation & Mission Critical.
 Collaborative Computing:
 Collaborative design, Data exploration, education.
 Service Oriented Computing (SOC):
 Computing as Competitive Utility: New paradigm, new
industries, and new business.
Ad Hoc Mobile Network Simulation

Ad Hoc Mobile Network Simulation: Network performance under

different microwave frequencies and different weather conditions
Drug Design: Data Intensive
Computing on Grid

Molecules

Chemical Databases
Protein (legacy, in .MOL2 format)

 It involves screening millions

of chemical compounds
(molecules) in the Chemical
DataBase (CDB) to identify
those having potential to
serve as drug candidates.
 MEG (MagnetoEncephaloGraphy)
Data Analysis on Grid: Brain Activity Analysis

64 sensors MEG Analysis All pairs (64x64) of MEG data by

shifting the temporal region of MEG data
over time: 0 to 29750: 64x64x29750 jobs
2

Data Generation
3

5
Data Analysis
Nimrod-G
Results
4

Life-electronics laboratory, World-Wide Grid

•[deadline, budget, optimization
AIST preference]
•Provision of expertise in [Collaboration with Osaka University, Japan]
the analysis of brain function
•Provision of MEG analysis
SETI@home: Search for
Extraterrestrial Intelligence at Home
Content Sharing – P2P
Collaborative Engineering
Access GRID: http://www-
fp.mcs.anl.gov/fl/accessgrid/

Components of an AG Node
RGB Video

Digital Video Display

Computer

NETWORK Digital Video Video

Capture
Computer NTSC Video

Audio
Capture
Computer Analog Audio
Digital Audio

Mixer
Control
Computer
Echo
Canceller

•Group to group interactions.

•Human collaboration across
distributed locations
•Remote visualizations, virtual meeting,
seminars,etc.
•Uses grid technologies for secure
communication etc.
•May have interaction with scientific apps.
 Image-Rendering
http://www.swin.edu.au/astronomy/pbourke/povray/p
arallel
Future of Grid
 Access to any resources, for anyone, anywhere,
anytime, from any platform – portal (super)
computing .
 Application access to resources from the wall
socket!
 Many applications provide solutions in real-time.
 Choice of working: office vs home vs . . .
 Collaboratories for distributed teams.
 Monitoring and steering applications through
wireless devices (PDAs etc.).
Conclusions
 The HPC will be dominated by Peer-to-Peer
Grid of clusters.
 Adaptive, scalable, and easy to use Systems
and End-User applications will be prominent.
 Access electricity, internet, entertainment
(music, movie,…), etc. from the wall socket!
 An Economics –based Service Oriented Grid
Computing computing needed for eventual success
of Grids!
 The impact of Grid on 21st century economy will be
the same as electricity on 20th century economy.