提纲
引言:什么是网格计算
网格计算研究现状
网格计算和对等计算之间的区别
国家教育科研网格计划ChinaGrid简介
结论
信息孤岛
没有任何单个地服务器或搜索引擎能有效地覆盖不断增长的web内容
Internet每年产生2 1018 bytes的信息
但每年只有3 1012 bytes信息可用 (0.00015%)
Google只能搜索1.3 108 个web页
(来源: Li Gong, IEEE Internet Computing, 2001)
第七个问题
摘自汪成为院士《请教关于我国计算机跨越发展的十二个问题》
请问-
这样一个预测准确吗 如果基本准确,这种Client和Server将采用哪种体系结构和基础软件 衡量它们性能优劣的指标又将是什么
有人认为 -
计算机体系结构应面向网络计算的需求.目前,应研制"简约的Client"和"功能强大的Server".Client应逐步实现Smart,而Server应遵循SUMA准则(即Scalability,Usability,Manageability,和Availability-可扩展,好用,可管理,可用).未来,Client将是P3C或算通机(Compunicator)而Server将最终发展成为Virtual
Server Environment.
引发这个问题的缘由之一
Client/Server
Client/Cluster
Client/Networking
Client/Virtual Server Environment (ViSE)
网络计算机是未来计算体系结构的方向
Vision for the Grid
Uniform, location independent, and transient access to the
Computational
Catalogued data
Instrument system
Human collaborator
Resources of science and engineering in order to facilitate the solution of
large-scale, complex, multi-institutional / multidisciplinary data and
computational based problems
These resources accessible through PSEs appropriate to the target user community
什么是网格
一个基础体系结构,它耦合了:
计算机 (个人计算机, 工作站, 集群, 传统的超级计算机, 甚至手提式电脑, 笔记本电脑, 移动式计算机, PDA等等)
软件(如根据要求租用有特殊用途的应用系统)
数据库 (如对人类基因数据库的透明访问)
特殊的设备(如无线电望远镜 - SETI@Home在星球中寻找生命, Austrophysics@Swinburne 寻找脉冲星)
人
通过局域网/广域网 (企业, 组织,或互连网) 并且使它们看起来象一个统一的集成(单一)资源
GRID
A computational grid is a hardware/software infrastructure that provides
dependable, consistent, pervasive, and inexpensive access to computational
capabilities
Analogies: electrical power, telephony, human nervous system, ...
Analogy: Automotive Power Grid
Distributed objects (cars, trucks, busses, bikes, ...)
Technology (chassis, motor, electronics, ...)
Infrastructure (roads, highways, ...)
Standards (technology, infrastructure, communication)
Interoperability, communication (traffic rules, traffic signs)
Services (bus, taxis, ..., gas stations, garages, ...)
Analogy: Electrical Power Grid
Dependable
Electricity is always there. When the switch is thrown, the light always comes
on. A power outage is an unusual situation in normal, day-to-day life
Consistent
The uniform interface means that all applications use the same connector. With
minor adaptations, a plug works almost everywhere in the world
Pervasive
Electrical service is virtually everywhere, throughout the world
Grid Computing
To treat CPU cycles and software like commodities, an application should be:
Ubiquitous -- able to interface to the system at any point and leverage whatever
is available
Resource aware -- capable of managing heterogeneity
Adaptive -- able to tailor its behavior dynamically so that it gets maximum
performance benefit from the services and resources at hand
网格的发展
1989 Metacomputing: computations across machines
1992 Gigabit Testbeds: focus on networking problems, connection and bandwidth
1995 Network Computing: CERN demo at HPCN 95, I-WAY demo at SC95: 30 centers, 60
applications
1997 High-Speed Research Networks: vBNS, Abilene, Al Gore's IT , GUSTO . . .
1998 NASA: The Information Power Grid
1999 DOE: ASCI DISCOM
驱动网格应用的动力
由于耦合了计算机,数据库,设备,人等,新旧应用程序都能得到使用
(分布式)超级计算
合作工程
高吞吐率计算
大规模模拟&参数研究
远低软件访问 / 租用软件
数据密集型计算
基于命令的计算
网格的分类
高性能计算机
系统的共享存取
计算网格
数据库和文件系统的共享存取
数据网格
应用软件和
信息资源的共享存取
信息服务网格
Grid Computing Challenges
Extends cluster computing toward LANs and WANs for parallel and distributed
computing
Problems to be solved
bandwidth and latency
reliability, availability, fault tolerance
wide area parallel processing
interoperability
heterogeneity
a single global name space
protection, security, authentication
efficient scheduling
and comprehensive resources management
Goals of Application Development Support
Applications should
be easy to develop
be portable
achieve high performance
as close to what is possible by hand
Application developer
should be able to concentrate on problem analysis and decomposition
System
should handle details of mapping abstract decomposition onto computing
configuration
Developer and system
should work together to debug and tune the program
Programming Languages and Environments: Challenges
Development of abstract grid programming models
design of an implementation strategy for those models
Development of easy-to-use programming interfaces
problem-solving environments
Compiler and language support for reliable performance
dynamic reconfiguration
optimization for distributed targets
Performance monitoring and control strategies
deep integration across compilers, tools, and runtime systems
performance contracts and dynamic reconfiguration
Robust reliable numerical and data-structure libraries
predictability and robustness of accuracy and performance
reproducibility, fault tolerance, and auditability
网格体系结构
网格
结构
网格
应用
网格中间件
网格
工具
组织间连网的资源
计算机
集群
数据资源
科学仪器
存储系统
局部资源管理器
操作系统
查询系统
TCP/IP & UDP
…
库&应用核心
…
分布式资源耦合服务
通信
签名&安全
信息
…
QoS
进程
数据访问
开发环境及工具
语言
库
调试器
…
Web 工具
资源代理
监视器
应用程序及入口
问题解决.
科学
…
合作
工程
Web使能应用
网格计算现状
网格计算
GIG,IPG,Apgrid, EUROGRID,DATAGRID,TeraGrid
Globus,Legion,Web Services,OGSA,.NET,Sun ONE
对等计算
Napster
全球计算
Xtremweb,SETI@home ,JET,Bayanihan,SuperWeb,Javelin,Popcorn,Charlotte
无处不在的计算/普适计算/无形计算
Globus
GRAM
MDS
GSI
GridFTP
MDS
GSI
GSI
Reliable invocation
Soft state
management
Notification
Compute
Resource
Data
Resource
Other Service
or Application
Job
manager
Job
manager
Service naming
Web Services
主要由XML,WSDL,SOAP,UDDI,WSIL等几种协议组成
OGSA
(Open Grid Services Architecture)
基于Globus和Web Service
提出了动态服务(即网格服务)的概念
动态服务的创建,注册,发现,生命期管理
动态服务的可靠性,故障检测,及服务交互的安全认证
Global Information Grid (GIG)
Information Power Grid (IPG)
TeraGrid
26
24
8
4
HPSS
5
HPSS
HPSS
UniTree
External Networks
External Networks
External Networks
External Networks
Site Resources
Site Resources
Site Resources
Site Resources
NCSA/PACI
8 TF
240 TB
SDSC
4.1 TF
225 TB
Caltech
Argonne
TeraGrid Detail
32
32
5
32
32
5
Cisco 6509 Catalyst Switch/Router
32 quad-processor McKinley Servers
(128p @ 4GF, 8GB memory/server)
Fibre Channel Switch
HPSS
HPSS
ESnet
HSCC
MREN/Abilene
Starlight
10 GbE
16 quad-processor McKinley Servers
(64p @ 4GF, 8GB memory/server)
NCSA
500 Nodes
8 TF, 4 TB Memory
240 TB disk
SDSC
256 Nodes
4.1 TF, 2 TB Memory
225 TB disk
Caltech
32 Nodes
0.5 TF
0.4 TB Memory
86 TB disk
Argonne
64 Nodes
1 TF
0.25 TB Memory
25 TB disk
IA-32 nodes
4
Juniper M160
OC-12
OC-48
OC-12
574p IA-32 Chiba City
128p Origin
HR Display & VR Facilities
= 32x 1GbE
= 64x Myrinet
= 32x FibreChannel
Myrinet Clos Spine
Myrinet Clos Spine
Chicago & LA DTF Core Switch/Routers
Cisco 65xx Catalyst Switch (256 Gb/s Crossbar)
= 8x FibreChannel
OC-12
OC-12
OC-3
vBNS
Abilene
MREN
Juniper M40
1176p IBM SP
Blue Horizon
OC-48
NTON
32
24
8
32
24
8
4
4
Sun E10K
4
1500p Origin
UniTree
1024p IA-32
320p IA-64
2
14
8
Juniper M40
vBNS
Abilene
Calren
ESnet
OC-12
OC-12
OC-12
OC-3
8
Sun
Starcat
16
GbE
= 32x Myrinet
HPSS
256p HP
X-Class
128p HP
V2500
92p IA-32
24
Extreme
Black Diamond
32 quad-processor McKinley Servers
(128p @ 4GF, 12GB memory/server)
OC-12 ATM
Calren
2
2
TeraGrid Timelines
SETI@home
第一个通过大规模并行计算完成来自其它宇宙文明社会电波信号的灵敏搜索
SETI@home主要集中在检测窄频段信号
根据频段对数据进行分块,这些分块在本质上是相互独立的
对太空一个位置的观察得到的结果和另外一个位置得到的结果是相互独立的
把很大的数据集分成大量的小块,每一个计算机能够比较快的分析出其中的一块
把工作分配到自愿贡献空闲cpu周期的机器处理
对等 (Peer-to-Peer)计算
对等计算是指在计算机系统之间通过直接交换共享资源和服务,包括对信息,处理能力,cache存储,磁盘存储等在内的资源和服务
对等计算利用现有桌面计算机的计算能力和网络的互连特性,使得经济型客户端的计算能力能够聚集起来完成整个企业的计算需求
在对等计算体系结构中,传统意义上被单独使用的计算机互相之间进行通讯,既可以作为客户机,又可以作为服务器,这完全取决于网络的需求.这大大减少了服务器的负荷,从而使得它们能够更有效地完成特定的服务.同时,对等计算也减少了IT行业为了支持某种服务而对基础设施不断增长地需求
Application Differences
Grid & P2P
Grid: providing a global problem solving environment for large and critical
scientific applications and professional collaborations, where each grid is a
server
P2P: providing a general and commercial information/computing services, where
each peer can be both server and client
Operation Differences
Grid & P2P
Grid: direct access to computing, software, and data resources in remote &
targeted sites (Servers-based)
P2P: random accesses to available computing, software, and data resources
without a specific target (Clients-based)
Different Participants
Grid & P2P
Grid: pre-determined and registered clients and servers
P2P: clients and servers are not distinguished and registered, which can come
and go by their choices
Different QoS
Grid & P2P
Grid: guaranteed and reliable services are required for each grid server
P2P: only partially reliable, because services from some peers are not
guaranteed and trusted
Security Differences
Grid & P2P
Grid: authentication, authority, and firewall protection to each grid
P2P: privacy, anonymity, authentication, authority, and firewall protection to
each peer is not guaranteed
Different Controls
Grid & P2P
Grid: centralized control plays important role in resource
monitoring/allocations and job scheduling
P2P: limited or no central controls, mainly rely on self-organization
CERNET的总体结构
ChinaGrid计划的总体目标
基于教育与科研网的基础,建立聚合能力超过每秒15万亿次量级的教育科研网格,总存储容量超过260TB,形成世界上最大的超级网格,其结点覆盖211建设的100所部属高等院校,争取在网格计算的基础研究和应用研究方面走在世界前列
充分利用CERNET和高校的大量计算资源和信息资源,开发相应的网格软件,配合网络计算机(NC)的使用,将分布在教育与科研网格上自治的分布异构的海量信息资源集成起来,实现CERNET环境下资源的有效共享,消除信息孤岛,提供有效的服务,形成高水平低成本的计算机服务平台,将高性能计算送到教育与科研网用户的桌面上,成为国家科研教学服务的大平台
ChinaGrid计划的总体目标(续)
建设几个支持科学研究与信息服务的专业网格,能为教育与科研网上的用户的科学研究提供强大的工具
坚持"统筹规划,持续发展,需求导向,分步实施"的指导思想,使ChinaGrid计划有利于促进自主知识产权的信息技术的发展和高级人才的培养,有利于高新技术产业化,有利于促进信息产业的发展,有利于加快国民经济信息化进程,有利于提高国家的综合竞争实力
ChinaGrid网格服务支持平台应用模式
ChinaGrid网格服务支撑平台体系结构
ChinaGrid的具体规划(第一期)
建立10个计算网格的主结点,提供具有高性能计算,资源共享,协同工作的服务平台
每个主结点将建立一个聚合计算能力超过每秒1000亿次,存储能力分别达到2TB的超级计算结点,并通过相应的计算网格软件将分布在10个主结点的高性能计算机连接起来,使整个ChinaGrid的聚合峰值计算能力超过每秒1万亿次,存储能力超过20TB
建设高性能计算网格与信息服务网格应用示范
密码解算
网络存储
数字奥运与虚拟视频会议
粉末冶金成形中的粉末压制过程数值模拟
大型地质灾害预测系统
飞行器遗传优化设计
分子动力学
大气多相流的数值模拟和灾害天气(沙尘暴)的预测
基因组数据的序列与结构分析
ChinaGrid的具体规划(第二期)
将第一期实现的总峰值计算能力为每秒1万亿次的计算网格升级为总峰值计算能力为每秒5万亿次,总存储空间为80TB的计算网格.参与单位包括10个主计算结点所在的单位,并扩展到全国其他20个部属重点高校.计划将10个主计算结点计算能力升级为至少每秒3000亿次,存储空间增长到4TB;其他20个高校每个计算结点的峰值计算能力至少为每秒1000亿次,存储空间为2TB
实现几个典型的信息服务网格,包括中国高校数字图书馆网格,数字博物馆网格,数字奥运网格,数字化流域网格,面向学科(课程)分类的教育信息资源网格,网格视频会议系统等,并开展第二批计算网格的各类应用研究
ChinaGrid的具体规划(第三期)
将第二期实现的总峰值计算能力为每秒5万亿次的计算网格升级为总峰值计算能力为每秒15万亿次,总存储空间260TB的计算网格.计划将10个主计算结点计算能力升级为至少每秒6000亿次,存储空间为8TB;其他70个新增高校的每个计算结点的峰值计算能力至少为每秒1000亿次,存储空间为2TB
完成计算网格的各类主要应用,开展生物信息,网络虚拟制造,计算流体力学,集成电路设计等专业网格,贵重仪器设备共享网格等各类应用网格研究
ChinaGrid的具体规划(第四期)
参与主结点建设的学校有条件的可以将其结点峰值计算能力提升到每秒1万亿次,存储容量达到10TB
完成所有的应用网格建设,进行各系统的集成,推广使用,组织验收
Grid Impact!
"The global computational grid is expected to drive the economy of the 21st
century similar to the electric power grid that drove the economy of the 20th
century"
计算网格
一种提供对资源进行容易,一致,廉价访问的基础体系结构
物理位置和访问点的无关性,类似与电力网格
Future Grid Scenarios
Access to any resources, for anyone, anywhere, anytime
Application power from the wall socket
Many applications provide solutions in real-time
Choice of working: office vs home vs . . .
Collaboratories for distributed teams
Monitoring and steering applications through wireless devices
Distance learning, training, education
Traffic Automation Grid
Health care: everybody gets the same high-quality treatment through WAN access
to central instruments (InSP) and experts
Reference Book
The Grid: Blueprint for a New Computing Infrastructure, I. Foster and C.
Kesselman (Eds), Morgan-Kaufmann, 1999
Thanks!
Any Questions
12
11（本文作者：金海）
|
