о뷢չ
JOURNAL OF COMPUTER RESEARCH AND DEVELOPMENT
1999Vol.36No.7P.830-835



ͲI/Oģ:IDDIO
¬ʿҢ¬
ժҪI/OģǽI/OϵͳṩĴI/OЧתΪûʵI/OĹؼ.DDIOģͶСߴI/OʵķܸƽСͨſľԣµIDDIOģ.IDDIOģԲпѧӦͬͶصͳһļʽ,ͬʱԴߴ硢СߴI/Oṩ˽ϺõŻ;ôˮȼͨſ,һ̶DDIOģеͨƿ˷ԭDDIOģͽʺڴͨſȱݣDDIOṩ˽ϸߵЧI/O.
ؼʣļϵͳ ϵͳ
ţTP316
IDDIO: A NEW PARALLEL I/O SERVICE MODEL
LU Kai
(Department of Computer Science, National University of Defence Technology, Changsha 410073)
JIN Shi-Yao
(Department of Computer Science, National University of Defence Technology, Changsha 410073)
LU Xi-Cheng
(Department of Computer Science, National University of Defence Technology, Changsha 410073)
AbstractA proper parallel I/O service model plays an important role in making an efficient use of the high I/O bandwidth provided by an I/O subsystem. It is pointed out in this paper, that the original DDIO makes little improvement of performance in the small size and fine-grained access mode, since the high communication overhead causes a new bottleneck. Then, a new I/O service modelIDDIO is proposed, which fits small-size and fine-grained as well as large-size access. IDDIO combines the packing and pipelining technology to reduce the communication overhead. The simulation shows that the IDDIO provides a higher bandwidth than the DDIO.
Key wordsparallel file system operating system
1
ģмϵͳMPPԿѧI/Oľ󣬹㷺òI/O.ӲͨI/OͨлĴ̷ṩߴI/O.òļϵͳӲϵͳṩĸߴתΪûʵʴ.ļϵͳһڷչļûI/Oģʽܴ󣬲ͬӦûûܻõʵI/OϵͳṩĲԶ.ˣԲͬӦоʵI/OģͳΪоȵ.
ĿǰҪ3ֲI/Oģͣͳcacheģ͡׶ģͺDDIOģͣ1.
ͳcacheģУI/OȷʼڵI/Oڵcacheδٷʴ.ڿѧI/Oߴһϴcacheϵͣ2ݣI/Oʱϳ.׶ģͽI/O̷Ϊ׶ΣI/Oڵ˳I/O׶κͼڵطֲ׶.׶ģͽϺõ˴˳ٶȿԻ˽ϴĴ̴˷˱ļڵڴ棬طֲ׶ײϢ.
KotzSPMDģʽI/OͬDDIOģԼӦĽ1.DDIOͬI/Oߴص㣬ڴϵֲ󣬻˽ϸߵĴ̴.ĿǰDDIOһֽϺõķģ.PANDA3ݣGalley4ݵȲļϵͳ˸ģ. DDIOȻޣҪڣ
(1) DDIOģͽԴߴṩ˴񣬵СߴʵӳԸƲ.ӦִʱǸĿģӳ١Ҫ˵Ŀꣻ
(2) DDIO治ṩԲͬļͬģʽŻ.DDIO֧SPMD̷ʽ޷֧MPMDʽ
(3) DDIOϢϴ.Ŀǰ˺ܴߣٶ뻥.ߵĹϵͨŽܿܳΪµƿ5.
DDIOľޣһ͵ĲI/Oģ͡ʽDDIOIDDIO. IDDIOI/OԴӳٵҪ󣬲Ϣͨſʹͨźʹ̷ʿ໥ǣӶI/Oִʱ.
2DDIOԭ
KotzSPMDļʽI/OCollective-I/ODDIOģԼؽ1.DDIOģ͵ĺ˼üʽI/OCIOṩķḻϢڴϵֲI/OŻ˳򣬴ӶƴI/O.
DDIOķģͼ1ʾ.MPPĴڵɰܷΪڵ㣨PNI/Oڵ㣨ION.PNIONֱһPNION.һCIOPN.DDIOһCIO̷Ϊ3׶Σ׶Ρ̷׶κͽ׶.ڽ׶ΣPNIONԼPN䡢IONжζԶ㲥ͨ.2׶Ǵ̷׶.PN㲥IONIONٸݵĴ̷ֲI/O.Զ׼ڵ϶ʱϢ.һCIOϴΪֹʱcacheʧЧĿDDIOIONcache. DDIO˫ƣһ黺ڴ̲ʱһ黺ɲеؽ紫.̲ʱ紫ʱʱ紫俪I/Oڸ.ڽ׶ΣIONڵ֪ͨIONPNڵ˳CIO.ʱͬڴĹ㲥ͨ.


ͼ1DDIOģͽṹ
DDIOCIOǻSPMDʽģ޷Բͬļͬʽķ.
DDIO̿ԿDDIO˻̼䴫ʵĲ࣬ͨſȡ̿˽ϺõI/O.
DDIOģͶԴṩ˽ϸߵЧcacheСߴI/OŻЧѣǲܶӳеĶI/Oṩӳٵķ.
DDIOģ͵ĿϢͨϴͽ׶ζдĹ㲥ͶϢ.PNڵϸȷʽֲʱDDIOֱÿPN.̼չѸ̵ͣķٶ뻥Чӽ(ST34520W˳ȡ8KݵʱԼ0.9msMPIͨſͨŽʱΪ0.05msֵΪ800MB).ϢʹͨźܿܳΪµƿ.ڶPANDAļϵͳʱѱ¶5.
3ʽDDIO(IDDIO)ģ
DDIOľԣһͲI/OģͣʽDDIOģͣIDDIO.IDDIOģ͵ûģͽṹͬʱṩ˶ԴߴСߴI/OŻֶ֧ԲͬļͲͬI/OʽCIO󣻼˽ͽ׶εϢͨˮȻƼϢͨſ˽ϸߵ.
3.1IDDIOģͽṹ
ͨпѧӦõI/OģʽǷI/OɷΪࣺͬI/OͶI/O6.ͬI/O̵ͨȻýϴI/OI/OСӳ.ԴԣIDDIOչDDIO CIOͷ.IDDIOģͽṹͼ2ʾ.


ͼ2IDDIOģͽṹ
IDDIOģPNڵ㡢IONڵMasterڵ㹹.PNڵпͻӿڿ⣬ ṩûʲļϵͳĽӿ.IDDIOûͬʱ֧SPMDMPMDı̷ʽIDDIOͨgroup()û֯ͬI/O飬ͬʶI/OΪͬͬ˿ʵֶԲͬļͬI/Oĳ鴦.IONִĵȺݷ.IDDIOģ͵ĺΪMasterڵ㣬ýڵҪʵֶͬI/OͬͶԶI/Oĵӳٷ.ҲļϵͳַҲλڸýڵ㣬ṩͳһĿ¼ṹ.
3.2IDDIOĴ
IDDIOģ͸ͬȲͬͷֱ䴦̼£
(1) ͬI/O󣬸PNֱӷMasterڵ㣬Masterڵ㸺ͬ. Masterڵڼ⵽ͬPNȫ󽫸PNϳһݰ㲥ION. IDDIOϢDDIOеĶԶ㲥ͨõչ.IONڵݵֲ󣬲ֱӷӦPN.ͬһϴļcacheԻýϴʣIDDIOIONͬδʹcache
(2) ԶI/OPNҲֱӷMasterڵ.Masterڵ㴦нϴcache.ڿѧӦļԵĹԣˡʶ̰cacheԤȡ̭һ廯㷨˽ϸߵcacheʣӳٽϵ. cacheݵĶдҲͬͬʵַʽ.IDDIOMasterڵͬͷСߴĶI/O˲Ϊƿ.ҸüʽṹṩDDIOģ޷ṩĹʷʽڵͨļָļܺõ֧˸PNڵĶ̬ƽ.DDIOģУPNڵݽܾ̬.
PNϸȷʽֲʱDDIOϢֱ.ʱDDIO˫ƽ޷ڸͨſ8KPN16BֲϢΪ512ͨſΪ25.64ms˳ȡ8Kݽ0.9msͨŽسΪƿ.IDDIOȽɢݴٷ͵Ŀʽblock_mode.BMʽͬһPNݴٷͣϢͨſ.
BMʽȻϢ䡢ʱͨΪҪì.IDDIOֽһˮݷֲʽPN ڵΪ飬ȺI/OĴ̷ݴ䡢ˮվվ䲢ִУһڸͨš̿ϼI/Oʱ.ˮʽ(pipeline_mode)УIONڵ3黺ֱͬʱմݡ紫.PNڵͬ黺ֱͬʱݺͽ.PMʽIDDIOĴI/OͨŸؾ⣬ͨƿ˽ϸߵI/O.
IDDIOģлؼBMPMʽΪһ壬ŻЧؼI/Oʵķʱ.
4ģ
ǷֱDDIOIDDIOģеBMPMʽķʱ˽ģģ.
T(i,j)ʾiڵjڵ㲥ʱ.Ti_j_kʾiڵjڵ㷢kϢʱ䣨Tpn_ion_reqʾPNڵIONڵ㷢ʱ䣩. SΪݵļֲߴ磬BΪPNIONϵĻСDDIOһ黺ݲϢNmsgB/S. TaccessΪ̵Ѱʱ䣬ݷʴΪBdisk. MPIͨſͨŽӳΪTdelay䴫ΪBnet. BLOCKΪһI/Oȡݿ.
DDIOģ͵I/Oʱɽʱ䡢ִʱͽʱɣʽ(1)ʾ. ִʱķʱʹ̷ʱTIOɣ̵ķʿͨʱDDIOI/OִʱTIOΪTIO_diskʽ(2)ʾ. ͨſڴ̷ʱʱ̷ʽͨڸǣI/OִʱΪTIO-networkʽ(3)ʾ.
TDDIO=Tsetupt+Texec+Tend
Tsetup=Twait+Tpnionreq(11)+Tionionreq(1,m)(1)
Texec=Tpnionreq(n,m)+TIO
Tend=Tionpnquit(1,1)+Tpnpnquit(1,n)
TIOdisk=(Taccess+B/Bdisk)BLOCK+(Tdelay+S/Bnet)B/S(2)
TIOnetwork=(Taccess+B/Bdisk)+(Tdelay+S/Bnet)BLOCKB/S(3)

IDDIOͬMasterڵ㴦㼯ͬ˵ִн׶.IDDIOĽ׶νȵߵĵȴκPNIONͨẒ́ʽ(4)ʾ. IDDIOͬʱTexecIDDIOCIOΪʽ(5)һԶ俪СDDIOĶԶഫ俪.IDDIOĶʱTexecIDDIOindependentΪʽ(6)TcacheΪcacheʱķʱ䣬TdiskΪδʱĴ̷ʱ.cachePϸʱƽӳ൱.
TsetupIDDIO=Twait(4)
TexecIDDIOCIO=Tpnionreq(1,m)+TIOdisk(5)
TexecIDDIOindependent=Tpnionreq(1,1)+Tcachep+Tdisk(1-p)+Tmasterpndata(6)

IDDIOBMʽͬһPNķɢݴٷͼϢ(Nmsg<Nmsg)ݵĴʱTpackͽʱTunpackִʱΪʽ(7).ʽ(7)еϢNmsgΪμI/OPNڵ.
TIOI_DDIO_BM=(Taccess+B/Bdisk)+(Tpack+Tion_pn_data+Tunpack)NmsgBLOCK(7)
IDDIOPMʽʱΪʽ(8).PNڵΪI/OȺˮߣPMʽͬһPNڵݴPNٽ. ڲ˴̷ݴ䡢4վˮߣҽڵĻּһηPNͨſ.Ȼһ̶Ͻ˴̵ķͨͨſƽ˴̺ͨŸأI/Oʱ.ʽ(8)еmax( )ʾˮվд̷ʱTdisk_ioʱTpackݴʱTion_pn_dataͽʱTunpackվʱ俪ˮеƿ.
TIO_IDDIO_PM=(Taccess+B/Bdisk)+Tpack+Tion_pn_dataNmsg+Tunpack+
max(Tdisk_io,TpackTion_pn_dataBmsgTunpack)BLOCK(8)
5ģ
ͨڹĳ;ͻ¶IDDIODDIOܲԣIDDIODDIOṩ˽ϸߵI/O.Ի1ʾ.
1Ի

PNĿ128ION Ŀ16
CPUAlpha21164,RISCͺST34520W
ڴ濽100MB/sת7200RPM 
MPI ӳ52usߴ/512Byte
800MBƽѰʱ7.4ms
·ɷʽWormhole4.3G
IDDIOļߴ8KB//ڵ4/ڵ


IDDIOʽ˽׶κϢ㲥ϢĿֹӵͬʽI/Oϵͳʱʱ䣩.ͼ3ʾPNION(P/I)ֱΪ48ʱͬIONڵIDDIODDIOģ͵ϵͳʱ.ͼ3ɼIDDIOϵͳԶԶСDDIO.ûʳߴСʱIDDIOṩ϶̵ķӳ.


ͼ3IDDIODDIO ϵͳʱ(ms)Ա
ͼ4ʾIDDIOBMʽDDIOڷ64MBI/OԱͼ.ûݰ16BѭֲڸPNϣ߼ļߴΪ8KBѭֲڸION. ڲݶ࣬BMʽĴƴϢͨſ.


ͼ4IDDIO_BMʽDDIOI/O(MB/s)Ա
BMʽУͨʱȻ٣Ϊƿ.ڸĽPMʽУͨˮһI/OǵĽڵˮվƽ˴̺ͨŸأ˷.ͼ5ʾPMʽP/N8ʱͬĿʱI/OԱͼ.
PMʽ£ڵʱI/OΪ˴̵ȵϢ˴Ѱʱ䣬.ŽڵӣӴЧؽͨſʵָˮվĸǣI/O.̵ͬϢļٶI/OĽ.ͨԱһ68ܴϴI/O.


ͼ5IDDIOPMͬʱI/O(MB/s)
ͼ6ʾIDDIOBMPMʽDDIO128PN16ION²ͬݼߴʱܶԱ.ݼߴDDIOϢ٣I/OѸ.IDDIO_BMϢļ٣ĺôѸʧڴܽDDIO.IDDIOPMʽΪˮͨںʵķ飬ЧI/OܼߴӰСԶDDIOģ.ͼ6ԿżߴIDDIODDIOܽ.


ͼ6IDDIO_BMPMʽDDIOڲͬߴµĴ(MB/s)Աͼ
6
IDDIOģͼBMPMΪһ壬ͬʱͬI/OͶI/OṩӳٵI/OʣDDIOϢȱݣͨBMPMʽϲСϢ˷ָƿεˮƽ˸أI/O.ңݼߴϢļ٣ܲδ.IDDIOһڵǰ̺͸ٻһЧĲI/Oģ.
߼飺¬У19737 ʿоҪоΪģֲ벢д.
ʿҢУ19366ʿʦڣҪоΪģֲ벢д.
¬ǣУ194611ʿʦڣҪоΪģֲ벢д.
ߵλ¬(ѧѧѧϵɳ410073
ʿҢ(ѧѧѧϵɳ410073
¬(ѧѧѧϵɳ410073
οף
1Kotz D. Disk-directed I/O for MIMD multiprocessors. In: Proc of 1994 Symp on Operating System Design and Implementation, 1994. 6174
2Kotz D, Nieuwejaar N. Dynamic file-access characteristics of a production parallel scientific workload. In: Proc of Supercomputing '94, 1994. 640649
3Seamons K E. Server-directed collective I/O in Panda. http://bunny.cs.uiuc.edu/CDR/panda.html
4Nieuwejaar N. Galley: A new parallel file system for scientific workloads. ftp:// ftp.cs. dartmouth. edu /TR/TR300.ps.Z
5Chen Y. Performance modeling for the Panda array I/O library. http://www.supercomp.org/ sc96/proceedings /SC98PROC /YING /INDEX. HTM
6¬. Сߴķģ͡EDDIO. ϵṹ, 1997. 3441
(Lu Kai. EDDIO: A server model facing the small size independent access. Annual Meeting of Computer Structures (in Chinese), 1997. 3441)
ոڣ1998-09-14
޸ڣ1998-09-14
