
Computer Engineering
1999ꡡ254ڡVol.25No.41999



ӢWWWϢ
 Ԯ ˼
ժҪ WWWס⣬WWWסĹؼۣڴ˻һӢWWWʵַϢ
ؼ WWW  ݿ Ϣ ĵ
The Structure and Implementation of Chinese-English Search Engine
Jiang Cheng Ma Fanyuan Jiang Sijie
Shanghai Jiaotong University 96033SB Shanghai 200030
AbstractThis paper Presents some relative problems on information processing of the WWW search engineesoecially the key technoloies on information processing of Chinese WWW search engine.A prototype of a chinese-English WWW search engine is implemented based on the Previous analysis and its information processing methods are described.
Key words:WWW;Search engine;Index database;Information retrieval;Relevance of document
Ѿкܶ棬AltaVistaYahooInfoSeekLycosȣĿǰϢ󲿷Ϊ,ļܴ,ǳҪ֧ĵ档
1WWWĽṹ
һWWWɷΪ¼֣
(1)WWWϢĲɼͨɶWWW(Robot)ԶظݳʼбһȥWWWվѼĵ(ͨΪHTMLĵ)
(2)WWWϢķRobotѼĵ,һ汾ؽзȡĵĹؼʡժҪϢĵеĹWWWṹĳ(HyperLink)
(3)WWWϢĴ洢ĵķһṹ洢汾ݿϣͬʱӦѯĸЧ
(4)WWWϢļһṩWWWĲѯ棬ûύĲѯ , ڱݿмûѯļ¼,ͨWWWظû
2WWWϢ
(1)ִʼϢķʣɱػִʣҲWWWҪҪ⡣еĺִ㷨кܶ࣬ڴʿƥ䷨ƥ䷨ƥ䷨Ƶȷִʷ﷨͹ķִʷƵȺͳƵķִʷķִʷרϵͳִʷȡЩ㷨ڲͬҪĳϣִڸȱݣھӦʱɿǼ㷨ϵķʽֲʹһִַʷĲ㡣ִʼеĻڴʿ㷨ĿǰʹýϹ,ҲΪ졣㷨ִʵȷԺ̶ܴȡĴʿ⡣һʿӦ߱걸Ժȫ档ʿ걸ԣ˵ǶһִܰʿҵзֵķʿȫԣζŴʿӦеĴʡͨȹһС걸ʿ⣬ȻϽչһȫʿ⡣
(2)WebݿļɡWWWṩϢѯģִ洢վݿϣͨWebݿļɲʵǿܡĿǰWebݿļɼCGIרõWebAPIԼ³ֵĻJavaԵJDBCͳCGIWeb˱дӦó򣬶ݿĲѯѯɾɴӦóɡӦı̹ÿʱһ̣ϵͳнϸҪЩWebṩרõAPIWebܽһĺ˳򣬲һ̣ЧʸߡWeb˳ͬһڣһ˳WebҪJDBCJavaռJavaݿһֻ֧SQLܵͨõͲAPIڿݿӦó
(3)ݵĴ洢ݿݴ洢ݸ¿죬ԲѯٶҪߡݿֲͬһݿ⣬ԵһΪıϢݲ򵥣ûرӵ߼ϵ򵥡ݶ༯ں̨ǰ̨١ص㣬ݿصӦһЧٵṹϡõбӦHashһЧٵݿ⣬ȽϷеİؼʲѯݴ洢Ҫ
(4)ĵԡһõ治ܰûѯҪ׼ȷذѽظûһȥα棬òƷϲѯҪʵûѯҪȥԶϢ˵ĵۣհûѯس̶ɸѡѯҪһͨĵ۲ûռģ͵TF*IDF㷨㷨ֻĵݣĵһ־̬ĶԡΪWWWϵһѯߣĵѲǾ̬͹ġWWWɿΪʱ仯ṹҲ仯Ķ̬WWWʵϿʱURL ıʽĵеıĵWWWеĽṹĳĵʱҲӦΪһ۱׼ĵWWWΪһֿʱϢɱʾʽ
INFORMATION(Doc)=TEXTINFOMATION(Doc)+HYPERINFORMATION(Doc)
ͨõ۱׼ǱʽеTEXTIN-FORMATIONHYPERINFORMATIONʵϽTEXTINFORMATION֮ϣϾһκκ壬ָĵǳϢԴHYPERINFORMATIONTEXTINFORMATIONֻеʱ(ûĵеURLȡĵ)HYPERINFORMATIONֳ֪ĵTEXTINFORMATIONôͿԸһȨֵHYPERINFORMATIONʵϵһƪĵʱڲܶĵкегͬһʱȥǣϾ͸ǸһȨֵһȼӦȨֵһЩĵʱĵгĸֵȫTEXTINFORMATIONȷѡ
HYPERINFORMATIONۿΪTEXTIN-FORMATION۵ĺֶĵۣضЧʺ׼ȷԽпơ
3һӢWWWϢ
һӢʵַṹμͼ1

ͼ1ӢĽṹ
(1)ռơͨ˶WWW80˿ڷѭHTTPЭʽָ(ҪΪGETHEAD)ܷHTMLĵӦϢ
ʱWWWܴ󸺵ͬʱҲäĿWWWθ⸺ءģǿƻ˵Χԡǵʵл(ϵͳҪڹ)WWWпƣԲͬԲͬȨֵĿĵطվ㡣ͬʱͬһURLظʣϸѭRobot Exclusion׼ȥWWWϣȥʵϢ
(2)ĵѼĵзĻȾҪִʡ԰ѻڴʿƥ䷨ͻƵͳƵ޴ʿִʷȱ֤Чʣֲ©רʡͬʱƥ䷨ͬΪȷְͬͬʱֵСȴרûƵͳƵķִʷȡ
ڹؼȡʱñԪءHTMLĵеĿ⡢֡бּǿݶؼȡΪӿĵٶȺͼݿֻǵݽйؼʺժҪȡȫͬʱάһרҵʿ⣬ڴʿеĴʼӴȨֵԱܸ׼ȷطӳĵʿⱾֹѧϰķ䡣
(3)ˡûύѯCGIвݲѯͨWebսظûݿԿֳɵݿ⣬רƵݿ⡣
ʵǰԵ÷ִСѡ÷ָߵĵĵķĵɡռģ͵TF*IDF㷨ĵTEXTINFORMAT-ION
 
termjǲѯQеĹؼʣOtermjtermjĵDеĳִNĵntermjĵ
HYPERINFORMATION(D)=
Ehyper()ĵЧȨֵNhyper()ĵЧȨֵνЧָĵУЧָĵûбͬʱִݿ֮Сm+n=Link-SuspLinkʽұǰΪĵΪĵ޷ȡӦĵĳTEXTINFOR()ΪָĵTEXTINFORMATION
һʵ(ͼ2)
гĵD1D2D3D4D5D6໥ùϵԵTEXTINFORMATION(ֱʾ)W1>W2>W3>W4>W5NWʾЧȨֵ

ͼ2һʵ
INFORMATION(D,Q)=0.5+W1*0.7+W2*0.5+W3*0.6+W4*0.4+W5*0.2-NW
ĿǰνSEP(SEARCH ENGINE PERSUA-SION)(ЩWebվΪ˰ѹҳǰ棬ĵ)԰ĵеĳΪָվָվ㣬ǰ߾ͿͨȨֵĴڣWeb վµ""
4
ǽϺӢWWWվĽдġڽеϺվҪĽУṩϢļ
߼飺ΡУ28꣬озΪݿ
ߵλϺͨѧ 96033SB Ϻ 200030
ο
1Berners-LeeT.Hypertext Transfer Protocol. Internet Working Draft1993-11
2Berners-LeeTConnolly D.Hypertext Markup Language.Internet Working Draft1993-07
3ƲӨ.Ϣר.廪ѧ磬1995
ոڣ1998-05-27
