Ӧо
APPLICATION RESERCH OF COMPUTERS
2000  Vol.17No.1P.87-90




WebLight-һĵۺ
ͤȺظ
ժ Ҫ WWWϢļͣЧػȡϢΪһؼĳ֣ʹWebõ˳ĹĵЧʵĲѯٶȣӵ﷨򣬽ϵ͵ϢʣȻûиı以ϢԡΪˣǿWebLightۺ棬ϵͳṩͳһĲѯ﷨棬ȡ̻߳ƴָӵĶлȡϢͬʱWebLightṩһĿ¼ҳĵߣʹûЧزѯϢ
ؼ  ۺ ıЭ ؽӿ ߳ ׼ ȫ Ŀ¼
1 
InternetռϢϢѾͻ˵ƣԽԽٶɡǣͬʱıըԵWWWԴټWWWķֲʽ̬ԼԵص㣬ʹĿϢĻȡΪƿYahoo!AltaVistaLycosInfoSeekִ̳WWWĳ򻯡ڲĿϢ֮ǰȽŻվ㣬ԼĿϢĹؼ(Key Words)ȻѰĿϢȻķչ䱾ľԣͬʱҲϢĸЧĻȡ
ȣûһܹеWWWԴҴ󲿷ƽֻ漰WWWԴ30%~50%(Search Engine Watch[2])ڵļ棬󲿷ֻҳңҳ˵Ҳܵͣҳһֻ30ңȻûϢ͵Ϣĸ£ĲȫʾͲܴﵽҪ
Σʹ˲ͬϢռʹøϢԴں̶ܴϾ൱Ĳ졣ʹõһ棬ǲܵõеWWWԴΪѰĿԴҪڼͬл鷳ӰûͶʡ
ķչҲӳWWWԴԡؼֲѯľ䷨ϣÿ涼ͬͬʹͨAltavistaʹá*Lycosȴá$档ʹ򵥵ġANDORȲҲһЩ治֧֡ûΪʹЩ棬òȥϤÿĲͬ䷨͹ܡ
֮ڽWWWϵٷչͬʱڲͨԡϢĲȫʺͲ׼ʡӦʱϣҲдĽĵط⣬ϢѼ㷨ĹԣʹøԵõԴϣ൱Ļԡ
ˣĶϢȡʳУһûͬʱܡͨáҪϢȡʽǣĿնλһĻۺ棬ҿ˷ͬϵͳĲ㣬ûķĵܡڵ2УȻعؿ⡣WebLightҪ˼3֣ϵͳۼ4֡
2 ؿ
2.1 (Search Engine)
ۺǽ֮ϵģоһĻڱԵ쳣Ҫν棬ָܹԶWWWԴ࣬ͨѯ﷨ΪûƥԴϵͳҲΪSpiderCrawlerIndexȵȡ
ͬʱ淢չͬʱҲYahoo! 9Ŀ¼Żһȫ˹WWWԴŻҲϵͼ?: Web Robotֹ档Ż߱ȷɿԣͬʱڼʱӳ˲ϢWWWֲ档
2.2 ۺ(Meta Search Engine or Meta Searcher)
νۺ棬ָͳһûѯϢʽ£ԴΪûṩϢϵͳۺͬûԼԴRobotsһĽɫûĲѯ󣬲ѲѯӦĲѯ﷨ڷֱ𷢳CGI󣬽ܶHTTPӦȡеĲѯͳһʽʾ
˹̹ѧKevin Chen-Chuan Chang1997STARTSЭ飬ӦѭͳһӿںͳһԴҪ⣻Э淶Ľĸʽ[2]Erik Selbergһġλڷ˵ۺMeta Crawlerģ[3]
2.3 Ϣ;ȡʽ
WebϻȡԴķʽһΣֱӴWWWȡԭʼԴվлȡԴۺʹڹԴпٻȡԴϾ͹WWWϢԴʳԼӦĻȡʽͬʱҲWebLightʱҪݡΪWWWϢʳӦļͼ(ͼ1)

ͼ1
3 ʵ˼
3.1 WebLightϵṹͼ
WebLightһ͵ۺ棬ѡAltaVistaӢYahooѺ(Sohu)(WebGather)ָ(Yippee)ԴۺãΪûṩһͳһĲѯ﷨Ϣ棬ͬʱҳԼɷΪϵṹͼ(ͼ2)

ͼ2WebLightϵṹ
3.2 ʵֲ
ϵĽṹͼУWebLightһĽṹȣƵûʾӿڣֻڲʵУԸûԺԤѡԤԴ⣬ͨHTTPЭ鷢ͺͽݡͬʱѯֲĻĵ־֣ǰ߰ѲѯĽҳֱҳԼɹ־ṩû鿴ʷؼӦʷܡ˵һ׼Ĳѯ̣²裺
(1)ûѯ󣬽о䷨Ѱʺϵ档
(2)ȷ󣬰ѲѯתΪض䷨
(3)γɲHTTP󣬵ȴܻӦ
(4)ؽʵĸʽ˳ʾ
3.3 Ҫ
ϵĲԿۺҪÿļԺ﷨ʽұܿٵطؽʾǣڴ󲿷滥಻ݣ໥Բûӿڲһ£һǿۺΪһӵ˵Ҫ⣺
(1)ѯ(QUERY LANGUAGE)ͳһ
һڱĵĽܲѾǵĽ취ǣܳȡĹֵ֧ͬĲѯ﷨ʽγWebLightĲѯ﷨ͬʱΪ˱ָĲͬԣҲֶ֧Եȫ﷨գ¹淶ʽ
@WebLightQuery{
Version(20): WebLight 1.0
Stemming(1): T/F
StopWords(1): T/F
CaseSensitive(1): T/F
QueryFields(20): [Title][+Body][+Author][+Date][All]
WholeWords(1): T/F
WordsRelevance(3): And/Or
QueryContents(100): [Sample Text...]
SearchEngines: [AltaVista][+Yahoo!][+WebGather][+Yippee][+Sohu]...
};
ϸʽSOIF(Summary Object Inter-change Format)[4]淶WholeWords=TQuery Contents俴򣬽QueryContentsֵʣWordsRelevanceȷĹϵϣγWebLight﷨ȻٽͳSearchEnginesɵ漯ϵ﷨ʽ͵ص
(2)HTTPĲ(PARAMETER SETTING)
Ϊ淢ͲѯҴзеķؽ뾫ȷÿĵCGIʽAltaVistaУѯTong Ji Universityؼʱ÷ǰνHTTP󣬵÷̨CGI
http: //www.altavista.com/cgi-bin/query?pg=q&kl=XX&q= 
Tong%2BJi%2BUniversity&search=Search
WebLightڽûĲѯͬγϵCGIʽֱӷ͵AltaVistaУͬʱȴHTTPӦӦĸʽжҳHTTPӦ
(Page2)http: //www.altavista.com/cgi-bin/query?pg=q&q= 
Tong%2bJi%2bUniversity&stq=10&c9k
(Page3)http: //www.altavista.com/cgi-bin/query?pg=q&q=
Tong%2bJi%2bUniversity&stq=20&c9k
...
(3)Դȡ(RESOURCE HARVESTING)
ܶӦ󣬵õҳݺWebLightҪǽϢȷҳԴĵַ⡣ҪϵγHTTPѵöࡣΪͬķĽҳĸʽܴⲿǲ˹̶ҺжϵĲԣϺõؽһ⡣
(4)Ϣ(RANK)
Ϊһۺ棬ϢԵӵ⡣Ϊͬڽһʱܴ㷨һЩδ֪㷨ۺЩʹòͬ㷨Ľͳһʽû
һĽ취֣
a)κУֱʹ淴ĽȽϼ򵥣ͬʱڸϢе㷨Ĳ죬ܿܵϢ򻯡
b)㷨еĻϣٽۺУȡеĵһϢγۺͷϢַ(1)(3)ԣĿǼÿıϢּ֮еƽ⡣׷ٵĲѯٶΪҪĿĵۺĿУԲַ
õķҳURLַϣֱȥصҳݣȻԼϢеͬ㷨ʾ˳һЧ㷨£

NpΪѯؼĵгֵĴ(ÿؼֻһ)
NtΪѯؼĵгֵܴ(ÿؼֿظ)
d(i, j)Ϊi͵jѯؼĵгֵ̾(ַ)
c1ǿRȫҪԵĳ
c2ǿòѯؼּĳ
c3ǿƹؼƵʵҪԵĳ
ȡc1=100 c2=5000 c3=10c1
ѯؼֻһһȡùؼĵеĵһγֵľ㷨ʵӦУܽϺõĵԡӶ˷ÿıȽϻҵϢﵽϢľȷλͬʱڽϵ͵£ַѯٶȡߵĲ׼ʡ
4 ϵͳ
(1)ûͳһȷûͨWebLightֻWebLightѯ淶öȡWWWԴ˵ҳȥϤͬѯ䷨ĸԺʾṹӶԼûͶʡͬʱWebLightʱȷģʵõĲѯͽʾӿڣԭνĹϢû
(2)ȫ(RECALL)ߣ߲ȫʵڶͬʱʹáڲƵ£WebLightĲȫӦΪ
Rwl=U(PavPyhPwgPyp)/P
*R=ȫʣP=ӦWWWҳwl=WebLightav=AltaVistayh=Yahoo!wg=WebGatheryp=Yippee
Ժ
U(PavPyhPwgPyp)>=Any(PavPyhPwgPyp)
(3)׼(Precision)ߣ˸õϢϢWebLightصϢȷûӶõ˸һĲ׼ʡ
(4)Ӧٶ(Response Time)죺WebLightڵʱȡ߳(MultiThread)ͬʱÿ棬ܿؼУӦӦʾûһ棬WebLightʹõʱĲҪĴͼƬWebԴʹӦٶȳͨʹõٶȡMotolora SM56ƽΪ31200 bpsͨ绰ݣͼ3ʾ
(5)/ӢĵԶû࣬WebLightԶб𣬲Ӧ/Ӣд

ͼ3Ĳѯٶȵƽֵλs
(6)ϵͳλڶͻ˳򣬶WEBҳʽWWWϵӢMeta SearcherWEBҳʽṩInternetûʹͻȺӣṩıҲ˴ᣬͬʱCGIƵȱӦٶȵߡЩ䲻ʺҹûˣWebLightƵĳʼȷλڿͻ˵ĹΡ
(7)໯ĵܵļɣΪһõϢȡϵͳɱĵĹ֯ڵWebӦó򣬴ֻǹWebȡԴٶȣȴṩһʵеĵߡ磬Internet Explorerṩ˓ʷղؔĵصĹܣǹڼ򵥣ʹòûϰߡˣWebLightṩһYahooĿ޷ĵߣûĻȡԴܹԶĵГͬʱû귽ĵȽķȥѻʽ£ĵͼ꣬õϴαݡҪݣûֻҼ˵ѡˢ¡ɡĵṩɾӵȹܣ󷽱WebԴĹ
ͼ4WebLight档
໯ĵڻߣԼұߣûڼϺѡҳϢõ𱣴档ͬʱϵͳҲֱ֧ҳַͬʱҲҳϢݡûʱ

ͼ4
5 չ
һʵƷҪWebLightûȷȻ಻㣻йʵʵӦУûʵʵĲӢˮƽϵͣWWWϢȱʹһԵĹؼʱӢȶԲң߼Ĳȫʡͬʱĵȫȱʹò׼ϴۿۡˣδĸĽУЧķִϵͳӢϵͳΪؼ
WebLightоУзҵWebͬһž޴н򻯻ƣĻȡϢԴ쳣ҪţĲ㣬WebLightȻӦWeb򻯵ЧŬ
ͤͬôѧѧ뼼ϵ Ϣϵ Ϻ 200092
Ⱥͬôѧѧ뼼ϵ Ϣϵ Ϻ 200092
ظͬôѧѧ뼼ϵ Ϣϵ Ϻ 200092
ο
1Search Engine Watch: http://searchenginewatch.com/
2Lambda Luis Gravano, Chen-Chuan K. Chang: STARTS - Stanford Protocol Proposal for Internet Retrieval and Search, ACM SIGMOD 97,1997
3Erik Selberg, Oren Etzioni: Multi-Engine Search and Comparison Using the Meta Crawler, In Proceedings of 4TH International WWW Conference, 1995
4SOIF淶, http://harvest.cs.colorado.edu/harvest/user-manual/
ոڣ199977
