ѧ
JOURNAL OF SOFTWARE
2000Vol.11No.2P.195-201



ʶԶʶ
ǿïɡƲ
ժҪͨ԰5573ӵıеʶķֲصͳƷ,ЧĺʶԶʶ㷨ڱֲ߽ʵʶ㷨ͻڲṹϵʶ㷨.ʵʾ,ߵʶȷʺٻʷֱﵽ85.4%82.3%,ȡ˽ϺõԶʶЧ.
ؼʣʶ,߽ʶ,䷨.
ţTP18
Automatic Identification of Chinese Maximal Noun Phrases
ZHOU QiangSUN Mao-songHUANG Chang-ning
(State Key Laboratory of Intelligent Technology and Systems Beijing,100084)
ZHOU QiangSUN Mao-songHUANG Chang-ning
(Department of Computer Science and Technology Tsinghua University,Beijing,100084)
AbstractBased on the statistical characteristics of Chinese maximal noun phrases (MNPs) in a Chinese corpus with 5 573 sentences,two efficient identifying algorithms for Chinese MNPs:(1) To identify MNPs by using boundary distribution probabilities; (2) To identify MNPs by using internal structure rules,are proposed in this paper.Experimental results show better performances:precision 85.4% and recall 82.3%,by using identifying algorithm (2).
Key wordsMaximal noun phrase,boundary identification,syntax parsing.
ȻԾӵ,ܷ׼ȷʶеʶ(np)źҪ.֪ѧĹ۵,ʶѧϰıеʵ(entity)߸Ļģ,ܺܺõȻı,Щʵ͸ıеʶ.,ıеʶ,Ϳں̶ܴϰıҪ˼.
ɽṹϿ,еʶɷΪ3ࣺ(1) ʶ(minimal noun phrase,mNP)κʶʶ(2) ʶ(maximal noun phrase,MNP)κʶʶ(3) һʶ(general noun phrase,GNP)вmNPMNPʶ.mNPGNPٵMNP,ԶʶѶڲӵ.ȻԴ,MNPԶʶиΪҪ.Ϊһܺõʶ˾еMNP,ͿԺܷذվӵṹ,Ӷܿ칹ӵ䷨ɭ֣.
ʶһ,,оԱMNPԶʶ̽,һЩ֮Чʶ𷽷.Ӣ﷽ĹҪУ(1) Churchļʶȡ1,øʾϢȷnpʼֹλ.(2) BourigaultȡLEXTER2,ͨ׶εԶıеMNP.(3) VoutilainenMNPȡߣNPTool3,״̬ƣNP-񶨻ƺNP-϶ƣıпܵMNP.(4) Kuang-hua Chen˵Ĺ4,ͳƷֿ(chunking)״̬ϵķ־еʶ.
ӢMNPԶʶѵڽɷ֮(attachment)ϵ.֮,MNPʶΪ,ɺ䷨ɷе׵5.Ӣﲻͬ,еκξ䷨ɷֶԲκ̬仯,ֻһṹʡġ,ͿԳ䵱һnpĶ(Ȼ,ǰ֮ԴκͱεĹϵ)γһnp.ʹ˺MNPԶʶѶ.
⼸һЩоʵ,ԶЧǺ.Ҫо(1) Ľݵˣ6ñֲ߽ϢģͶеMNPԶʶʵ,俪ŲԵʶȷʴﵽ71.3%30ƪűУ.(2) Angel S.T.Tseˣ7ͳƺ͹ϵķ,ˡġʶԶ.ʵΪʶȷΪ75%,ٻΪ90%15ƪıУ.
һԶʶMNP·.ڶıԤĻ,ͨnpֲ߽Ϣnpڲṹ֪ʶ,γ˸ЧĺMNPԶʶ.Լ7ʵĺϽеMNPʶʵʾ,ȷʴﵽ85.4%,ٻΪ82.3%,ȡ˽ΪʶЧ.
1ʶԶʶ
ͼ1ǽкʶԶʶĻͼ.ԾȷзֺʹԱעĺΪ,MNPԶʶҪ׶ν.,ӽзֲεԤ,ԶһЩĳɷ,ָṹнṹ̶ṹ,ڸɷڲɷ֮дĳɷֱ߽Ԥ,ȷÿǴڳɷֵ߽硢ұ߽绹мλ.Ȼ,ͨ첻ͬMNPʶ㷨,׼ȷȷЩɷֱ߽MNPұ߽.һķʵ.

Fig.1Overview of maximal noun phraes identifier
ͼ1MNPԶʶĻͼ
(a) ӣ/rְ/n/v/v/n,/w/r/n/vְ/nȥ/t/v/u/w.
(b) Ԥ {{/rְ/nݣۻ/v/v/n},/w{/r/nݣ/v۰ְ/nȥ/t/vݵ/u}/w}.
(c) MNPʶӦʶ3MNP{MNP/rְ/n} /v /v /n,/w{MNP/r/n}/v{MNPְ/nȥ/t/v/u}/w.
ļ,ǽйصݽϸ˵.
1.1MNPʶԤ
ӽԤĿ,ǽġ,ࡵԵתΪɷֱ߽Ϣ,ӶΪһľ䷨,ʶеMNP,ӵľ䷨ṹṩ֧.
S=W,T,W=w1,w2,...,wnΪӵĴﴮ,T=t1,t2,...,tnΪӦĴǴ,ԤӦõУCH={CG,WS},CG={cgij},ʾɴӴλiʼλjĳɷ,WS=ws1,ws2,...,wsn,ʾע˳ɷֱ߽ԤϢĴ,wsi=wi,ti,bpi ,bpiȡֵ0,1,2,ֱʾ˴ﴦڳɷֵмλá߽ұ߽.йسɷʹϸɲף8еĺעϵ.
Ŀǰϵ,Ҫ¼ɷ飺
(1) ָṹ(punctuation seperated structure,PS),{ ³Ѹ ȫ }.
(2) ̶ṹ(collocation structure,COS),{ ʦ   ʱ}.
(3) нṹ(CS)гɷ(conjunction constituentCC),{{ ˲}{ ˲}{ʹ ˲}}.
һľ䷨(1) ɷڲĳɷֻɷڲɷַ䷨.(2) ɷΪһеɷַ䷨.ͨһЩ,м»ߵĴ,һЩĽṹ,粢нṹĲгɷ֮ŽǿԵ,Թ򵥶Чʶ㷨Զ־пܵĳɷ,йصϸ㷨ɲף9.
ڴеĳɷֱ߽Ϣ,ͨµͳģԶԤ⣺ǴСWij,Tij ,ѡʵĳɷֱ߽BPij=bpi,bpi+1,...,bpj,ʹP(BPij|Wij,Tij)ﵽ.ף10ݸ˾Ԥ㷨.
1.2MNPԶʶ
MNPԶʶĴĿڳɷֱ߽ԤϢĻ,׼ȷȷMNPұ߽.òͬͳ֪ʶ,ǹʶ㷨(1) npֲ߽ʵʶ㷨㷨1.(2) npڲṹϵʶ㷨㷨2.
1.2.1npֲ߽ʵʶ㷨
,ھԤõгɷұ߽,´
.P(npL|ti-1,bpi=1,ti)>,ô˱߽ΪܵMNP߽.
.P(npR|ti,bpi=2,ti+1)>,ô˱߽ΪܵMNPұ߽.
ֵͦµҪĿ꣺(1) ųͳӰ,ڷMNP߽磻(2) ܱMNP߽.ʵеMNPұ߽ĸʷֲϢ,ѡֵ=0.05,=0.15,ϴﵽĿ.
ΪȷϷֵĸұ߽ԿγMNP,һNPջṹ.ڴ˻ʵµMNP㷨,˳пܵMNP߽磻߽,ѹNPջ,,ǷջԪ,,򵯳ջԪأγһܵMNP,˹̲Ͻ,ֱΪֹγɵMNPѹNPջ,һMNPֱ߽ӽ.ͨNPջ,ͿʶMNP.
1.2.2npڲṹϵʶ㷨
ʵһڣ,npֲ߽,MNPԶʶЧǺ.Ϊ,һµMNPʶ㷨㷨2,˼·npڲṹ֪ʶԵϵMNPַ.
ף11,һֻƥԭĺ䷨,ھɷֱ߽Ԥ,ͨƥ־пܵľ䷨ɷ,γӵɭ֣.ͨһ㷨ʵ,ֿMNPĽṹֲص,ǹһµMNPʶ㷨,.
֪ʶ䷨ṹԼ{ṹϡ{䷨}}

 ϻɷ(bc)
wsi,...,wsj,ǿγһɷ֣
(a) bpi=1,bpj=2
(b) wsk,k(i,j),bpk=0
 ֿܵMNPұ߽ɷ
MNPĽṹֲ,ĳɷֿܳΪMNPұ߽磺
(a) wsi=ġ,uJDE,2
(b) wsi=*,n,2
(c) ԼΪnpĻɷ
(d) ܳΪMNPĳɷֵĴɷ
 չMNP
ӿܵMNPұ߽ɷֳ,ڳɷγµMNP,ֱΪֹ.
ؽṹNPջ㷨1
̣ɨ,˳ִл١ں͢,ֱӽ.
2ʵ
ǵMNPʶʵԼ9ֹģΪ.ȷעMNPΪ۱׼,ԷضԶʶ,ӶԶMNPԶʶĴһ͹۶ȫʶ.
2.1Ԥ
ֲͬεԤ,Ƕ¼ָ꣺
(1) ɷ߽ȷ(constituent group precision,CGP)=ȷı߽λõĳɷ(CortCG)/ʶĳɷ(sum of identified constituent group,SCG).
(2) ɷ߽罻(constituent group crossing ration,CGR)=ɷֽĳɷ(CrossedCG)/ʶĳɷ(SCG).
(3) 綨Ԥȷ(word stem precision,WSP)=ȷɷֱ߽ԤĴ(CortBP)/еĴ(WSSum).
1ͱ2гйصʵ.пԿ,˲нṹ(CS)߽,ĳɷֱ߽Ԥⶼﵽ˺ܸߵ׼ȷ,ӶΪһMNPԶʶṩ˽ΪɿĻ.
Table 1Preprocessing results of WS prediction
1Ԥ׶εĴ߽Ԥ

WSSumCortBPWSP(%)
644266240296.86

Table 2Preprocessing results of CG identication
2Ԥ׶εĳɷ߽ʶ

CG TypesSCGCortCGCrossedCGCGP(%)CGR(%)
PS108451059122897.72.1
COS125912013695.42.9
CS76360614279.418.6
CC109610187092.96.4

ٳɷ.
2.2ʶʶʵ
MNPԶʶĴ,Ҫָ꣺
(1) MNPȷ(precision of the maximal noun phrase,MNPP)=ȷʶMNP(CortMNP)/ԶʶMNP(EMNP);
(2) MNPٻ(recall of the maximal noun phrase,MNPR)=ȷʶMNP(CortMNP)/еMNP(CMNP).
3гĿǰʵ,мMNPĴʳ<5,MNPĴʳ5.пԿ,ֻñֲ߽֪ʶ㷨1,MNPʶȷʺٻʷֱΪ69.2%70.9%,ЧǺ.øΪḻѧ֪ʶ,npڲṹϹMNPԶʶʱ㷨2,ȷʺٻʶнϴ,ֱﵽ85.4%82.3%,ʾΪԶʶЧ.
Table 3Maximal noun phrase identifying results
3MNPԶʶ

CMNPEMNPCortMNPMNPP(%)MNPR(%)
Algorithm 1Simple MNP43594724335271.076.9
Complex MNP86562635055.940.5
Total52245350370269.270.9
Algorithm 2Simple MNP43594279376387.986.3
Complex MNP86575453470.861.7
Total52245033429785.482.3

㷨,ڼMNP,۸MNP,ܺϼ.
,ʵʾĿǰ㷨ԸMNPԶʶЧȽϲ,ϳĸMNPʶȷһҪȼMNPԼ16ٷֵ,ٻ.ĿǰԶʶڶԸMNPʶϻŽϴȱ,дڽһԸĽ.
2.3ʵ
㷨2ʶʵз֣ԭҪɹΪһԤı߽Ԥģʶ㷨ƶ.Ȼ,Ǹĵеĵ2,ΪпܽһЩڸĽʶ㷨Ľ.Ϊ,һʵ,ͨMNPԶʶϢ´(1) ɾԶֵеĳɷཻĳɷ(2) ȷĳɷֱ߽Ϣ滻ԶԤĴΪMNPԶʶṩɷֱ߽ȫȷ.4гڴ㷨2Զʶ3ĽȽϿԷ֣ԭʶвһԤı߽.
Table 4Maximal noun phrase identifying results (A2) by using correct preprocessor data
4ʹȷԤݵMNPԶʶ(㷨2)

CMNPEMNPCortMNPMNPP(%)MNPR(%)
Simple MNP43594301407394.793.4
Complex MNP86573462684.972.4
Total52245038469993.390.0

ټMNP,ڸMNP,ۺϼ.
ʣ525ʵ339ʶ186δٻش,Ǵ¼ͬĽǶȶǽϸµķ.
,ͨԶʶMNPұ߽Ƿһȷ,ԽЩʵΪ¼ͣ
(1) ߽ʶȷĴʵ,һϸΪ
(a) ʵұ߽>ȷʵұ߽磨͢񣩣
(b) ʵұ߽<ȷʵұ߽磨͢.
(2) ұ߽ʶȷĴʵ,һϸΪ
(a) ʵ߽<ȷʵұ߽磨͢,
(b) ʵұ߽>ȷʵұ߽磨͢.
(3) ԶʶMNPұ߽綼ȷ͢
5гЩͬ͵MNPʵķֲ.пԿ,ұ߽ʶȷĴʵĿ(199)ԶԶ߽(56),һʾMNPʶѵȷ߽.ѰҸõķ,׼ȷȷӶ߽λ,ǽоһص.
Table 5Distribution of maximal noun phrase identifying errors (total 339) with different error types
5вͬ͵MNPʶֲ339

Error types٢
Distribution frequency25314515484
Distribution ratio(%)7.49.113.345.424.8

ٴ,ڷֲƵ,۷ֲ.
ԴʵڲṹϽз,ԷкһЩṹռ˺ܴ.ǽṹϣv np  n,вͬĴ,ֺķṹ(1) npvp v npݵ n,磺npvpμ ѧۻ  ʦݣ(2) vp v np np nݣ,磺vp  np ϹŶ  Ӱݣ.Ŀǰǵʶ㷨ֻͳһʶΪṹ(2),⵼˶ƽṹʶȷʺٻʵĽ.μǿṹʶ,ǽоһص.
3
ΪһҪӦûо,MNPԶʶȻԴеӦо,䷨ϢϢȡ,Ҫʵ.ںMNPԶʶһЩ̽,ͨıеʶķֲصͳƷ,ЧĺʶԶʶ㷨ڱֲ߽ʵʶ㷨㷨1ͻڲṹϵʶ㷨㷨2,ȡ˽ϺõԶʶЧ.ڽо,ǽ¼Щ㷨иĽ:
(1) ѰȷӶ߽ĸ÷;
(2) ǿṹʶ.
ĿоõȻѧ(No.69705005)йʿѧ(No.97005)
.
߼飺ǿ,1967,ʿ,оԱ,ҪоΪѧ.
ï,1962,,ҪоΪϢ,ѧ.
Ʋ,1937,,ʿʦ,ҪоΪѧ,Ϣ.
ߵλǿ(ܼϵͳصʵҡ100084)
ï(ܼϵͳصʵҡ100084)
Ʋ(ܼϵͳصʵҡ100084)
ǿ(廪ѧѧ뼼ϵ100084)
ï(廪ѧѧ뼼ϵ100084)
Ʋ(廪ѧѧ뼼ϵ100084)
οף
1Church K.A stochastic parts program and noun phrase parser for unrestricted text.In:Proceedings of the 2nd Conference on Applied Natural Language Processing.Austin:Association for Computational Linguistics,1988.136143
2Bourigault D.Surface grammatical analysis for the extraction of terminological noun phrases.In:Boitet C ed.Proceedings of the 15th International Conference on Computational Linguistics (COLING'92).Nantes:Academic Press,1992.977981
3Voutilamen A.NPTool,a detector of English noun phrases.In:Church K ed.Proceedings of the Workshop on Very Large Corpora:Academic and Industrial Perspectives.Columbus:Association for Computational Linguistics,1993.4857
4Chen Kuang-hua,Chen Hsin-hsi.Extracting noun phrases from large-scale texts:a hybrid approach and its automatic evaluation.In:Proceedings of the 32nd Annual Meeting of Association of Computational Linguistics.New York:Association for Computational Linguistics,1994.234241
5Lu Jian-ming.Special nesting phenomena of Chinese constituents.In:The Optional Papers of Lu Jian-ming.Zhengzhou:He'nan Education Press,1993.174192
(½.䷨ɷе׵.½ѡ.֣:Ͻ,1993.174192)
6Li Wen-jie,Zhou Ming,Pan Hai-hua et al.Corpus-based maximal-length Chinese noun phrase extraction.In:Chen Li-wei,Yuan Qi eds.Advances and Applications on Computational Linguistics.Beijing:Tsinghua University Press,1995.119124
(Ľ,,˺.ϿʶԶȡ.Ϊ,Ԭ.ѧչӦ.廪ѧ,1995.119124)
7Tse A S Y,Wong Kam-fai et al.Effectiveness analysis of linguistics- and corpus-based noun phrase partial parsers.In:Choi Key-sun ed.Proceedings of the Natural Language Processing Pacific Rim Symposium (NLPRS'95).Taljon:Academic Press,1995.252257
8Zhou Qiang,Huang Chang-ning.Chunk parsing scheme for Chinese sentences.Technical Report,TR98002,Department of Computer Science and Technology,Tsinghua University,1998
(ǿ,Ʋ.עϵ.Ƽ,TR98002,廪ѧѧ뼼ϵ,1998)
9Zhou Qiang.Phrase bracketing and annotating on Chinese language corpus Ph.D.Thesis.Beijing:Beijing University,1996
(ǿ.ϿĶԶֺͱעо۲ʿѧλģ.:ѧ,1996)
10Zhou Qiang.A model for automatic prediction of Chinese phrase boundary location.Journal of Software,1996,7(supplement):315322
(ǿ.һԶ綨ģ.ѧ,1996,7():315322)
11Zhou Qiang.Implementation of Chinese parsing algorithm based on bracket matching principle.In:Chen Li-wei,Yuan Qi eds.Language Engineer.Beijing:Tsinghua University Press,1997.194200
(ǿ.ƥ㷨ʵ.Ϊ,Ԭ.Թ.廪ѧ,1997.194200)
ոڣ1998-11-10
޸ڣ1999-03-09
