ѧ
JOURNAL OF SOFTWARE
1999 2 No.2 1999



һʶĿ״ʶ𷽷*
ӡ1ܲ21
ժҪ﷢εص,ʶһˮƽϽкĸʶ,һֿ״ʶͻҶȵͳƷʵ.ʵ۹Ǻ,5ԪĿȷ80%,ΪԵʶṩһĸֶ.
ؼʡʶ,ȡ,￴,ͳģʽʶ.
ͼšO235
A Method of Mouth-shape's Shape Recognition Forward to Chinese Speech Recognition 
ZHONG Xiao1ZHOU Chang-le2YU Rui-zhao1 
AbstractThis paper presents in detail a statistical method and implementation of mouth-shape's shape recognition and gray level based on perfect image features of mouth shapes at the level of vowels recognition. The results of the recognition experiments approximately accord with the reasoning evaluations. The recognition rate is over 80% for five vowels' mouth-shapes. It proposes a beneficial associate approach for language's voice wave recognition.
Key wordsChinese speech recognition, mouth-shape features abstracting, lip-reading, statistical pattern recognition.
Խʱ,رڻǳǿ,ʹѧź,ͨϢԴ,￴Lip-reading,沿飨Facial-expression,ƣHand-gestureԣBody-languageʶ.ϰ,￴һָˮƽԽֶ.ѧо,,˾Ϳһӵ70%80%.μ䶯̬仯еҪɴ˿ɼ.ʵ,ͨʶ,ĳЩӦ,ڸԶµӾԶװõơ˵ߵơĸʶԼ沿ٴӦõ.
ʶзоԶʶ沿оΧ.ܲĵ,ݿʶо,,ҪһЩо17,˼붼Ǹݵͼ,ͨһ״ȡ,ԣرԪĸʶΪʶĺ,Աʶϵͳʶ.ͱĽ,õϢǴԼ򵥵,ߵʶȴ൱ɹ.ϧ,Щоֻͣڵͼͼ,ź.Ϊ,ǶԿı仯˽Ϊϵͳо,һֻ״֮ϵͳģʽʶ𷽷,ΪЧؽһ.
1ط
,˼Խп̬仯,ѧķ.һֱѧϵоҪǴо,Ҳƾзо,εоһֱδ״̬,ǰ˻Ϊ˼͵Σ810ݺһЩϵͳоɹ.
о,ｻγɵ4ֱӻӵμб仯й:
(1) 촽Ŀڡǰ״λйأ
(2) δСʱйأ
(3) ιαйأ
(4) пڽṹй.
ǵķ,صĿص,Թɳɱ1,ҲǽпʶҪ,ָԪ͸.
1ؿ 

ؿδͼʾֻ棩ܺ
Aa1
Bo1
Ce1
Dj q x i n ng l z c s10
Eu1
F?/TD> 1
Ger r2
HDCd t g k h5
IJCb p2
J˫m1
Kϳҧ´f1
Lϳҧ³zh ch sh3
ܼơ29

2ѡȡ
֪,ѡĿڶ޲Ӱʱ,Ŀ.ʵӦ,ȡѡ֪SϢ.ڴӾӦ÷,ǰͼȱݵԻ򲿼еϵ,ͼӳΪйصϢıʽ,ֱʽֱӶ׵Ƶ11.
ѧ֪ʶͼʵ,ѡԲȡչȡڶȺͿΪʶĻ.иȷȡ£
(1) Բȣָǰ.ȶȡĿͼԤ,õԵԵͼ,ٲ˫Ŀ⼼пռ,ȡԲһ.
(2) չȣָĿ.,ȽпͼԤ,ٶͼɨ,ȡչһ.
(3) ڶȣָĴֱ߶.,ԾԤͼɨ,ȡڶһ.
(4) ָ.ʵ,Ǿͨͼ߽صĸȡһ.
ǿѡȡĻ˼,4. 
3ͳƷĿʶԭ
ģʽʶоһԶ,ּ,ԶѴʶģʽ䵽Եģʽȥ.ͳģʽʶĴͳݰη෨ʷ෨;12.Ǿʵ,ѡþ෨пʶ.
һ޽ʦķ෨.ַ,ԭϽ,ǿѡƽ׼۾.Ϊ˵õѷ,ѡڽ̽һ㷨.
1. Ǽ裺
(1) 5Ԫ/a/o/e/i/u/ı׼ͼѾԤ,5ΪS1,S2,S3,S4,S5
(2) һͼ,Ԥ,õʶͼ,ΪX
(3) ѡȡǸֵTΪ5֮ƽ֮һ.
2. Ȼ,ĹʽֱXS1,S2,S3,S4,S5֮ľD(X,Sj),j=1,2,3,4,5.

(1) XΪʶ,SΪ
(2) iΪȨϵ,ȡֵɸʶö,ڽҪ,Ȩϵͽϴ󣻷֮,ȨϵͽС.i=1;
(3) ϸʽ,nʾ,ĸ.
3. ,бȽ.|D(X,S)|<T,D(X,Sk)=minD(X,Sj),j=1,2,3,4,5,k{1,2,3,4,5},жXSk,SkԪأ,жزԪ.
,ԵԡģʶĿͼ,ͼ⼼д,ף13.
4㷨ʵ
ϸ㷨ľʵ,ʵԷ.
1. 洢׼βտͼ񣺹̶˵֮ľ,һ£δȡʩ,Ҳ䣩,ͬʱ׼˵¼,ȡ5Ԫصı׼ͼ,10,.tiffļʽͶдģ.׼αעΪal,ar,ol,or,el,er,il,ir,ul,ur.Ȼ,ʵʱ˵,ٴͨȡ䵥ͼ,Ҳ.tiffļ.
2. ͼԤѵõ˿εϢ,Ϊ˸õؽȡ,ͨͨռ˲ͼе,˹ԵǿķǿԵ,бԵԤ.
3. ȡǰѡȡĻ˼,ڿʼȡ.
(1) ͼԭ㿪ʼ,ɨ,¼ÿػҶֵλãi,j1,i,j2,|j1j2|,max|j1j2|Ϊչ,£
(2) ͼԭ㿪ʼ,ɨ,¼ÿػҶֵλãi1,j,i2,j|i1i2|,max|i1i2|ǿڶ,ã
(3) ͼԵصĸ,ΪS,
S=(ji1-ji2), jm1-jm2=jn1-jn2=0, nm. 
˫Ŀ⼼Levine14ģ崰ڷ,ͼпռ,ֱȡ5ԪԲ,.
巽Ϊ:FlFrֱʾĳһԪҶͼ,fl(i,j)fr(i,j)ΪӦĻҶȺ,(i,j)Ϊ,ȡ(2u+1)(2+1)ΪӦƥĻ,زȼ:
 
ЦdΪƵλӲ,ҪŹ۲߷ȡֵ;l,r,l,rֱΪ
 
 
˦(d)ļ,Ȼȡd*,ʹ
(d*)={(d)}, 
d*ǸԪԲ.
4. ͳģʽʶл̽ľ㷨,νоɷжǷԪ,ǺԪ.㷨ͼͼ1ʾ.

ͼ1㷨ʾͼ
5ʵ
ʶC,PC586иó.ʵǲѿ,˹ԵǿԺܺõؽбԵǿ,˫Ŀ⼼ϳ,ڽ̽,ԽϿػúľ.ô㷨еεԪʶ,ɹʿɴ80%.23.
2гñ׼֪ʶ 

aoeiu
Բ14112
չ()7227818030
չ()7629878535
ڶ()421320107
ڶ()451523129
()2 3252601 225650180
()2 4802961 487776236

3ʵݷͳƱ 

ŲԿÿһȡʶʶ(%)
12071785
22071680
32071575
42071785

6ᡡ
 ѧо,ͼģʽʶͼ⼼,Եͼ״ʶ,رǶ5ԪصĿνʶ,ʵȷʴ80%.Ϊʶṩһõĸͼ.Ȼ,ʶСˮƽȷдڽһоĿ. 
оõ㽭ʡȻѧͱѧӾʵһ.
߽ܣ,Ů,1973,ʿ,ҪоΪ.
ܲ,1959,ʿ,,ҪоΪѧ֪߼ѧý.
,1937,,ʿʦ,ҪоΪ.
ͨѶϵˣ, 310027,㽭ѧ243
ߵλӡȡ㽭ѧϵʵҡݡ310027
ܲ֡ݴѧϵʵҡݡ310028
ο
1Lewis J P, Parke F I. Automated lip-synch and speech synthesis for character animation. In: Proceedings of Human Factors Computer System Graphics Interface'87. Toronto, Canada, 1987. 143147
2Wu Jian-tong, Tamura S, Mitsumoto H et al. Neural network vowel-recognition jointly using voice features and mouth shape image. Pattern Recognition, 1991,24(10):921927
3Hight R L. Lip-reader trainer. Tohns-Hopkins Apl. Technical Digest, 1982,10(3):213237
4Petajan E D. Automatic lip-reading to enhance speech recognition. Procedure IEEE Computer Association Conference Computer Vision Pattern Recognition, 1985,12(3):4447
5Matsuoka K, Furuya T, Kurosu K. Speech recognition by image processing of lip movements. Journal of Association Instrument Control Engineers, 1986,22(10):6774
6Uchimura K, Michida J, Tokou M et al. Discrimination of Japanese vowels by image analysis. Transactions of Institute of Electronics, Information Communication, Engineers J 71D, 1988,12(12):27002702
7Kurosu K, Furuya T, Matsuoka K et al. Word-recognition by mouth shape and voice. In: Proceedings of the 1st Symptom Advanced Man-Uech. Interface Through Spoken Language. Tokyo, Japan, 1988. 205206
8.ͨʾͼ.Ϻ:Ϻ,1979(Xu Shi-rong. An Illustration of Chinese Speech Pronunciation Schematic Diagram. Shanghai: Shanghai Education Press, 1979)
9ݵ˹ P B,ƽɭ E N.˵Ŀѧ.:йѧ,1983(Dense P B, Pingson E N. The Language LinkSpeaking and Listening's Science. Beijing: Chinese Social Science Press, 1983)
10,.ѧ̳.:ѧ,1992(Lin Tao, Wang Li-jia. Phonetics Lectures. Beijing: Beijing University Press, 1992)
11콨.ͼ.:ѧ,1992(Xu Jian-hua. Image Processing and Analyzing. Beijing: Science Press, 1992)
12,.ģʽʶ.ɳ:Ƽѧ,1992(Shen Qing, Tang Lin. Introduction of Pattern Recognition. Changsha: National University of Defense Technology Press, 1992)
13ܲ,ʩ.ͼλ.Ӧ,1996,13(2):3641(Zhou Chang-le, Shi Xiang-jun. An introduction of understanding different meanings' graph with machine. Computer Software and Application, 1996,13(2):3641
14Levine M D, O'Handley D A, Yagi G M. Computer determination of depth maps. Computer Graphics and Image Processing, 1973,13(2):134150
1997-10-23յԭ,1998-01-23յ޸ĸ
