ѧ
JOURNAL OF SOFTWARE
1999ꡡ109ڡVol.10No.91999



󽻲عƸ˹ģͲķ*
ӿ
ժҪͳĻȻƸ˹ģͲķһ޵ʦѧϰ.÷Ҫȱѧϰ㷨ڹһģʽģеĲʱֻ˸ģʽеѵδѵֲӰ죬ˣַʶЧ.󽻲عƸ˹ģͲķַ˲֮ͬ.ͬʱ,Ϊ߻ȫŽĿԣ¸һý滮Ų㷨ַڷ޶ıĻʶ.ʵ÷ȴͳĲƷʶЧҪ.
ؼʡ󽻲أ˹ģͣ滮ʶ.
ͼšTP391
An Approach for Estimating Parameters in Gaussian Mixture Model Based on Maximum Cross Entropy
MA Ji-yong,GAO Wen
(Department of Computer Science and Engineering Harbin Institute of Technology Harbin 150001)
AbstractThe traditional approach for estimating parameters in Gaussian mixture models (GMM) based on maximum likelihood is a kind of unsupervised learning method, its shortage is that the parameters in GMM are derived only by the training samples in one class without taking the effect of sample distributions of other classes into account, hence, its recognition is usually not ideal. In this paper, an approach is presented for estimating parameters in GMM based on the maximum cross entropy of different classes, this method takes the discriminations of samples in different classes into account. To increase the possibility of obtaining the global optimal solution, this paper proposes an approach for estimating the optimal parameters in GMM based on evolutionary programming. An experiment has been carried out using the method for the text-independent speaker recognition, the results have shown that the recognition accuracy is higher than that of the traditional approach. The method has also fast convergent speed.
Key wordsMaximum cross entropy, Gaussian mixture model, evolutionary programming, speaker recognition.
˹ģ(Gaussian mixture model,GMM)㷺Ӧģʽʶݷ1ݵ.,G.Hintonˣ2ַодʶ.⣬GMMҲ㷺Ӧڷ޶ıĻʶ3,ȡõʶЧ.ͳGMM޵ʦѧϰģеĲֻһѵͨȻƷõδѵķֲӰ.һѵֵ£ģ͵.Ͻѵռֲ㹻ܣַƳĲȻŵ.ȻʵӦУʵƣܵõ޶ѵ.⣬ȡȱʹòͬѵռд໥ǵˣбҪԵĹģͲԸĽģ͵ʶ𾫶.ϽѵĹƲӦʹģ͵ĺʶСȻʵӦк.ף4ñҶ˹бһԵĹƸ˹ģͲķõõĽ.ף5һģרһ(mixtures of experts)ģеӦ.ĸܶȵĽԵһֶ.ԽԽ.ģѵʱ,Բͬ彻ΪŻĿ꣬ģͲ.˼룬˻󽻲صGMMģͲƷ.ַ˲ͬԣʹģʶ𾫶ȽϺ.
1󽻲صĲƷ
覸1,2,...,NNͬ࣬iѵɵļϼΪXiٶÿֵ.xiгֵܶΪ
(1)
ʽpik(x)Np(ik,ik)(ֲ̬)xpάik,ikֱǾֵЭ,cik0,, ҪƵĲ.
iԵjĽأҲΪϢɢȻKullback-Leibler룩Ϊ6
(2)
Nͬ彻ضΪ
(3)
 彻ΪŻĿ꣬ԼΪcik0,£ģеĲ,ճŻ,Ϊ˵õcik,ik,ikĵ5(x)ʾxת.
d2(x,u,)=(x-)䦲-1(x-)=(ij)pp.
1.

ʽ{-1(x-)(x-)䦲-1}ijʾ-1(x-)(x-)䦲-1iеjеԪ,d2(x,,)=--1(x-)(x-)䦲-1.
2.

ʽ{-1(x-)}iʾ-1(x-)ĵi,d2(x,,)=-2-1(x-).
3. A=(ij)ppһp׷

ʽAijǾAԪaijĴʽA*=(Aij)ppAİAA*=A*A=|A|I,A-1=A*Iǵλ.
4. pik(x)Np(ik,ik)2ɵ
ikpik(x)=pik(x)-1ik(x-ik).
5. pik(x)Np(ik,ik)13ɵ

ʽ(3)ԹĿ꺯
(4)
ʽЦ,,һķӣһȡһϴֵ.
Ϊ˼Ŀ꺯HTotal,Ӧ
(5)
ʽУ

,ʽ(5)ͬcik󣬶kͿɵ

ʽ.
ʽ(5)ϵɵHTotalcikƫ
(6)
ʽ(4)4ɵ
(7)
ʽUik(x)=-1ik(x-ik).
ʽ(4)5ɵ
(8)
ʽVik(x)=-1ik(x-ik(x-ik)䦲-1ik--1ik.
õڲcik,ik,ikݶϢԭϿݶȴͳŻ⣬ڴͳŻһ̰㷨㷨һ㲻ܵõȫŽ.Ϊ߻ȫŲcik,ik,ikĿԣɲִŴ㷨ͽ滮.Ŵ㷨һģƵŻ㷨J.H.Holland70.㷨ȫԡͨԼԣӶΪһŻһЧ㷨.Ŵ㷨ҪԲ룬ȽӦں΢ĺٶȽ.滮(evolutionary programming,EP)7ģŻ㷨һ֧.Ľý滮Ż⣬ΪŴ滮㷨ҪԲ룬ûͬĽ.
ĸһ㷨£
(1) ʽ(4)Ŀ꺯ӦԺ.
(2) ѡȡʼȺ壬Ngcik,ik,ikĳֵNgȺģ.ikĳֵVQGMM,ikĳֵѡΪসi뱾ĵk֣ikõkָѵЭƱƽѡΪԽcikĳֵ趨Ϊ1/M.
(3) ĸcik,ik,ik·ʽ

(9)
(10)
(11)
cik,ik,ikĺΪ,,൱ڱ.
(4) ѡԭcik,ik,ik,,ѡNgӦԺõĸһ.
(5) ظ,ֱֹΪֹĲŴ.
ɴ˿ɼ㷨ĵ3ʹͳݶ㷨㷨ʽƣڸĸcik,ik,ikʱݶȵķӴ,,ˣڸ.ڽе4ѡʱһЩԽĸµĺ.ԽĸĺȽ϶࣬ܲĸĺ.⣬ڽУ2ikƱ,ŴӶ𽥼С.
ɼ㷨봫ͳݶ㷨ȵŵǣö·ͬʱŽ⣬ȫŽĿ.
ʶʱڸĲxȷ

,ģͻMȷɸʵЧȷM̫Сʹģ͵ľȲ룬M̫ᷢ(over fitting),һҪѡȡ.
2GMM
ȱѵ,ڼ˹ܶеЭ-1ʱֵܻᷢȶԵΪֵȶԣЭΪ=+IIǵλ󣬦һ򻯲һַæMoore-Penrose+榲-1

ʽЦ1ݦ2...ݦr>0ekǾ󦲵ĶӦֵkĵλ.ʽ||ֵæk棬˹ܶȺе(2)p/2(2)r/2.򻯷У󦲵ĽСֵk(k>r)ضϣֵȶ.3ַ,ڼֻþ󦲵ĶԽϵԪعɵľͬʱڶԽϵÿԪϼһСڸ˹ģжԲͬЭȡһͬЭֵַͬȶ.
3ʵ
Ӧڱռķ޶ı߱ϣGMMģеĻMȡΪ32ĲƵΪ8kHzĶʱ͹ȡأ16LPC(linear predictive coefficients)ϵ16LPCϵһĲֺ16ͨĹһLPCʣ൹ɵ48ά֡Ϊ32ms֡Ϊ16ms.߼60ɣ30ˣŮ30ˣÿߵѵڲͬ4ʱڲɼģܳΪ150sңĳȷֱȡΪ30sʵͲԼʱΪ3.öԽǾģеЭ.ȺģNgȡΪ9ŴȡΪ16.
GMMģͽлʶʱϽʽ(1)ĸ˹ܶеĸܶpik(x)Ӧһصƽȶεĸܶ.ٶÿسֵĸǵȿܵ.ٶɣȻѧͳϿÿسֵĿԲһһģѵƣǲ޵ѵ׼ȷػÿصĳֵĸ.ģѵʱʽ(9)cik¹ƣõcikֻӳѵÿسֵĿ.ʶûģΪʶʱõĲسֵĿѵʱһ.ˣģѵʱûʽ(9)cik¹ƣϱcikʼȡΪ1/M.ʵҲԻعʶ𾫶Ƚϸ߲ѵļ.
1ML-GMMMCE-GMMֱӦȻƷ󽻲عƵGMM.ʶǶ30sĲ֡ͳƵģɼMCE-GMMʶʱML-GMMʶʸ߳2%.
1 ͬѧϰ㷨ʶ

ʶ𷽷ML-GMMMCE-GMM
ȷʶ(%)9799

4 
ĸ˻󽻲عƸ˹ģͲķ.ø÷˷޶ʶʵ飬ʵַʶЧȻȻGMMʶЧ.ͬʱ㷨봫ͳݶ½㷨ȵŵǣö·ͬʱŽ⣬ȫԡͨԺͲԣʹȫŽĿ.ȻƷַȱǼϴ󣬶Ի߼Ҫѵ.
*оõصȻѧ͹863߿ƼĿ.
ͨѶϵˣӿ100080йشѧԺ·8ţͨƼչ޹˾
߼飺ӿ1963ʿоԱҪоΪʶ𣬻ʶ.
ģ1956ڣʿʦҪоΪ๦ܸ֪ͼѹ.
ߵλҵѧѧ빤ϵ150001E-mail: mjy@cti.com.cn
οף
1Nandakishore Kambhatla. Local models and Gaussian mixture models for statistical data processing Ph.D. Thesis. Department of Computer Science and Engineering: Oregon Graduate Institute of Science & Technology, 1996
2Hinton G, Revon M, Dayan P. Recognizing handwritten digits using mixtures of linear models. In: Tesauro, Touretzky, Leen eds. Advances in Neural Information Processing Systems. Cambridge, Massachusetts: MIT Press, 1995. 10151022
3Reynolds D A. Speaker identification and verification using Gaussian mixture speaker models. Speech Communication, 1995,17(1):91108
4Ormpneit D, Tresp V. Improved Gaussian mixtures density estimates using Bayesian penalty terms and network averaging. Advances in Neural Information Processing Systems. Cambridge, Massachusetts: MIT Press, 1996
5Moerland Perry. Mixtures of experts estimate a posteriori probability. In: Gerstner W ed. Proceedings of the International Conference on Artificial Neural Networks. Berlin: Springer-Verlag, 1997. 499504
6Cover T, Thomas J. Elements of information theory. New York: John Wiley & Sons, Inc., 1991
7Fogel D B. An introduction to simulated evolutionary optimization. IEEE Transactions on Neural Networks, 1994,5(1):314
ոڣ1998-04-16޸ڣ1998-09-21
