Զѧ
ACTA AUTOMATICA SINICA
1999ꡡ255ڡVol.25 No.5 1999



ϢεͳƻӦԤ1)
ܻ
ժҪоͳƻ(SRNN)ķӦԤ.ڻͳģͣϢεĴSRNNĲתΪһARMAϵͳС㷨ջSRNN.RNNԤĲԺ󣬿ʮַARMAϵͳĿƹSRNNԤƹɣ˷SRNNԤơӷϵͳƹƵ.оԪSRNNĻϣһ̽˶ԪSRNNӦԤ.
ؼʡͳƻ磬ϢΣԽ.
RECURRENT NEURAL NETWORK PREDICTION CONTROLLER BASED ON THE INFORMATION GEOMETRY
DAI Xianhua
(Dept.of Electronic Eng.,Shantou University,Guangdong515063)
AbstractThe paper primarily investigates the adaptive predication controller modeled by the statistic recurrent neural network(SRNN).Using a mixed statistic model to approximate single neuron of the SRNN,one can easily obtain both parameter estimation and control rule of the adaptive controller based on the procedure of information geometry.On the basis of adaptive controller of single hidden unit,the paper further investigates the controller problems of the complicated SRNN with multiple hidden units.
Key wordsStatistic recurrent neural network, information geometry, linear
approximation.
1
ʵʹӦУʵϵͳٵشڷ.ڶķģУͳƻһʮЧķϵͳоԼڿơϢȷӦһֱڶѧоص.
ϵͳԤƼȣϵͳӦƵоһֱڳڷչ׶.ĿǰձõĿƷЧؽһԿȫԵ.
ڴڶ෽УھֲԻֲԽƵĴһֱʮЧĴ.ŵǣϵͳֲԻ󣬿ֱôϵͳƼϵͳӦ.ľǻһ˼룬ϢεĴϵͳתΪһϵϵͳĿ⣬ϵͳӦƷRNNӦ.
2ԪRNNԤ
2.1ԪRNNԤԽģ
¾еԪļRNNϵͳ
(1)
x(t),y1(t)ֱΪRNNźţai(t),bm(t)ΪܿϵͳĲδ֪ģҪԤйƣg(.)ΪRNNļn(t)Ϊٶ˹ֲ.
õԪRNNԤģͣʽ(1)ϵͳԤΪ
(2)
в(t),(t)ΪRNNԤĴƲ.
ʽ(1)(2)Уg(.)ΪӣֱӸʽ(2)ĿźͨҲʮѵ.ʽ(1)(2)֪g(.)ΪʱԪRNN˻ΪARMAϵͳ.һ£ARAMϵͳԤĲƿȫŽ234.ARMAϵͳԤҲΪ򵥣ֱԤƻ򼫵ÿƷȫĿƹɣ4.ˣܽԪRNNתΪARMAϵͳܻýϺõĲ򵥵Ԥƹ.оνʽ(2)תΪϵͳ.
ΪʱڶĳС򽫿һTaylorչ.ˣRNNļȡΪԺSigmodialʱʽ(2)һϵϵͳ
(3)
(4)
(5)
УAkʾg(.)y01,kΪĵС򣬲㣬
Ϊg(.)Ķ.kΪRNN״ָ̬.
ģԿԽƵĿKִʱʽ(3)(5)ϵͳȫԽԤϵͳ(2)K.ͿԽRNNԤתΪһϵARMAϵͳԤ.
2.2ԪRNNԽģ͵ļ
SRNNԤģ͵ѵѧϰУѵݻָźΪy1(t).Խģ֪ͣѵy1(t)Ϊʽ(3)(5)ϵͳָźţôƼΪһARMAϵͳԤĲ234.ǣͨԤ֪ѵy1(t)ΪKϵͳеһϵͳָź.ˣģͲһȫ۲(incompletedata)Ĳ.ôư棺1)ȷǰѵy1(t)ΪһϵͳָźŻźţλ״ָ̬kȷ⣬ɢ¼ģʽʶ.2)ȷÿϵͳѵݺARMAϵͳԤ㷨(t),(t).
Ϊ⣬ͳģ.Խģ(3)(5)֪ĳһʱRNNѵy1(t)ΪKԽϵͳеһϵͳָź.ͬʱRNN˹ţôܿϵͳy1(t)ͳģ156
(6)
(7)
.ķ֪״ָ̬kһɢ(ҲΪ)ΪôԪRNNͳƷֲдΪ
(8)
ӦĶȻΪ
(9)

r1,k(t)=k(z),r2,k(t)=y1(t)k(z),r3,k(t)=y21(t)k(z),(10)
(11)
r(t)=r1,1(t),r1,K(t),r2,1(t),,r2,K(t),r3,1(t),,r3,K(t)T,
(t)=ۦ1,1(t),1,K(t),2,1(t),,2,K(t),3,1(t),,3,K(t)T.(12)
ôͳƷֲʽ(8)дΪ
pr(t),(t)=py1(t),z=expT(t)r(t)-զ(t).(13)
Ȼ۲r(t)Ȼ(t)Ϊһֲָ.(t)ΪSRNN(t),(t)ķԺˣSRNNɵļΪȻεΣһָ17.
ڷֲ(13)ʽУ۲r(t)۲ͳΪһ(t)ԵȵĲϵͳйָָļʿɲμף17.
2.3RNN
ϵķȻƷȻ(t)Ĺ.Ȼæ(t)(t),(t)ĹϵRNN.ǣֲָУ۲r(t)ˣֱȻƷƲ.ûϢεem-ͶӰ㷨.
a)e-ͶӰ
Ĺ۲ʸ״ָ̬kΪδ֪ģy1(t)֪ɹ۲.ôtʱ̵Ȼ(t)ù۲r(t)ĹΪ
(14)
(15)
Цk=pky1,k(t)ʾѵy1(t)ΪkARMAϵͳָźŵĸʣʽ
(16)
b)mͶӰ
Ϣεֲָۣ֪У۲ΪһͳˣȻĹ.ݹ۲ıʾʽ۲Ϊ
(17)
=0ʱʽ(17)Ϊ޼.ôȻΪ
(18)
GTΪFisherϢG󣬦((t))Ϊ(t)µ.


ΪRNNԤĲƣȻ(t)ĹϵΪ.ôĹΪ
(19)
ף17ݣʽ(2)(10)ʽ(19)е.ʽ(17)еĦ=0۲ȡΪ޼ʱʽ(19)ĿչΪ
=y(t)e(t),(20)
y(t)=y1(t),,y1(t-I),x(t),,x(t-M+1)T.
ʽ(20)ʽ(19)ף234еһARMAϵͳԤ㷨ȽϣԿSRNNԤƹʽһARMAϵͳԤĲƻֻͬеARMAϵͳвͬ.ˣͨƴSRNNԤתΪARMAϵͳԤ.ARMAϵͳԤƵĸ㷨㷨ʷֱӦõm-ͶӰ.ͬϢεæk,2Ĺƣ167.עem-ͶӰ㷨ǽòƵģ17.
2.4RNNСԤƹ
ԽģͼϢε£ַԼRNNСԤƹ.RNNźΪd1(t).
1.ڻRNNƺRNNԤдΪ
(21)
,ôʽ(21)ɼΪ
(22)
Ȼʽ(22)ΪһʱϵARMAϵͳĿƹɣźſɲARMAϵͳӦԤƹɵƷã4.
2.RNNԤѵݵķֲ(13)ʽУȻ(t)Ϊܿϵͳźx(t)ĺ.ˣͨȹȻȻȻܿϵͳźŵĹϵRNNź.
ֲָ(13)ʽеĹ۲r(t)Ϊ
r(t)=ۦ1,K,1d1(t),Kd1(t),1d21(t),Kd21(t)T.(23)
em-ͶӰ㷨y1,k(t),k=1,K(Ϊܿϵͳx(t)ĺ)Ȼݹϵʽ(4)RNNϵͳź.
ۿRNNĿȫתΪARMAϵͳԤ.ͬʱҲɸARMAϵͳȶԷRNNϵͳȶ.ȶԷƪĲ.
3жԪĸRNNԤ
ݵԪSRNNģͣԪRNNԤľֲԽģʾͼ1.ڶԪRNNԤֽܷΪԪRNNԤ⣬ôԪRNNԤҲתΪԪRNNԤ.


ͼ1ԪRNNԤֲԽģ

ģУԪRNNϳΪRNNԤ.ˣRNNÿһԪӦѵݻźȷ֪ôԪRNNĲơӦƼΪԪRNNķֱƼ.ǣԪRNNԤÿһԪѵݡźŲûиҪڲƵĹйÿһԪѵݡź.¼򵥵ĹƷʽԪѵݡź.lԪt+1ʱ̵ѵݡźŷֱΪ
(24)
Ϊtʱ̲ƵĵiԪһǰԤźţ4.
ڶԪRNNÿһԪѵݼźź󣬶ԪRNNԤĲԼԤҲתΪԪĲơ⣬ñĵڶķ.
עڶԪRNNתΪԪԺһ롢(MIMO)ARMAϵͳMIMOARMAϵͳĲơƷƼ4.
4ģʵ
ʵģ
(25)
y(t)=y(t)+n(t),(26)
n(t)=0.8n(t-1)+w(t),w(t)Ϊֵ˹.RNNԤȡʽ(25)(26)ͬ׵ԤʽÿһԪľֲԻĿȡΪ11.RNNƵźȡΪ.
ͼ2Ϊʵ(a)ΪRNNƵźd(t)ܿϵͳ(25)(26)Ŀźţ(b)ΪRNNźţ(c)RNNԤϵͳĲѧϰߣ30ζļģƽ߱RNNԤƻϵͳʶ.ͼ2ɿRNNԤƷȫе.


ͼ2Ԥʵ
5ᡡ
SRNNķԼķֶξֲԽƷSRNNתΪͳģͱʾͳԤ.Ϣεem-ͶӰ㷨SRNNԤתΪARMAϵͳԤ⣬ARMAϵͳԤƵĽ۽SRNNԤ.
߼飺ܻ1992ҵڶϴѧʿѧλ.19931994ڻѧͨϵͳʿվ1995ͷѧӹϵν̣ͬøְʸ.Ҫ򣺷źŴźŴͨϵͳ⣬ӦƵ.
ߵλͷѧӹϵ㶫515063
ο
1Shuninch Amari.Information geometry of the EM and em algorithms for neural networks.Neural Network,1995,8(9):13791408
2Anderson B D O.Stability of adaptive systems:passivity and averaging analysis.Cambridge,London,The MIT Press,1984
3Kumar P R.Stochastic systems estimation,identificaion and adaptice control. Englewood Cliffs,NJ:PrenticeHall,Inc.1986
4Graham C.Goodwin Adaptive filtering prediction and control.Englewood Cliffs,NJ:PrenticeHall,Inc.1984
5Xuan J.Information geometry of maximum partial likelihood estimation for channel equalization.In:ICASSP'96:35333566
6Roy L.Streit maximum likelihood training of probabilistic neural network.IEEE Trans.Neural Network,1994,5(5):764783
7Shuninch Amari.Differential geometry of curved exponential familycurvature and information loss.The Annual of statistics,1982,10(2):357385
ոڣ1997-06-04
޸ڣ1998-02-06
