Զѧ
ACTA AUTOMATICA SINICA
1998 24 5  Vol.24  No.5 1998



һֻعĿѧϰ㷨
ΤΡ
ժҪԻعBPѧϰ㷨ȱݣһµĿߵѧϰ㷨.㷨Ŀ꺯ӣڷϵͳȻԭɹؽ˶̬ϵͳعģȨϵѧϰʵʱԺͿ.㷨ȴͳĻعBPѧϰ㷨иٶ..
ؼʡع磬ѧϰ㷨.
A NEW ON-LINE RECURSIVE LEARNING ALGORITHM FOR
RECURRENT NEURAL NETWORK
WEI WEI
(Department of Electrical Engineering, Zhejiang University, Hangzhou 310027)
AbstractIn this paper a new on-line recursive learning algorithm for recurrent neural network is proposed. It overcomes the disadvantage of the slow convergence of the recurrent BP algorithm. The real-time learning ability and the fast convergence of the recurrent network model of nonlinear dynamical system have been obtained by introducing the forgetting factor in the objective function and the maximum likelihood estimation principle. Simulation results show that the proposed algorithm performs better than the traditional recurrent BP algorithm.
Key wordsRecurrent neural network, on-line learning, fast algorithm.
1
ͳǰ򴫲ڶ̬ʱźŴԶ̬ϵͳƵȴǿʱΪϵͳӦд൱.ĿǰȻͨʱͻϵͳĶ̬ܣ1ݣֱܹӸطӳϵͳ̬ԵӦǶ̬磬ڲ״̬ķ.عڲ״̬ϵͳķԶѧΪ.ɻعģ͵ķкܶ࣬ܵ˼붼ͨǰмһЩӵġڲķͨ籾̬Ϣ.״̬Ϣķ;ͬɹֲͬĻعṹģͣJordan(ͼ1(a))Elman(ͼ1(b)).


(a)Jordanṹ(b)Elmanṹ
ͼ1عṹģ
ڻعдڲ˴ͳǰѧϰ㷨BPѲֱӦڻعѧϰ.1987Pineda˻عBPѧϰ㷨2.źPinedaķȻݶȷеģ˲ɱػֲٶȱ.ѰһֿٵѰ㷨عѧϰרѧߵĹע.ĽϵͳС˵㷨عȨѧϰô㷨ʵʱԺͿѧϰ.ݻعBPѧϰ㷨ԭͷƵعĿѧϰ㷨.ʵһ֤˴ѧϰ㷨ЧԺͿ.
2عģͼBPѧϰ㷨
һ̬Ŀtk,tk-1,tk-2,,ͳԪ綯̬ϵͳͨϵͳǰyk-1,yk-2,,yk-nΪǰ򴫲ģ͵źţӶϵͳĶѧԵ.͵ĴģͽṹNarendra K.S.ĴʱͻڵMLPṹ1.Ȼٵʱͻڼn̫Си߽ͺϵͳĶ̬ʱ.һ棬ʱͻڼn̫Ҫϵͳļ䵥ԪеϢҲǲȡ.عһȱݶ.úٵļ䵥ԪκϵͳĶѧ.
YHURֱʾmάʸhάԪ״̬ʸnάʸrάڲ״̬ʸعģͿ·
Yk=(b+¡Hk),Hk=(c+áUk+ġRk),Rk=(Uk,Rk-1,W)(1),(2),(3)
kʾkʱֱ̣ΪԪԪͻ񾭵ԪʸWΪsάȨϵ󣬰(b,',c,',').ȡRk=Yk-1,Hk-10ʱֱΪJordanElmanMLPṹ.
˻ع鵥Ԫ(3)ʵֶݵļ¼֮عĻعRܹYԪ״̬HϢýյʽϵͳǰϢ.ʽ(3)ƿɵ
Rk=(Uk,Rk-1;W)=(Uk,(Uk-1,Rk-2;W);W)==(Uk;W),(4)
Uk=U0,U1,,UkΪǰϢ.
(4)ʽԿعģԼģͣ洢еϢзӳϵͳеʷźŶԵǰϵͳӦӰ.ڻعһʹڶ̬Ӧþкܴ.
(12)ʽɵ
Yk=(b+¡䦷)(c+áUk+ġRk(W))=F(Uk,Rk(W);W).(5)
ʽ(4)ʽ(5)ԿYkҲϵͳȥUk-1ĺ.ҪʹĻعʵеõӦãҪѧϰ.봫ͳǰ򴫲һعѧϰָ꺯ȻѡƽͺռZk,k=0,1,2
(6)
ڶ̬ZkȨϵWɹΪ
(7)
E.ݱʾѧ.
ʽ(7)ļ޴ڣȻƷ֪Ȩϵĸ¹ʽΪ
Wk+1=Wk-k+1Pk+1eTk+1.ek+1,(8)

ek+1=Zk+1-F(Uk+1,Rk+1;Wk)(9)
ek+1Ϊek+1ԲWݶʸ.
Pk+1=IʱΪݶȷPk+1=ETk+1.ETk+1ʱΪţٷ.Ek+1=
עǣڷ(9)ʸWԺRk+1ĺW.ݶʸek+1ʱҪek+1Wʽ󵼶Ҫʽ.ЩԺ󣬲ѵóعģ(1-3)BPѧϰ㷨(ף3).
3عѧϰ㷨
ڿǵǶ̬ϵͳڴʱӦµ罨ģе.عĴͳBPѧϰ㷨ָ꺯ͬǣĸĽ㷨ӶJkеĸϲвƽмȨ취߶̬ģ͵ʵʱ.ȡĿ꺯
(10)
WkĿ꺯(10)ļС⣬
.(11)
Ȼk߱õĽʣkȽϴʱkֵWܽӽ.=kĸe(l,)k̩չȡһ
(12)
ȡһµĹ۲Zk+1ɸl=k+1ʱǰȫ۲죬óʹĿ꺯
(13)
ļСk+1.ʱӦ
(14)
kܴʱk+1kʰʽ(12)
(15)
(16)
ʽ(11)(15)(16)Ŀ꺯(10)ʽ󵼵
(17)
ʽ(15)(17)ʽ(14)пɵ
(18)
ʽ(18)ɵ
(19)

(20)
عϵĸ¹ʽΪ
k+1=k-Pk+1e(k+1,k)e(k+1,k)(21)

(22)
þݹʽɽ(22)ʽһ򻯣ӶɵóعĿѧϰ㷨
1)ѡȡһʼϵʾڵһʱȨϵݶȷиµĲ.
2)k0.
3)ȡǰµ(Uk+1,Zk+1)иʽȨϵ
Rk+1=(Uk+1,Rk;Wk),(23)
e(k+1,k)=Zk+1-F(Uk+1,Rk+1,k),(24)
k+1=W(Uk+1,Rk;Wk)+k.R(Uk+1,Rk;Wk),(25)
e(k+1,k)=-FW(Uk+1,Rk+1;Wk)-k+1FR(Uk+1,Rk+1;Wk),(26)
k+1=ۦImm+eT(k+1,k)Pke(k+1,k)-1,(27)
Pk+1=ۦPk-Pke(k+1,k)k+1eT(k+1,k)Pk/,(28)
k+1=k-Pk+1e(k+1,k)e(k+1,k).(29)
4)kk+1,ת3).
4о
ͨһӵķ۱Ŀѧϰ㷨Ч.ģѡף4и˫DGPģͣ
zk=0.5-0.4zk-1+0.4zk-1uk-1+uk,k=1,2,,kmax,(30)
ukѡN(02).
ѿϵͳϵͳȥϢ.ѡElmanعṹȡԪΪ4.ԪļΪSigmoidԪΪԺ.ӵķоȡ=0.02,=0.95,=0.68.W0R00Ϊ01֮.뵥ԪΪzk-1,uk.ѡElmanعṹʱϵܹ33.ôṹ͸Ķ̬ģͽзоͼ2ͼ3ʾ.


ͼ2ѧϰ㷨ƽ


ͼ3ѧϰ㷨ıʶ
ͼ2еĺʾĴʾ25ƽֵ.ʵ߱ʾñ㷨ѧϰƽߡ߱ʾôͳعBPѧϰ㷨ѵƽߣͼ3ʾһεѧϰ25ϵͳӦ.кʾʱֵΪϵͳʵ߱ʾ̬ģ͵ʵֵ㻮߱ʾعѧϰ㷨óıʶ߱ʾͳعBPѧϰ㷨óıʶ.ӷԿعѧϰ㷨߱ȴͳعBP㷨Ժ͸ߵıʶ.عѧϰ㷨ѧϰP0ʱʱ̵̿㣬˲ɱسһЩ.ڱ㷨вȡһЩʩ㷨ֵܳķɢ統εĿ꺯ȹʱȨϵĸ½ݶȷ.P0ΪԽ.Ӷ֤˻عѧϰ㷨ܹȥ.⣬ڳʼѧϰ׶ΣWkԶֵWʽ(12)ĽƶܵӰ.ڷобĲȡڳʼļѡݶȷתߵ㷨ѵѧϰ.
1)ҽίѧعԱ.
߼飺ΤΡ1964.1986㽭ѧ˶ʿѧλ1994ʿѧλ.Ϊ㽭ѧϵ.ѧ20ƪ.ĿǰҪоΪܿۺӦá˿.
ߵλ(㽭ѧϵݡ310027)
ο
1Narendra K S, Parthasarathy K.Identification and control of dynamical systems using neural networks. IEEE Trans. on N.N. 1990, 1(1):427
2Pineda F J. Generalization of back-propagation to recurrent neural networks. Physical Rev.Lett., 1987, 59:22292232
3Kuan C M, Hornik K, White H. A conversgence results for learning in recurrent neural networks. Neural Computation, 1994, 6(2):420440
ոڡ1996-08-12
