Զѧ
ACTA AUTOMATICA SINICA
1997ꡡ236ڡVol.23No.61997




һǰѧϰ㷨
ϵͳʶеӦ
ŷֳ
ժҪһֻС˵ǰѧϰ㷨.ͬ㷨ȣ㷨κξ棬СѧϰϵͳʶӦ.Ƶ㷨һָΪľֲ㷨.ϵͳʶķʵ㷨.
ؼʡǰ磬ѧϰϵͳʶ.
A FAST LEARNING ALGORITHM OF FEEDFORWARD NEURAL
NETWORKS AND ITS APPLICATION TO
SYSTEM INDENTIFICATION
WANG ZHENG'OU
(Institute of Systems Engineering,Tianjin University,Tianjin300072)
LIN CHEN
(Fujian Asia Bank Limited,Fuzhou350001)
AbstractIn this paper,we propose a fast learning algorithm of feedforward networks based on the least squares.Compared with existing similar algorithms,the present algorithm does not require any matrix inversion,therefore,it has a less computational cost and can be better suited for system indentification and other areas where fast learning is required.We derive the algorithm and also give an even simpler and more convenient localized algorithm.Simulation results for system identification show the effectiveness of the algorithm.
Key wordsFeedforward neural networks,fast learning,system identification.
1
ǰBP㷨1ڿƽõ㷺Ӧã23.BP㷨;ֲС⣬Ľ㷨в㿨˲ѧϰ㷨4˹㷺.㷨ѧϰʺ̬ϵв²⣬ٶȿ졢ȸ.˼ǰȨֵΪһӦ̬ϵͳ״̬㿨˲ƣɵõõЧ.Ȼ㷨ά棬Ĺģ.־ֲ㷨56ݵ併άⲻ˾.һֻС˵Ŀѧϰ㷨κξ棬Ӧľֲ㷨ؼ˼˼ٶȣϵͳʶʵӦó.
2ӦĶ̬ϵͳ
һľһ㡢һǰ磬һȫȨɵȨȡRMΪһԪֵɵλǿӵõԪȨ.ٶеԪļ()Sκ(x)=1/(1+e-x)xΪõԪĴ룬뵽õԪźŵļȨ.ڵҪȷȣʹöһi(k)Rn(k=1,,p)ʵo(k)Rm(k=1,,p)Ӧܽӽd(k)Rm(k=1,,p).￼ѧϰ㷨ģʽ㣬ȨֵƵظĽ.Ϊ·Զ̬ϵͳ״̬
(k+1)=(k)=0,(1)
d(k)=f(0,i(k))+e(k).(2)
f(0,i(k)=o(k,0)ʾ롢ȨֵϵķԺe(k)Ϊģ.f(0,i(k))ڵǰֵ(k-1)չ(1)(2)ʽɸдΪ
(k+1)=(k)=0,(3)
d(k)=f((k-1),i(k))+GT(k)(0-(k-1))+(k)+e(k).(4)
G(k)MmάݶȾ
(5)
(k)fչʽĸ߽.
(k)=f((k-1),i(k))-GT(k)(k-1)+(k),(6)
(4)ʽɼдΪ
d(k)=GT(k)0+(k)+e(k).(7)
罫еԪ1NУȡȨֵΪ=[T1T2TN]T(7)ʽдΪ
(8)
iΪӵiԪȨc(k)=(k)+e(k).G(k)еƫͨ練o(k).
3ѧϰ㷨
3.1ȫֵС㷨(GRLS)
ģ(8)һ/ж(i(j),d(j))(j=1,,k)ѡµĿ꺯
(9)
.ʾŷ÷ei(j)(8)ʽеiģΪ0ˡ1ֵ(k)ʹ(k)ﵽСõ.˽ɴ򷽳Ƶ
(10)
1MάgTi(j)ǾGT(j)ĵi.(10)ʽɵ
(k)=S-1(k)t(k).(11)
С (12)
(13)
(11)ʽƵGRLS㷨£
(14)
(15)
㷨ֵȡp(0)=r-1IrΪһСIΪλ(0)Ϊ.(14)(15)ʽƵ¼.
ע⣬㷨Ҫ洢͸ĽһMMά󣬵Ҫκξ.
3.2ֲС㷨(LRLS)
GRLS㷨Pά൱ߣʵ.һЧļ򻯷ǽϵͳֽɽСϵͳֽ⵽һԪһȨһ.ԪһƵLRLS㷨.
еjԪMj+1άȨjMj+1άjа˸õԪֵ.õԪO(k)ӰݶȾGjT(k)GT(k)ĵjӿ
(16)
˾ĵiǣgjTi(k)ʾ.ڵjԪȨֵԻf(4)ʽ
d(k)=f((k-1),i(k))+GjT(k)[j-j(k-1)]+j(k)+e(k)=GjT(k)j+j(k)+e(k).(17)
Сj(k)=f((k-1),i(k))-GjT(k)j(k-1)+j(k).(18)
̨չԵjԪȨֵУȨֵk-1ʱǰĹֵ.˷ԺԻɶÿԪƽеؽ.
ÿԪĽȨֵľֲ㷨(14)(15)ʽɵ
(19)
(20)
ע⵽LRLSҪ洢͸Ľһ(Mj+1)(Mj+1)ά(浥ԪͬMjɲͬ)˾άСGRLSӦάLRLSĴ洢ͼԶСGRLS.
4LRLSϵͳʶеӦ
ϵͳʶͨӦϵҪټʱرʶڿƣͨѧϰ㷨ٶƣЧ.LRLS㷨ɽϺõֲһȱ.
Ϊо㷨ϵͳʶеܣｫͬԭʼBP㷨˷Ƚ.ΪʵüƣֻоLRLS.ǰɶԺͷԺӳ.ʧһԣоϵͳʶ.Ϊ򵥼ƣڷȥϵͳģͻ˵㷨Ժ;.ɾѡȡ=0.995㷨.4˵㷨Խ.
1.Ǽ򵥵ϵͳ
y(k)=0.8y(k-1)+0.2u(k),
u(k)(01)оȷֲṹΪu(k)y(k-1)һ15Ԫ㼰һy(k)(2-15-1).ʼȨֵΪ00.01ȷֲֵPj ʼr=0.001.ԭʼBP㷨ѧϰʺ̬ϵֱȡΪ0.90.3.ѵΪ30.ѵֹͣ׼Ϊei<0.05eʾ30ֵ±iʾ.ͼ1ʾLRLSԭʼBP㷨ڲͬʼ10ѵƽı仯.

ͼ11LRLSBPƽ
2.Ƕ̬ϵͳ
y(k)=1.368y(k-1)-0.368y(k-2)+0.368u(k-1)+0.364u(k-2),
u(k)01ȷֲṹΪ4-10-1ʼֹͣ׼ͬ1ѵΪ30.ͼ2ʾLRLSԭʼBP㷨ٶȱȽϣBP㷨ȡͬ1.

ͼ22LRLSBPƽ
3.ǵϵͳ
y1(k)=u(k-1),
y2(k)=u(k-1)+u(k-2).
ȡΪ˫ṹ2-10-10-2u(k)ȡΪ00.5.Pjʼȡr=1000ʼȨȡΪ00.5ֵȫȡΪ0.5.жȡΪn=30eiΪiѧϰ.ͼ3ʾLRLSԭʼBP㷨ٶȵıȽϣBP㷨ȡѧϰΪ0.83̬ϵΪ0.23.

ͼ33LRLSBPľ
ע.LRLS129ξȴ0.004979BP18700ξȲŴ0.01347.
4.Ƕ̬ϵͳ
y1(k)=0.8y1(k-1)+3u1(k-1)+u2(k-1),
y2(k)=-0.8y2(k-1)-4u1(k-1)+u2(k-1).
ȡΪ˫ṹ4-10-10-2u1(k),u2(k)ȡΪ01.ģͳʼֵȡΪy1(0)=y2(0)=0㷨ĳʼֵͬ3жõ׼Ϊ.ͼ4ʾLRLSԭʼBP㷨ٶȵıȽϣBP㷨Ĳͬ3.

ͼ44LRLSBPľ
ע.LRLS112ξȴ0.007984BP9651ξȲŴ0.007981.
ĸӿɼLRLSмõܣٶԶԭʼBP㷨.
5
ƵһǰĿѧϰ㷨GRLSһָΪľֲ㷨LRLS.㷨˾棬˼LRLS㷨ʹ洢ͼΪ.㷨١ȸߵص㣬һʵõǰѧϰЧ㷨ϵͳʶʵӦó.
1)Ȼѧ.
߼飺ŷ1961ҵѧԶǱרҵ.ΪУںŦԼѧԺԱ IEEE ԱڴϵͳʶϵͳģۼӦõĽѧй.ڹⷢ60ƪ.оΪۼӦá˹ܵ.
֡19691991ҵѧ1994ѧϵͳоѧ˶ʿѧλ.ְڸ.оȤΪ硢ϵͳʶͿơýڵ.
ߵλŷѧϵͳо300072
֡Сݡ350001
ο
1ݡRumelhart D EHinon D E,Williams R J.Learning internal representations by error propagation,parallel distributed processing:Explorations in the microstructure of cognition,Cambridge MA:MIT Press,1986,318362.
[2]Narendra K S,Parthasarathy K.Identification and control of dynamical systems using neural networks.IEEE Trans. Neural Networks,1990,1(1):427.
[3]Chen S,Billings S A,Grant P M.Non-linear system identification using neural networks.Int.J.Control, 1990,51(6):11911214.
[4]Singhal S,Wu L.Training feed-forward networks with the extended Kalman algorithm.In:Proc.of the IEEE Int.Conf.on Acoustics,Speech and Singal Processing,Glasgow,1989,11871190.
[5]Kollias S,Amastassion D.An adaptive least squares algorithm for the efficient training of artificial neural networks.IEEE Trans.Circuits and Systems,1990,36:10921101.
[6]Shah S,Palmieri F,Datum M.Optimal filtering algorithms for fast learning in feed-forward neural networks.Neural Networks,1992,5:779787.
ոڡ1995-03-13
¼
㷨(14)(15)Ƶ
(12)ʽ
(A1)
ͬɵ
(A2)
ٶĵʽ
(A3)
h(k)gi(k)ͬά.Ҫָϸ˵(A3)ʽܾȷ˴ΪƵ棬h(k)hT(k)Խági(k)gTi(k)ʲ.Ӧþ湫ʽ
A=B-1+CCT,A-1=B+BC(I+CTBC)-1CTB,
B=S-1(k-1)-1C=h(k).(A1)(A3)ʽɵ
S-1(k)=-1S-1(k-1)--2S-1(k-1)h(k)hT(K)S-1(k-1)[1+-1hT(k)S-1(k-1)h(k)]-1.(A4)
P(k)=S-1(k)(A4)ʽдΪ
P(k)=-1P(k-1)--2P(k-1)h(k)hT(k)P(K-1)[1+-1hT(k)P(k-1)h(k)]-1.(A5)
(A3)ʽ֪
(A6)
سh(k)hT(k)ĶԽԪ֮͵йh(k)hT(k)ľĶԽԪ֮ͣ
(A7)
(A3)(A5)(A7)ʽɵ
(A8)

a(k)=-1P(k-1)h(k)[1+-1hT(k)P(k-1)h(k)]-1.(A9)
(A5)ʽ
P(k)=-1P(k-1)--1a(k)hT(k)P(k-1),(A10)
(A9)ʽдΪ
a(k)+-1a(k)hT(k)P(k-1)h(k)=-1P(k-1)h(k)(A11)
򡡡a(k)=[-1P(k-1)--1a(k)hT(k)P(k-1)]h(k)=P(k)h(k).(A12)
⣬(11)ʽеĲ(k)дΪ
(A13)
(A10)ʽ(A13)ʽ
(A14)
(11)(A12)(A14)ʽɵ
(A15)
ȥ߽i(k),i=1,,m,
(A16)
Ƶѵõ(14)(15)ʽ.
