Ϣ
INFORMATION AND CONTROL
1999 28 6 Vol.28 No.6 1999



ѧϰԪѧϰBP
ƽ ܺ
ժҪ˹ԪBPȱ㣬ʵֽǶһԪBP硪ѧϰԪѧϰBP磮ѧϰԪͻص֮һڲͨͨѧϰܹϢ򴫲ķ򴫲ԪѧϰԪɵѧϰBP򴫲Ϣ򴫲ѧϰĲлĻѧϰBPѧϰ⣬һµѧϰԣǵķѧϰкܺõѧϰЧ
ؼʣѧϰԪѧϰBP磬ѧϰԣԪ
ͼţTP13ױʶ룺A
SELF-EARNING NEURONS AND SELF-LEARNING BP NETWORKS
LAI Xiao-ping1 ZHOU Hong-xing2
(1. Department of Control Engineering, Shandong University at Weihai, Weihai 264209; 2. Department of Mathenatics, Shandong University, Jinan 250100)
Abstract: A new king of neurons and new kind if BP networks, self-learning neurons and self-learning BP networks, are presented in this paper with a wiew to their implementation. A self-learning BP network is composed of self-learning neurons. One of the prominent characteristics of the self-learning neuron is thatit has a forward-channel, a backward-channel and a leamer, This makes each neuron in a self-learning BP network accomplish the forward propagation of message, backward propagation of errors, and modification of its parameters independently, and hence makes the network implement parallel computing easity. A new training policy for BP networks is also presented in this paper. Simulation results demonstrate the effectiveness of the policy.
Key words: self-learning neurons, self-learning BP networks, training policy, neuron-oriented algorithm
1
ǰоӦõ㷺Ҳɹ˹Ԫģ֮һǰһӳ磬ϣSigmoidǰ⾫ȱƽһԺԪԸ뻷໥öеڼѧϰһBP缴һǰ򴫲ѧϰ㷨һBPӦܣ1Ϣ뵥ԪԪ󴫵ԪϢ򴫲2ʵ֮ԪԪ󴫵뵥Ԫ򴫲3򴫲Ϣͷ򴫲Ȩϵѧϰ̣ĿǰǰȨϵѧϰ㷨󶼲BP㷨BP㷨ĸĽ㷨BP㷨ͬʱȨϵBP㷨[1,2,3,4]ӦѧϰBP㷨[2,3]ȣЩ㷨УͬʱȨϵBP㷨ӦѧϰBP㷨нϺõԣBPڷϵͳıʶԤ⼰ƵкܺõӦã
ǣϸBPṹȨϵѧϰ㷨ʵʱͻᷢµľԣ1еBPṹֻϢ򴫲ķ򴫲ȨϵѧϰΪظϵģ2ӱϿеBPвнṹʵѧϰ㷨ģĲл3еBPԪ֮ǴȨӣʵֽǶȿǣԪʱ̫㣬ӸϽɵ㣨򵥸ԪʱԪӣȨϵìܵģ4ĳԪȨϵܵƫѧϰʱ㷨ģòȨϵѧϰ5еBPѧϰ㷨ԪȨϵͬһ㷨ѧϰȱԣ
BPЩ⣬һ͵Ԫ͵BP磮͵ԪдϢͨķͨ򴫲Ϣ򴫲ѧϰѧϰźȨϵƫȨϵĶȶΪԪڲѧϰǵЩ͵Ԫһ𹹳һ͵BP磬ص㣺1ܹʵBPܣ򴫲Ϣ򴫲ѧϰ2Ϣ򴫲ķ򴫲ѧϰǲеģӶĲл3Ԫ֮Ȩģʵַǳ㣻4ԪѧϰԪ㷨ӶĳԪܵƫʱԵԪѧϰӿѧϰٶȣ5ͬԪԲòͬѧϰ㷨ӶɸѧϰԣBPѧϰԣ
ĻоBPѧϰ⣬һµѧϰԣѧϰԵԱͨͬʱȨϵBP㷨[1,2,3,4]ӦѧϰBP㷨[3,4] õö࣮ĸķҲ˵һ㣮
2ѧϰԪ
1 һԪΪѧϰԪ򡢷ѧϰڲ״̬ҵ״̬ʱֱܷʵ򴫲Ϣ򴫲ܣ
(1)Ϣ򴫲 ѧϰԪ״̬ʱԪϢ򴫲źu=(u1, u2,, un)ƫ1һ=(1, u1, u2,, un)ӣȨϵw=(w0, w1, w2,, wn)ӵˣnet=զwnetϵ˵õz=netzͨf(.)òԪĻo=f(z)ȫһ=(1,1,,1)˵õo=(o, o,, o)ӡRm
Ȩϵwеw0ΪƫȨϵw1, w2,, wnΪźȨϵͨȡSigmoidǶԳƵf(x)=[1+exp(-x)]-1ԳƵf(x)=[1-exp(-x)][1+exp(-x)]-1ҲȡԺf(x)=xϵ̳ΪĶȣ̿ԸıԪĻoھnetı仯ʣ൱ڸıԪļ


ͼ1״̬

ͼ2״̬
״̬ʱѧϰԪⲿͼ1ʾSIźˣSOźˣ
(2)ķ򴫲 ѧϰԪڷ״̬ʱԪķ򴫲ⲿͼ2ʾ״̬ʱSOʱΪEI״̬ʱSIʱΪEOe=(e1, e2,, em)EI뵽Ԫڲȫһҵ˺f(z)(f(x)Ϊf(x)ĵ)˲wȨõ=(1,2,,n)=Ħ(w0, w1, w2,, wn)EO
(3)ѧϰԪѧϰ㷨 ѧϰԪĲźȨϵw1,w2,,wnƫȨϵw0ĶẠ̇Ʀ=(w0,w1,w2,,wn, )=(w,)ΪԪĲʸԪѧϰ״̬ʱڲѧϰԲʸȽ򴫲Ϣ(աnet)򴫲ļɵĲʸold=(wold,old)ͨĳһѧϰ㷨µĲʸnew=(wnew,new)ӣõѧϰ㷨ԪġԪڴ洢ϢԪԼĲѧϰ㷨Ʋ趨Ԫѧϰ㷨ͳһʽ
new=old +new(1)
еĦnew =(wnew,new)ӣͬ㷨вͬļ㹫ʽ±һЩԪѧϰ㷨
1 Ԫѧϰ㷨

㷨wnewnewƲ
㷨1ǦĦold0ǣ0
㷨2wold+ǦĦold001, ǣ0
㷨30æold+knet0ã1, k0
㷨4wold+ǦĦoldզæold +knet01, ǣ0, 0ã1, k0

ΪʵѧϰԪܣԪڲӦдϢǰͨĺͨԪѧϰͼ3ѧϰԪһʵ֣K1K2FλʱǰͨѧϰԪǰ״̬K1K2BλʱɺͨѧϰԪں״̬ѧϰԪ״̬佻任״̬ͷ״̬֮ѧϰ״̬ʱϢ״̬ʱԪĲʱѧϰԪѧϰ״̬


ͼ3 ѧϰԪ
3ѧϰBP
31ѧϰBP缰ͨBP
2 ѧϰԪȨӹͼ4ʾһ㡢ɸ㡢һһȽϲԪ磬ΪѧϰBP磬üǺűʾΪN(n1, n2, , nM-1, nM)M3niʾiԪi=1ʾ㣬i=2, ,M-1ʾ㣬i=Mʾ㣮


ͼ4 ѧϰBP
ѧϰBPıȽϲԪ״̬ͷ״̬״̬ʱԪyбȽϣ״̬ʱ״̬ʱıȽy-ȨγɼȨ䷴䣮
ѧϰBPԪļһȡΪԺf(x)=x ԪSI/EOֻϢ򴫲ÿԪֻһάźţźȨϵ1ƫȨϵ0Ķȿɱ䣬ԪȨϵɱʾΪw1j=(0,1) (j=1, 2, , n1)ϱ1ʾһ㼴㣬±jʾjԪԪֻмĶҪѧϰ
ȻӱϿѧϰBPͨBPƣ֮ģͨBPȣѧϰBPŵ㣮
1ѧϰԪڵȨϵԶԪźżȨӶѧϰBPԪ֮Ȩģ۶ʵֻǶӲʵ֣ⶼǷǳģ
2ѧϰBP籾򴫲Ϣ򴫲ԪѧϰÿԪ״̬ʱ򴫲ϢÿԪڷ״̬ʱɷ򴫲ÿԪѧϰ״̬ʱɶ
3ѧϰBPĸԪ֮ӹϵԶӶʵϢ򴫲򴫲Ĳл
4ѧϰBPеÿԪԼѧϰõѧϰ㷨ԪģԪԲȡͬѧϰ㷨
5Ԫѧϰ໥ģ˵ĳԪܵҪѧϰʱֻһԪĲԪĲֲ䣮ѧϰʽ޸Ԫѧϰʽиٶȣ
32 ѧϰBPĹԭ
ÿѧϰԪ򴫲Ϣ򴫲ѧϰĹܣԪ֮ȨģBPĴ󲿷ֹɢ˸ԪѧϰԪɵѧϰBPҲ״̬״̬״̬ѧϰ״̬Ԫ״̬ʱΪ״̬Ԫڷ״̬ʱΪ״̬Ԫѧϰ״̬ʱΪѧϰ״̬״̬ת籾⣬Ԫѧϰ㷨Щⲿ趨ƲҪ
һѧϰԪѧϰBP磬Ԫ֮źŹϵȷģΪx=(x1, x2,, xn1)ӣΪy=(y1, y2,, ynM)ӣʵΪy=(1, 2,, nM)ӣ򴫲ϢĹУԪź֮¹ϵ
ulj=xj (j=1,2,,n1)(2)
(3)
j=oMj (j=1,2,,nM)(4)
uijʾijԪźţi=1ʱǱi>1ʱni-1άoijʾijԪźţڷ򴫲ĹиԪ֮¹ϵ
eMj=j(yj-j) (j=1,2,,nM)(5)
(6)
eijʾijԪi=MʱǱi<Mʱni+1άeijkkj>0ΪȽϲjԪԱȽyj-jļȨӣikjʾikԪikĵj
1 ѧϰBPΪN(n1, n2,  , nM-1, nM )(x, y)ƥΪ
(7)
xµʵ=diag(1,2,,nM), j >0j=1,2,,nMȽϲjԪļȨӣ
(8)
(9)
(10)
֤ ֤8ʽi=Mʱ

i=M-1, M-2, , 1ʱ

8ʽ֤
ɣ8ʽУ

֤ϣ
1ƥ磨7ʽ壬ѧϰBPԪľͨBPеķ򴫲ģӶԪѧϰ㷨1ѧϰ㷨4 Բĸ¶ƥԲĸݶȷ½еģ
4ѧϰBPѧϰ
ѧϰBPеԪڲвѧϰеѧϰ㷨ԪģеĸԪԲȡͬѧϰ㷨ӶɸѧϰԣѧϰBPͨBPȵһͻŵ㣮Ԫѧϰ㷨㷨1-㷨4еκһ֣Ҳ͵㷨С㷨ȣ
ѧϰ1Ԫ̶ԪԪ㷨1еƲ=
ѧϰ2Ԫ̶ԪԪ㷨2еƲǾȡ
ѧϰ3Ԫ̶ԪԪ㷨4еƲǡükֱȡͬĳֵ
ɶ1 ģ9ʽ10ʽ֪ԪΪԪѧϰ1൱ͨǰBP㷨ѧϰ2 ൱ͨǰBP㷨ѧϰ3൱ͬʱȨϵͼBP㷨[1,2,3,4]
ѧϰ4:ӦBPѧϰԣԪ̶ԪԪ㷨1ѧϰѧϰ(x(t),y(t))仯ģʱѧϰ(t)ʽ
(11)
(12)
PʾԪԪĲɵʸ
(13)
JP(t, P)ʾƥJ(t, P)=0.5(y(t)-(t))Ӧ(y(t)-(t))PݶȣӶ
(14)
0<<1ǿɵP0PĳʼֵJ(0)ƥĳʼֵ
2 Ϊ{(x(t), y(t)), t =1,2, }ѧϰBPN(n1, n2,  , nM-1, nM)ѧϰ4ԪĲ
(15)
֤ 

ѧϰ4 1

Ӷ

ɣ12ʽ0<<1֪۳֤ϣ
ѧϰ3öȲȨϵƵӶɱЩԪͣĿ꺯ѧϰ4Ӧѧϰ(x(t),y(t))ı仯µĿҪʹ²(x(t),y(t))ƥСھɲ(x(t-1),y(t-1))ƥ2ѧϰԱѧϰ1 ѧϰ2 пöٶȣҲиԵľԣѧϰ4 ʱѧϰʺܴ󣬵ƽָСһֵԺѧϰʱСָСֵѧϰ3 ƽָСһֵԺܿ죬ѧϰ𲽽׶νҶȨϵĳʼֵȽУЩԴǵķĽѧϰԽܿԵĶ̴γһµѧϰԡ׶ѧϰԣȶѧϰ3 һС޸ģõѧϰ5
ѧϰ5:ȿɵBPѧϰԣԪ㷨3ԪԪ㷨4㷨3㷨4еƲǡükȡֵ
Ԫǽźű任һʵķΧٴ䵽㣮ԪĻԺΪԳƵSigmoidɰźű任(-1,1)ΪԺͨ䶸Ƚźű任ҪķΧѧϰ5ԪļȺԪԪļһ⽫һѧϰ̵ٶȣ
ѧϰ6:׶ѧϰԣ={(x(k), y(k)), k =1,2,,N}ѧϰ̷׶Σʱѧϰ4ѧϰƥ
(16)
СĳһֵJc>0󣬸ѧϰ5ѧϰֱJСĳһֵJ0Ϊֹ
Կ׶ѧϰԼӦBPѧϰԺͼȿɵBPѧϰ ŵ㣬ܿǵĶ̴бѧϰԸõԣΪ= {(x(k), y(k)), k =1,2,,N}ѧϰBPΪN(n1, n2,  , nM-1, nM)׶ѧϰԶѧϰľ£
(1) ʼ磺趨ԪĻԺͣԪĲóֵ
(2) NPѧϰڣJ0Jcnp = 1
(3) k=1,2,, N
a. t=np*N+k, x(t)=x(k), y(t)=y(k)
b. 㣻
c. ķ㣻
d. ӦBPѧϰ Ԫ
(4) JJ<Jcת(5)np(np+1)ת(3)
(5) k= 1,2,, N a. t=np * N+k, x(t)=x(k), y(t)=y(k)
b. 㣻
c. ķ㣻
d. üȿɵBPѧϰ Ԫ
(6) Jnp(np+1)np>NPJ<J0ת(7)ת(5)
(7) 
5ʵ
ӦBPѧϰ൱[2]иӦѧϰBP㷨ȿɵBPѧϰǶ[1,2,3,4]иͬʱȨϵͼBP㷨Լ޸ĶһڣǷֱΪѧϰAѧϰB
һϵͳıʶ⣮ϵͳΪ
y(k+1)=y(k)/(1+y2(k))+u2(k), y(0)=0
u(k)Ǿȷֲ[-22]źţΪ˷棬400{(u(k),y(k)), y(k+1)}ΪѧϰýṹΪN( 2, 5, 1, 1 )ѧϰBPнģ
ԪԪΪԪԪΪԳSigmoidͣԪȵĳֵȡ1.0Ȩϵĳʼֵȡȷֲ(-0.5, 0.5)ͬһȨϵʼֵ,ֱѧϰAѧϰB׶ѧϰԶѧϰѧϰAЦ=0.8ѧϰBԪȡ=0.9, =0.005, =0.5, k=0.005׶ѧϰȡJc=10.0ͨʮεķ棬ѧϰA ʱѧϰʺܴҶȨϵĳʼֵУJСһֵԺѧϰʱСJСֵѧϰB𲽽׶ҶȨϵĳʼֵȽУʱJһСһֵԺܿ죻׶ѧϰѧϰ׶ζкܿٶȣȨϵĳʼֵ̫У׶ѧϰԵٶȱѧϰAѧϰBٶȴıϣͼ5һε͵ķͼJ(np) Ƕ̶ȵģnp̶ȣ10nΪ̶ȣο̶Ϊȿ̶ȣ
6
ѧϰԪ򴫲Ϣ򴫲ѧϰĹܣһʵ壮ѧϰԪɷعѧϰBP磬BPͨBPûеŵ㣮׶ѧϰԾٶȿ졢ľָСص㣮ĵĽΪBPʵּBPʵӦô˼ʵĻ


ͼ5 ѧϰԵܱȽ
Ŀ:ȻѧĿ,Ŀ׼:69774002
߼
ƽ(1965-)У˶ʿڣоΪϵͳʶźŴӦã
ܺ(1941-)УڣʿʦоΪֲϵͳDEDSۣۣ
ߵλƽ ɽѧУƹϵ  264209
ܺ ɽѧѧϵͳѧѧԺ  250100
ο
1L V Fausett. Fundamentals of Neural Networks-Architectures, Algorithms and Applications, Prentice Hall, Englewood Cliffs, 1994
2S Zhou, D Popociv, G Schulz-Ekloff. An Improved Learning Law for Back-propagation Networks. In 1993 IEEE International Conference on Neural Networks, San Francisco: IEEE press, 1993: 573579
3S Zhou, D Popociv, G Schulz-Ekloff. A Motivation-Followed Learning. Proceedings of 1993 International Joint Conference on Neural Networks, 1993:549552
4ʿ£ģ.񾭿ƺۣܿҵѧ磬1996
ո:1998-11-09
