Զѧ
ACTA AUTOMATICA SINICA
1997 23 1 Vol.23 No.1 1997



ǰо
ŴBP㷨ͽṹŻ
١ɭ
ժҪBP㷨ֲԭ̽˽;.ͨŴ㷨еĻȺʵBP㷨лؼŴBP㷨.⣬ڽ֦˼Ļϣ¶ṹӶȺŴ㷨ֱṹŻ⣬ṹŻ.ʵӦþоĳЧ.
ؼʡ˹磬Ŵ㷨BP㷨ṹŻ.
RESEARCH ON MULTILAYER FEEDFORWARD NEURAL NETWORKS
GENETIC BACKPROPAGATION ALGORITHM & STRUCTURE
OPTIMIZATION STRATEGY
CHEN RONG XU YONGMAO
(Department of Automation, Tsinghua University, Beijing 100084)
LAN HONGSEN
(Fujian Refinery, Fujian Hui'an 362100)
AbstractIn this paper, when classical genetic algorithms are applied to the region of real number and integrated with BP algorithm, a hybrid GA-BP learning algorithm is proposed. Besides, with the aid of genetic algorithms as well as the improvement of network complexity function, a structure optimization strategy which is on the basis of network stripping is proposed. The efficiency of research work mentioned above has been shown by numerical simulations and industrial applications. 
Key words Artificial neural networks, genetic algorithms, backpropagation algorithm, structure optimization.
1
ڵǰе˹УǰʵӦΪ㷺һ.о棺ѧϰ㷨ͽṹŻ.УBP㷨(Backpropagation Algorithm)ĿǰʹΪձѧϰ㷨ٶֲСͻ.һԣԲȡ̬ȷѧϰӦıϵȴʩӿBP㷨ٶȣΪ˿˷ֲ볹װݶϢָȨֵַʽ.
ĽṹƾΪϾƫ࣬⽫ɣһѵʱӳѧϰ㷨ѵٶϵĸӦԣڶĸ߾ȺܿڵڵĽ׾в̬Ϊѵ֮侫ȼ½ķ.ԣӦѧȷṹ.
2ŴBP㷨
Ŵ㷨(Genetic Algorithms, GAs)ǸݴĵȻ˼룬õŻһѰ㷨134ݣŵ㣺
1)ڿнռͬʱɶʼ㿪ʼЧʸߣ
2)ѰẒ́ھֲ⣻
3)Ҫ׼ɵŻ.
ȱڣ
1)ͨĶƱַ㣬Ϊռɢ¼㾫ַȡ֮ìܣҶЩȡֵΧȷ(ȡ˹Ȩֵ)Ҳ޴ӱ.
2)ȫеѰŲֲС㣬޴ѰʱһֻܵõȫַΧĴŽ⣬׻ȡŽ.
Ȿ¹
1)ȡƱ룬ֱӽĲֵ(ʵ)λֵתΪַ(γַ)֮ȫŴ㷨ԶַĴͬʽԴʵַѡ(selection)(crossover)֮ͬڷ(mutation)ʱԭַ01֮䣬ģ10͵MеĲ.磬3.1415922.718030ַ֡תɵһ

3.141592



2.718030

ȡһĺλнݻΪڶ

3.141030



2.718592

ȡڶĵһλģ10ͽбγɵ

7.141030



8.718592

Ȼֻ򵥵ʾ⣬оף1.
2)ʵ֮Ŵ㷨ֱBP㷨༯ɣǰߵȫѰֲֹС㣬ͬʱߵݶ½֤޴ҵȫŽ.
ݴˣŴBP㷨Ϊ
 NڲͬʵȡֵĳʼȨֵ.
 BP㷨NʼȨֱֵԤѵԤѵNȨֵһ㾫Ҫ㷨ת벽.
 ֱݾԤѵNȨֵӦȷȡֵ䣬r*NµȨֵͬԤѵNȨֵһ𣬹ĻȺ壬(r+1)*NȨֵ.
 (r+1)*NȨֵѡ񡢽桢Ŵ.
 ܵĲٵõһϾҪȨֵ㷨ӾŴ(r+1)*NȨֵѡNϺõģظ.
ƪŴBP㷨꾡㷨ʵ1.

.˹缰ڳѹеӦá廪ѧ˶ʿģ1994.
3ṹŻ
˹ĽṹŻ𽥿չ.Bhat֦ΪĿ2ݣ˼ǣṹӶȺEcomṹŻ̶ȣ֮ڱ֤ѧϰEbpnǰ£ĽṹӶȣEcomֵѧϱΪһԼļֵ.ΪBhatBP㷨ΪѵֶΣֱӴַʽԲòڷ˼뽫ֵԱ任Ϊ
E=Ebpn+Ecomˣ0(1)
һάԼֵ.ʽУEΪۺָꣻ
ΪӣԱEbpnEcomҪ̶.
EbpnEcom֮䲢ϵȴϵ˵ĲͬȡֱֵӰṹյŻ뾫ȣȷ˵ȡֵΪһΪֵ.Bhat˹е⣬˻Ŵ㷨ṹŻԣΪ
¶ṹӶȺEcomΪ
Ecom2Nin+Nhid-Nk)ijw2ij(2)
УNinΪڵĿ NhidΪڵĿ w2ijΪӽڵiָڵjȨֵNk ΪṹŻָʼֵΪ㣬ÿɾȥһڵֵ2ɾȥһڵֵ1.ĳڵ㱻ɾı׼ǣɸýڵָһڵȨֵ(ֵ)֮ʱýڵ㱻ɾȥ(˴ΪԤ趨Ȩֵ羫һֵ).
(2)ʽ¶Ecomָܹѵɾڵ㣬ʵṹ򻯶ڵӵϡ軯.
ڸ׳̶ȣԤȸһڸ˵㹻ӡĶǰ.
 ȫѰŵŴ㷨ѵи磬ֱṹŻԭʼʽ
minEcom(3)
ԼΪ
Ebpnţţ0(4)
УEbpnΪӳӳ侫ȵԾΪEbpnֵʵҪ.
 Ŵ㷨ѰẒ́EbpnԼǰ£EcomʱṹѾԶʵŻ().
Ҫָǣ
(4)ʽΪԼ(3)ʽĿŻֻɾڵ㣬ᵼṹĹڼ.
4ʵӦþ
4.1
ѡȡһڵΪ2ڵΪ1ڵΪ5ǰ磬ֱȡ׼BP㷨Ŵ㷨ŴBP㷨Ϊѵʵ¾жֵķԺ
(5)
ѵΪ30Ϊ30ҪΪEbpn=0.002.
1.1㷨ѵ龫

׼BP㷨Ŵ㷨ŴBP㷨
ѵ
Ebpn0.00460.00340.0018
Emax0.07130.05260.0216
졡
Ebpn0.00480.00340.0019
Emax0.07540.05300.0283

ע1.EbpnΪԾEmaxΪ
2.ѵڲϵ.
1.2㷨Ӧѵ

Ծ׼BP㷨Ŵ㷨ŴBP㷨
0.005090334510210
0.004617725
0.00408400460
0.003415820
0.0030ޡ690
0.0018ޡ1020

ע1.ޡʾ㷨ﲻӦȣʾ㷨ֱ˶Ӧȣ
2.Ŵ㷨ŴBP㷨ӦѵϵCPUʱɱ׼BP㷨еѵʾ.
1.1㷨ѵͼеľȱȽϽǿԿѵֻŴBP㷨ﵽԤ趨ľҪ󣻴ӼŴBP㷨ܹϺõظѧϰɹȡõĽϸ߾Ȳǲ̬.
1.2¼㷨ڲͬʱ辭ѵпԿ׼BP㷨ٶŴ㷨ڽΪѸٵҵŽȡŽʱȴѣ֮£ֻŴBP㷨ʼձſݵѵٶȣѸٵ辫ȵĽ.
õṹʵǾṹŻԴõģһڵΪ3(ֱΪx1x2x3=x1+2x2),ڵΪ1ڵΪ10ǰ磬ṹŻڵx3ɾȥͬʱڵҲ10Ϊ5ṹΪ.
ô֮ӳ亯ǷҲ?Ϊ˲ͬԽṹŻǰֱѵͼ飬2ʾ.
2ṹŻǰѵ龫

δṹŻ羭ṹŻ
ѵ
Ebpn0.00170.0018
Emax0.02070.0216
졡
Ebpn0.00210.0019
Emax0.03010.0283


ӱ2вѵóۣ羭ṹŻ֮侫Ȳδܵ𺦣෴ܹڵڵ޳ʹʵֵĺϵıʶøΪ׼ȷ룬˵òͬʱ侫Ȼάֲ.
4.2ʵӦþ
ԭͳѹŻУڳѹ߳ƷϢ޷߻ȡбջƴ˺ܴѣ˹缼ѹģʮֱҪ.ڽģʱѡȡڵĿĶٶģ͵ľȼӦǿֱӵӰ죬ǲṹŻȷ֮.ڳѹΪͷԴ͸ӹҵռ伫ΪӣʲŴBP㷨ѵ֮.
Գһ͸ɵģ͵ĽΪͼ14ʾ.ͼ1Ϊݹպͻȷṹͼ2ΪṹŻ֮õģͣͼ3ͼ4ֱΪͼ2ʾģ͵ѵͼ龫ȣǾС4ĹҪ.ͼ1ʾģ͵ѵȻ㹤Ҫ龫ȴɢˣƪȥδʾ.
Ŀ˶ǰһȫֿѰ㷨ŴBP㷨ͬʱṹŻ.оĳЧֵʵʹҵӦõļ.ݵ˼벢ڶǰоκشĶͿչŻͼֵ.



ͼ1ʼ͸ɵģ



ͼ2Ż͸ɵģ


ͼ3Ż͸ɵģ͵ѧϰ


ͼ4Ż͸ɵģ͵ļ龫
עĲֽ1994һйܿԶѧ.
߼飺١1969ڽʡϾУ1992廪ѧԶѧʿѧλͻѧʿѧλ1994УԶۼӦרҵ˶ʿѧλ.ĿǰڸУԶϵν̣Ҫ˹硢̽ģŻCIMSȷо.
廪ѧԶϵڣʿʦڴ¹̿ƵĽѧڡ̿ơȿγ̣зǹҵ̽ģŻȽ.е˹ҶصƼؿ⣬λ񲿡ίƼ.רСģۺĻӦá40ƪ.
ߵλ廪ѧԶϵ100084ͳݰ362100
ο
1ݡHolland J H. Genetic algorithms and the optimal allocations of trials. SLAM Jounal of Computing, 1973,2:88-105.
2ݡBhat N V,McAvoy T J.Determining model structure for neural models by network stripping. Computers and Chem.Engng., 1992,16(4):271-281. 
3ݡGoldberg D E. Genetic algorithms in search, optimization and machine learning. Mass:Addison-Wesley, 1989.
4ݡDavis L. Handbook of Genetic Algorithms. London:Pitman, 1991.
ոڡ1994-07-16
