о뷢չ
JOURNAL OF COMPUTER RESEARCH AND DEVELOPMENT
1999 36 5 Vol.36 No.5 1999



һӱȽƷ
Զԡ塡
ժҪӦʵʹʱѵݼٶ쳣ֵ.ΪʹиõȼԣиȽͳѧԭǰ磨FNNһȽƣREΪµĿ꺯.ԺƽΪ۷ͼģʵ֤REԽԣȻС˹ƣLSEŵ㣬ڵֿ쳣ֵoutliersŷLSEʹFNNʵ.
ؼʡȽƣǰ磬ƽ쳣ֵ
ͼšTP18
A NOVEL ROBUST ESTIMATION METHOD FOR NEURAL NETWORK
LIU Guang-Yuna, QIU Yu-Hui, and QIN Chao-Ling
(Department of Computer Science, Normal University of Southwest China,Chongqing400715)
LIAO Xiao-Feng,
(Department of opto-elec.UESTC,Chengdu610054)
AbstractWhen neural network (NN) is applied to practical engineering there are some noises or outliers more or less in training data sets. To make NN hold more robust, a novel robust estimation function is proposed to serve as new target function of feedforword neural network (FNN) according to the theorem of Robust Statistics in this paper. The results of the theoretical analysis and simulation for function approximation problem show superiority of the method which not only holds basically the merit of least square estimate (LSE) method but has robust against outliers.
Key wordsrobust estimation, feedforword NN, function approximation, outliers
1
ֹͨһģܹһѵݼΪҳõĿģͣģ֮䶨һĿ꺯ȥһʮֱҪ.ھMSEľԺ׼ԣʰ磨NNڵݽģMSEΪĿ꺯ƺһѡ.ǣͳѧָ1ݣһĺûӦж෽Ŀ飬һԡƫԡԡЧԡ걸ԼȽԣrobustnessε.С˹ƣLSEĻ˼ͨСѵݲвƽʹһģ͵ϴﵽѳ̶ȣٶķֲΪ̬.ʵֲŷֲΪβֲ̬ʱLSEڵֿ쳣ֵoutliersʱͷǳⷽԵΪ˵LSEȽԣ14.ˣȽƣrobust estimationΪREͱ.
磨NNӦʵʹʱѵݼٶ쳣ֵУֲ̬Ҳ.һΪʹNNõӦʵʹУⲿѧ߿չ˽RENNϵо58ݣǰڹ910ݵĻϣһչⷽоԺƽΪ۷ģ˵REԽ.
2Ӱ캯Ƚ
3ǰ磬ʸΪXpʱĿӳ亯ӦΪ
Yp=F(XTpW)+ep(p=1,2,N)(1)
WΪͨѧϰõһȨepʸ.
ͨͳķLSEԭWйƣ漫ֵ
(2)
Ľ.ӦIFΪ
ps(rp)=rp(3)
Ҫֵ0Eeps=2<.epķֲǳβѵݴ쳣ֵʱLSEĹƾǲȽģ1.ˣһЩREˣM-ƾеһ.漫ֵ
(4)
Ľ1,11.Ц()Ϊĳһѡĺ.()͹ģһ׵()4ʽȼ


ͼ1IFͼ
(5)
ף1ݶԺ()Ҫ󣬶Ŀ꺯
(6)
СʽķΪLCElease cosine estimate.ʽCΪĳһ.ӦIFΪ
(7)
ʽAndrewsף4иȽƺ.
ͼ1ˣ3ʽ루7ʽͼͼ֪쳣ֵʹrp|rp|=Cʱps(rp)󣬶pc(rp)ȴʼrpĴڣ|rp|>Cʱps(rp)ʹ󣬶pc(rp)ʹ0.˵epc(rp)쳣ֵУȽģ,epsȴӦУȽģ.
3Ȼ
LSELCEַȽԲҲɼȻԭƵ.ѸݵĳһģͣȻڸģʱݵĸʣһȻʵʻ߸µһֵ12.
ֲΣģ͵Ȼȡ۲ĵʷֲPDF.ٶÿݵypһrp=yp-prpĶֲΪf(rp)ôһģ͵Ȼÿĸ֮
(8)
ʽf(rp)rpڵĸ.Ȼ8͵ȼڼĶСĶĸֵ
(9)
Wǿʸ.ΪNͦǳʽ͵ͬ
(10)
3.1˹ֲ
¶ٶǳʸ˹ֲ.׼ֲ̬ܶȺΪ
(11)
루10ʽе
(12)
ʵ൱Ŀ꺯r2pLSE
(13)
ֲ̬ͳ֪ʶ99.73%Ƿֲھֵ3ҷΧ֮.Ȼʵʹϵݶ1%10%ĳ3ҷΧĴִ.ڸ˹ģͼ֮£쳣ֵֵĸʵϾͼ0LSEЩ쳣ֵΪЧݴʱģ;.
3.2ҷֲ
Ϊ˵쳣ֵãҪĳڿCauchyгβ͵ֲ.ǶһֳΪҷֲcosine distributionĺ
(14)
rpR, C>0.
루10ʽ
(15)
ͼ2fg(rp)fc(rp)ͼ.|rp|Cʱfc(rp)=exp(-2C)ΪԳƳβͷֲ.ڴ˷ֲ£쳣ֵֵĸǴڵģ.ڸ˹ֲ̬£3ҵֵֵĸʼΪ㣬˷ֲδ쳣ֵֵĿ.


ͼ2ֲͼ
ɴ˷֪LCEҪݵΪȥģʹﵽСֵϺùƵĿ.LSEֻȥ쳣ֵܻ֮ȷƣ111.
4ģ
Ϊ˱ȽLSELCEַҺƽΪ˼ģ.ַھṹԼѧϰ㷨֮ΨһǼٶ˲ֲͬʽ.ȻѡǺƽREȻƹ㵽ȥ.
4.1ģⷽ
Ϊ˵⣬ģУÿһɿƵĲṹʼȨֵѧϰʣѧϰ㷨׼ȷرͬΨһĿ꺯һ׵ͬ.
ʵУѡȡһ1-10-13ṹBP磬ļfx=1/(1+e-x).ѧϰ㷨Ϊ׼BP㷨ѧϰʦ=0.2ϵ=0.8ȡ=1.5AndrewsΪ1.51.8Щֵ.ƽΪ
y=0.4*sin(2Ц)+0.5 xʣ01ݡ(16)
ѵݼ3ֲ͵ݹɣ
(1) ɣ16ʽĸݣ100
(2) (1)мС˹N00.1ݣ
(3) (2)ټ5%쳣ֵN100.1ɵ.
ע⣬f(x)ΪĳһѡĵxpypΪѵģʽԣѵϵΪyp=f(xp)+ep˴epһִڵƫֵ(2)ȡΪֲ̬(3)ȡΪֲֵ̬.
ΪѵܣһǰѵݼصĲݼǱģʵԣ16ʽΪxʣ01ڲ150Ծݵ㣬ڶ3ѵĲ.
4.2ģ
LSELCEֱַǰ3ѵѧϰѧϰΪ8000.Խֱʾͼ3ͼ4ͼ5.3ͼзֱɣ16ʽʵʺΪȽ.


ͼ3ַݼ(1)Ľ


ͼ4ַݼ(2)Ľ


ͼ5ַݼ(3)Ľ
ʹõ(1)ѵĽͼ3ʾ.ͼLSELCEַڱƽ16ʱЧܺ.ɵ(2)ѵĽʾͼ4Уַ൱õرƽ16ʽ.ֵһǣʹLSEԸ˹ŵѡ񣬵LCEֵҲ൱.(3)ݣ5%쳣ֵѵĽʾͼ5УַĲʮԵرֳLCELSE.LSEѵĽںƽʱʮֲ׼ȷȫƫʵֵLCEıƽȴϺõشԭ.
ͼ6ͼ7ͼ8ֱʾַ3ѵе.


ͼ6ݼ(1)ѵ


ͼ7ݼ(2)ѵ


ͼ8ݼ(3)ѵ
ͼԶȣڹ۲죩.ݼ(1)(2)ͼ6ͼ7˵ַ¾ĳһСֵݼ(3) LSE޷LCEȴͼ7.
Ӧ˵ͳѧеȽƷĿıСвʵϣС˹ʹвƽʹﵽСķСвϲܳýȷĹƣҲ˵.
5ᡡ
ѵNNʱලѧϰ㷨LSEַݼ׼ȷС˹ʱȷ.Ȼݼ쳣ֵʱʹø÷ýͲȷ.һ棬ʹLCEȴ3ܵóȷĽ.ԭLCEREʵ.ڴˣǽչⷽо.
õпѧίԱо(Ŀ 96-4335).
߼飺ԶУ19614ڣҪ¼ܼ·ϵͳĽѧо20ƪ.
ߵλԶԡᡡʦѧѧϵ졡400715
塡ɶӿƼѧϵɶ610054
ο
1Huber D J. Robust Statistics. New York: Wiley, 1981
2ҫģMƵĽ.Ӧѧѧ1994,7(3):401410
(Wu Yaohua. Asymptotic behavior of M-estimators in linear models (in Chinese). ACTA Mathematicae Applicatae Sinica, 1994,17(3):401410)
3֣ҹģλòͿ̶ȲȽƣӦѧѧ1985,8(1):1828
(Zheng Zhongguo. On robust stimators of location and scale parameters in a linear model (in Chinese). ACTA Mathematicae Applicatae Sinica, 1985,8(1):1828)
4Andrews D. A Robust Method for Multiple Linear Regression. Technometries, 1974:523531
5Liano K. Robust error measure for supervised neural network learning with outliers. IEEE Trans on Neural Network, 1996,7(1):264250
6Chen D S, Jain R C. A robust back propagation learning algorithm for function approximation. IEEE Trans on Neural Network, 1994,5(3):464469
7Oja E, Wang L. Robust fitting by nonlinear neural units. Neural Networks, 1996,9(3):435444
8ڣŰ飬.³عѧϰ㷨.ѧѧȻѧ棩1996,24(4):4855
(Huang Lei et al. Learning algorithms of neural networks for robust interval regression analysis (in Chinese). Journal of South China University of Technology (Natural Science), 1996,24(4):4855)
9Զԣʰ.ȽĿBP㷨ѧ199724(2):6668
(Liu Guangyuan et al. Fast BP algorithm based on robust-estimators (in Chinese). Computer Science, 1997,24(2):6668)
10Զ壬ʰ.һȽƷƹо.ӿѧѧ199921(1):128131
(Liu Guangyuan et al. Generalization study of a robust estirnation method for neural network (in Chinese). Journal of Electronics, 1999,21(1):128131)
11ϣ棬ɹ.ʵûع.磬1984.301321
(Cheng Xiru. Modern Practicable regression analysis (in Chinese). Guangxi People Press, 1984.301321)
ԭյڣ1998-10-27
޸ĸյڣ1999-03-27
