Ϣ
Information and Control
1999ꡡ281ڡVol.28No.11999



۵ķչǰ


ժҪϵͳ۷չʷ ״ڴ˻ϣҪչ漰ǰ˲лһ ۣµĹ۵㣮
ؼʡۣ񾭼㣬㣬񾭿ѧѧо
THE DEVELOPMENT AND FORWORD PROBLEMS 
OF NEURAL NETWORK THEORY
LIU Yonghong
(Dept. of Automation, Wuhan University of Technology, Wuhan 430 070)
AbstractThis is a survey paper of the development and forwo rd problems of neural network theory. It is divided into four parts: 1. Introduc tion to neural network; 2. The history and present condition; 3. The development trend and forword problems; 4. Conclusions. In the paper, a new comment and vie w for neural network theory is presented.
Key wordsneural network theory, neural computing, evolutionary computing, study based on neurosciences and mathematics

1
һŻԾıԵԽѧ.оķչ̺ǰ⣬Ҫ .
ǾϢдʹģƽмĻǸ߶ȷԶ ѧϵͳӦ֯ϵͳ֪߼ƵΪ ֪ܵģ⣮ӽѧѧһӵĲϵͳͬڴͳ NeumannʽҪС֪ʶ͡顱ȸ߼Թ ˹Щܣʣڼ˼άܵʶ80 Ѷ֪ıʵĻоҵ˿ǰĴ̼ ͼƶã
ʮʵעĿĽչһչ˼ںʹ 񾭼㡢Ϊµѧƣģõ˹㷺ӦãƼ ҵҪ˾оƬоƬӣIntel˾IBM˾AT &T˾HNC˾ȡ˶רвƷгҵͿвѡ ãҲӵʵûĹߣҵ˹裮񾭼ѧ ƹļԺͳԣһʹοڻֻ ǳ¶âо޴Ǳᣬǰõģ
ʵϣ̽ԡ˼ά֮Ĺϵոտʼ·ʮԵļԭ 临ԣѧϰͼ̵ĻģȷоܵǵĹע δķչؽǼĵģ۵ǰ⽫͸21Ϳѧս Уȡشͻƣ
2չʷ״
絮5׶Σ
(1)׶.40񾭽ѧѧѧԼ ԪĵоȶгɹУѧMcCullochᳫֻ 壮ѧPitts1ݣϢ۵ ģ͵ķоϸĶͽṹԪһЩԣ˵ һ񾭼ģͣԪֵԪģͣMPģͣʶģԿ ߼е磬һЩ㣬֮໥ϵһģ ͣҪڣĲмǿΪΪĳ˷ṩ˿ ӶоһԵ˼룬˺ܴӰ죮
˵ѧKleeneڴ˻ϳһԶۣWienerǿ۵Ĵ ʼ֮һ1948רCybernetics2,̽˶ͻĿ ͨѶ⣬1961ݣҪѧϰֳ⣬ѡѧϰ Ϊо󣬶Ե粨֯ϵͳ̽ǣMPģյ NeumannӼҪ֮һѧNeumann3ִѧ Ĵʼ֮һ֮һоҷԶ֤ ٴһȷʵܹҷܵķԶģͣ1966ԪԶģ ϵͳеԸƹܣģȻ󣮵кܶԪԶҲ һĳϵͳɢúãǣNeumannһµļ ԶۡȻ˹֪ʶļ
⣬ѧTuringͨüĳģͣ45ݣPost 6ݶ֤һҪԭϴһ֡Զʶκα ܹʶķŴTuringۣΪдļʽԵķ ṩۿܣҪǣо㷨ǹϵͳЧʣдʹд ԭϾͬڣЧʻٶȲֵͬעǣTuring ߼֮ٵĵȼֵõ֤ʹǶڴԺͼ֮ ԵһǿˣϧʱΪƶǻ߼ԪԣΪ Ϣߵϵͳǻȱʶ
1949ѧHebb7ݵThe Organization of BehaviorԴϸѧϰ˴󵨵ؼ裬ΪHebbѧϰĻ˼ǣ Ծͻ΢ı仯ͻϵǿȿɱѧϰͼĻǿ̵ ˴֯γϸ(ǧԪӽ)ѭ񾭳嶯ǿѭκһԪͬڶϸϣ˵ϸǴ˼άϢĻ ӣͻģͣ˷ֲ䣬Ϊ(connectionist )ģǱѧϰֻ̣ʸо߰ͻı仯 ͻǰλĻ˱κ䣮˵Hebbķչ شƶãȻ֤
50۾߱˳ģʵRochesterHollandIBM˾оԱͨȡǿȣַʽģHebbѧϰIBM701Уȡ˳ɹڳͻ󣬼дԵĴ񣮵ģģҲֻ1000ԪÿԪֻ16ϵ㣮 飬ܵƣHebbѧϰۻӰIBMʵϰ оMcCarthyIBMһС飬̽йϷܳ򣬺Ϊ ˹ܵҪʼ֮һ˹ܵһҪʼMinsky1954ϵͳ ܹѧϰо뷨дĲʿУRosenblatt ĸ֪(Perceptron)ѧϰģ
1952ӢѧHodgkinHuxley˳ǹ޴Զѧ΢ַ H-Ḥ硡


ԣHodgkinHuxleyоĳɹشۼӦüֵٻŵҽѧǵѧߵĹעа˷ḻݣۺʵ˼ãЩѧ߶H-Hоõ˺ܶĽ磬ĤķԼ񵴡缰ȶԵȣ 
1954ѧEcclesʵͻķģͣ8ݣͨͻĵʵõ֤ʵҪǣΪģͻĹṩԭͺѧ֤ݣ1956UttleyһɴԪɵִԪΪϢ(informo n)ģΪһԷShannonֵ ֮ȵȻ70ڰӦӦģʽʶ ΪģʵϵͳĹԭרInformation Transmission in the Nervous System
(2)һθ߳׶Σ1958ѧRosenblatt9ݻMPģͣѧϰƣƹMPģ֤֪ܹͣΪ࣬Բɷ֣Ҳһƽܽռָ֪֮Ȩصĵڼֵ֮ĸ֪ģͣ״ΰ۸ʵ֣磬1957굽1958˧˵һ̨񾭼Mark ĸָ֪˴㴦Ԫ3֪ һҪо򣬲Խ֪ƹ㵽3㣮δҵȽϸѧѵ㴦Ԫָ֪һѧϰ֯ѧģͣṹѧ֪ʶģ͵ѧϰʱڲṹӦϵָ֪ѧϰͻǿɣӦģʽʶȷ森˵ģͰһЩִ񾭼Ļԭ緽ͼϵشͻƣִҪ֮һRosenblatt֮ټѧ߶оļȤ ϰټӰʵҷ׷Ͷ򣬾޶ʽ磬ɲʶ ѸȷзǱˮͧλãһʱоڻһĳɹЩʵ˵ γ״θ߳
1960WidrowHoffӦԪADACINEģͣ10ݣһ ȡֵ磬ҪӦϵͳоһΪԿɷ ӦӦȫСֵ羭ѵͨеĻز ӦԤ森ǵһʵõ磮ǻ3 йԣԸѧ⣮˵ǶԷֶѵһã ӦƵۻWidrow70ԴΪADALINEѧϰ 80ǵõһֶѧϰ㷨
Уһֳ˥˷ܲΪ(reverberation) ǹĵǲĲԻڡȵʵӰ죬Լ ͨⲿƻ첨ӶĽԣԵ粨еĦɣ1961 ѧCaianielloԪģͣ˲Ӧԣһ񾭷 

ʽ1.ǵλԾSiǵiԪ̼ܵiǵiԪֵt-rǹȥĳһʱ̣HӰʱ䣬Wij(r)ӰȨأ˷Ժ룮÷ͬʱԪĿռʱʣȡɢֵڲṹķʽеĻľڣܷӳѧϰͼ̣ڸ÷гһԺоʱͻ һڷϵͳڽ⣬һ൱ӵ⣮CaianielloHebb˵չԼģͣѧϰͼظǿأԼΪʣһּ䷽

ȤǣCainanielloԵĳһ״̬ӦͬĲ˾˵磬Ӧ⡢롢ǡ˯ߺεȳֵķ⣬һδϵͳĽṹϣϵͳĹһЩԷȻķʱ䷽飬һЩ򻯵¿Եõ⣮ѧڼģⷽо൱ӵ⣮
1962Rosenblattĸ֪ܽ11ݣЩѧҲѧģ ͣ磬ôȷо磮ֵһǣҹпԺ19 65þһЩģͣصоӾϵͳϢݹ̺ͼӹĻ Լйѧģͣ⣬FogelOwensWalsh1966һڽ רArtificial Intelligience Through Simulated Evolutionڸᳫ ˼뷽ϵʱ˹ܵܵѧĻɣһֱ90ű 
60СڣGrossberg1213ݴϢĽǶȣо˼άʹԽϵ⣬ѧо֯ԡȶԺԵԣԼֱӴȡϢйģͣһṹ(instar)(outstar)ѩ(av alanche)ΪСĽṹѩڿռģʽѧϰͻԼʱģʽĴ 棬磬ִʶͿƻֱ۵˶ЩɹԵʱӰܴ ЩѧGrossberg齨Ӧϵͳȡ˷˶ĳɹ漰 ĸձۼAmariעΪϸѧ ϣǶηоõҪWillshawˣ14һģͣźźֻ룬ָźţȫϢ( holophone)ģͣΪùѧԭʵ춨ۻΪȫϢͼ ϵıоһ;NilssonԶĹ ֪˾15ݣΪʵһ任һӳ䣮ѶϵͳĽṹ͹бȽʶûиһʵõѧϰ㷨ʱ֪ѵɶֵ߼Ԫ(TLUs)ɵĶʵ÷60ӸпƷȹһӣʶӾĹһֱòǿܵ䣬Ҫȱ۵ȷָԶҪӵӦýĻ ֱۣ80Ullmanָ˶ƵļۡMarrӾۣΪ һķչ춨˼ʵۻ
(3) ֽ׶ΣңԶңɼĿʵ˺ܶ˵Ŀ⣬ Ϊ繤Ӧȡԭӵ̡ҪͶȤǳ󣬶ʵ ЧҲȽ⣮ʱMinskyǣ˹ܵķչǰ;⣮˳ƵΪྶͥѧ飬¶˹ͶʵӣӸ֪Ĺܼ֣ѧϽ˷֤˸֪ʵXOR ߼⣬Ҳʵνʺʶ֪ʽļ ֪ȺΪ1969MinskyPapertMITһPercertrons ʱ֪йصо䷢չ˶ӵӰ죬Щѧ߰оȤתƵ˹ ּйصۺӦ÷森ƶ˹ܵķչʹռλڴ˺15δо⣬ǰйоҲܵȾֹѾоĿ⣮
MinskyԸ֪һԣȻƫ󵼣ѧSimon 1984һPatterns and Operators The Foundation of Data Re presentationли֪̣˵7080й ֪ĳɹȱ˽⣮źǣMinskyPapertûпձѧAmari 1967ηѧһҪɹ֪дǱͻҲ ;ԲʱӰ죮Minskyϯ׽(1987) ˵ȥRosenblattĸ֪µĽ̫̫ڿ͹ϣ谭 ķչ
ǣȻŵľԶ׿ʶĿѧڼֲϵоۣ еĿѧڴڼͶ뵽򣬴µĻȡϵһϵҪ £
Holland1960Ŵ㷨ѡѧͻ۵оУ̽ʵŴ㷨ۣ1617ݣŴ㷨һֽȻѡȻŴƵĸ߶ȲСӦ㷨Ӷ۵һµоȻʱָֻĲʿ˹ѧߣڼѧûаġŹ֡ĵһ£ͬʱHolland ҲûаԼӦõ㷺עЩʵⲢȡٵͶʣӲԼȾֻȤԼоչȻHollandо˹ܵо෴ΪѧϰԻķ ⣬һ¸ΪʣƵķϵͳĹʱıͽ Ҫڼģͻģͣϵͳпɡ̽ʽѧϰһ ѧGoldbergʿ1983꽫۳ɹӦõúܵģϵͳ. һЩ ѧʵҲõ֤
1974Stein, Lenng, MangeronOguztoreliһԪģͣ18ݣ÷΢ַͨ͵ԪĻ

ʽаtʱ̹淶ͻ嶯Ƶx(t) (0x(t)1)ϸf(t)ϵ a0, 0pqbǸԪԻĳԪԣbֵȡ(ͨ) ʾ³嶯Ĳ1976Heidenо̵ʣ 

ǵķ̻Ϊ3άϵͳ

HeidenõһЩҪĽۣ19ݣ磬ϵͳɻΪϵͳһڹFloquetģ1ϵͳΨһĽȶڹȣԪдΪһѭģ
1976Grossberg20Ӧ(ART)Ǹ֪Ƶģͣsuperrisedѧϰʽ˵һunsuperrisedѧϰʽCarpen terһоART磬ֽṹART1ART2ܹʶӵĶԪͼѧϰ֯ȶһΪһȽѧϰģͣ
ƵΣWerbos21BPԼķ򴫲ԭFukushima 22ӾͼʶNeocognitronģͣ¶Neocognitron; Am ari23ݶѧоܵһЩѧߵĹעFeldmannBallardRu melhartMcClellandѧƣзֲ(PDP)ļԭ㷨о ҪĸģͣRechenber˽ԣɹӦõ綴УKohonenһרAssociative Memory-A System Theoretic Approach (1977)ȫϢ洢洢Ĺϵϸ˾洢 ִ洢ԵģԻķʽʵȽף
оɹӰ󣬼ᶨۼڼоΪڶθ߳ ׼Կ϶ڼֽ׶βлĶԷǳǿĺרѧ֪ʶУ
(4) ڶθ߳׶ΣKohonen֯ӳģͣ24-26ݣӳʣһάάȷģڼϽģ⣬ͨʵ չʾӦѧϰЧΪпƹ㵽άʱ֯ ľֲȫȶ⻹ûеõֵһǣHintonAndersonPara llel Models of Associative MemoryһӰ죮Ԫɵ ģҲ洢ܣر壮40ѧо Ȼͬʱµʶ1982ѧHopfield27ϸ ԣ洢øӾϸʶ㷨ǽ洢Ϊĳ ۺСֵ⣬ʺڵݹ⣬Lyapunovз ڵһ첽ʽ໥ʣֵʵ֣ ȶоݣģʽŻĹܣϵͳ˶ ̣HopfieldԪģһ΢ַ

uiǵiԪĤλCiRiֱݺ͵裬Iiǵ· TijǵJԪԵiԪϵǿȣf[u]uķԺһ ȡS߻ԾLyapunov֤Tij=Tji£ƽ㸽ȶԣģԵӵ·ʵ֣оȡشͻƣ۵ķչԶӰ죮1982HopfieldѧԺύ˹ı棬Ҫǣռǰоָ˸ģ͵ʵԣӴˣڶθ߳Ļˣ
MarrӾ񾭿ѧоƪ£Ӿ۶ӾϢӹĹ̽ ȫ桢ϵͳ̵Լۡ㷨ʵֻƼӲɵ3 ˲1982Marr28ݵVisionʹѧ棬ΪȨ Ժ;ԵMarrۿܵʾ£Hopfield19821986 缯㹦ܵۿܣ29-31ݣѧоHopfield ȳĽߡ䡢εȣڽУƶķչ 磺1986Lee32߽ͻӣʹһĴ洢൱ ߣ죬ŽӣӼĿӣʵԽѣò ֧֣Lapedes33Ƕķչ 估ԼŻ˫عܣƹ㵽ʱ仯Ķ̬ڴN άܸߣҲΪһʵѣһЩо߷,Hopfieldеƽλδ֪ʹһƽλãķҲȷȶԣֻõСֵıҪǳHopfieldTSPҲһЩ⣬ҪĽ ⣬Щѧͼʵȶԣһɣݴ
1983Kirkpatrickˣ34ʶģ˻㷨ӦNPȫŻ⣮˼Metropolis1953ģƽ⣬ͨģ˻̵ķȫŻȫţ㷨 ׼һֺЧĽ㷨ʵϣǻMonte Carloһ ʽ㷨1984Hintonˣ3536Boltzmannģͣ ͳѧеĸͷģ˻𷽷Ʒѧϰ㷨棬 αǿѵģһģͣ޸֮ΪԪ Ԫһǿȵӹɣһ񾭼ģͣSejnowski371986 ˸Ľ˸߽BoltzmannͿ˻ȣЩΪĻ ۣ
Poggioˣ3839MarrӾΪӾ㷨о1984 1985˳Ӿ򻯷ʹӾоͻԽչҹ ѧƾӾԪĹGabor(EG)ģͣԼйӾ ⡢˶⡢Ӿļģͣƾŵ˻˳Ӿ񾭶 ѧܣǿһµ;
Hecht-Nielsenһλصѧʽҵң񾭼֮һ ۡӦüҵҪף1979꣬ʼƶMotorola񾭼 о뷢չƻڴ˻ϣ1983ֽһƶTRWļƻһ ԴĶģʽʶ磬ҪͼѹͳƷɹһ ΪTRW Mark1987꽫ͶҵӦãGrossbergʽʱ ƥ˲1988֤˷򴫲㷨ڶӳԣ40ݣ
1986RumelhartMcClelland Parallel Distributed Processing: Exploratio n in the Microstructures of Cognition棬Ľչ˼ ãչʾPDPоŵˮƽѧѧѧ ѧѧͼѧѧƵѧߴӲͬоȡõĳɹ ǽ˲зֲۣҪ֪΢оǣRumelhart ˶Back-PropagationError PropagationǺBP㷨 ܵѧߵӣ
⣬ҹϵͳѧǮѧɭ80о˼άѧ1986 ˼άѧ41ݣй緽ģԡ˼ ά񾭻֣̽ԡ˼άһȷɢѧģ͡оصĳƪ ¡ʼ⡱Ȿ˹ѧйʿļ죮
1987ʥŸٿ׽ᣬϻ(INNS) úINSSĿJournal Neural Networksʮֹ ѧ۾͵ķչǿѾ˶ۣǣ ߼Ԥмƻ(DARPA)֯һרҡڽеУ߷ǧλй оߺѧߣ19882һݳٶҳоƻ֤森 11¿ʼִһչ缰ӦõİƻͶ4Ԫҿѧ (NSF)1987겦50Ԫ1989NSFONR & AFOSRͶʴ1ǧԪDARP AĿǣǽܵΨһϣһЩѧ׷׳ оƶйؽƻӦѧᣬٿԡ 飬磺1990ŷٿ׽ʻParallel Problem Solving from Nature (PPSN), 1994IEEEѧٿ˵һʻ飮
ҹѧԼ80ڹעһЩѧȵãп ԺѧƾţҦֵȣ42ݣѧо 19889·ٰBeijing International Workshop on Neural Networks: Learning and Recognition, a Modern ApproachINNS鳤Szuʿڻڼһ ϵнЩݳˣ43ݣʱҹЩѧҺͼѧ ʼһȤչһо
1989ٿȫһʽ飬1990ҹİ˸ѧڱٿ ׽ѧᣬý׷ױһشʢᣬҹ緢չԼ ÿˣ1991Ͼٿйѧ(ڶ)ϳй ѧᣮҹ863߼оƻ͡ʵǡƻ1990׼˹ 3⣬Ȼѧ͹ƼԤлҲоѡָϣ ȫѧһЩѧۼӦ÷Ϊص㣮Щ ΪҹչõʹǼӿ첽ҹĲ INNSʼҹ1992ѧᡢIEEEίԱĹѧ IJCNNڱٿ
1988ChuaYangϸ(CNN)ģͣ4445ݣһģ ԼϵͳϸԶĶѧĳֶ۵ķչ ˺ܴӰ죮⣬Kosko˫洢ģ(BAM)4648ݣ зǼලѧϰһʵʱѧϰͻģʽȫȶԵĶѧϵͳ 
MeadVLSIϵͳĴߣConwayMahowald˺һֶϵͳĵ ӵ·ģ⣬ƹϵͳ磬һкǧԪVLSIоƬ˵ Ĥ׶ϸķʽһVLSIоƬԴˣ1989רAnalog VLSI and Neural System MeadMuhlenbein4950һֽϵͳ۵ʽģ ͣһŴģͣ˼ԴWaddington1974귢ģԻ ͹ϵAleksander51˸߼Markov chain ۣԡṹԼоΪ߼Ԫķչṩµķ ;
֤˵θ߳ѧоۣشɹ Ӧ̼ܵԱļͣ
(5) ·չ׶Σ۵ķչʷĸ߳׶Ǻ׶ ģIJCNN91ϯRumelhartʶһ㣬ĿĻһ۵㣬 ķչѵһת۵ʱڣķΧڲӦ򼸺森 ѧƵۺͼѴﵽһģΪ絽· չ׶ΣҪƺͻƣʹ似Ӧõõ֧֣
90ķչ˺ܴӰŵEdelmanDarwini smģͣҪ3ʽDarwinism񡢢򡢢һϵͳۣ DarwinismĽṹɰСDarwinNallance磬 ǲеģְ˲ͬܵһЩ磮HebbȨֵ򣬵һ ˶̼ģʽúϵͳͨѧɨ͸Ŀ꣮ϵͳйȺ(group) ã1984Ͳˣģʽѡ׶ȺơȺѡȺ
1994꿣52ݶϸ罨µѧóһϵн 磬ɢԡƽλõĿʾƽ̬ȫȶԡȶԡڽĴ ԺԵȣʹȡµĽչΪо޴ǱӦǰ дԴΪչϵͳ
ĹѧַܳӹѧǿĻͲдʵ ֵĹģӶǿӦܺѧϰܣ˽ѧӣWunsch 90 OSAһAnnual MeetingùִARTҪǿɹѧӲ ɣARTԪĻΪ˫͸ѧùռɶֵ λ˲ͼĶάFourier任ѧϰӦ˲ܣ԰ѹ ѧлУпٺȶѧϰص㣬ԪĿ٣ ΪڲҲٺ࣮ܶ1995Jenkinsˣ53о˹ѧ(PNN) ˹ѧάлѧϵĹѧϵͳʵ˹ѧԪǽѧ ʵȡ¶ֵӵǣ90McAulayJewelѧ ڵӷӦڹѧо5455ݣڹ洢ȷȡһ ܵǵĹע껵˲ƵCas (Eu, Sm)ӸʵIPA( Interpattern Association)HopfieldЩȨزģͣ56 Ϊַʽʵ֪Щͨѧϰı以Ȩصģ ЩԹѧķչ𵽺ܴƶã
ڲģʽʶƵ⣬ǶۼҵһսǶڶĿת ʶой㷺Ӧǰ¡ĸˣ57 һ·洢WTAģ͵תʶ𣮵ʶģʽʱͿ һģʽԸ⣬ǲȫʽԻȨؽжֵȡ 洢ģWTAģͼӶ˴洢ݴԣʵ˶Ŀת ʶѡĴͷΪģ⣬䷽кЧ
⣬Haken1991һSynergetic and CognitionA Top-Down Approac h to Neural NetsЭͬѧ磮Ϊģо һӱۿУǿԣ֪Էģʽγɵģ ԣģʽʶģʽγɣһ²⣺֪ģʽγƿģ ʽʶβʹַʶԾʹģʽ
ֵӵǣٵˣ58һּɵ ģͣ۵ķչҪ壮Ϊ֪ʶ񲻿ϧ ֪ʶʱһץסʵʣһԣˣƼ (TAF)ʱҪһ͵ģʽ⣬˫TAF ơƵѵȣʵ緽ЧԺȷ ԣһЩѧ⣮⣬ģʽʶедʶо ҪۺӦüֵºʹΪ59ݰͳʶ𷽷֪ۺ һ缯ɷ4ͬдַмɣһ ƹԣṩһ90ѧ߶Hopfield һоܸȤʹõһƺͷչ
90ǽ϶عעϵͳĿ⣬ͨ緽 ȡͻĳɹһҪо1990NarendraParthasarathy һƹĶ̬ϵͳȨѧϰ 60ݣɱʾԣǿ³ԣǸһµıʶƷ multilayerrecarrentͳһģԶ̬ϵͳ˶̬BP ߵڷоǣٶΪԻɢʱϵͳЩѧ߶ ѧϰ㷨ٶһĽֵӵǣʱԶ̬ϵ ͳ磬ϵͳֱӦڹ㷺ʵϵͳֵһǣе 61ϵͳϵͳƷһ þ̬ƽ̬Ժһû΢ϵͳĶ̬ ϵͳۺͷӶһϵͳҪĸϿ˵ֿ ƲԾһԺԣ
Miller62ݵ˻Albus1975Сģ͹ؽڿ (CMAC)о˷Զ̬ϵͳ⣬оֲƽ޸ȨС ص㣬ʵʱƷ棬һȱݣ˼ϳƽԷԳıƽ ʱ־ȲҲпܵòӦĵƣַһʼͱMiller 63ݳɹӦû˵ʵʱ̬켣ĸٿУTouretzkyҲɹӦڷ ʱзϣ1992Lane64ݶ˸Ľø߽B-ʹ ƽĹ⻬ԸãȻӣ̷Χ֮ڣƽһߣ 1993Bulsari65Գ˻SigmoidalΪ˷ϵͳ ƽĹõ˽ڵĿһϽƣĸ66 С任ʽ˷Lp£Ҳ֤˵ǰ ƽ1997ҵˣ67ݶCMACԼhashӰ ˾֤⣬Щѧͨѵʵַϵͳ ״̬ƣѧϰ㷨ΪϵͳѧϰṩЧķҪȫ״̬ ϢʵӦôһѣͬʱѧϰٶȻдߣ
Davis1991༭Handbook of Genetic AlgorithmsⱾԸ ƺӦԱкܴİǽĸ1992γɣٽһ۵ķչ 1993굮˹־Evolutionary ComputationIEEEίԱᶨ ʻ飮Ϊһȵо1993YipPaoһִ ָĽģ˻㷨ǽָѡẒ́մﵽ ĿģЩѧ߶˸Ľ磬εнˣ68 ָCauchy滻Gaussianʹ㷨ĿԼ ٶߣǽַɹӦתԲƽ⣮ֵһ 1997Žˣ69ݽMarkovʾŴ㷨ģ˻ķ ƵĻϷĽٶȣͬʱݽֲֵ 
ָļ㸴ԷҪ壮Щѧ߲˼Ȥ19 91Hertz70̽񾭼ۣ1992Anthony71ݳһ Computational Learing Theory 1995ƽ72 ѧϰԼ㸴ԣ˵ⷽ۳ɹԽ࣬ӦõþԽ 
׶ηչ켣иǿѧʺѧ ѧѧʶѧȷһЩش⣬оս Ҳչᣮ90øעѧ֮ ໥ãϲҪĸͷγõĹߣ
3չǰ
չ21ͳڽʮоı£Ϊ۵Ҫǰ 
(1) ܺͻϵʶһ
оһֱǿѧչ壬Ҳǿǰѵս⣮֪ Ψһϵͳи֪ʶѧϰ롢䡢ܣ80ڳˡ 塱ĸ򡰲зֲ(PDP)ֱձسΪ磬ѧϰ Ӧ֯ص㣬ҲҪǿҪܣһоڶ ֪㷨ʹģͺѧϰ㷨ΪӦߣ֪ ֯ͼĸ磬ǿʵһЧ; ĿɱͨоӶٽܿѧķչͨ̽ ܵıԼƣ˹ϵͳֻ򲿷ָ֣ܻʹ иʱͻ¸ΪӡԵĹ
ܵĲͱ仯Ḷ̌Ƕܴ·Ҫ񾭿 ѧѧϰʵͻĹܣõСϵͳԪоѧϰͻ ƵȻģͣϸͷˮƽоΪṩʵ֤磬Եǰ Ҷ߶ȷռ30%Եı 丽γųֵ˶ܷأʹϵͳķ ͬĹϵУԵĿԺܴӦ컷 ܹ߷ԵķӳʵϣԵĿԽԵӰ Խ󣬶̶֪̺ܴϲܾӰ죬һģͺ ͣˣܺͻĹϵӦôӽĽǶȣܻɶ̬չḶ́ķӾãͬʱӦôӻԼԼʷĻԼĽǶȼ 
ɴԪɵķԡӦ֯ϵͳִ񾭿ѧо ĻģͼģӹϢķʽһµĻʹ֮ ԷϢԵĿԡ⡪Ŀġм ջѹķչͽǣѻۡڷ רϵͳۺͻڽ۵˹оڹͬ׷Ŀ£ лĽֻظһǵƲ⣺21ͳܵĻ ʵоµĽչͻƣ
(2) 񾭼ͽ㽫شķչ
㷨Թʮӵо򣬱30߼оǳ ԾChurchKleeneGodelPostTuringѧҶ˿ɼ㷨ľȷ ѧ壬Ժļ㷨ķչӰܴ50ѧMarkovչPostϵͳ80 Ժڼ۷ȡ עĿĳɹγ񾭼ͽ¸ۼҵǿȤǰ ģƽмǶԻTuringɢ۵ĸԵĳ90Ǹ Ľ߽񾭼ͽܻԾµ չڴϵͳϸתڽѧۻǲ̽ ļ㷨ƶܻչ21ཫȫϢ ϢĻȡʹ⣻·Ż⣻ݰȫͱȵȽ µҪЩΪеҪˣ񾭼ͽϢ ϵУڼзӾ޴ã磬Χ ֯ʵ־Ҫн㣮
еһЩģͲûйϱը⣬ֻǰѼתѧϰ㷨 ˵ӴĿһ㲻ӽĹģ˵Щģ ļ(ѧϰ)ʱԪжʵϹϵ̫ȴѧϰԵϵ ֵעǣܲôģдҪǻӦѰ Чм㸴ԡݴԺͼԵļۣ
˼άʽת䣺˼άת˼άԪ綼зԡ ԡǶԡ͹ԺͻԣڼܵĲоԶϵͳ  ԼоһоӦӲ񾭳˷ ģʽ񾭼ŵĺѧȣΪо۷չһ Ҳٵս⣬ֿƷлϾкܴչǰ  ģ㷨ͿϵͳȶԵоΪȵ⣬̿ơоܳΪµ ȵ⣮չ㷨ȶԷƷоٽķչ ѧϰԲ㷨㸴ϵ Щģ͵ļ㸴ԼȷԣӶȷǷ񾭼úע ϢԼǵۺϷо¶̬յԹȸ ¼·磬1994Science־ϣﻯѧAdleman һƪģMolecular Computation of Solutions to Combinatorial Problems  еDNA⣬DNAĴ֮һĳDNAT T-100ȡ˼ϵͻƣһشֵۺͷǶNPȫݼ ׼ϵͳȷҵĹˣDNAзϵӦûǵĹ ע
ԣɢż㡢񾭼ͽ໥ٽӵ˵Ҳյ3 ͳһǻرܲ˵һش⣮Ԥ21ͳ оµĸͷӾ㷽õֵطչӦץס ᣬȡشۺӦóɹ
(3) ṹԪоƬý
ṹоʵԼɹʵӦõǰᣬԽǰᣮ 㷨ͽṹͳһӲĻ壬ӲϽṹģͿΪʶ úͻṩͣδоҪϢ壬ϵͳṹ· Ͳϵȷ֪ʶлйص¸¼磬ᾧ塢 ЧӦ塢߷ӹȣӲʵϣоϵĽṹ֯ʹȻ Ϣ磬Ԫϵͳ֯ϵͳȣĿǰЩѧоӲ Ӧ֮ӳۣһЩµķ
񾭼ҪǾвзֲʽѧϰܣһ߼ܵЧ; ʹĹܻչ˵ĴԵĹƣרҵص㣬ͨ ķӦݣ˼ܣѧ񾭼Ԫ֮Ӳ޴ҽ ǿȿԶ̬ƣΪⲨĴ޽ʧ棬󣬲ʵֳ㣬 һҪ ķչѧ漰ѧԹѧȣЩ ֮ѧϴԣǲý⻥֤֮  ûдţŹķչǰڼҪг١ģĹѧϰ  Լȶ⣬ʹ֮õͻԽչһǲLSIоƬ ԼάVLSIڴоֲ͹⣮δһʮ뵼 оƬԽܼ ӲҪ壬Ԫʵֻŵĸܶȡ߽滥 õ⣬о֮̽Уоģ͵۸ 輯ɿԪľӽ0.01΢ʱӴڽԪĸʽޣ ЧӦǸ߼ɵ·鹤ɿԭ֮һоƬԪ ǷӴСģװܶȿɳӣźŴʽǹµӣ Ҽȣˣи˵ǰŴ񾭼ԪоƬӦ ߿Ƽ򣬸ۺͷµݣͬʱҲһЩµۿ  Ѹٷչһ
4
ҹ863ƻؼƻʵǡƻ͹Ȼѧȣ о˴˲Ŵоٽҹ ȡϵȵλ
ѧƵķΧܹ㷺ֻЩзչǰ;Уṩ ֻУоٳ򣬲Ʋ⣮Ӧ˵ǣоٵ⣬ ɫɫġģɹ۵оڽУ֮21ͿѧչУ ۵ķչվ
עͣȻѧ

߼飺죬У35꣬˶ʿʦоΪۡܿƵȣ
ߵλ人ҵѧԶϵ人430070
ο
1McCulloch W S, Pitts W. A Logical Calculus of the Ideas Immanent in Nervous 
Activity, Bulletin of Mathematical Biophysics, 1943, (5):115133
2N.ά¼룬ۣѧ棬1985
3Von Neumann J. The General and Logical Theory of Automata, Cerebral Mechanisms in
Behavior; The Hixon Sympsium, 1951
4Turing A M. On Computable Numbers, with an Application to the Entscheidungs 
problem, Proc. London Math. Soc., 1936, 2(42):230265; (43):544546 
5Turing A M. Systems of Logic Based on Ordinals, Proc. London Math. Soc., 19 39,2
(45):161228
6Post E L. Finite Combinatory Process-formulation I, J.Symbolic Logic, 1936, 
(1):103105
7Hebb D O. The Organization of Behavior, New York:Wiley, 1949
8Eccles J C. Cholinergic and Inhibitory Synapses in A Pathway from Motor-axon 
Collaterals to Motorneurones, J.Physiol., 1954,(126):524
9Rosenblatt, F., The Perceptron: A Probabilistic Model for Information storage 
and Organization in The Brain. Psychological Review, 1958, (65):386408 
10Widrow B, M E Hoff. Adaptive Switching Circuits. 1960 IRE WESCON con vertion 
record: part 4. computers: Man-machine Systems, Los Angeles:96104
11Rosenblatt F. Principles of Neurody Namics: Perceptrons and The Theory of Brain 
Mechanisms, Spartan, New York, 1962
12Grossberg S. On the Serial Learning of Lists, Math. Biosci., 1969,(4):201253
13Grossberg S. Some Networks that Can Lean, Remenber, and Reproduce any Number of
Compialted Space-time Patterns, , stud. Appl. Math. 1970,(49):135166
14Willshaw D J, Buneman O P. Longuest-higgins, H.C Nature 1969,(222):960
15Nilsson N J. Learning Machines: Foundations of Trainuble Pattern Classifying 
Systems, McGraw-hill, New York,1965
16Holland J H. Genetic Algorithms and the Optimal Allocations of Trials, SIAM 
Journal of Computing, 1973,(2):88105
17Holland J H. Adaptation in Natural and Artificial Systems, Ann Arbor: The 
University of Michigan Press, 1975
18Stein R B, Leung K V, Mangeron D, Oguztoreli M N. Improved Neuronal Model s for 
Studying Neural Networks, Kybernetik, 1974,(15):19
19Heiden U. an der, Existence of Periodic Solutions of a Nerve Equation, Biol . 
Cybern., 1976,(21):3739
20Grossberg S. Aaptive Pattern Classification and Universal Recoding: Part I : 
Parallel Development and Coding of Neural Feature Detectors, Biological Cybern 
etics, 1976,(23):121134
21Werbos P. Beyond Regression: New Tools for Prediction and Analysis in the 
Behavioral Sciences. Ph D Dissertation, Harvard University,1974
22Fukushima K. Neocognitron: A Self-organizing Multilayered Neural Network. 
Biological Cybernetics, 1980,(36):193202
23Amari. S-I., Characteristics of Random Nets of Analog Neuron-like Elements , 
IEEE Transactions on Systems, Man and Cybernetics, 1972a, SMC-2:643657
24Kohonen T. Automatic Formation of Topological Maps in Self-orgnizing Systems: 
Proceedings of the 2nd Scandinavian Conf. on Image Analysis, 1981:214220
25Kohonen T. Self-organizing Formation of Topologically Correct Feature Maps , 
Biol. Cybern. 1982,(43):5969
26Kohonen T. Self-organization and Associative Memory. Berlin: Springer-Verlag,
1984
27Hopfield J J. Neural Networks and Physical Systems with Emergent Collective 
Computational Abilities, Proc. Natl. Acad. Sci., USA, 1982,(79):22542558
28Marr D. Vision, San Francisco: W.H.Freeman, 1982
29Hopfield J J. Neurons with Graded Respone have Collective Computational Pr 
operties Like those of Twostate Neurons, Proc. Natl, Acad. Sci., 1984,(81):3088
3092
30Hopfield J J, Tank D W. Neural Computation of Decisions in Optimization Prob
lems, Biol. Cybern. 1985,(52):141152
31Hopfield J J, Tank D W. Computing with Neural Circuits: A Model, Scien ce, 1986,
(233):625633
32Lee Y C. Physica 22D, 1986,86.,276
33Lapedes A. Physica 22D,1986,247
34Kirkpatrick S, Gellat Jr C D, Veechi M P. Optimization by Simulated An nealing.
Science, 1983,220(4598):671681
35Hinton G E, Sejuowski T J, Ackley D H. Boltzmann Machiues: Cotraint Satisfaction
Networks that Learn, Carnegie-Mellon University, Tech, Report CMU-CS -84-119, 
1984
36Ackley D, Hinton G, Sejnowski T. A Learning Algorithm for Boltzmann machines. 
Cognitive Science, 1985,(9):147169
37Sejnowski T. Higher Order Boltzmann Machines. In: Denker J ed. AIP Conf 
Prpoceeding 151:Neural Networks for Comptuing, New Youk: American Institute of 
Phys ics, 1986,398403
38Poggio T, et al. An analog model of Computation for Ill-posed Problems of Early 
Vision. Artif Intell Lab Memo, 783,MIT, 1984
39Poggio T, et al. Computational Vision and Regularization Theory. Neture (Lond), 
1985,(3):314319
40Hecht-Nielsen, R., The Theory of Backpropagation Neural Network, In Review, 1988
41Ǯѧɭ () ˼άѧϺ磬1986
42Ҧƾţļ㣬Ϣƣ198918(2)31 40
43˹ݣԻ硪(ռ)ѧ磬19 92
44Chua L O, Yang L. Cellular Neural Networks: Theory. IEEE Transactions on 
Circuits and Systems, 1988,(35):12571272
45Chua L O, Yang L. Cellular Neural Networks: Application, IEEE Transactions on 
Circuits and Systems 1988, (35):12731290
46Kosko B. Adaptive Bidirectional Associative Memories, Appl. Opt., 1987,26
(23):49474860
47Kosko B. Constructing an Associative Memory, Byte, 1987,12(10):1 37144
48Kosko B. Bidirectional Associative Memories, IEEE Trans. on Man, Systems and 
Cybernitics, 1988,(18):4959
49Muhlenbein H. Parallel Genetic Algorithms, Population Genetics and Combina 
torial Optimization, in J.D.Schaffer, Ed. Proceedings of the Third International
Conference on Genetic Algorithms (ICGA), 1989,416421
50Muhlenbein H. Limitations of Multi-layer Perceptron Networks-steps Towards 
Genetic Neural Networks, Parallel Computing, 1990,(14):249260
51Aleksander I. The Logic of Connectionist Systems, Neural Computing Archite 
ctures, MIT Press, 1989
52. ϸѧ()(),йѧ(A)199424 (9)902910
24(10)10371046
53Jenkins B K, A R Tanguay. Jr. Optical Architectures for Neural Network 
Implementation, Handbook of Neural Computing and Neural Networks, MIT Press, 
Bosto n, 1995:673677
54McAulay A D, Wang J, Ma C. Optical Heteroassociative Memory Using Spatial Light 
Rebroadcasters. Appl, Opt., 1990,29(14):20672073
55Jewel J L, Lee Y H, Scherer A, et al. Surface Emitting Microlasers for Photonic 
Switching and Interclip Connection. Opt. Eng., 1990,29(3):2 10214
56껣ãǧ. õӷʵֹѧIPAģͣѧѧ,
199717(6)766771
57£ʤӻĸ. 洢 WTAģ͵תʶ𣬹ѧѧ
199717(10)13521356
58٣. һּɵ˹缰Ӧãйѧ(E)
199727(1)5560
59ºΪ. дʶ𷽷ϵͳйѧ(E) 199727(16)556
559
60Narendra K, Parthasarathy K. Identification and Control of Dynamical Systems 
Using Neural Networks. IEEE Trans on Neural Networks, Mar.1990,1(1):427
61У봿. ϵͳϵͳƷԶѧ 
199824(4)463468
62Miller W T. Real-time Application of Neural Networks for Sensor-based Control 
of Robots. with Vision, IEEE, Trans. Syst., Man, Cybern., 1989,(19):825831 
63Miller T W, et al. (Eds), Neural Networks for Control, Cambridge, MA, MIT Press,
1990
64Lane SH, et al. Theory and Development of Higher Order CMAC Neural Ne tworks, 
in Special Issue on Neural Networks in Control Systems, (Antsaklis, P.J. ,Ed.), 
IEEE Control Systems Magazine 1992,12(2):2330
65Bnlsari A. Some Analytical Solutions to the General Approximation Problem for 
Feedforward Neural Networks. Neural Networks 1993,(6):991996
66ĸ. С任˹еӦãӦѧѧ199720 (2)
67ңлع. CMACѧϰоԶѧ199723(4)455461
68εнҦ . ĽĽ㼰ӦãԶѧ199824(2)
262265
69Ž磬ڱ. ˻Ŵ㷨Ҫйѧ(E)1997 
27(2)154164
70Hertz J, et al. Introduction to Theory of Neural Compution. Sant Fee Complexity 
Science Series, 1991:156
71Anthony M, Biggs N. Computational Learning Theory. Combridge University Press, 
1992
72ƽ. ˹ѧϰ㸴ԣѧ199523 (4)6367
73ţ. й׽ѧļ1990
74. 㣬ӿƼѧ磬1993
75ʷֲ. 񾭼㣬ӹҵ磬1993
ոڣ1998-08-12
