о뷢չ
JOURNAL OF COMPUTER RESEARCH AND DEVELOPMENT
1999ꡡ3610ڡVol.36No.101999



ṹѧϰ֪ʶ󾫷
񿭡һ 
ժҪ֪ʶ֪ʶȡбزٵĲ.е֪ʶ󾫵KBANN(knowledge based artificial neural network)Ҫѵʱܸı˽ṹ.һֻṹѧϰ֪ʶ󾫷ȽһתΪʼ磬Ȼѵͽṹѧϰ㷨ѵʼ磬ȡ󾫵Ĺ֪ʶ.˽ṹĸıͨѵʱûڶ̬ڵɾĽṹѧϰ㷨ʵֵ.ʵ÷Ч.
ؼʡ֪ʶ󾫣˹磬ṹѧϰȡ
ͼšTP18
A KNOWLEDGE BASE REFINEMENT METHOD BASED
ON STRUCTURAL LEARNING OF NEURAL NETWORKS
LIU Zhen-Kai, GUI Zhong-Hua*, and CAI Qing*
(School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an 710049)
(Northwestern University of Technology, Xi'an 710072)
AbstractKnowledge base refinement is necessary for knowledge acquisition in building expert systems. The KBANN (knowledge based artificial neural network) for knowledge base refinement has been proposed in literatures. The key limitation of KBANN is that there is no mechanism for changing the topology of the network. In this paper, a novel approach to knowledge base refinement based upon structural learning of neural networks is presented. In this approach, a set of rules are mapped into a neural network (i.e., the initial neural network), and then this reformulated knowledge is refined using structural learning (i.e., the initial neural network is trained by structural learning algorithm and a set of training examples). Finally, the refined rules are extracted from the trained neural network. To accomplish the topology change of the initial neural network, structural learning algorithms based on dynamic node creation and network pruning are used in this approach. Many simulation experiments have proved the effect of this approach.
Key wordsknowledge base refinement, artificial neural networks, structural learning, rule extraction
1
֪ʶȡרϵͳƿ.֪ʶȡҪ輰һģʽѧּ⣬ֽбȽ3ֽ£
(1) Buchananڹרϵͳһģ1һ֪ʶȡģʽҪǣ ʶ׶Σ ׶Σ ʽ׶Σ ʵֽ׶Σ Խ׶Σ ʼ֪ʶģ͵޸Ľ׶.
(2) Kidd˸ѧһ֪ʶȡģʽ2ݣΪ3׶Σ ֪ʶṹʶ𣻢 ϸ֪ʶĳȡ ֪ʶĵԺ.
(3) Ginsberg˰֪ʶȡΪ׶Σ3ݣ ʼ֪ʶȡ׶Σ ʼ֪ʶ󾫽׶.
֪ʶ֪ʶȡزٵĲ.һ˵õĳʼ֪ʶⳣЩ⣬֪ʶȫ֪ʶ֮䲻һ¡е֪ʶȷȣҪԡ޸ĺͲ.ʵ֤ʼ֪ʶ󾫺רϵͳܣ֪ʶϵͳSEEK2ϷʪרϵͳEXPERT֪ʶ󾫺ȷ21.2%3.֪ʶܵרϵͳߵļ.
ǽ˹֪ʶ󾫣ȡһЩչ.1990꣬TowellShavlik֪ʶ󾫵Ļ֪ʶ˹(knowledge based artificial neural networkKBANN)4ݣͨʵ֤KBANN֪ʶ󾫷ȴϵͳҪ.KBANNҲһЩȱ45ݣҪKBANNѧϰʱܸı˽ṹȫĹµĹ.1993OptizShavlikһKBANNĽѧϰ㷨6ݣѵڼ÷Żָͺ˵ζ̬½ڵ㣬ںδ½ڵһ⣬Ҫһо.ͬ꣬FuKBANNƵKBCNN(knowledge based conceptual neural network)֪ʶ󾫣7ݣָѵʱ˹ָ½ڵ.
⣬һֻṹѧϰ֪ʶ󾫷.
2ṹѧϰ֪ʶ󾫷
2.1֪ʶ
ָ֪ʶ£
֪(1) ʼ֪ʶ(ָ)
(2) ר֤.
⣺֪֤ʼ֪ʶ⣬޸ġɾͲ䣬ʹӹ֪ʶﵽԤڵ.
2.2ṹѧϰ֪ʶ󾫷
KBANNƣṹѧϰ֪ʶ󾫷ͼͼ1ʾͼеĳʼ򼯼ʼ֪ʶ⡢ѵר֤.3ɣ
(1) תʼתΪʼ磻
(2) 󾫣ѵѧϰ㷨ѵʼ磻
(3) ȡ󾫵Ĺ֪ʶ.ѵʱȨֵֵѧϰ(Ӧڹ޸)ҿ˶̬ڵ(ӦڹĲ)ɾ(Ӧڹɾ).


ͼ1ṹѧϰ֪ʶ󾫷ͼ
2.3ʼתΪ
ʼתΪ磬ӦϵΪ սۣڵ㣻 ֵ֧ʵڵ㣻 мۣڵ㣻 ϵȨֵֵ.
ϵԽ󾫵Ĺ֪ʶתΪʼ.ǣЩ(1)ν(2)ɻ·.
ʼķкܶ࣬ͬķͬ˽ṹͬģĳʼ.ĲһTowellKBANNʹõġתΪ㷨Ƶ㷨.
1. д. ʹÿȡʽʾΪһǰĹ.жҾͬĽۣЩֱдΪһԭĽΪۣһ½Ϊǰ᣻һԴ½ΪۣԭǰΪǰ.ĿǱӦֲ.ͼ2һдʵ֪ʶеһΪ(BC)(DEF)A򽫴˹дΪ򣬼BCADEFAȻǷֱдΪAA, BCAAA, DEFA.


ͼ2дʵ
2. תΪ. ӳϵ߼ϵANDORNOTĹתУһԪһһӦĳʼ.ЩΪȡʽȡʽһѱдΪȡʽ޻·ޱԪ.ǰ۵ϵתΪжӦȨֵ.
3. мڵ. Ϊʹиڵ㴦ȷĲΣҪʵһЩ½ڵ.
4. . ԭûйϵ²ڵ֮ȨֵΪӣʹеκһмڵսڵÿһ²ڵ.
5. ȨֵͽڵŶеȨֵֵһС.
裬յõһȫ.
2.4ѵʼ
KBANNKBCNNBPѧϰ㷨BP㷨ֻܶȨֵѧϰܸı˽ṹ.֪ʶʱڳʼ򼯿ܲƻһЩĹ³ʼȱٽڵһЩ.˱ʹ˽ṹк仯ӽڵ㡢Լɾڵ㡢ӵ.ֻͨṹѧϰʵ˽ṹĸı.
Ľṹѧϰ㷨̬ڵɾ.
(1) ̬ڵ
ǰ֪ʼ˽ṹǸݳʼ򼯹ȫ.һ˵ʼ򼯲ƣɴ˹Ϊȱڵ㣬˾BP㷨ѵһ.Ϊ⣬ѵж̬ڵ.˵BP㷨ѵֱһѧϰƽ̨ѵʱСʱÿһһĽڵ(磬ǰڵ10%һ)¼ڵڲڵȫнСȨֵҪԽϵͣѧϰʱµڵҪӲʼӰ.ظֱﵽ涨ʲ.
Ȼ̬ڵпʹ˶ĽڵӣбҪɾĽڵӣͬҲɾʼеĴ.
(2) ɾ
Ŀǰṹѧϰɾ㷨8ݣе㷨ֻɾӣе㷨ֻɾڵ㣬е㷨ͬʱɾڵ.ǲIshikawaṹѧϰ㷨9(structural learning with forgettingSLF)ɾڵҲɾӣ3ɣѧϰLF(learning with forgetting)ڵѧϰLHUC(learning with hidden units clarification)ѡѧϰLSF(learning with selective forgetting).SLFĻ˼ľMSEӳͷʵLF, LHUCLSF.㷨3ɣ
 LFõһǼṹ
 LHUCʹڵΪȫ(ӽ1)ȫ(ӽ0)
 ͬʱLSFLHUCõõѧϰЧ.
SLF㷨ɾ˶ڵ㡢ӣʹڵӽ10Ϊһȡ˻.Ϊڵǽӽ10ȡǺѵģһЩȡ㷨ӦĴ10.
2.5ȡ
ͨ㷨ѵõʵ󾫵磬֪ʶֲʾṹȨֵУһʽ.ֻͨȡ򣬲ʹ֮Ϊͽܵʽ.
TowellShavlikһִKBANNȡ󾫹ķ11ݣNofM.ַȡĹʽ£
If N of the following M antecedents are true then 
ֹʵʽ֪ʶһֱʾʽֱڹרϵͳ֪ʶ.ǲFuKT12ݣ÷ȡĹ֪ʶ㷨£
ÿڵYWWֱΪȨֵļϺиȨֵļ.
(1) ĳȨֵ֮ͼϳĳָȨֵκȨֵ֮;ֵ¹

(2) ĳָȨֵ֮ͼϳĳȨֵκȨֵ֮;Сֵ¹

УֱͨȨֵ,YڵYڵǰһĽڵ.
3ʵ
ñ֪ʶ󾫷ǽʵ.һʵʼΪ
R1:BZAR2:EFGBR3:CġDB
R4:YXZR5:HIYR6:GJX
дõͼ3ʾͨӡŶȲյõһȫ.繲4㣬8ڵ㣬CDEFGHIJֱ42ڵ㣬мB1B2YXֻһڵսA.


ͼ3תΪ
ѵ256ʼ򼯶63ó.BP㷨ѵһƽ̨ʱֱ21ڵ㣬Ȼѵ.SLF㷨ѵ磬ɾڵ2ӵĶڵ㲢ɾȫӣյõṹͼ4ʾ.KT㷨ȡ󾫺Ĺ֪ʶ£


ͼ4󾫺
R1:BZAR2:XYZR3:EFB
R4:CġDBR5:CDBR6:GHY
R7: IġJX
Щȫ256óȷ.Կ÷޸һЩĹ򣬶һһЩ©Ĺ.
4ᡡ
KBANNľһֻṹѧϰ֪ʶ󾫷.÷KBANNҪǣ ѵʼ˽ṹѧϰ㷨̬ڵɾ KT㷨ȡ󾫵Ĺ֪ʶʽͽܣֱڹרϵͳ֪ʶ.ַӦڹʵ߽һо.
õȻѧ(Ŀ69483005).
߼飺񿭣У19679ʿоΪ˹硢רϵͳCADͷ󹤳.
һŮ196911ʿооΪ졢FMSCIMS.
࣬У193410ڣʿʦоΪCAD/CAMCADCAGDͿӻ.
ߵλͨѧеѧԺ710049
ҵѧʮϵ710072
ο
1Buchanan B G et al. Constructing an expert system. In: Hayes-Roth F, Waterman D A, Lenat D B eds. Building Expert Systems, Chapt 5. Reading, MA: Addison-Wesley, 1983, 127168
2Kidd A, Welbank S M. Knowledge acquisition in expert systems. Berkshire Pergamon Infortech Ltd, State of Art Rept: 1984, 7180
3Ginsberg A, Weiss S M, Politakis P. Automatic knowledge base refinement for classification system. Artificial Intelligence, 1988, 35(1): 197226
4Towell G G, Shavlik J W, Noordewier M O. Refinement of approximate domain theories by knowledge based neural networks. In: Proc of AAAI-90, Menlo Park: AAAI Press, 1990. 861866
5Towell G G, Shavlik J W. Knowledge based artificial neural networks. Artificial Intelligence, 1994, 70(1/2): 119165
6Optiz D W, Shavlik J W. Heuristically expanding knowledge-based neural network. In: Proc of IJCAI-93, San Mateo, 1993. 13601365
7Fu L. Knowledge-based connectionism for revising domain theories. IEEE Trans on Systems, Man, Cybernetics, 1993, 23(1): 173182
8Reed R. Pruning algorithmA survey. IEEE Trans on Neural Networks, 1993, 4(5): 740747
9Ishikawa M. Structural learning with forgetting. Neural Networks, 1996, 9(3): 509521
10Yoon B, Lacher R C. Extracting rules by destructive learning. In: Proc IEEE ICNN'94, New York: IEEE Press, 1994. 17661771
11Towell G G, Shavlik J M, Extracting refined rules from knowledge based neural networks. Mach Learn, 1993, 13(1): 71101
12Fu L. Rule generation from neural networks. IEEE Trans on Systems, Man, Cybernetics, 1994, 24(8): 11141124 
ԭյڣ1998-10-06
޸ĸյڣ1999-05-17.
