Abstract
Artificial Neural Networks (ANNS) are used successfully in industry and commerce. This is not surprising since neural networks are especially competitive for complex tasks for which insufficient domain-specific knowledge is available. However, interpretation of models induced by ANNS is often extremely difficult. BP-SOM is an relatively novel neural network architecture and learning algorithm which offers possibilities to overcome this limitation. BP-SOM is a combination of a multi-layered feed-forward network (MFN) trained with the back-propagation learning rule (BP), and Kohonen's self-organizing maps (sorts). In earlier reports, it has been shown that BP-SOM improved the generalization performance as compared to that of BP, while at the same time it decreased the number of necessary hidden units without loss of generalization performance. In this paper we demonstrate that BP-SOM trained networks results in uniform and clustered hidden layer representations appropriate for interpretation of the networks functionality.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Andrews, R., Diederich, J., and Tickle, A. B. (1995).A Survey And Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge Based System, 8: 6, 373–389.
Hinton, G. E. (1986). Learning distributed representations of concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society, 1–12. Hillsdale, NJ: Erlbaum.
Kohonen, T. (1989). Self-organisation and Associative Memory. Berlin: Springer Verlag.
Prechelt, L. (1994). Proben1: A set of neural network benchmark problems and benchmarking rules. Technical Report 24/94, Fakultät fur Informatik, Universität Karlsruhe, Germany.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986).Learning internal representations by error propagation. In D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1: Foundations (pp. 318–362). Cambridge, MA: The MIT Press.
Schaffer, C. (1993). Overfitting avoidance as bias. Machine Learning, 10, 153–178.
Setiono, R. and Liu, H. (1997). NeuroLinear: A system for extracting oblique decision rules from neural networks. In M. Van Someren and G. Widmer (Eds.), Proceedings of the Ninth European Conference on Ma chine Learning. Lecture Notes in Computer Science 1224. Berlin: Springer Verlag, 221–233.
Thornton, C. (1995). Measuring the difficulty of specific learning problems. Connection Science, 7, 81–92.
Thrun, S. B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Džeroski, S., Fahlman, S. E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I.,Kreuziger, J., Michalski, R. S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., Van de Velde, W., Wenzel, W., Wnek, J., and Zhang, J. (1991). The MONK's Problems: a performance comparison of different learning algorithms. Technical Report CMU-CS-91-197, Carnegie Mellon University.
Thrun, S. b. (1994). Extracting Provably Correct Rules From Artficial Neural Networks. Technical Report IAI-TR-93-5. Institut fur Informatik III. Universitat Bonn.
Tickle, A. B., Orlowski, M., and Diederich, J. (1994) DEDEC: Decision Detection by Rule Extraction from Neural Networks. QUT NRC.
Weijters, A. (1995). The BP-som architecture and learning rule. Neural Processing Letters, 2, 13–16.
Weijters, A., Van den Bosch, A., Van den Herik, H. J. (1997). Behavioural Aspects of Combining Backpropagation Learning and Self-organizing Maps. Connection Science, 9, 235–252.
Weijters, A., Van den Herik, H. J., Van den Bosch, A., and Postma, E. O. (1997). Avoiding overfitting with BP-SOM. Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, IJCAI'97, San Francisco, Morgan Kaufmann, 1140–1145.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Weijters, T., van den Bosch, A. (1998). Interpretable neural networks with BP-SOM. In: Pasqual del Pobil, A., Mira, J., Ali, M. (eds) Tasks and Methods in Applied Artificial Intelligence. IEA/AIE 1998. Lecture Notes in Computer Science, vol 1416. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-64574-8_442
Download citation
DOI: https://doi.org/10.1007/3-540-64574-8_442
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64574-0
Online ISBN: 978-3-540-69350-5
eBook Packages: Springer Book Archive