Abstract
In various applications of autoencoders, an auxiliary subnetwork is used to improve the performance of a neural network with an autoencoder as the key component. For the specific task of anomaly detection, we have observed that in certain cases, when the reconstruction performance reaches a high level, the auxiliary subnetwork becomes ineffective in further improving the autoencoder’s performance. This phenomenon results in oscillation and degradation of the overall system. To address this issue, we propose an adaptive auxiliary training method (AAT) that ensures continuous improvement in the autoencoder’s reconstruction performance throughout the entire training procedure. AAT enhances the monitoring of the autoencoder’s training, enabling adaptive adjustment of the training strategy without a validation set. Additionally, an anomaly detection scheme is devised based on the proposed adaptive auxiliary training method. Experimental results on multiple datasets prove that the proposed methods produce autoencoders with better reconstruction and detection performances comparing to the state-of-the-art (SOTA) methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, K., et al.: History matching of naturally fractured reservoirs using a deep sparse autoencoder. SPE J. 26(04), 1700–1721 (2021)
Lai, Z., Liu, S., Efros, A.A., Wang, X.: Video autoencoder: self-supervised disentanglement of static 3d structure and motion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision 2021, pp. 9730–9740 (2021)
Parmar, G., Li, D., Lee, K., Tu, Z.: Dual contradistinctive generative autoencoder. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 823–832 (2021)
Nguyen, H., Tran, K.P., Thomassey, S., Hamad, M.: Forecasting and anomaly detection approaches using LSTM and LSTM autoencoder techniques with the applications in supply chain management. Int. J. Inf. Manage. 57, 102282 (2021)
Kim, J., Kong, J., Son, J.: Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In: International Conference on Machine Learning PMLR, pp. 5530–5540 (2021)
Shao, H., Xia, M., Wan, J., de Silva, C.W.: Modified stacked autoencoder using adaptive morlet wavelet for intelligent fault diagnosis of rotating machinery. IEEE/ASME Trans. Mechatron. 27(1), 24–33 (2021)
Wang, C., Lucey, S.: Paul: procrustean autoencoder for unsupervised lifting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 434–443 (2021)
Ngairangbam, V.S., Spannowsky, M., Takeuchi, M.: Anomaly detection in high-energy physics using a quantum autoencoder. Phys. Rev. D 105(9), 095004 (2022)
Le, V.-T., Kim, Y.-G.: Attention-based residual autoencoder for video anomaly detection. Appl. Intell. 53(3), 3240–3254 (2023)
Chen, X., et al.: Context autoencoder for self-supervised representation learning University of Trento, pp. 1–16. Trento Italy University of Amsterdam, Amsterdam The Netherlands (2023)
Yan, S., Shao, H., Xiao, Y., Liu, B., Wan, J.: Hybrid robust convolutional autoencoder for unsupervised anomaly detection of machine tools under noises. Robot. Comput.-Integr. Manufactur. 79, 102441 (2023)
Fanai, H., Abbasimehr, H.: A novel combined approach based on deep autoencoder and deep classifiers for credit card fraud detection. In: College of Hydroelectric and Digitalization Engineering, Huazhong University of Science and Technology, Hubei Province 430074 Wuhan, China; Dispatch and Communication Center, Hunan Electric Power Company, Hunan Province C, vol. 217, p. 119562 (2023)
Liang, Y., Liang, W.: ResWCAE: Biometric Pattern Image Denoising Using Residual Wavelet-Conditioned Autoencoder arXiv preprint arXiv:2307.12255 (2023)
Kuzmanovic, M., Hatt, T., Feuerriegel, S.: Deconfounding temporal autoencoder: estimating treatment effects over time using noisy proxies. In: Presented at the Proceedings of Machine Learning for Health, Proceedings of Machine Learning Research (2021). https://proceedings.mlr.press/v158/kuzmanovic21a.html
Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Asian Conference on Computer Vision, pp. 622–637. Springer (2018)
Akçay, S., Atapour-Abarghouei, A., Breckon, T.P.: Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)
Latif, S., Rana, R., Khalifa, S., Jurdak, R., Epps, J., Schuller, B.W.: Multi-task semi-supervised adversarial autoencoding for speech emotion recognition. In: IEEE Transactions on Affective computing (2020)
Yuan, F., Yao, L., Benatallah, B.: Adversarial collaborative auto-encoder for top-n recommendation. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019
Zhong, G., Gao, W., Liu, Y., Yang, Y., Wang, D.-H., Huang, K.: Generative adversarial networks with decoder–encoder output noises. Neural Netw. 127, 19–28 (2020)
Zhang, Z., Chen, Y., Wagner, D.: Seat: Similarity encoder by adversarial training for detecting model extraction attack queries. In: Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, pp. 37–48 (2021)
Blance, A., Spannowsky, M., Waite, P.: Adversarially-trained autoencoders for robust unsupervised new physics searches. J. High Energy Phys. 2019(10), 1–19 (2019)
Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: MVTec AD--A comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592–9600 (2019)
Kimura, D., Chaudhury, S., Narita, M., Munawar, A., Tachibana, R.: Adversarial discriminative attention for robust anomaly detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2172–2181 (2020)
Marafioti, A., Majdak, P., Holighaus, N., Perraudin, N.: GACELA: a generative adversarial context encoder for long audio inpainting of music. IEEE J. Select. Top. Sign. Process. 15(1), 120–131 (2020)
Li, C., et al.: Fusing convolutional generative adversarial encoders for 3D printer fault detection with only normal condition signals. Mech. Syst. Signal Process. 147, 107108 (2021)
Sahu, S., Gupta, R., Sivaraman, G., AbdAlmageed, W., Espy-Wilson, C.: Adversarial auto-encoders for speech based emotion recognition, arXiv preprint arXiv:1806.02146 (2018)
Chouchane, O., et al.: Differentially private adversarial auto-encoder to protect gender in voice biometrics. In: Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security, pp. 127–132 (2023)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)
LeCun, Y.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/ (1998)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
Gong, D., et al.: Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1705–1714 (2019)
Zong, B., et al.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International Conference on Learning Representations (2018)
Zhai, S., Cheng, Y., Lu, W., Zhang, Z.: Deep structured energy based models for anomaly detection. In: presented at the Proceedings of the 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research (2016). https://proceedings.mlr.press/v48/zhai16.html
Scholkopf, B., Williamson, R., Smola, A., Shawe-Taylor, J., Platt, J.: Support vector method for novelty detection. Adv. Neural. Inf. Process. Syst. 12(3), 582–588 (2000)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes, arXiv preprint arXiv:1312.6114 (2013)
Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A.: Conditional image generation with pixelcnn decoders. Adv. Neural Inf. Process. Syst. 4790–4798 (2016)
Acknowledgement
This work is funded by Zhejiang “Lingyan” Research and Development Program (No. 2022C03121). We are grateful for the support of this program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Niu, L., Liao, J., Sha, F., Cheng, Z., Qiu, Y. (2024). An Adaptive Auxiliary Training Method of Autoencoders and Its Application in Anomaly Detection. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1961. Springer, Singapore. https://doi.org/10.1007/978-981-99-8126-7_41
Download citation
DOI: https://doi.org/10.1007/978-981-99-8126-7_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8125-0
Online ISBN: 978-981-99-8126-7
eBook Packages: Computer ScienceComputer Science (R0)