Abstract
Time series data generated by environmental sensors are typically “messy,” with unexpected anomalies that must be corrected prior to extracting useful information. This paper addresses automatic detection of such anomalies and discusses two lines of study for achieving efficient and accurate detection using AI techniques with a focus on peak anomalies. One study uses the classic knowledge-engineering process and the other uses a deep-learning method to mimic how a trained watershed scientist detects anomalies. These two approaches were applied to time series data collected from a research watershed in Vermont, U.S.A., and their performances were assessed with respect to detection accuracy and computational efficiency. The two approaches had different anomaly detection accuracy depending on the peak type. The knowledge engineering approach was readily tunable to achieve competitive or better detection accuracy while computationally far more efficient than the deep learning approach. Results indicate the advantage of using the two approaches in combination, while a more general study involving other watersheds’ time series data would be needed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The watershed name is not mentioned due to a data management policy of the agency that owns the datasets.
- 2.
Note that anomalous peaks are positive instances, and non-anomalous peaks are negative instances in this anomaly detection problem.
References
Cook, A., Mısırl, G., Fan, Z.: Anomaly detection for IoT time-series data: a survey. IEEE Internet Things 7(7), 6481–6494 (2020)
Sgueglia, A., Di Sorbo, A., Visaggio, C., Canfora, G.: A systematic literature review of IoT time series anomaly detection solutions. Futur. Gener. Comput. Syst. 134, 170–186 (2022)
Hill, D.J., Minsker, B.S.: Anomaly detection in streaming environmental sensor data: a data-driven modeling approach. Environ. Model. Softw. 25(9), 1014(2010)
Kim, J.-M., Cho, Y.W., Kim, D.-H.: Anomaly detection of environmental sensor data using recurrent neural network at the edge device. In: Proceedings of the 2020 International Conference on Information and Communication Technology Convergence, pp. 1624–1628. IEEE (2020)
Conde, E.: Environmental Sensor Anomaly Detection Using Learning Machines. MS Thesis. Utah State University (2011)
Hayes, M., Capretz, M.: Contextual anomaly detection in big sensor data. In: Proceedings of the 2014 IEEE International Congress on Big Data, pp. 64–71. Anchorage, AK, USA (2014)
Hill, D.J. and Minsker, B.S.: Automated fault detection for in-situ environmental sensors. In: Proceedings of the 7th International Conference on Hydroinformatics. Nice, France (2006)
Russo, S., Lürig, M., Hao, W., Matthews, B., Villez, K.: Active learning for anomaly detection in environmental data. Environ. Model. Softw. (134), 104869 (2020)
Jones, A.S., Jones, T.L., Horsburgh, J.S.: Toward automating post processing of aquatic sensor data. Environ. Model. Softw. (151), 105364 (2022)
Lai, K., et al.: Tods: an automated time series outlier detection system. In: Proceedings of the AAAI conference on artificial intelligence, vol. 35. no. 18, pp. 16060–16062 (2021)
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)
Cho, H., Fryzlewicz, P.: Multiple-change-point detection for high dimensional time series via sparsified binary segmentation. J. Royal Statist. Soc. 475–507 (2015)
Enikeeva, F., Harchaoui, Z.: High-dimensional change-point detection under sparse alternatives. Ann. Statist. 2051–2079 (2019)
Fearnhead, P., Rigaill, G.: Changepoint detection in the presence of outliers. J. Am. Statist. Assoc. 169–183 (2019)
Fryzlewicz, P.: Wild binary segmentation for multiple change-point detection. Ann. Statist. 2243–2281 (2014)
Tveten, M., Eckley, I.A., Fearnhead, P.: Scalable change-point and anomaly detection in cross-correlated data with an application to condition monitoring. Ann. Appl. Statist. 721–743 (2022)
Pang, G., Shen, C., Cao, L. Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. 1–38 (2021)
Yu, Y., Wan, D., Zhao, Q., Liu, H.: Detecting pattern anomalies in hydrological time series with weighted probabilistic suffix trees. Water 1464 (2020)
Sun, J., Lou, Y., Ye, F.: Research on anomaly pattern detection in hydrological time series. In: 14th Web Information Systems and Applications Conference (WISA), pp. 38–43 IEEE (2017)
Qin, Y., Lou, Y.: Hydrological time series anomaly pattern detection based on isolation forest. In: IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 1706–1710. IEEE (2019)
Lin, Y., Lee, B., Lustgarten, D.: Continuous detection of abnormal heartbeats from ECG using online outlier detection. In: Lossio-Ventura, J., Muñante, D., Alatrista-Salas, H. (eds.) Information Management and Big Data. SIMBig 2018. Communications in Computer and Information Science, vol. 898. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-11680-4_33
Li, H., Boulanger, P.: A survey of heart anomaly detection using ambulatory Electrocardiogram (ECG). Sensors 1461 (2020)
Hong, S., et al.: Holmes: Health online model ensemble serving for deep learning models in intensive care units. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1614–1624. ACM, New York, USA (2020)
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Deep learning for time series classification: a review. Data Min. Knowl. Disc. 33(4), 917–963 (2019). https://doi.org/10.1007/s10618-019-00619-1
Cerqueira, V., Torgo, L, Mozeti ̆c, I.: Evaluating time series forecasting models: an empirical study on performance estimation methods. Mach. Learn. 109, 1997–2028 (2020)
Firebug: Practical hyperparameter optimization: random vs. grid search (2016). https://stats.stackexchange.com/q/209409
Zagorukyo, S., Komodakis, N.: Wide Residual Networks. In: Proceedings of the British Machine Vision Conference, p. 87.1–87.12. BMVA Press (2016)
Kaiguang: Bayesian Changepoint Detection & Time Series Decomposition. MathWorks (2022). https://www.mathworks.com/matlabcentral/fileexchange/72515-bayesian-changepoint-detection-time-series-decomposition
Chatterjee, S., Bopardikar, R., Guerard, M., Thakore, U., Jiang, X.: MOSPAT: AutoML based Model Selection and Parameter Tuning for Time Series Anomaly Detection. arXiv preprint arXiv:2205.11755 (2022)
Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No. EAR 2012123. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government. The work was also supported by the University of Vermont College of Engineering and Mathematical Sciences through the REU program.
The authors would like to thank the US Geological Survey (USGS) for offering the domain expertise that was crucial to identify the peak anomaly types that are of practical importance.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lee, B.S., Kaufmann, J.C., Rizzo, D.M., Haq, I.U. (2023). Peak Anomaly Detection from Environmental Sensor-Generated Watershed Time Series Data. In: Lossio-Ventura, J.A., Valverde-Rebaza, J., Díaz, E., Alatrista-Salas, H. (eds) Information Management and Big Data. SIMBig 2022. Communications in Computer and Information Science, vol 1837. Springer, Cham. https://doi.org/10.1007/978-3-031-35445-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-35445-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35444-1
Online ISBN: 978-3-031-35445-8
eBook Packages: Computer ScienceComputer Science (R0)