{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T04:40:17Z","timestamp":1711600817895},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T00:00:00Z","timestamp":1613692800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T00:00:00Z","timestamp":1613692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003170","name":"Stiftelsen f\u00f6r Kunskaps- och Kompetensutveckling","doi-asserted-by":"crossref","award":["20170297"],"id":[{"id":"10.13039\/501100003170","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100014791","name":"University of Sk\u00f6vde","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014791","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"published-print":{"date-parts":[[2021,4]]},"abstract":"Abstract<\/jats:title>Basic oxygen steel making is a complex chemical and physical industrial process that reduces a mix of pig iron and recycled scrap into low-carbon steel. Good understanding of the process and the ability to predict how it will evolve requires long operator experience, but this can be augmented with process target prediction systems. Such systems may use machine learning to learn a model of the process based on a long process history, and have an advantage in that they can make use of vastly more process parameters than operators can comprehend. While it has become less of a challenge to build such prediction systems using machine learning algorithms, actual production implementations are rare. The hidden reasoning of complex prediction model and lack of transparency prevents operator trust, even for models that show high accuracy predictions. To express model behaviour and thereby increasing transparency we develop a reinforcement learning (RL) based agent approach, which task is to generate short polynomials that can explain the model of the process from what it has learnt from process data. The RL agent is rewarded on how well it generates polynomials that can predict the process from a smaller subset of the process parameters. Agent training is done with the REINFORCE algorithm, which enables the sampling of multiple concurrently plausible polynomials. Having multiple polynomials, process developers can evaluate several alternative and plausible explanations, as observed in the historic process data. The presented approach gives both a trained generative model and a set of polynomials that can explain the process. The performance of the polynomials is as good as or better than more complex and less interpretable models. Further, the relative simplicity of the resulting polynomials allows good generalisation to fit new instances of data. The best of the resulting polynomials in our evaluation achieves a better$$R^2$$<\/jats:tex-math>R<\/mml:mi>2<\/mml:mn><\/mml:msup><\/mml:math><\/jats:alternatives><\/jats:inline-formula>score on the test set in comparison to the other machine learning models evaluated.<\/jats:p>","DOI":"10.1007\/s42979-021-00488-w","type":"journal-article","created":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T23:19:34Z","timestamp":1613776774000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Using Reinforcement Learning for Generating Polynomial Models to Explain Complex Data"],"prefix":"10.1007","volume":"2","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-2128-7090","authenticated-orcid":false,"given":"Niclas","family":"St\u00e5hl","sequence":"first","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7106-0025","authenticated-orcid":false,"given":"Gunnar","family":"Mathiason","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0127-2767","authenticated-orcid":false,"given":"Dellainey","family":"Alcacoas","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,2,19]]},"reference":[{"key":"488_CR1","doi-asserted-by":"crossref","unstructured":"Bae J, Li Y, St\u00e5hl N, Mathiason G, Kojola N. Using machine learning for robust target prediction in a Basic Oxygen Furnace system. Metall Mater Trans B. 2020;51.","DOI":"10.1007\/s11663-020-01853-5"},{"key":"488_CR2","unstructured":"Bello I, Zoph B, Vasudevan V, Le QV. Neural optimizer search with reinforcement learning. In: Proceedings of the 34th international conference on machine learning, vol 70. 2017. p. 459\u2013468."},{"issue":"10","key":"488_CR3","doi-asserted-by":"publisher","first-page":"1892","DOI":"10.1016\/j.camwa.2013.06.031","volume":"66","author":"SMH Fard","year":"2013","unstructured":"Fard SMH, Hamzeh A, Hashemi S. Using reinforcement learning to find an optimal set of features. Comput Math Appl. 2013;66(10):1892\u2013904.","journal-title":"Comput Math Appl"},{"key":"488_CR4","doi-asserted-by":"crossref","unstructured":"Gao C, Shen M, Wang L. End-point prediction of BOF steelmaking based on wavelet transform based weighted TSVR. In: 2018 37th Chinese control conference (CCC). IEEE. 2018. p. 3200\u20133204.","DOI":"10.23919\/ChiCC.2018.8484194"},{"key":"488_CR5","unstructured":"Graves A. Generating sequences with recurrent neural networks. 2013. arXiv:1308.0850."},{"key":"488_CR6","unstructured":"Jomaa HS, Grabocka J, Schmidt-Thieme L. Hyp-rl: hyperparameter optimization by reinforcement learning. 2019. arXiv:1906.11527."},{"key":"488_CR7","doi-asserted-by":"crossref","unstructured":"Khurana U, Samulowitz H, Turaga D. Feature engineering for predictive modeling using reinforcement learning. In: Thirty-second AAAI conference on artificial intelligence. 2018.","DOI":"10.1609\/aaai.v32i1.11678"},{"issue":"3","key":"488_CR8","doi-asserted-by":"publisher","first-page":"211","DOI":"10.6029\/smartcr.2014.03.007","volume":"4","author":"V Kumar","year":"2014","unstructured":"Kumar V, Minz S. Feature selection: a literature review. SmartCR. 2014;4(3):211\u201329.","journal-title":"SmartCR"},{"key":"488_CR9","unstructured":"Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Advances in neural information processing systems. 2017. p. 4765\u20134774."},{"key":"488_CR10","doi-asserted-by":"crossref","unstructured":"Ogutu JO, Schulz-Streeck T, Piepho HP. Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. In: BMC proceedings. vol.\u00a06. Springer; 2012. p. S10.","DOI":"10.1186\/1753-6561-6-S2-S10"},{"issue":"1","key":"488_CR11","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1186\/s13321-017-0235-x","volume":"9","author":"M Olivecrona","year":"2017","unstructured":"Olivecrona M, Blaschke T, Engkvist O, Chen H. Molecular de-novo design through deep reinforcement learning. J Cheminform. 2017;9(1):48.","journal-title":"J Cheminform"},{"issue":"19","key":"488_CR12","doi-asserted-by":"publisher","first-page":"2229","DOI":"10.1103\/PhysRevLett.59.2229","volume":"59","author":"FJ Pineda","year":"1987","unstructured":"Pineda FJ. Generalization of back-propagation to recurrent neural networks. Phys Rev Lett. 1987;59(19):2229.","journal-title":"Phys Rev Lett"},{"key":"488_CR13","unstructured":"Pi\u00f1ol M, Sappa AD, L\u00f3pez A, Toledo R. Feature selection based on reinforcement learning for object recognition. In: Adaptive learning agent workshop. 2012. p. 4\u20138."},{"key":"488_CR14","doi-asserted-by":"crossref","unstructured":"Rehse JR, Mehdiyev N, Fettke P. Towards explainable process predictions for industry 4.0 in the dfki-smart-lego-factory. KI-K\u00fcnstliche Intelligenz 2019;33(2):181\u2013187.","DOI":"10.1007\/s13218-019-00586-1"},{"key":"488_CR15","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should i trust you?\u201d explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. p. 1135\u20131144.","DOI":"10.1145\/2939672.2939778"},{"issue":"11","key":"488_CR16","doi-asserted-by":"publisher","first-page":"2491","DOI":"10.1016\/j.ijleo.2013.10.094","volume":"125","author":"Y Shao","year":"2014","unstructured":"Shao Y, Zhou M, Chen Y, Zhao Q, Zhao S. BOF endpoint prediction based on the flame radiation by hybrid SVC and SVR modeling. Optik. 2014;125(11):2491\u20136.","journal-title":"Optik"},{"key":"488_CR17","doi-asserted-by":"crossref","unstructured":"Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den\u00a0Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et\u00a0al. Mastering the game of go with deep neural networks and tree search. Nature 2016;529(7587):484.","DOI":"10.1038\/nature16961"},{"issue":"4","key":"488_CR18","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1177\/004912417800600405","volume":"6","author":"JA Stimson","year":"1978","unstructured":"Stimson JA, Carmines EG, Zeller RA. Interpreting polynomial regression. Sociol Methods Res. 1978;6(4):515\u201324.","journal-title":"Sociol Methods Res"},{"issue":"3\u20134","key":"488_CR19","first-page":"229","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn. 1992;8(3\u20134):229\u201356.","journal-title":"Mach Learn"},{"key":"488_CR20","doi-asserted-by":"crossref","unstructured":"Xu L, Li W, Zhang M, Xu S, Li J. A model of basic oxygen furnace (BOF) end-point prediction based on spectrum information of the furnace flame with support vector machine (SVM). Optik. 2011;122(7):594\u201398.","DOI":"10.1016\/j.ijleo.2010.04.018"},{"key":"488_CR21","doi-asserted-by":"crossref","unstructured":"Yu L, Zhang W, Wang J, Yu Y. Seqgan: sequence generative adversarial nets with policy gradient. In: Thirty-first AAAI conference on artificial intelligence. 2017.","DOI":"10.1609\/aaai.v31i1.10804"},{"key":"488_CR22","doi-asserted-by":"crossref","unstructured":"Zoph B, Vasudevan V, Shlens J, Le QV. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. p. 8697\u20138710.","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00488-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s42979-021-00488-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00488-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,18]],"date-time":"2022-12-18T07:49:11Z","timestamp":1671349751000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s42979-021-00488-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,19]]},"references-count":22,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["488"],"URL":"https:\/\/doi.org\/10.1007\/s42979-021-00488-w","relation":{},"ISSN":["2662-995X","2661-8907"],"issn-type":[{"value":"2662-995X","type":"print"},{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,19]]},"assertion":[{"value":"31 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with Ethical Standards"}},{"value":"The authors of this article state that there are no conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"103"}}
