iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: https://doi.org/10.1007/s10270-023-01084-7
Assessing the usefulness of a visual programming IDE for large-scale automation software | Software and Systems Modeling Skip to main content
Springer Nature Link
Log in

Assessing the usefulness of a visual programming IDE for large-scale automation software

  • Special Section Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Industrial control applications are usually designed by domain experts instead of software engineers. These experts frequently use visual programming languages based on standards such as IEC 61131-3 and IEC 61499. The standards apply model-based engineering concepts to abstract from hardware and low-level communication. Developing industrial control software is challenging due to the fact that control systems are usually unique and need to be maintained for many years. The arising challenges, together with the growing complexity of control software, require very usable model-based development environments for visual programming languages. However, so far only little empirical research exists on the practical usefulness of such environments, i.e., their usability and utility. In this paper, we discuss common control software maintenance tasks and tool capabilities based on existing research and show the realization of these capabilities in the 4diac IDE. We performed a walkthrough of the demonstrated capabilities using the cognitive dimensions of notations framework from the field of human–computer interaction. We then improved the tool and conducted a user study involving ten industrial automation engineers, who used the 4diac IDE in a realistic control software maintenance scenario. Based on lessons learnt from this study, we adapted the 4diac IDE to better handle large graphical models. We evaluated these changes in a reassessment study with automation engineers from seven industrial enterprises. We derive general implications with respect to large-scale applications for developers of IDEs that we deem applicable in the context of (visual) model-based engineering tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://youtu.be/K9iItQBC-ac

  2. https://youtu.be/xishphcgYmc

  3. ELK 0.6.1, https://www.eclipse.org/elk/

  4. https://youtu.be/by02Xu2yrPE

References

  1. Jolak, R., Savary-Leblanc, M., Dalibor, M., Wortmann, A., Hebig, R., Vincur, J., Polasek, I., Le Pallec, X., Gérard, S., Chaudron, M.R.V.: Software engineering whispers: the effect of textual versus graphical software design descriptions on software design communication. Empir. Softw. Eng. 25(6), 4427–4471 (2020)

    Article  Google Scholar 

  2. Meliá, S., Cachero, C., Hermida, J.M., Aparicio, E.: Comparison of a textual versus a graphical notation for the maintainability of MDE domain models: an empirical pilot study. Softw. Qual. J. 24(3), 709–735 (2016)

    Article  Google Scholar 

  3. National Instruments, “Labview 2020,” 2020. www.ni.com/en-us/shop/labview.html (2021). Accessed 23 Dec 2022

  4. The MathWorks, “Simulink R2021a,” 2021. www.mathworks.com/products/simulink.html (2021). Accessed 23 Dec 2022

  5. Harrison, R., Vera, D., Ahmad, B.: Engineering methods and tools for cyber-physical automation systems. Proc. IEEE 104(5), 973–985 (2016)

    Article  Google Scholar 

  6. International Electrotechnical Commission (IEC), “IEC 61131 - Programmable controllers, Part 3: Programming languages,” Geneva (2014)

  7. International Electrotechnical Commission (IEC), TC65/WG6, “IEC 61499-1, Function Blocks - part 1: Architecture: Edition 2.0,” Geneva (2012)

  8. Vogel-Heuser, B., Fay, A., Schaefer, I., Tichy, M.: Evolution of software in automated production systems: challenges and research directions. J. Syst. Softw. 110, 54–84 (2015)

    Article  Google Scholar 

  9. Vogel-Heuser, B., Diedrich, C., Fay, A., Jeschke, S., Kowalewski, S., Wollschlaeger, M., Göhner, P.: Challenges for software engineering in automation. J. Softw. Eng. Appl. 7(5), 440–451 (2014)

    Article  Google Scholar 

  10. Törngren, M., Grogan, P.: How to deal with the complexity of future cyber-physical systems? Designs 2(4), 40 (2018)

    Article  Google Scholar 

  11. Eclipse 4diac. (2020) Eclipse 4diac—the open source environment for distributed industrial automation and control systems. www.eclipse.org/4diac (2021). Accessed 23 Dec 2022

  12. Weber, T., Zoitl, A., Hußmann, H.: Usability of development tools: a case-study. In: 2019 ACM/IEEE 22nd international conference on model driven engineering languages and systems companion (MODELS-C), 228–235. IEEE (2019)

  13. Holzinger, A.: Usability engineering methods for software developers. Commun. ACM 48(1), 71–74 (2005)

    Article  Google Scholar 

  14. Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1994)

    MATH  Google Scholar 

  15. Seaman, C.B.: Qualitative methods in empirical studies of software engineering. IEEE Trans. Softw. Eng. 25(4), 557–572 (1999)

    Article  Google Scholar 

  16. Rabiser, R., Vierhauser, M., Grünbacher, P.: Assessing the usefulness of a requirements monitoring tool: A study involving industrial software engineers. In: 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), 122–131 (2016)

  17. Blackwell, A., Green, T.: CHAPTER 5 - Notational systems–the cognitive dimensions of notations framework. In: Carroll, J.M. (ed.) Interactive Technologies. HCI Models, Theories, and Frameworks. Morgan Kaufmann, pp. 103–133 (2003) https://doi.org/10.1016/B978-155860808-5/50005-8 ( ISBN 9781558608085)

  18. Wiesmayr, B., Zoitl, A., Rabiser, R.: Assessing the usefulness of a visual programming ide for large-scale automation software. In: 2021 ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS). ACM. https://epub.jku.at/obvulioa/content/titleinfo/6603944 (2021)

  19. Holwerda, R., Hermans, F.: A usability analysis of blocks-based programming editors using cognitive dimensions. In: 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 217–225. IEEE (2018)

  20. Obermeier, M., Braun, S., Vogel-Heuser, B.: A model-driven approach on object-oriented PLC programming for manufacturing systems with regard to usability. IEEE Trans. Industr. Inf. 11(3), 790–800 (2015)

    Article  Google Scholar 

  21. Bayrak, G., Ocker, F., Vogel-Heuser, B.: Evaluation of selected control programming languages for process engineers by means of cognitive effectiveness and dimensions. J. Softw. Eng. Appl. 10(5), 457–481 (2017)

    Article  Google Scholar 

  22. Vogel-Heuser, B., Feldmann, S., Folmer, J., Ladiges, J., Fay, A., Lity, S., Tichy, M., Kowal, M., Schaefer, I., Haubeck, C., Lamersdorf, W., Kehrer, T., Getir, S., Ulbrich, M., Klebanov, V., Beckert, B.: Selected challenges of software evolution for automated production systems. In: IEEE 13th International Conference on Industrial Informatics (INDIN) 2015, 314–321 (2015)

  23. Obermeier, M., Braun, S., Sommer, K., Vogel-Heuser, B.: Fundamental aspects concerning the usability evaluation of model-driven object oriented programming approaches in machine and plant automation. In: Marcus A., (ed) Design, User Experience, and Usability. Theory, Methods, Tools and Practice, ser. Lecture Notes in Computer Science, 6770, 497–506. Springer, Berlin Heidelberg (2011)

  24. Legat, C., Folmer, J., Vogel-Heuser, B.: Evolution in industrial plant automation: a case study. In: IECON 2013—39th Annual Conference of the IEEE Industrial Electronics Society, 4386–4391. IEEE (2013)

  25. Fischer, J., Vogel-Heuser, B., Haben, F., Schaefer, I.: Reengineering workflow for planned reuse of IEC 61131-3 legacy software. In: 2020 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE, 1126–1130 (2020)

  26. Hopsu, A., Atmojo, U. D., Vyatkin, V.: On portability of IEC 61499 compliant structures and systems. In: 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE), 1306–1311. IEEE (2019)

  27. Lyu, G., Brennan, R.W.: Towards IEC 61499-based distributed intelligent automation: a literature review. IEEE Trans. Industr. Inf. 17(4), 2295–2306 (2021)

    Article  Google Scholar 

  28. nxtControl GmbH.: Nxt technology IDE. www.nxtcontrol.com/engineering/ (2021). Accessed 23 Dec 2022

  29. Rockwell Automation: ISaGRAF development toolkit. www.isagraf.com (2021). Accessed 23 Dec 2022

  30. Holobloc, Inc.: Function block development kit (FBDK). www.holobloc.com (2021). Accessed 23 Dec 2022

  31. Sorokin, Radimir: (2021) FBME (Function Blocks Modelling Environment). https://github.com/JetBrains/fbme (2021). Accessed 23 Dec 2022

  32. Sorokin, R., Patil, S., Vyatkin, V.: Novel development tool for iec 61499 based on domain-specific languages. In: 14th IFAC Workshop on Intelligent Manufacturing Systems, 433–438. Springer (2022)

  33. Oberlehner, M., Sonnleithner, L., Wiesmayr, B., Zoitl, A., VaSiCS, C.: Catalog of refactoring operations for IEC 61499. in: 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA ), 01–04 (2021)

  34. Sonnleithner, L., Oberlehner, M., Kutsia, E., Zoitl, A., Bácsi, S.: Do you smell it too? towards bad smells in iec 61499 applications. In: 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA ), 1–4 (2021)

  35. Sonnleithner, L., Zoitl, A.: A software measure for IEC 61499 basic function blocks. In: 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 1, 997–1000 (2020)

  36. Zoitl, A., Strasser, T., Ebenhofer, G.: Developing modular reusable IEC 61499 control applications with 4DIAC. In: 2013 11th IEEE International Conference on Industrial Informatics (INDIN), 358–363. IEEE (2013)

  37. Ovsiannikova, P., Vyatkin, V.: Towards user-friendly model checking of iec 61499 systems with counterexample explanation. In: 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA ), 01–04 (2021)

  38. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13(3), 319 (1989)

    Article  Google Scholar 

  39. Ko, A.J., LaToza, T.D., Burnett, M.M.: A practical guide to controlled experiments of software engineering tools with human participants. Empir. Softw. Eng. 20(1), 110–141 (2015)

    Article  Google Scholar 

  40. Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131–164 (2009)

  41. Wiesmayr, B., Zoitl, A., Rabiser, R.: Assessing the Usefulness of a Visual Programming IDE for Large-Scale Automation Software: Documents for Performing a User Study on Maintaining Hierarchical Control Applications with 4diac IDE. Zenodo. https://doi.org/10.5281/ZENODO.4758816 (2021)

  42. Rubel, D., Wren, J., Clayberg, E.: The Eclipse Graphical Editing Framework (GEF). The eclipse series, Addison-Wesley, Boston (2012)

    Google Scholar 

  43. Zoitl, A., Prähofer, H.: Guidelines and patterns for building hierarchical automation solutions in the IEC 61499 modeling language. IEEE Trans. Industr. Inf. 9(4), 2387–2396 (2013)

    Article  Google Scholar 

  44. Fuhrmann, H., von Hanxleden, R.: Taming graphical modeling. In: Petriu, D. C., Rouquette, N., Haugen, Ø. (eds) Model Driven Engineering Languages and Systems, ser. Lecture Notes in Computer Science, 6394, 196–210. Springer (2010)

  45. Cicchetti, A., Ciccozzi, F., Pierantonio, A.: Multi-view approaches for software and system modelling: a systematic literature review. Softw. Syst. Model. 18(6), 3207–3233 (2019)

    Article  Google Scholar 

  46. Balzer, R.: Tolerating inconsistency (software development). In: Proceeding 13th International Conference on Software Engineering, 1991, 158–165. IEEE (1991)

  47. Balzer, R.: Tolerating inconsistency. In: Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001, 665 (2001)

  48. Kritzinger, L.M., Krismayer, T., Rabiser, R., Grünbacher, P.: A user study on the usefulness of visualization support for requirements monitoring. In: Working Conference on Software Visualization (VISSOFT) 2019, 56–66 (2019)

  49. Mackay, W. E.: Triggers and barriers to customizing software. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 153–160 (1991)

  50. Hofstetter, P., Mettier, T.M.: What users want and may need: insights from a survey of users of a life-cycle tool. J. Ind. Ecol. 7(2), 79–101 (2003)

    Article  Google Scholar 

  51. Xu, T., Jin, L., Fan, X., Zhou, Y., Pasupathy, S., Talwadker, R.: Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 307–319 (2015)

  52. Zoitl, A., Lewis, R.W.: Modelling Control Systems Using IEC 61499. IET Control Engineering Series, 2nd edn. IET, London (2014)

    Book  Google Scholar 

  53. Nielsen, J., Landauer, T. K.: A mathematical model of the finding of usability problems. In: Proceedings INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems, ser. CHI ’93. Association for Computing Machinery, New York, 206-213 (1993)

Download references

Acknowledgements

The financial support by the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, and the Christian Doppler Research Association is gratefully acknowledged. We want to thank our industry partner for their continuous support. We particularly thank all participants of the study.

Author information

Authors and Affiliations

  1. LIT CPS Lab, Johannes Kepler University Linz, Linz, Austria

    Bianca Wiesmayr, Alois Zoitl & Rick Rabiser

  2. CDL VaSiCS, Johannes Kepler University Linz, Linz, Austria

    Alois Zoitl & Rick Rabiser

Authors
  1. Bianca Wiesmayr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alois Zoitl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rick Rabiser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bianca Wiesmayr.

Additional information

Communicated by Shiva Nejati and Daniel Varro.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wiesmayr, B., Zoitl, A. & Rabiser, R. Assessing the usefulness of a visual programming IDE for large-scale automation software. Softw Syst Model 22, 1619–1643 (2023). https://doi.org/10.1007/s10270-023-01084-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-023-01084-7

Keywords

Search

Navigation