Abstract
When growing the software infrastructure for a large-scale scientific project (namely ALeRCE, Automatic Learning for the Rapid Classification of Events), we observed an “internal API hell” phenomenon in which numerous and various API issues coexist and are inextricably interwoven with each other. Driven by this observation, we conducted a set of investigations to help both understand and deal with this complicated and frustrating situation. Through individual interviews and group discussions, our investigation reveals two root causes of the “internal API hell” in ALeRCE, namely (1) an internal API explosion and (2) an increased “churn” of development teams. Given the nature of the system and the software project, each root cause is inherent and unavoidable. To demonstrate our ongoing work on tackling that “hell”, we discuss five API issues and their corresponding solutions, i.e., (1) using a multi-view catalog to help discover suitable APIs, (2) using a publish-subscribe channel to assist API versioning management and negotiation, (3) improving the quality of API adoption through example-driven delivery, (4) using operation serialisation to facilitate API development debugging and migration, and (5) enhancing the usability of long and sophisticated machine learning APIs by employing a graphical user interface for API instantiation. We also briefly consider the threats to validity of our project-specific study. On the other hand, we argue that the root causes and issues are likely to recur for other similar systems and projects. Thus, we urge collaborative efforts on addressing this emerging type of “hell” in software development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
There can be APIs used both internally and externally, e.g., https://api.alerce.online/ztf/v1 for accessing Alerts with the corrected magnitudes, object statistics and object data.
- 3.
The ALeRCE pipeline: http://alerce.science/alerce-pipeline/.
- 4.
A Forecast API uses a particular machine learning technique to make predictions on future supernova light curve brightness. Light curve is the variation of brightness of a star as time goes by. The prediction here shows how the light curve would behave in the future based on the existing measurements.
- 5.
The snippets of some serialised operations are exemplified at https://doi.org/10.5281/zenodo.5914452.
- 6.
- 7.
References
Adzic, G.: Specification by Example: How Successful Teams Deliver the Right Software. Manning Publications, Shelter Island (2011)
Bajaj, G.: PowerPoint version hell, 04 March 2009. https://www.indezine.com/products/powerpoint/ppversionhell.html. Accessed 3 Aug 2022
Bogart, C., Kästner, C., Herbsleb, J., Thung, F.: How to break an API: cost negotiation and community values in three software ecosystems. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2016), pp. 109–120. ACM Press, Seattle, 13–18 November 2016
Brooks, F.P., Bullet, N.S.: Essence and accidents of software engineering. IEEE Comput. 20(4), 10–19 (1987)
Deinum, M., Serneels, K., Yates, C., Ladd, S., Vanfleteren, C.: Configuring a spring development environment. In: Pro Spring MVC: With Web Flow, chap. 1, pp. 1–23. Apress, Berkeley (2012)
Dick, S., Volmar, D.: DLL Hell: Software dependencies, failure, and the maintenance of Microsoft Windows. IEEE Ann. Hist. Comput. 40(4), 28–51 (2018)
Eisenbach, S., Sadler, C., Jurisic, V.: Feeling the way through DLL Hell. In: Proceedings of the First International Workshop on Unanticipated Software Evolution (USE 2002) Co-located with ECOOP 2002, Malaga, Spain, pp. 1–11, 11 June 2002
Esch-Laurent, P.: Versioning hell, 19 February 2021. https://paul.af/versioning-hell. Accessed 3 Aug 2022
Fan, G., Wang, C., Wu, R., Xiao, X., Shi, Q., Zhang, C.: Escaping dependency hell: finding build dependency errors with the unified dependency graph. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2020), pp. 463–474. ACM Press, Los Angeles, 18–22 July 2020
Förster, F., Cabrera-Vives, G., Castillo-Navarrete, E., et al.: The automatic learning for the rapid classification of events ALeRCE alert broker. Astronom. J. 161(5) (2021). Article no. 242
Garcia, J., Kouroshfar, E., Ghorbani, N., Malek, S.: Forecasting architectural decay from evolutionary history. IEEE Trans. Softw. Eng. (2022). Early access
Ginsbach, P., Remmelg, T., Steuwer, M., Bodin, B., Dubach, C., O’Boyle, M.F.P.: Automatic matching of legacy code to heterogeneous APIs: an idiomatic approach. In: Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2018), pp. 139–153. ACM Press, Williamsburg, 24–28 March 2018
Goodyear, J.: Welcome to multiple version hell, 26 March 2004. https://visualstudiomagazine.com/articles/2004/05/01/welcome-to-multiple-version-hell.aspx. Accessed 3 Aug 2022
Lamothe, M., Shang, W.: When APIs are intentionally bypassed: an exploratory study of API workarounds. In: Proceedings of the 42nd IEEE/ACM International Conference on Software Engineering (ICSE 2020), pp. 912–924. ACM Press, Seoul, 23–29 May 2020
Markan, Z.: Android versioning hell in the CI-land, 6 May 2014. https://markan.me/android-versioning-hell/. Accessed 3 Aug 2022
Mellor, S., Clark, A., Futagami, T.: Model-driven development - guest editor’s introduction. IEEE Softw. 20(5), 14–18 (2003)
Nesteruk, D.: Design Patterns in .NET Core 3, 2nd edn. APress (2021)
Nevatech: API discovery and description (2022). https://nevatech.com/api-management/api-discovery-description. Accessed 3 Aug 2022
Ofosu-Amaah, K.: Version hell revisited, 01 July 2020. https://koranteng.blogspot.com/2020/07/version-hell-revisited.html. Accessed 3 Aug 2022
Sabir, F., Gueheneuc, Y.G., Palma, F., Moha, N., Rasool, G., Akhtar, H.: A mixed-method approach to recommend corrections and correct REST antipatterns. IEEE Trans. Softw. Eng. (2022). Early access
Seo, H., Sadowski, C., Elbaum, S., Aftandilian, E., Bowdidge, R.: Programmers’ build errors: a case study (at Google). In: Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), pp. 724–734. ACM Press, Hyderabad, 31 May–7 June 2014
Shmerlin, Y., Hadar, I., Kliger, D., Makabee, H.: To document or not to document? An exploratory study on developers’ motivation to document code. In: Persson, A., Stirna, J. (eds.) CAiSE 2015. LNBIP, vol. 215, pp. 100–106. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19243-7_10
Stoica, I.: Systems and ML at RISELab, 13 July 2020. https://www.usenix.org/system/files/hotcloud20_hotstorage20_slides_stoica.pdf. Keynote speech at the 12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 2020)
Tanabe, Y., Aotani, T., Masuhara, H.: A context-oriented programming approach to dependency hell. In: Proceedings of the 10th International Workshop on Context-Oriented Programming: Advanced Modularity for Run-time Composition (COP 2018), pp. 8–14. ACM Press, Amsterdam, 16 July 2018
Thomas, D.: The API field of dreams - too much stuff! It’s time to reduce and simplify APIs! J. Object Technol. 5(6), 23–27 (2006)
VanSlyke, T.: API hell: the rise of application frameworks and the fall of sanity (2021). http://www.teejayvanslyke.com/api-hell-the-rise-of-application-frameworks-and-the-fall-of-sanity.html. Accessed 3 Aug 2022
de Villamil, F.: Always broken, inconsistent and non versioned, welcome to API hell, 22 August 2014. https://thoughts.t37.net/always-broken-inconsistent-and-non-versioned-welcome-to-api-hell-a26103b31081. Accessed on 3 Aug 2022
Wan, C., Liu, S., Hoffmann, H., Maire, M., Lu, S.: Are machine learning cloud APIs used correctly? In: Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE 2021), pp. 125–137. IEEE Press, Madrid, Spain, 22–30 May 2021
Zibran, M.F., Eishita, F.Z., Roy, C.K.: Useful, but usable? Factors affecting the usability of APIs. In: Proceedings of the 18th Working Conference on Reverse Engineering (WCRE 2011), pp. 151–155. IEEE Computer Society, Limerick, 17–20 October 2011
Acknowledgement
We want to thank the ALeRCE Engineering Team for their hard work and effort on developing and maintaining all the services, Alberto Moya, Javier Arredondo, Esteban Reyes, Ignacio Reyes, Camilo Valenzuela, Ernesto Castillo, Daniela Ruz and Diego Mellis. ALeRCE and this paper would not exist without them.
We acknowledge support from CONICYT/ANID through the grants FONDECYT Initiation Nž 11191130 (G.C.V.) and Nž 11180905 (Z.L.); BASAL Center of Mathematical Modelling (AFB-170001, ACE210010 and FB210005) and FONDECYT Regular Nž 1200710 (F.F). This work has been partially funded by ANID -Millennium Science Initiative Program - ICN12_009 awarded to the Millennium Institute of Astrophysics (MAS). Powered@NLHPC: This research was partially supported by the supercomputing infrastructure of the NLHPC (ECM-02). This work has been possible thanks to the use of AWS credits managed by the NLHPC.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cabrera-Vives, G., Li, Z., Rainer, A., Athanasopoulos, D., Rodríguez-Mancini, D., Förster, F. (2022). Managing the Root Causes of “Internal API Hell”: An Experience Report. In: Taibi, D., Kuhrmann, M., Mikkonen, T., Klünder, J., Abrahamsson, P. (eds) Product-Focused Software Process Improvement. PROFES 2022. Lecture Notes in Computer Science, vol 13709. Springer, Cham. https://doi.org/10.1007/978-3-031-21388-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-21388-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21387-8
Online ISBN: 978-3-031-21388-5
eBook Packages: Computer ScienceComputer Science (R0)