Abstract
Multi-access edge computing (MEC) enables mobile applications, which consists of multiple dependent subtasks, to be executed in full parallelism between edge servers and mobile devices. The idea is to offload some of the subtasks to edge servers while utilizing the directed acyclic graphs (DAGs) based subtask dependency. In some cases involving multiple users running the same application, users can cooperate to improve the application’s performance (e.g., object detection accuracy in connected and autonomous vehicles) by sharing the intermediate results of the subtasks, which is referred to as cooperation gain. However, inter-user cooperation also introduces additional dependencies between the subtasks of different users, which inevitably increases the execution delay of the application with DAG tasks offloading. Therefore, how to jointly optimize execution delay and cooperation gain remains an open question. In this paper, we present an approach for DAG tasks offloading in MEC with inter-user cooperation to optimize the execution delay of the application and the cooperation gain. First, we introduce the concept of application utility, which incorporates both delay and cooperation gain. We formulate the DAG tasks offloading problem with inter-user cooperation as a Markov decision process (MDP) to maximize the total application utility. Next, we propose a quantized soft actor-critic (QSAC) algorithm to solve the formulated problem. Specifically, QSAC generates multiple potential solutions according to the order-preserving quantizing method, increasing the exploration efficiency to avoid falling into local optimal while maintaining the overall stability. Finally, simulation results validate that the proposed algorithm significantly improves the total application utility compared with the other benchmark algorithms.
This work is support in part by National Key R&D Program of China under Grant 2019YFB2102404, and in part by the National Natural Science Foundation of China under Grant 62072330.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Each core can execute only one task at same time.
References
Dai, B., Xu, F., Cao, Y., Xu, Y.: Hybrid sensing data fusion of cooperative perception for autonomous driving with augmented vehicular reality. IEEE Syst. J. 15(1), 1413–1422 (2021)
Du, J., Yu, F.R., Lu, G., Wang, J., Jiang, J., Chu, X.: MEC-assisted immersive VR video streaming over terahertz wireless networks: a deep reinforcement learning approach. IEEE Internet Things J. 7(10), 9517–9529 (2020)
Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2016)
Song, F., Xing, H., Luo, S., Zhan, D., Dai, P., Qu, R.: A multiobjective computation offloading algorithm for mobile-edge computing. IEEE Internet Things J. 7(9), 8780–8799 (2020)
Sahni, Y., Cao, J., Yang, L., Ji, Y.: Multihop offloading of multiple DAG tasks in collaborative edge computing. IEEE Internet Things J. 8(6), 4893–4905 (2021)
Fu, X., Tang, B., Guo, F., Kang, L.: Priority and dependency-based DAG tasks offloading in fog/edge collaborative environment. In: 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 440–445 (2021)
Wang, T.-H., Manivasagam, S., Liang, M., Yang, B., Zeng, W., Urtasun, R.: V2VNet: vehicle-to-vehicle communication for joint perception and prediction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 605–621. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_36
Chen, Q., Ma, X., Tang, S., Guo, J., Yang, Q., Fu, S.: F-cooper: feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds. In: Chen, S., Onishi, R., Ananthanarayanan, G., Li, Q. (eds.) Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, SEC 2019, Arlington, Virginia, USA, 7–9 November 2019, pp. 88–100. ACM (2019)
Yan, J., Bi, S., Zhang, Y.J., Tao, M.: Optimal task offloading and resource allocation in mobile-edge computing with inter-user task dependency. IEEE Trans. Wireless Commun. 19(1), 235–250 (2020)
Chen, Q., Tang, S., Yang, Q., Fu, S.: Cooper: cooperative perception for connected autonomous vehicles based on 3D point clouds. In: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 514–524 (2019)
Fu, F., Kang, Y., Zhang, Z., Yu, F.R., Wu, T.: Soft actor-critic DRL for live transcoding and streaming in vehicular fog-computing-enabled IoV. IEEE Internet Things J. 8(3), 1308–1321 (2021)
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, 10–15 July 2018, vol. 80. Proceedings of Machine Learning Research, pp. 1856–1865. PMLR (2018)
Huang, L., Bi, S., Zhang, Y.A.: Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans. Mob. Comput. 19(11), 2581–2593 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, P., Ge, S., Zhou, X., Zhang, C., Li, K. (2022). Soft Actor-Critic-Based DAG Tasks Offloading in Multi-access Edge Computing with Inter-user Cooperation. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13157. Springer, Cham. https://doi.org/10.1007/978-3-030-95391-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-95391-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95390-4
Online ISBN: 978-3-030-95391-1
eBook Packages: Computer ScienceComputer Science (R0)