Abstract
In adversarial games like VR hunting which involves predators and preys, locomotive behaviour of the non-player character (NPC) is crucial. For effective and realistic quadruped locomotion, major technical contributions of this paper are made to inverse kinematics embedded motion control, quadruped locomotion behaviour adaptation and dynamic environment informed reinforcement learning (RL) of the NPC agent. Behaviour of each NPC can be improved from the top-level decision making such as pursuit and escape down to the actual skeletal motion of bones and joints. The new concepts and techniques are illustrated by a specific use case of predator and prey interaction, in which the objective is to create an intelligent locomotive predator to reach its autonomous steering locomotive prey as fast as possible in all the circumstances. Experiments and comparisons are conducted against the Vanilla dynamic target training; and the RL agent of the quadruped displays more realistic limb movements and produces faster locomotion towards the autonomous steering target.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhao, K., Lin, F., Seah, H.S.: Steering autonomous animals in VR hunting. In: SPIE Proceedings of International Workshop on Advanced Image Technology (IWAIT’21), Japan, pp. 5–6 Jan 2021
Zhao, K., Lin, F., Seah, H.S.: Collective intelligence of autonomous animals in VR hunting. In: IEEE VR 2021 Workshop on 3D Content Creation for Simulated Training in Extended Reality (IEEE VR21 TrainingXR), Portugal, 27 Mar–3 Apr 2021
Omar, S., Benatitallah, R., Duvivier, D., Artiba, A.H., Belanger, N., Feyzeau, P.: Path planning: a 2013 survey. In: IEEE International Conference on Industrial Engineering and Systems Management (IEEE IESM 2013) (2013)
De, J., Zhang, X., Lin, F., Cheng, L.: Transduction on directed graphs via absorbing random walks. IEEE Trans. Pattern Anal. Mach. Intell. 40(7), 1770–1784 (2018)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)
Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4(2), 100–107 (1968)
Cui, X., Shi, H.: A*-based pathfinding in modern computer games. Int. J. Comput. Sci. Netw. Secur. 11(1), 125–130 (2011)
LaValle, S.M.: Rapidly-exploring random trees: a new tool for path planning, CiteSeer, pp. 98–11 (1998)
Reynolds, C.W.: Steering behaviors for autonomous characters. Game Dev. Conf. 1999, 763–782 (1999)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Fan, J., Wang, Z., Xie, Y., Yang, Z.: A theoretical analysis of deep Q-learning. In: Learning for Dynamics and Control, PMLR, pp. 486–489 (2020)
Kai, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34(6), 26–38 (2017)
Lou, C., et al.: Dynamic balance measurement and quantitative assessment using wearable plantar-pressure insoles in a pose-sensed virtual environment. Sensors 18(12), 4193 (2018)
Zhongke, W., Lin, F., Seah, H.S., Yun, C.K.: Evaluation of difference bounds for computing rational Bézier curves and surfaces. Comput. Graph. 28(4), 551–558 (2004)
Cai, J., Lin, F., Seah, H.S.: Graphical Simulation of Deformable Models. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-51031-6
Leong, M.C., Prasad, D.K., Lee, Y.T., Lin, F.: Semi-CNN architecture for effective spatio-temporal learning in action recognition. Appl. Sci. 10(2), 557 (2020)
Stepanova, S., Lin, F., Lin, V.C.-L.: A hopfield neural classifier and its FPGA implementation for identification of symmetrically structured DNA motifs. J. VLSI Signal Process. Syst. Signal Image Video Technol. 48(3), 239–254 (2007)
Ma, J., Lin, F., Wesarg, S., Erdt, M.: A novel Bayesian model incorporating deep neural network and statistical shape model for pancreas segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 480–487. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_55
Acknowledgement
This work is partially supported by the MOE AcRF RG93/20 grant by the Ministry of Education, Singapore, as well as the 206-A021006 grant by China-Singapore International Joint Research Institute.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, K., Lin, F., Seah, H.S. (2021). Reinforcement Learning for Quadruped Locomotion. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2021. Lecture Notes in Computer Science(), vol 13002. Springer, Cham. https://doi.org/10.1007/978-3-030-89029-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-89029-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89028-5
Online ISBN: 978-3-030-89029-2
eBook Packages: Computer ScienceComputer Science (R0)