ADAPTIVE CONTROL STRATEGIES FOR ROBOTIC MANIPULATORS USING DEEP REINFORCEMENT LEARNING
Keywords:
Adaptive Control, Robotic Manipulator, Deep Reinforcement LearningAbstract
This research examines adaptive control strategies for robotic manipulators using Deep Reinforcement Learning (DRL) through a systematic literature review approach. The main focus of the study is the identification of commonly used DRL algorithms, implementation challenges, and the direction of developing DRL-based adaptive control systems. The study results show that algorithms such as DDPG, SAC, and PPO are effective in addressing the non-linear dynamics and uncertainties of robotic manipulators, both in simulation and real-world environments. However, there are significant challenges such as the need for large training data, the simulation-to-real gap, and the limitations in the interpretability of control policies. The integration of hybrid control strategies, the development of more sample-efficient algorithms, and the application of hierarchical and meta-reinforcement learning have been identified as promising future research directions. This study provides a foundation for the development of more flexible, efficient, and safe robotic control systems to support various industrial applications.
Downloads
References
Abiola, A., Manzano, F. S., & Andújar, J. M. (2023). A Novel Deep Reinforcement Learning (DRL) Algorithm to Apply Artificial Intelligence-Based Maintenance in Electrolysers. Algorithms, 16(12), 541–541. https://doi.org/10.3390/a16120541
Arshad, M., & Bazzocchi, M. (2024). Collision Avoidance and Disturbance Minimization Through Deep Reinforcement Learning Control of a Free-Floating Space Manipulator. 22nd IAA Symposium on Space Debris, Query date: 2025-07-17 15:35:41, 1881–1887. https://doi.org/10.52202/078360-0181
Atti, M., & Yogi, M. K. (2024). Application of Deep Reinforcement Learning (DRL) for Malware Detection. International Journal of Information Technology and Computer Engineering, 43, 23–35. https://doi.org/10.55529/ijitc.43.23.35
Bashendy, M. S., Tantawy, A., & Erradi, A. (2024). Autonomous Response Agent for Cyber Physical System Attacks: A Model-Free Deep Reinforcement Learning Approach (Drl-Irs). Query date: 2025-07-17 15:35:41. https://doi.org/10.2139/ssrn.4716080
Calderon-Cordova, C., & Sarango, R. (2023). A Deep Reinforcement Learning Algorithm for Robotic Manipulation Tasks in Simulated Environments. XXXI Conference on Electrical and Electronic Engineering, Query date: 2025-07-17 15:35:41, 12–12. https://doi.org/10.3390/engproc2023047012
Campos, F. R., Fidencio, A. X., Domingues, J., Pessin, G., & Freitas, G. (2022). Application of Reinforcement Learning to The Orientation and Position Control of A 6 Degrees of Freedom Robotic Manipulator. 2022 Latin American Robotics Symposium (LARS), 2022 Brazilian Symposium on Robotics (SBR), and 2022 Workshop on Robotics in Education (WRE), Query date: 2025-07-17 15:35:41. https://doi.org/10.1109/lars/sbr/wre56824.2022.9995835
Feng, X., He, M., Zhuang, L., Song, Y., & Peng, R. (2023). DRL-G: SFC Deployment Algorithm Based on Deep Reinforcement Learning in Space–Air–Ground Integrated Network. Query date: 2025-07-17 15:35:41. https://doi.org/10.21203/rs.3.rs-3450191/v1
Ge, D. (2024). Research on obstacle avoidance path planning of robotic manipulator based on deep reinforcement learning. Fourth International Conference on Advanced Algorithms and Neural Networks (AANN 2024), Query date: 2025-07-17 15:35:41, 46–46. https://doi.org/10.1117/12.3049584
Heaton, J., & Givigi, S. (2023). A Deep Reinforcement Learning Solution for the Low Level Motion Control of a Robot Manipulator System. 2023 IEEE International Systems Conference (SysCon), Query date: 2025-07-17 15:35:41, 1–7. https://doi.org/10.1109/syscon53073.2023.10131174
Kashkash, M., Saddik, A. E., & Guizani, M. (2024). Optimizing 7-DOF Robot Manipulator Path Using Deep Reinforcement Learning Techniques. 2024 IEEE International Symposium on Robotic and Sensors Environments (ROSE), Query date: 2025-07-17 15:35:41, 1–7. https://doi.org/10.1109/rose62198.2024.10591186
Liu, D., & Li, L. (2023). A traffic light control method based on multi-agent deep reinforcement learning algorithm. Query date: 2025-07-17 15:35:41. https://doi.org/10.21203/rs.3.rs-2626589/v1
Majumder, S., & Sahoo, S. R. (2024). Enhancing 3D Trajectory Tracking of Robotic Manipulator Using Sequential Deep Reinforcement Learning with Disturbance Rejection. 2024 European Control Conference (ECC), Query date: 2025-07-17 15:35:41, 2512–2517. https://doi.org/10.23919/ecc64448.2024.10591259
Malik, A., Lischuk, Y., Henderson, T., & Prazenica, R. (2022). A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator. Robotics, 11(2), 44–44. https://doi.org/10.3390/robotics11020044
Pan, Z., Zhou, J., Fan, Q., Feng, Z., Gao, X., & Su, M. (2023). Robotic Control Mechanism Based on Deep Reinforcement Learning. 2023 2nd International Symposium on Control Engineering and Robotics (ISCER), Query date: 2025-07-17 15:35:41, 70–74. https://doi.org/10.1109/iscer58777.2023.00018
Pei, H., & Lan, Z. (2023). Optimization of High-Speed Train Operation Control Based on Soft Actor-Critic Deep Reinforcement Learning Algorithm. Query date: 2025-07-17 15:35:41. https://doi.org/10.2139/ssrn.4573607
Sehgal, A., Ward, N., La, H. M., Papachristos, C., & Louis, S. (2022). GA+DDPG+HER: Genetic Algorithm-Based Function Optimizer in Deep Reinforcement Learning for Robotic Manipulation Tasks. 2022 Sixth IEEE International Conference on Robotic Computing (IRC), Query date: 2025-07-17 15:35:41. https://doi.org/10.1109/irc55401.2022.00022
Shen, T., Liu, X., Dong, Y., & Yuan, Y. (2022). Energy-Efficient Motion Planning and Control for Robotic Arms via Deep Reinforcement Learning. 2022 34th Chinese Control and Decision Conference (CCDC), Query date: 2025-07-17 15:35:41, 5502–5507. https://doi.org/10.1109/ccdc55256.2022.10033563
Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. https://doi.org/10.1016/j.jbusres.2019.07.039
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management, 14(3), 207–222. https://doi.org/10.1111/1467-8551.00375
Wang, Y., Zhong, S., & Yuan, T. (2024). Grasp Control Method for Robotic Manipulator Based on Federated Reinforcement Learning. 2024 7th International Conference on Advanced Algorithms and Control Engineering (ICAACE), Query date: 2025-07-17 15:35:41, 1513–1519. https://doi.org/10.1109/icaace61206.2024.10549724
Xiao, X. (2023). DVNE-DRL: dynamic virtual network embedding algorithm based on deep reinforcement learning. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-47195-5
Yan, Z., Tan, J., Liang, B., Liu, H., & Yang, J. (2022). Active Fault-Tolerant Control Integrated with Reinforcement Learning Application to Robotic Manipulator. 2022 American Control Conference (ACC), Query date: 2025-07-17 15:35:41, 2656–2662. https://doi.org/10.23919/acc53348.2022.9867641
Yang, Z. Y. Z., Yang, Y. K. Z., & Kong, C.-H. H. Y. (2024). DERLight: A Deep Reinforcement Learning Traffic Light Control Algorithm with Dual Experience Replay. 網際網路技術學刊, 25(1), 79–86. https://doi.org/10.53106/160792642024012501007
Yanover, A., & Choukroun, D. (2024). Line-of-Sight Control of a Spherical Parallel Manipulator via Deep Reinforcement Learning. 2024 IEEE 19th Conference on Industrial Electronics and Applications (ICIEA), Query date: 2025-07-17 15:35:41, 1–6. https://doi.org/10.1109/iciea61579.2024.10665087
Zeng, L., Cen, J., Liu, X., Wu, Y., & Tan, X. (2023). Deep Reinforcement Learning Based HVAC Optimization Control Algorithm Application. 2023 IEEE International Conference on Electrical, Automation and Computer Engineering (ICEACE), Query date: 2025-07-17 15:35:41, 439–446. https://doi.org/10.1109/iceace60673.2023.10442090
Zhang, T., & Mo, H. (2023). Robust Grasp Operation in Clutter for Multi-Objective Robotic Tasks Using Deep Reinforcement Learning. 2023 42nd Chinese Control Conference (CCC), Query date: 2025-07-17 15:35:41, 4751–4755. https://doi.org/10.23919/ccc58697.2023.10240314
Zhong, S., Wang, Y., & Yuan, T. (2024). Grasp Control Method for Robotic Manipulator Based on Federated Reinforcement Learning. Query date: 2025-07-17 15:35:41. https://doi.org/10.2139/ssrn.4687810
