Autonomous Mobile Robot Navigation Optimization in Dynamic Warehouse Environments Using Reinforcement Learning and Sensor Fusion Techniques

Authors

  • Yogiek Indra Kurniawan Universitas Jenderal Soedirman
  • Siti Shofiah Politeknik Keselamatan Transportasi Jalan
  • Rosalina Yani Widiastuti Sekolah Tinggi Ilmu Komputer Yos Sudarso
  • Teguh Arifianto Politeknik Perkeretaapian Indonesia Madiun
  • Ribut Julianto Universitas Indonesia Mandiri

DOI:

https://doi.org/10.61132/ijiime.v1i2.395

Keywords:

Autonomous Mobile Robot, Path Planning, Reinforcement Learning, Sensor Fusion, Warehouse Automation

Abstract

Background: The rapid growth of warehouse automation and autonomous mobile robots has increased the need for adaptive navigation systems capable of operating safely and efficiently in dynamic industrial environments. Classical path planning algorithms such as A* and RRT perform well in structured settings but exhibit limitations when handling moving obstacles and environmental uncertainty. Objective: This study aims to develop and evaluate a reinforcement learning based navigation framework integrated with sensor fusion to improve path efficiency, collision avoidance, and robustness in dynamic warehouse scenarios. Method: An experimental research design was implemented combining high-fidelity simulation and real-world warehouse prototype testing. Deep Q-Network and Proximal Policy Optimization models were developed and trained using multi-sensor inputs from LiDAR, camera, and inertial measurement units. Performance was evaluated using path efficiency, collision rate, computational cost, and robustness metrics, with benchmarking against classical algorithms. Results: The results demonstrate that the Proximal Policy Optimization model achieved the highest path efficiency and lowest collision rate while maintaining stable computational performance under dynamic conditions. Reinforcement learning models significantly outperformed classical planners in adaptability and robustness, confirming their suitability for scalable industrial warehouse automation.

References

Ai, C., Qi, Z., Zheng, L., Geng, D., Feng, Z., & Sun, X. (2021). Research on mapping method based on data fusion of LiDAR and depth camera. Proceedings of the 2021 4th International Conference on Advanced Electronic Materials, Computers and Software Engineering. https://doi.org/10.1109/AEMCSE51986.2021.00082

Cadete, T., Pinto, V. H., Lima, J., Goncalves, G., & Costa, P. (2024). Dynamic AMR navigation: Simulation with trajectory prediction of moving obstacles. 2024 7th Iberian Robotics Conference (ROBOT 2024). https://doi.org/10.1109/ROBOT61475.2024.10797420

Chewu, C. C. E., & Manoj Kumar, V. (2018). Autonomous navigation of a mobile robot in dynamic indoor environments using SLAM and reinforcement learning. IOP Conference Series: Materials Science and Engineering, 402(1), 12022. https://doi.org/10.1088/1757-899X/402/1/012022

Choi, J., Kim, N., & Hong, Y. (2023). Unsupervised Legendre-Galerkin neural network for solving partial differential equations. IEEE Access, 11, 23433-23446. https://doi.org/10.1109/ACCESS.2023.3244681

Das, D., Adhikary, N., & Chaudhury, S. (2022). Sensor fusion in autonomous vehicle using LiDAR and camera sensor. IEEE Region 10 Humanitarian Technology Conference. https://doi.org/10.1109/R10-HTC54060.2022.9929588

Divya Vani, V., & others. (2024). Digging deeper: The role of big data analytics in geotechnical investigations. E3S Web of Conferences, 529, 4012. https://doi.org/10.1051/e3sconf/202452904012

Dixit, P., Nargundkar, A., Suyal, P., & Patil, R. (2023). Intelligent warehouse automation using robotic system. In Lecture Notes in Electrical Engineering (Vol. 959, pp. 435-443). https://doi.org/10.1007/978-981-19-6581-4_34

Dumonteil, G., Manfredi, G., Devy, M., Confetti, A., & Sidobre, D. (2015). Reactive planning on a collaborative robot for industrial applications. ICINCO 2015-12th International Conference on Informatics in Control, Automation and Robotics, 2, 450-457. https://doi.org/10.5220/0005575804500457

Ellithy, K., Salah, M., Fahim, I. S., & Shalaby, R. (2024). AGV and Industry 4.0 in warehouses: A comprehensive analysis of existing literature and an innovative framework for flexible automation. The International Journal of Advanced Manufacturing Technology, 134(1–2), 15-38. https://doi.org/10.1007/s00170-024-14127-0

Fahmy, T. A., & Maged, S. A. (2021). Teaching quadruped to walk using fault adaptive deep reinforcement learning algorithm. MIUCC 2021. https://doi.org/10.1109/MIUCC52538.2021.9447643

Gradu, M. (2021). The benefits of advanced 3D LiDAR for autonomous mobile robots. https://doi.org/10.4271/2021-01-1015

Hanh, L. D., & Cong, V. D. (2023). Path following and avoiding obstacle for mobile robot under dynamic environments using reinforcement learning. Journal of Robotics and Control, 4(2), 157-164. https://doi.org/10.18196/jrc.v4i2.17368

Huo, Y., & Liang, Y. (2022). Offline reinforcement learning application in robotic manipulation with a COG method case. ACM International Conference Proceeding Series. https://doi.org/10.1145/3522749.3523075

Jiang, H., & Ke, H. (2024). Research progress in multi-sensor data fusion algorithms. ACM International Conference Proceeding Series, 547-552. https://doi.org/10.1145/3703187.3703279

Khetani, N., Sharma, A., & Kulkarni, A. (2023). Real-time logistics and warehouse integration via networks of the future and dynamic digital twins. 2023 IEEE Engineering Informatics (EI 2023). https://doi.org/10.1109/IEEECONF58110.2023.10520525

Loganathan, A., & Ahmad, N. S. (2023). A systematic review on recent advances in autonomous mobile robot navigation. Engineering Science and Technology, an International Journal, 40, 101343. https://doi.org/10.1016/j.jestch.2023.101343

Lonklang, A., & Botzheim, J. (2022). Improved rapidly exploring random tree with bacterial mutation and node deletion for offline path planning of mobile robot. Electronics, 11(9), 1459. https://doi.org/10.3390/electronics11091459

Mac, T. T., Copot, C., Tran, D. T., & De Keyser, R. (2016). Heuristic approaches in robot path planning: A survey. Robotics and Autonomous Systems, 86, 13-28. https://doi.org/10.1016/j.robot.2016.08.001

Nam, D. V, Danh, P. T., Park, C. H., & Kim, G.-W. (2024). Fusion consistency for industrial robot navigation: An integrated SLAM framework with multiple 2D LiDAR-visual-inertial sensors. Computers and Electrical Engineering, 120, 109607. https://doi.org/10.1016/j.compeleceng.2024.109607

Nomanfar, P., & Notash, L. (2023). Brief review of reinforcement learning control for cable-driven parallel robots. In Mechanisms and Machine Science (Vol. 132, pp. 161-172). https://doi.org/10.1007/978-3-031-32322-5_13

Raina, N., Chuetor, S., Charoenkool, P., Jiradechakorn, T., Sereenonchai, C., Phojaroen, J., Boonmee, R., Pathak, A. K., & Singh, H. M. (2024). Opportunities and challenges in the production of biofuels from waste biomass. In Waste Valorization for Bioenergy and Bioproducts: Biofuels, Biogas, and Value-Added Products (pp. 23-43). https://doi.org/10.1016/B978-0-443-19171-8.00006-7

Raj A., K. A. S. V. R. S. S. A. K., & Singh, T. (2023). Applications of genetic algorithm with integrated machine learning. In Proceedings of the 3rd International Conference on Innovative Practices in Technology and Management. https://doi.org/10.1109/ICIPTM57143.2023.10118328

Sapan, G. N., Stanikzai, A. N., Sanjar, S., & Anwari, G. (2022). International Journal of Social Science Research and Review, 5(1), 113-128. https://doi.org/10.47814/ijssrr.v6i11.642

Sari, A., & Butun, I. (2021). A case study of decision support system and warehouse management system integration. In Decision Support Systems and Industrial IoT in Smart Grid, Factories, and Cities (pp. 111-138). https://doi.org/10.4018/978-1-7998-7468-3.ch006

Sheikh, M. S., Ali, A., Baig, I., Bhanodia, P. K., Rathore, N. P. S., Khamparia, A., & Al-Turjman, F. (2024). Harnessing logistic industries using autonomous carebot for smart surveillance, protection and security. In Advances in Science, Technology and Innovation (pp. 191-200). https://doi.org/10.1007/978-3-031-63103-0_20

Sun, L. (2024). Digital transformation in the manufacturing industry: The impact of Industry 4.0 and the case of Tesla. In Exploring the Financial Landscape in the Digital Age - Proceedings of ICFMDE 2023 (pp. 697-703). https://doi.org/10.1201/9781003508816-100

Tian, S., Ouyang, Y., & Wei, C. (2024). Collision avoidance approach with heuristic correction policy for mobile robot navigation in dynamic environments. CAAI Transactions on Intelligent Systems, 19(6), 1492-1502. https://doi.org/10.11992/tis.202304056

Wallace, R. L., Cai, Z., Zhang, H., & Guo, C. (2024). Numerical investigations into the comparison of hydrogen and gas mixtures storage within salt caverns. Energy, 311, 133369. https://doi.org/10.1016/j.energy.2024.133369

Wang, Y., Li, X., Zhang, J., Li, S., Xu, Z., & Zhou, X. (2021). Review of wheeled mobile robot collision avoidance under unknown environment. Science Progress, 104(3). https://doi.org/10.1177/00368504211037771

Xu, Y. (2024). The comparison of advanced algorithms for pathfinding robots. AIP Conference Proceedings, 3194(1), 20012. https://doi.org/10.1063/5.0223914

Yu, S. (2024). Comparison of typical traditional path planning algorithms in a two-dimensional environment. AIP Conference Proceedings, 3194(1), 50019. https://doi.org/10.1063/5.0222884

Zhang, X., Gao, S., Yao, C., Chu, Y., & Peng, S. (2023). Reinforcement learning and its application in robot task planning: A survey. Pattern Recognition and Artificial Intelligence, 36(10), 902-917. https://doi.org/10.16451/j.cnki.issn1003-6059.202310004

Zhen, L., & Li, H. (2022). A literature review of smart warehouse operations management. Frontiers of Engineering Management, 9(1), 31-55. https://doi.org/10.1007/s42524-021-0178-9

Zhu, J., Ortiz, J., & Sun, Y. (2024). Decoupled deep reinforcement learning with sensor fusion and imitation learning for autonomous driving optimization. 2024 6th International Conference on Artificial Intelligence and Computer Applications (ICAICA), 306-310. https://doi.org/10.1109/ICAICA63239.2024.10823066

Zou, Y., Liu, F., Qu, J., Jing, H., Kuang, B., Wang, G., & Li, H. H. (2022). Overview of multi-sensor fusion in autonomous vehicles. MEMAT 2022 Conference Proceedings.

Żuchowski, W. (2022). The smart warehouse trend: Actual level of technology availability. LogForum, 18(2), 227-235. https://doi.org/10.17270/J.LOG.2022.702

Downloads

Published

2024-05-31

How to Cite

Yogiek Indra Kurniawan, Siti Shofiah, Rosalina Yani Widiastuti, Teguh Arifianto, & Ribut Julianto. (2024). Autonomous Mobile Robot Navigation Optimization in Dynamic Warehouse Environments Using Reinforcement Learning and Sensor Fusion Techniques . International Journal of Industrial Innovation and Mechanical Engineering, 1(2), 15–28. https://doi.org/10.61132/ijiime.v1i2.395