Bridge Crane Monitoring using a 3D LiDAR and Deep Learning
Keywords:
Bridge crane, Collision detection, Convolutional neural network, Deep learning, 3D LiDARAbstract
The use of overhead cranes in warehouses and factories has advantages for handling and transporting bulky and/or heavy loads. But it also involves risks such as collisions with other fixed or mobile elements in the working environment. Different types of sensors have been used for monitoring its operation, mainly artificial vision. In this paper, it is employed a three-dimensional (3D) LiDAR to capture the workspace of a bridge crane. The point clouds generated by this laser sensor are delivered to a convolutional neural network to detect the position of the bridge and its carriage, which allows to locate the hook and the suspended load afterwards. Additionally, the laser scans can also be used to warn the operator of possible collisions with fixed elements of the warehouse. The tests carried out show that the proposed system can be successfully used for monitoring overhead cranes.
Downloads
References
O. Lengerke, J. Niño, M. Suell, and M. Morales, "Diseño y automatizacion de un puente grúa para procesos secuenciales," in 8vo Congreso Iberoamericano de Ingeniería Mecánica, Cusco, 2007, pp. 1-8.
A. Akhtulov, M. Kirasirov, and O. Kirasirov, "The construction of the system of automation of designing of load-lifting cranes of bridge type," Journal of Physics: Conference Series, vol. 1399, no. 4, pp. 1-5, 2019. DOI:10.1088/1742-6596/1399/4/044031.
J. López, "Actualización del sistema de automatización, control y supervisión de una grúa pórtico," Universidad de Oviedo, España, Trabajo fin de máster 2018.
J. Carcedo, "Ingeniería de automatización, control y supervisión de una grúa semi-pórtico," Universidad de Oviedo, España, Trabajo Fin de Máster 2020.
S. Beller and H. Yavuz, "Crane automation and mechanical damping methods," Alexandria Engineering Journal, vol. 60, pp. 3275–3293, 2021. DOI:10.1016/j.aej.2021.01.048.
H. Bao, J. An, M. Zhou, and Q. Kang, "A Data-driven MPC Algorithm for Bridge Cranes," in International Conference on Advanced Mechatronic Systems, Hanoi, 2020, pp. 328-332. DOI:10.1109/ICAMechS49982.2020.9310150.
P. Hyla, "The crane control systems: A survey," in 17th International Conference on Methods & Models in Automation & Robotics (MMAR), Miedzyzdroje, 2012, pp. 505-509. DOI:10.1109/MMAR.2012.6347867.
P. Hyla and J. Szpytko, "The vision technique concept support crane safety exploitation process," Journal of KONBiN, vol. 49, no. 9, pp. 37-47, 2019. DOI:10.2478/jok-2019-0074.
J. Huang, W. Xu, W. Zhao, and H. Yuan, "An improved method for swing measurement based on monocular vision to the payload of overhead crane," Transactions of the Institute of Measurement and Control, vol. 44, no. 1, pp. 1-10, 2020. DOI: 10.1177/0142331220921318.
N. Sun, J. Zhang, T. Yang, Y. Fu, and M. Zhou, "A Flexible Rope Crane Control Method," in IEEE International Conference on Real-time Computing and Robotics, Kandima, 2018, pp. 254-258. DOI:10.1109/RCAR.2018.8621682.
A. Kaneshige, T. Akamatsu, and K. Terashima, "The development of an autonomous overhead traveling crane with real time path planning based on the potential method," in International Conference on Control and Automation (ICCA), Budapest, 2005, pp. 1079-1084. DOI:10.1109/ICCA.2005.1528282.
K. Zhao, Q. Zhou, X. Xiong, and J. Zhao, "The construction method of the digital operation environment for bridge cranes," Mathematical problems in engineering, pp. 1-13, 2021. DOI: 10.1155/2021/5528639.
C. Zhengyan, Z. Zhengwei, D. Qiang, and L. Zheren, "Route planning of intelligent bridge cranes based on an improved artificial potential field method," Journal of Intelligent & Fuzzy Systems, vol. Pre-press., pp. 1-8, 2021. DOI:10.3233/JIFS-189696.
C. Zhimei, L. Min, S. Xuejuan, and Z. Zhicheng, "Obstacle avoidance path planning of bridge crane based on improved RRT algorithm," Journal of system simulation, vol. 33, no. 8, pp. 1832-1838, 2021. DOI: 10.16182/j.issn1004731x.joss.20-0272.
J. Yang, M. Huang, W. Chien, and M. Tsai, "Application of machine vision to collision avoidance control of the overhead crane," in International Conference of Electrical, Automation and Mechanical Engineering, 2015, pp. 361-364. DOI:10.2991/eame-15.2015.100.
A. Kaneshige, S. Nagai, S. Ueki, T. Miyoshi, and K. Terashima, "An algorithm for on-line path planning of autonomous mobile overhead traveling crane based on obstacle information," in IEEE International Conference on Automation Science and Engineering (CASE), Seul, 2012, pp. 914-919. DOI:10.1109/CoASE.2012.6386356.
S. Nagai, A. Kaneshige, and S. Ueki, "Three-dimensional obstacle avoidance online path-planning method for autonomous mobile overhead crane," in 2011 IEEE International Conference on Mechatronics and Automation, Beijing, 2011, pp. 1497-1502. DOI:10.1109/ICMA.2011.5985971.
H. Chen, G. Liu, G. Tian, J. Zhang, and Z. Ji, "Adaptive Safe Distance Prediction Using MPC for Bridge Cranes Considering Anti-Swing," in 2020 Chinese Automation Congress (CAC), Shanghai, 2020, pp. 1914-1919. DOI:10.1109/CAC51589.2020.9326855.
Y. Zhou, H. Guo, L. Ma, Z. Zhang, and M. Skitmore, "Image-based onsite object recognition for automatic crane lifting tasks," Automation in construction, vol. 123, pp. 1-13, 2021. DOI:10.1016/j.autcon.2020.103527.
Q. Xie, H. Wen, and Q. Wang, "Research and Reliability Analysis of Double Bridge Crane Control System," International Journal of Mechatronics and Applied Mechanics, no. 4, pp. 188-195, 2018.
T. Raj, F. H. Hashim, A. B. Huddin, M. F. Ibrahim, and A. Hussain, "A survey on LiDAR scanning mechanisms," Electronics, vol. 9, no. 5, pp. 1-25, 2020. DOI: 10.3390/electronics9050741.
W. J. Zhang, G. Yang, Y. Lin, C. Ji, and M. M. Gupta, "On Definition of Deep Learning," in World Automation Congress, Stevenson, 2018, pp. 1-5. DOI: 10.23919/WAC.2018.8430387.
S. Cohen, "The basics of machine learning: strategies and techniques," in Artificial Intelligence and Deep Learning in Pathology.: Elsevier, 2021, pp. 13-40. DOI:10.1016/B978-0-323-67538-3.00002-6.
L. Soto-Ayala and J. Cantoral-Ceballos, "Automatic blood-cell classification via convolutional neural networks and transfer learning," IEEE Latin America Transactions, vol. 19, no. 12, 2021. DOI: 10.1109/TLA.2021.9480144.
A. Asvadi, L. Garrote, C. Premebida, P. Peixoto, and U. Nunes, "DepthCN: vehicle detection using 3D-LIDAR and ConvNet," in IEEE 20th International Conference on Intelligent Transportation Systems, Yokohama, 2017, pp. 1-6. DOI: 10.1109/ITSC.2017.8317880.
P. Wei, L. Cagle, T. Reza, J. Bal, and J. Gafford, "LiDAR and camera detection fusion in a real-time industrial multi-sensor collision avoidance system," Electronics, vol. 7, no. 0, pp. 1-34, 2018. DOI:10.3390/electronics7000000.
S. Mihai, P. Shah, G. Mapp, H. Nguyen, and R. Trestian, "Towards autonomous driving: A machine learning-based pedestrian detection system using 16-layer LiDAR," in 13th International Conference on Communications, Bucharest, 2020, pp. 271-276. DOI: 10.1109/COMM48946.2020.9142042.
J. Szpytko and P. Hyla, "Material handling devices operation environment 3D-type presentation based on laser scanning systems," Journal of KONES Powertrain and Transport, vol. 17, no. 2, pp. 451-458, 2010.
H. Jeong, H. Hong, G. Park, M. Won, M. Kim, and H. Yu, "Point cloud segmentation of crane Parts using dynamic graph CNN for crane collision avoidance," Journal of Computing Science and Engineering, vol. 13, no. 3, pp. 99-106, 2019. DOI: 10.5626/JCSE.2019.13.3.99.
C. R. Qi, L. Yi, H. Su, and L. J. Guibas, "PointNet++: Deep hierarchical feature learning on point sets in a metric space," in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 2017, pp. 5105–5114. DOI:10.5555/3295222.3295263.
J. Qiao, Y. Lv, C. Cao, Z. Wang, and A. Li, "Multivariate deep learning classification of Alzheimer’s disease based on hierarchical partner matching independent component analysis," Frontiers in Aging Neuroscience, vol. 10, pp. 1-12, 2018. DOI:10.3389/fnagi.2018.00417.
