Computer Vision-Assisted Object Detection and Handling Framework for Robotic Arm Design Using YOLOV5
Abstract In recent years, there has been a surge in scientific research using computer vision and robots for precision agriculture. Productivity has increased significantly, and the need for human labor in agriculture has been dramatically reduced owing to technological and mechanical advancements. However, most current apple identification algorithms cannot distinguish between green and red apples on a diverse agricultural field, obscured by tree branches and other apples. A novel and practical target detection approach for robots, using the YOLOV5 framework is presented, in line with the need to recognize apples automatically. Robotic end effectors have been integrated into a Raspberry Pi 4B computer, where the YOLOV5 model has been trained, tested, and deployed. The image was taken with an 8-megapixel camera that uses the camera serial interface (CSI) protocol. To speed up the model creation process, researchers use a graphical processing computer to label and preprocess test images before utilizing them. Using YOLOV5, a computer vision system-assisted framework aids in the design of robotic arms capable of detecting and manipulating objects. The deployed model has performed very well on both red and green apples, with ROC values of 0.98 and 0.9488, respectively. The developed model has achieved a high F1 score with 91.43 for green apples and 89.95 for red apples. The experimental findings showed that robotics are at the forefront of technological advancement because of the rising need for productivity, eliminating monotonous work, and protecting the operator and the environment. The same discerning can be applied to agricultural robots, which have the potential to improve productivity, safety, and profit margins for farmers while reducing their impact on the environment. The system’s potential could be seen in an assortment of fields, including sophisticated object detection, nuanced manipulation, multi-robot collaboration, and field deployment.
- Referencias
- Cómo citar
- Del mismo autor
- Métricas
“Apple Historical Dataset” Apple Historical Dataset | Kaggle. /datasets/prasoonkottarathil/apple-lifetime-stocks-dataset (accessed Dec. 22, 2022).
Abubeker, K. M., & Baskar, S. (2023). B2-Net: an artificial intelligence powered machine learning framework for the classification of pneumonia in chest x-ray images. Machine Learning: Science And Technology, 4(1), 1-23. https://doi.org/10.1088/2632-2153/acc30f
Chen, Z., Su, R., Wang, Y., Chen, G., Wang, Z., Yin, P., & Wang, J. (2022). Automatic Estimation of Apple Orchard Blooming Levels Using the Improved YOLOv5. Agronomy, 12(10), 2483. https://doi.org/10.3390/agronomy12102483
Cognolato, M., Atzori, M., Gassert, R., & Müller, H. (2021). Improving Robotic Hand Prosthesis Control With Eye Tracking and Computer Vision: A Multimodal Approach Based on the Visuomotor Behavior of Grasping. Frontiers In Artificial Intelligence, 4, 744476. https://doi.org/10.3389/frai.2021.744476
Dewi, T., Risma, P., & Oktarina, Y. (2020). Fruit sorting robot based on color and size for an agricultural product packaging system. Bulletin Of Electrical Engineering And Informatics, 9(4), 1438-1445. https://doi.org/10.11591/eei.v9i4.2353
Hasan, S., Jahan, M. S., & Islam, M. I. (2022). Disease detection of apple leaf with combination of color segmentation and modified DWT. Journal Of King Saud University - Computer And Information Sciences, 34(9), 7212-7224. https://doi.org/10.1016/j.jksuci.2022.07.004
Intisar, M., Khan, M. M., Islam, M. R., & Masud, M. (2021). Computer Vision based robotic arm controlled using interactive GUI. Intelligent Automation And Soft Computing, 27(2), 533-550. https://doi.org/10.32604/iasc.2021.015482
Ji, S-J., Ling, Q., & Han, F. (2023). An improved algorithm for small object detection based on YOLO v4 and multi-scale contextual information. Computers & Electrical Engineering, 105, 108490. https://doi.org/10.1016/j.compeleceng.2022.108490
Junos, M. H., Khairuddin, A. S. M., & Dahari, M. (2022). Automated object detection on aerial images for limited capacity embedded device using a lightweight CNN model. Alexandria Engineering Journal, 61(8), 6023-6041. https://doi.org/10.1016/j.aej.2021.11.027
Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. arXiv. https://doi.org/10.48550/arXiv.1405.0312.
Maroto-Gómez, M., Marqués-Villarroya, S., Castillo, J. C., Castro‐González, Á., & Malfáz, M. (2023). Active learning based on computer vision and human–robot interaction for the user profiling and behavior personalization of an autonomous social robot. Engineering Applications Of Artificial Intelligence, 117, 105631. https://doi.org/10.1016/j.engappai.2022.105631
Reis, D. H. D., Welfer, D., De Souza Leite Cuadros, M. A., & Gamarra, D. F. T. (2019). Mobile Robot Navigation Using an Object Recognition Software with RGBD Images and the YOLO Algorithm. Applied Artificial Intelligence, 33(14), 1290-1305. https://doi.org/10.1080/08839514.2019.1684778
Savio, A., Dos Reis, M. C., Da Mota, F. A. X., Marciano Martinez, M. A., & Auzuir Alexandria, A. R. (2022). New trends on computer vision applied to mobile robot localization. Internet Of Things And Cyber-Physical Systems, 2, 63-69. https://doi.org/10.1016/j.iotcps.2022.05.002
Starke, J., Weiner, P., Crell, M., & Asfour, T. (2022). Semi-autonomous control of prosthetic hands based on multimodal sensing, human grasp demonstration and user intention. Robotics And Autonomous Systems, 154, 104123. https://doi.org/10.1016/j.robot.2022.104123
Wang, D., & He, D. (2021). Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning. Biosystems Engineering, 210, 271-281. https://doi.org/10.1016/j.biosystemseng.2021.08.015
Wang, Q., Cheng, M., Huang, S., Cai, Z., Zhang, J., & Yuan, H. (2022). A deep learning approach incorporating YOLO v5 and attention mechanisms for field real-time detection of the invasive weed Solanum rostratum Dunal seedlings. Computers And Electronics In Agriculture, 199, 107194. https://doi.org/10.1016/j.compag.2022.107194
Xia, R., Li, G., Huang, Z., Meng, H., & Pang, Y. (2023). Bi-path Combination YOLO for Real-time Few-shot Object Detection. Pattern Recognition Letters, 165, 91-97. https://doi.org/10.1016/j.patrec.2022.11.025
Xu, B., Li, W., Liu, D., Zhang, K., Miao, M., Xu, G., & Song, A. (2022). Continuous Hybrid BCI Control for Robotic Arm Using Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking. Mathematics, 10(4), 618. https://doi.org/10.3390/math10040618
Yan, B., Fan, P., Lei, X., Liu, Z., & Yang, F. (2021). A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5. Remote Sensing, 13(9), 1619. https://doi.org/10.3390/rs13091619
Zhang, X., Fu, L., Karkee, M., Whiting, M., & Zhang, Q. (2019). Canopy Segmentation Using ResNet for Mechanical Harvesting of Apples. IFAC-PapersOnLine, 52(30), 300-305. https://doi.org/10.1016/j.ifacol.2019.12.550
Zhao, K., Li, H., Zha, Z., Zhai, M., & Wu, J. (2022). Detection of sub-healthy apples with moldy core using deep-shallow learning for vibro-acoustic multi-domain features. Measurement: Food, 8, 100068. https://doi.org/10.1016/j.meafoo.2022.100068
Abubeker, K. M., & Baskar, S. (2023). B2-Net: an artificial intelligence powered machine learning framework for the classification of pneumonia in chest x-ray images. Machine Learning: Science And Technology, 4(1), 1-23. https://doi.org/10.1088/2632-2153/acc30f
Chen, Z., Su, R., Wang, Y., Chen, G., Wang, Z., Yin, P., & Wang, J. (2022). Automatic Estimation of Apple Orchard Blooming Levels Using the Improved YOLOv5. Agronomy, 12(10), 2483. https://doi.org/10.3390/agronomy12102483
Cognolato, M., Atzori, M., Gassert, R., & Müller, H. (2021). Improving Robotic Hand Prosthesis Control With Eye Tracking and Computer Vision: A Multimodal Approach Based on the Visuomotor Behavior of Grasping. Frontiers In Artificial Intelligence, 4, 744476. https://doi.org/10.3389/frai.2021.744476
Dewi, T., Risma, P., & Oktarina, Y. (2020). Fruit sorting robot based on color and size for an agricultural product packaging system. Bulletin Of Electrical Engineering And Informatics, 9(4), 1438-1445. https://doi.org/10.11591/eei.v9i4.2353
Hasan, S., Jahan, M. S., & Islam, M. I. (2022). Disease detection of apple leaf with combination of color segmentation and modified DWT. Journal Of King Saud University - Computer And Information Sciences, 34(9), 7212-7224. https://doi.org/10.1016/j.jksuci.2022.07.004
Intisar, M., Khan, M. M., Islam, M. R., & Masud, M. (2021). Computer Vision based robotic arm controlled using interactive GUI. Intelligent Automation And Soft Computing, 27(2), 533-550. https://doi.org/10.32604/iasc.2021.015482
Ji, S-J., Ling, Q., & Han, F. (2023). An improved algorithm for small object detection based on YOLO v4 and multi-scale contextual information. Computers & Electrical Engineering, 105, 108490. https://doi.org/10.1016/j.compeleceng.2022.108490
Junos, M. H., Khairuddin, A. S. M., & Dahari, M. (2022). Automated object detection on aerial images for limited capacity embedded device using a lightweight CNN model. Alexandria Engineering Journal, 61(8), 6023-6041. https://doi.org/10.1016/j.aej.2021.11.027
Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. arXiv. https://doi.org/10.48550/arXiv.1405.0312.
Maroto-Gómez, M., Marqués-Villarroya, S., Castillo, J. C., Castro‐González, Á., & Malfáz, M. (2023). Active learning based on computer vision and human–robot interaction for the user profiling and behavior personalization of an autonomous social robot. Engineering Applications Of Artificial Intelligence, 117, 105631. https://doi.org/10.1016/j.engappai.2022.105631
Reis, D. H. D., Welfer, D., De Souza Leite Cuadros, M. A., & Gamarra, D. F. T. (2019). Mobile Robot Navigation Using an Object Recognition Software with RGBD Images and the YOLO Algorithm. Applied Artificial Intelligence, 33(14), 1290-1305. https://doi.org/10.1080/08839514.2019.1684778
Savio, A., Dos Reis, M. C., Da Mota, F. A. X., Marciano Martinez, M. A., & Auzuir Alexandria, A. R. (2022). New trends on computer vision applied to mobile robot localization. Internet Of Things And Cyber-Physical Systems, 2, 63-69. https://doi.org/10.1016/j.iotcps.2022.05.002
Starke, J., Weiner, P., Crell, M., & Asfour, T. (2022). Semi-autonomous control of prosthetic hands based on multimodal sensing, human grasp demonstration and user intention. Robotics And Autonomous Systems, 154, 104123. https://doi.org/10.1016/j.robot.2022.104123
Wang, D., & He, D. (2021). Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning. Biosystems Engineering, 210, 271-281. https://doi.org/10.1016/j.biosystemseng.2021.08.015
Wang, Q., Cheng, M., Huang, S., Cai, Z., Zhang, J., & Yuan, H. (2022). A deep learning approach incorporating YOLO v5 and attention mechanisms for field real-time detection of the invasive weed Solanum rostratum Dunal seedlings. Computers And Electronics In Agriculture, 199, 107194. https://doi.org/10.1016/j.compag.2022.107194
Xia, R., Li, G., Huang, Z., Meng, H., & Pang, Y. (2023). Bi-path Combination YOLO for Real-time Few-shot Object Detection. Pattern Recognition Letters, 165, 91-97. https://doi.org/10.1016/j.patrec.2022.11.025
Xu, B., Li, W., Liu, D., Zhang, K., Miao, M., Xu, G., & Song, A. (2022). Continuous Hybrid BCI Control for Robotic Arm Using Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking. Mathematics, 10(4), 618. https://doi.org/10.3390/math10040618
Yan, B., Fan, P., Lei, X., Liu, Z., & Yang, F. (2021). A Real-Time Apple Targets Detection Method for Picking Robot Based on Improved YOLOv5. Remote Sensing, 13(9), 1619. https://doi.org/10.3390/rs13091619
Zhang, X., Fu, L., Karkee, M., Whiting, M., & Zhang, Q. (2019). Canopy Segmentation Using ResNet for Mechanical Harvesting of Apples. IFAC-PapersOnLine, 52(30), 300-305. https://doi.org/10.1016/j.ifacol.2019.12.550
Zhao, K., Li, H., Zha, Z., Zhai, M., & Wu, J. (2022). Detection of sub-healthy apples with moldy core using deep-shallow learning for vibro-acoustic multi-domain features. Measurement: Food, 8, 100068. https://doi.org/10.1016/j.meafoo.2022.100068
Maideen, A., & Mohanarathinam, A. (2023). Computer Vision-Assisted Object Detection and Handling Framework for Robotic Arm Design Using YOLOV5. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 12(1), e31586. https://doi.org/10.14201/adcaij.31586
Downloads
Download data is not yet available.
+
−