TY - JOUR
T1 - On-tree apple fruit size estimation using stereo vision with deep learning-based occlusion handling
AU - Mirbod, Omeed
AU - Choi, Daeun
AU - Heinemann, Paul H.
AU - Marini, Richard P.
AU - He, Long
N1 - Funding Information:
This study was supported by the USDA National Institute of Food and Agriculture Multistate Research under Project #FLA-GCR-006262 and Accession #7003555 . The authors would also like to thank Dr Robert Crassweller, Mr Don Smith, and Dr James Schupp for their help on this project.
Publisher Copyright:
© 2022 The Author(s)
PY - 2023/2
Y1 - 2023/2
N2 - Apple (Malus domestica) fruit size plays an integral role in orchard management decision-making, particularly during chemical thinning, fruit quality assessment, and yield prediction. A machine vision system was developed using stereo cameras synchronised to a custom-built LED strobe to perform on-tree sizing of fruit in images with high measurement accuracy. Two deep neural network models (Faster R–CNN and Mask R–CNN) were trained to detect fruit candidates for sizing followed by extrapolation of occluded fruit regions to improve size estimation. The segmented fruit shapes were converted to metric surface areas and diameters using spatial resolutions and depth information from the stereo cameras. Monthly field trials from June to October using the camera system were conducted, measuring fruit diameters ranging from 22 to 82 mm, and compared against ground truth diameters. Diameter estimates had a mean absolute error ranging from 1.1 to 4.2 mm for the five-month trial period, an average error of 4.8% compared to ground truth diameter measurements. Standard deviation errors ranged from 0.7 to 1.9 mm. Using neural network models for intelligent sampling of fruit in images followed by extrapolation of missing regions can be an alternative method of handling fruit occlusion in agricultural imaging and improving sizing accuracy.
AB - Apple (Malus domestica) fruit size plays an integral role in orchard management decision-making, particularly during chemical thinning, fruit quality assessment, and yield prediction. A machine vision system was developed using stereo cameras synchronised to a custom-built LED strobe to perform on-tree sizing of fruit in images with high measurement accuracy. Two deep neural network models (Faster R–CNN and Mask R–CNN) were trained to detect fruit candidates for sizing followed by extrapolation of occluded fruit regions to improve size estimation. The segmented fruit shapes were converted to metric surface areas and diameters using spatial resolutions and depth information from the stereo cameras. Monthly field trials from June to October using the camera system were conducted, measuring fruit diameters ranging from 22 to 82 mm, and compared against ground truth diameters. Diameter estimates had a mean absolute error ranging from 1.1 to 4.2 mm for the five-month trial period, an average error of 4.8% compared to ground truth diameter measurements. Standard deviation errors ranged from 0.7 to 1.9 mm. Using neural network models for intelligent sampling of fruit in images followed by extrapolation of missing regions can be an alternative method of handling fruit occlusion in agricultural imaging and improving sizing accuracy.
UR - http://www.scopus.com/inward/record.url?scp=85145979782&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85145979782&partnerID=8YFLogxK
U2 - 10.1016/j.biosystemseng.2022.12.008
DO - 10.1016/j.biosystemseng.2022.12.008
M3 - Article
AN - SCOPUS:85145979782
SN - 1537-5110
VL - 226
SP - 27
EP - 42
JO - Biosystems Engineering
JF - Biosystems Engineering
ER -