In-field apple size and location tracking using machine vision to assist fruit thinning and harvest decision-making

Monitoring of fruit size development has important implications for apple orchard management decision making such as scheduling for fruitlet chemical thinning and allocating resources for harvest. The current method for tracking fruit size development is by tagging a sample of fruits and using calipers or sizing rings to make measurements, which can be labor-intensive and time-consuming. In this study, a stereo vision system was developed which sized fruits on a tree and kept track of their growth during the season by matching fruit in images across time. Neural network models including Faster R-CNN and Mask R-CNN were used for fruit detection and on-tree fruit sizing. Camera pose estimation using feature matching of apples was used for tracking individual fruit growth. The best performance on fruit matching for the ‘Golden Delicious’ variety during the growing season in an apple orchard was observed in September and October; 74% of all detected fruits which were fully visible were matched between the two months. Fruitlets averaging 25 mm in diameter also had a matching accuracy of 73% during two imaging trials performed on the same day for the month of June.