Wheat Head Classification in 3D Point Clouds for Fusarium Head blight Detection

Abstract

Deep learning (DL) has become one of the most efficient tools for data processing in computer vision and is a popular technique for tasks such as classification, segmentation, and detection. Although most of these techniques have been applied to data with a structured grid, 3D data such as point clouds have shown proficient results and increased popularity due to the growing availability of acquisition devices. This has led to their application in areas such as robotics, autonomous driving, medicine, agriculture, and more. A point cloud is a set of points defined in a 3D metric space, characterized by its unstructured nature. The unstructuredness of point clouds makes the use of DL for direct processing challenging and 3D object detection has become an active research topic. 3D object detection is an important functional method as it can simultaneously predict surrounding objects' categories, locations, and sizes. In fields like agriculture, this technique offers the potential to analyse various plant attributes, such as plant height, biomass, and the number and size of relevant plant organs.Plant detection and recognition represent a difficult challenge due to the plants' size, posture, shape, illumination, and texture, which vary depending on the varieties and growth stages. One major challenge is presented in wheat plants. As a fundamental source of food, the interest in its analysis has increased. Detection of wheat spikes can help validate spikelet fertility, spike characteristics, and evaluate high-yield wheat cultivars. In this thesis, we created a dataset of 576 point cloud data samples of multiple wheat plants, which we manually labeled for computer vision tasks such as object detection and wheat head classification. Utilizing a 3D neural network model specialized for point clouds, called PointNet, we developed a 3D object detection model to identify and detect wheat heads. This model allowed us to use point clouds directly as input data to preserve the detailed point information. The results demonstrated a test accuracy of 80% in the best model. Finally, a 3D CNN-based classification model was integrated to develop a wheat head classification model for 3D point clouds for Fusarium Head blight (FHB) detection. The classification model was fine-tuned for disease detection to automatically identify wheat infected with FHB from 3D images of wheat heads. The model for FHB detection in wheat spikelets achieved 91% accuracy in a multiple wheat plants test set. Extensive cross-validation experiments were performed to evaluate the performance ability of the model with promising detection results. In addition, the drawbacks of the proposed method were analyzed, and directions for future work are provided.