Estimating Compositions and Nutritional Values of Seed Mixes based on Vision Transformers

The cultivation of seed mixtures for local pastures is a traditional mixed cropping techniques of cereals and legumes for producing at a low production cost, a balanced animal feed in energy and protein in livestock systems. By considerably improving the autonomy and safety of agricultural systems, as well as reducing their impact on the environment, it is a type of crop that responds favorably both to the evolution of the European regulations on the use phyto-sanitary products, and the expectations of consumers who wish to increase their consumption of organic products. However, farmers find it difficult to adopt it because cereals and legumes do not ripen synchronously and the harvested seeds are heterogeneous, making it more difficult to assess their nutritional value. Many efforts therefore remain to be made to acquire and aggregate technical and economical references to evaluate to what extent the cultivation of seed mixtures could positively contribute to secure and reduce costs on herd feeding. The work presented in this paper proposes to evaluate recent deep learning techniques that could be transferred to an online or smartphone application to automatically estimate the nutritive value of harvested seed mixes to help farmers better managing the yield and thus engage them to promote and contribute to better knowledge of this type of cultivation. For this purpose, we have built an original image dataset containing 4,749 images of seed mixes, covering 11 seed varieties, with which we have compared 2 types of deep learning models. Our results highlight the potential of this method, and show that the best performing model is a recent state-of-the-art Vision Transformer pre-trained with self-supervision (BeiT). It allows an estimation of the nutritive value of seed mixtures with a coefficient of determination \(R^2\) Score of 0.91, which demonstrates the interest of this type of approach, for its possible use on a large scale.