Contrastive multimodal alignment of Sentinel-2 and community radio transcripts for pest-outbreak early warning: a climate-analytics evaluation of false-alarm burden on extension services
Authors/Creators
- 1. University of Lomé
Description
Pest-outbreak early warning in smallholder systems depends on the timely fusion of satellite vegetation signals with farmer-reported observations, yet the two streams are misaligned in vocabulary, cadence and reliability. The framework is formalised as a contrastive objective over paired satellite tiles and transcribed radio segments, with a calibration layer that converts embedding similarity into outbreak probabilities. Because no paired operational corpus is available, the evaluation is analytical: a propagation model translates classifier operating characteristics into extension-worker visit loads, and a threshold-selection rule is derived that bounds expected false alarms per season. False-alarm burden is governed less by classifier accuracy than by the base rate of outbreaks and the cost asymmetry between missed detections and wasted visits. Under plausible smallholder base rates, a system with high recall still generates visit loads that exceed typical extension capacity unless the decision threshold is raised, which in turn suppresses recall.
Notes
Files
article.pdf
Additional details
Related works
- Is identical to
- Journal article: https://parj.africa/ajai_artificial_intelligen_26/article/view/contrastive-multimodal-alignment-of-sentinel-2-and (URL)
References
- Anderson, J. R., & Feder, G. (2007). Agricultural extension. In Handbook of Agricultural Economics (Vol. 3, pp. 2343–2378). Elsevier.
- Ayush, K., Uzkent, B., Meng, C., Tanmay, K., Burke, M., Lobell, D., & Ermon, S. (2021). Geography-aware self-supervised learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, 10181–10190.
- Clevers, J. G. P. W., & Gitelson, A. A. (2013). Remote estimation of crop and grass chlorophyll and nitrogen content using red-edge bands on Sentinel-2 and -3. International Journal of Applied Earth Observation and Geoinformation, 23, 344–351.
- Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8440–8451.
- Davis, J., & Goadrich, M. (2006). The relationship between precision-recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, 233–240.
- Elizalde, B., Deshmukh, S., Al Ismail, M., & Wang, H. (2023). CLAP: Learning audio concepts from natural language supervision. IEEE International Conference on Acoustics, Speech and Signal Processing, 1–5.
- Elkan, C. (2001). The foundations of cost-sensitive learning. Proceedings of the 17th International Joint Conference on Artificial Intelligence, 973–978.
- Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning, 1321–1330.
- López-Más, I., Escobet, T., & Quevedo, J. (2018). Early warning system for pest outbreaks using predictive models. Computers and Electronics in Agriculture, 151, 1–10.
- Mulla, D. J. (2013). Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps. Biosystems Engineering, 114(4), 358–371.
- Oord, A. van den, Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
- Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10(3), 61–74.
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. Proceedings of the 38th International Conference on Machine Learning, 8748–8763.
- Robinson, J., Ching, S., Sra, S., & Jegelka, S. (2021). Contrastive learning with hard negative samples. International Conference on Learning Representations.
- Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10(3), e0118432.
- Whitcraft, A. K., Vermote, E. F., Becker-Reshef, I., & Justice, C. O. (2015). Cloud cover throughout the agricultural growing season: Impacts on passive optical earth observations. Remote Sensing of Environment, 156, 438–447.