Published January 15, 2026 | Version v2
Model Open

Web Platform for B2-GraftingNet: A Hybrid Deep-Machine Learning Framework with Explainable AI for Automated Grape Leaf Disease Detection

Description

A web + ML pipeline for grape-leaf disease diagnosis with ROI-aware XAI and a local chat assistant (gpt-oss:20b).

This repository contains the runnable demo used for figures and supplementary materials in the paper.

What this project does

  • Classifies grape-leaf images (e.g., black rot) with our B2-GraftingNet pipeline.

  • Explains each prediction with ROI-aware Grad-CAM, LIME, and Occlusion Sensitivity overlays.

  • Summarizes evidence in plain language using a local LLM (gpt-oss:20b via Ollama) and provides an integrated chat panel for follow-up questions.

  • Exposes a REST API so mobile or third-party apps can send an image and receive: the predicted label, explainability overlays, and assistant insights.

Folder layout

 
project-root/ ├─ matlab/ # MATLAB code (computeExplainableAI.m + helpers) ├─ models/ # Trained MATLAB model(s), e.g., B2-GraftingNet.mat ├─ outputs/ # Generated overlays and metrics (auto-created/cleared) ├─ static/ # Front-end assets (logo, background image) ├─ templates/ # Flask HTML template(s), e.g., index.html ├─ uploads/ # Temporary uploaded images (auto-created/cleared) ├─ app.py # Flask server (API + web UI + LLM integration) └─ windows_cnn_matlab.yaml # Example run configuration (env & options)

Key paths (in code)

  • MODEL_PATH = ./models/B2-GraftingNet.mat

  • MATLAB_FOLDER = ./matlab

Uploads go to /uploads/; visuals and metrics are written to /outputs/.

Requirements

1) Python

  • Python 3.9–3.11

  • Packages: flask, werkzeug, requests, (optional) python-dotenv

  • MATLAB Engine for Python (see below)

Create a venv and install:

 
python -m venv .venv # Windows .venv\Scripts\activate # macOS/Linux source .venv/bin/activate pip install flask werkzeug requests python-dotenv

If you keep a requirements.txt:

 
flask==3.* werkzeug==3.* requests>=2.31 python-dotenv>=1.0

2) MATLAB (for XAI + model inference)

  • MATLAB R2023a+ with Deep Learning Toolbox (+ Image Processing Toolbox recommended).

  • MATLAB Engine for Python:

     
    cd "MATLABROOT/extern/engines/python" python -m pip install .

  • Place your trained network at models/B2-GraftingNet.mat.

3) Ollama (local LLM runtime)

  • Install: https://ollama.com/download

  • Pull and serve the model:

     
    ollama pull gpt-oss:20b ollama serve

    Default API: http://127.0.0.1:11434

4) System notes

  • Tested on Windows 11 and Linux/WSL (GPU optional for MATLAB speed-ups).

  • Once pulled, the LLM runs offline (no network required).

Configuration

You can use environment variables (recommended) or copy values from the provided YAML.

Environment variables

  • OLLAMA_BASE (default http://127.0.0.1:11434)

  • LLM_MODEL (default gpt-oss:20b)

  • OLLAMA_OPTIONS (JSON string, e.g. {"temperature":0.4,"top_p":0.9})

Example (PowerShell):

 
$env:OLLAMA_BASE="http://127.0.0.1:11434" $env:LLM_MODEL="gpt-oss:20b" $env:OLLAMA_OPTIONS='{"temperature":0.4,"top_p":0.9}' python app.py

The provided windows_cnn_matlab.yaml mirrors these settings for reproducible runs.

Running the app

  1. Start Ollama and ensure the model is available:

 
ollama serve ollama run gpt-oss:20b # quick sanity check; Ctrl+C to exit

  1. Launch the Flask server:

 
python app.py

  1. Open your browser at: http://127.0.0.1:5000

What happens on upload

  • The server clears /uploads and /outputs.

  • Your file is saved as uploads/image_1.<ext>.

  • MATLAB computeExplainableAI.m:

    • classifies the image,

    • generates Grad-CAM / LIME / Occlusion overlays,

    • writes a *_metrics.json with ROI-aware numeric summaries.

  • The app converts metrics → plain phrases and asks the LLM for a short, non-technical AI Assistant Insight.

  • The UI shows all images + the insight; the right-side chat panel supports follow-ups grounded in the latest analysis.

REST API

POST /upload

Body: multipart/form-data with field image (png/jpg/jpeg/bmp, ≤64 MB)

Response (JSON):

 
{ "predictedLabel": "Grape__Black_rot", "confidenceScore": 0.9956, "inputImageUrl": "/uploads/image_1.jpg?v=...", "labelImageUrl": "/outputs/..._label.png?v=...", "gradcamUrl": "/outputs/..._GRADCAM.png?v=...", "limeUrl": "/outputs/..._LIME.png?v=...", "occlusionUrl": "/outputs/..._OCCLUSION.png?v=...", "llmText": "Verdict ... Where to look ... What to do now ... Caution ...", "chatContext": "Latest image analysis: ..." }

cURL example:

 
curl -F "image=@/path/to/leaf.jpg" http://127.0.0.1:5000/upload

POST /chat

Body (JSON):

 
{"messages":[{"role":"user","content":"How do I separate black rot from sunburn?"}]}

Response (JSON):

 
{"reply":"Short, plain guidance ..."}

The server injects a system style and the latest analysis context so responses stay grounded in your most recent upload.

MATLAB pipeline (summary)

matlab/computeExplainableAI.m:

  • Loads B2-GraftingNet.mat

  • Resizes input and performs classification

  • Computes Grad-CAM, LIME, Occlusion (ROI-aware)

  • Writes overlays (*_GRADCAM.png, *_LIME.png, *_OCCLUSION.png) and metrics (*_metrics.json)

Metrics include coverage, centroid distance, inside/outside contrast, and inter-method agreement within the ROI.
The Flask app never shows raw numbers to end users—it translates them into short phrases and passes those to the LLM.

LLM assistance (summary)

  • Local model: gpt-oss:20b (Ollama)

  • Style: plain, friendly, field-ready; no tables, no percentages, no metric names (IoU/SNR/etc.)

  • Grounding: numeric metrics → phrases (e.g., “highlights tightly cluster over the spot”), injected into the prompt so the LLM explains what to check and what to do next

  • Chat: right-side panel keeps history; each turn includes the latest image context for consistent answers

Using this as a backend only

Skip the UI and:

  • POST /upload with an image → receive URLs to overlays + assistant insight text.

  • POST /chat with your own chat UI → get grower-focused answers grounded in the last analysis.

This makes mobile integration straightforward.

Troubleshooting

LLM unavailable (404 /api/generate or /api/chat)

  • ollama serve is running

  • ollama pull gpt-oss:20b completed

  • OLLAMA_BASE matches your instance (default http://127.0.0.1:11434)

MATLAB Engine not found

  • Install the engine for your active Python; confirm MATLAB license and local install.

Slow or blank overlays

  • Check GPU in MATLAB; reduce NumSamples for LIME or adjust occlusion mask/stride in computeExplainableAI.m.

Large images

  • Uploads ≤ 64 MB (configurable via MAX_CONTENT_LENGTH).

Data & model

  • Kaggle grape-leaf dataset (see manuscript).

  • Place the trained network (B2-GraftingNet.mat) in models/.

  • If you train your own, export to .mat (the script auto-detects the network variable name).

Security & privacy

  • Uploaded images are stored locally and overwritten on each new request (folders are cleared by default).

  • The LLM runs offline, keeping grower data on the device/server.

Citation & acknowledgments

If you use this code or ideas, please cite our Zenodo record:
DOI: 10.5281/zenodo.17353657 (placeholder—replace with your actual Zenodo DOI if different).

Expert validation: Two independent horticulturists reviewed and confirmed the clarity and safety of the AI Assistant’s guidance on representative vineyard scenarios.

 

Quick Start (TL;DR)

 
# 1) Python env python -m venv .venv # Windows .venv\Scripts\activate # macOS/Linux source .venv/bin/activate pip install flask werkzeug requests python-dotenv # 2) MATLAB Engine for Python (see MATLAB docs) # 3) Ollama + model ollama serve ollama pull gpt-oss:20b # 4) Optional env set OLLAMA_BASE=http://127.0.0.1:11434 set LLM_MODEL=gpt-oss:20b # 5) Run python app.py # Open http://127.0.0.1:5000

Contact

Questions or issues about reproducing the figures or running the demo?
Open an issue in your tracker or contact the corresponding author listed in the manuscript.

Chicago (Author–Date)

Article (in submission)

Shah, Syed Baqir Hussain, Farwa Naseer, Syed Adil Hussain Shah, Kashif Razzaq, Tahir Javaid, Qandeel Asghar, Giacomo Di Benedetto, Syed Bilal Hussain, Syed Taimoor Hussain Shah, and Marco Agostino Deriu. 2025. “B2-GraftingNet: End-to-End Grape Leaf Diagnosis with Explainable AI and a Chat-Guided Agronomy Assistant.” Manuscript submitted for publication.

Web platform / code (Zenodo)

Shah, Syed Baqir Hussain, Farwa Naseer, Syed Adil Hussain Shah, Kashif Razzaq, Tahir Javaid, Qandeel Asghar, Giacomo Di Benedetto, Syed Bilal Hussain, Syed Taimoor Hussain Shah, and Marco Agostino Deriu. 2025. Web Platform for B2-GraftingNet: A Hybrid Deep-Machine Learning Framework with Explainable AI for Automated Grape Leaf Disease Detection. Zenodo. https://doi.org/10.5281/zenodo.17353657.

BibTeX

Article (in submission)

 
@unpublished{Shah2025_B2GraftingNet_Article, author = {Shah, Syed Baqir Hussain and Naseer, Farwa and Shah, Syed Adil Hussain and Razzaq, Kashif and Javaid, Tahir and Asghar, Qandeel and Di Benedetto, Giacomo and Hussain, Syed Bilal and Shah, Syed Taimoor Hussain and Deriu, Marco Agostino}, title = {B2-GraftingNet: A Hybrid Deep-Machine Learning Framework with Explainable AI for Automated Grape Leaf Disease Detection}, note = {Manuscript submitted for publication}, year = {2025} }

Web platform / code (Zenodo)

 
@software{Shah2025_B2GraftingNet_WebPlatform, author = {Shah, Syed Baqir Hussain and Naseer, Farwa and Shah, Syed Adil Hussain and Razzaq, Kashif and Javaid, Tahir and Asghar, Qandeel and Di Benedetto, Giacomo and Hussain, Syed Bilal and Shah, Syed Taimoor Hussain and Deriu, Marco Agostino}, title = {Web Platform for B2-GraftingNet: A Hybrid Deep-Machine Learning Framework with Explainable AI for Automated Grape Leaf Disease Detection}, year = {2025}, publisher = {Zenodo}, doi = {10.5281/zenodo.17353657}, url = {https://doi.org/10.5281/zenodo.17353657} }

Files

B2-GraftingNet_Plant_Web_Project_v2.zip

Files (137.4 MB)

Name Size Download all
md5:c5436cc6db1dec83103e6b66f6a77aed
137.4 MB Preview Download