Design and Comparative Evaluation of Rule-Based and Machine Learning Classification for Service Health Incident Detection from Diagnostic Logs

Monitoring service health is a critical business function, yet many organizations struggle to communicate system status clearly to non-technical customers. At AgroVision, diagnostic logs are continuously generated to monitor application health, but these logs are highly technical, contain sensitive information, and make it difficult to distinguish actual incidents from routine operational events. This gap between internal diagnostics and customer-facing communication creates confusion and reduces trust.

This thesis addresses three challenges in transforming diagnostic data into customer-facing service health information: identifying and sanitizing sensitive technical details while preserving meaning, classifying diagnostic events into meaningful service incidents, and evaluating the trade-offs between rule-based and machine learning classification.

The proposed solution is an event-driven middleware pipeline that processes diagnostic logs through parsing, validation, normalization, sanitization, and parallel classification paths. The machine learning path evaluated classical models including Support Vector Machine, Logistic Regression, Gradient Boosting, and Random Forest, trained on over 1.4 million historical diagnostic records. The middleware was evaluated using benchmark datasets and validated on 13,752 live production records across AgroVision applications using metrics such as parsing accuracy, precision, recall, F1-score, and consistency.

The research produced a working middleware prototype and dashboard for AgroVision. The findings show that both rule-based and machine learning classification are viable, but their suitability depends on data quality, explainability requirements, and operational overhead.