AI-Based MITRE ATT&CK Detection System: A Feasibility Study

The rise in cyber threats necessitates automated detection systems that can effectively identify and respond to hostile techniques. This paper presents a feasibility assessment of adopting Large Language Models (LLMs) to enhance cyber- security operations within the MITRE ATT&CK framework. We research how AI can automate Kusto Query Language (KQL) development to better cyber threat detection in Microsoft Sentinel. We start with prompt engineering to improve AI-
generated queries, then compare LLMs to determine the top models. Through successive breakthroughs, we progressed from a natıve prompting method to an advanced Chain of Thought (CoT) prompting technique, enabling AI models to give more contextually accurate and structured KQL queries. We extensively tested
both open-source and closed-source models, evaluating their performance using two separate accuracy scoring formulae. Our results demonstrate that CoT significantly enhances the precision of AI-generated queries, while ChatGPT-4o-mini surpasses other models in generating structured KQL queries. Our technology
leverages real-time MITRE ATT&CK Intelligence and Microsoft Sentinel log analysis for automated threat identification and response in order to minimize human effort and enhance productivity. Our approach applies AI to automate cybersecurity tasks, whereas most other research on LLM-assisted security
analytics remains theoretical and thus fills an important gap between theory and practice.