LLMs in Security Testing and Monitoring: An Initial Study

Cyber threats are becoming increasingly complex, causing
traditional security systems to struggle in keeping up and highlighting
the need for advanced solutions. Large Language Models (LLMs), such
as OpenAI’s ChatGPT and Meta AI’s LLaMA, have shown great potential
to transform cybersecurity workflows with their abilities in natural
language understanding, pattern recognition, and automated reasoning.
These models are particularly promising for tasks like network
monitoring, threat detection, and security alert triage. However, challenges
related to the reliability of outputs, adversarial risks, and ethical
concerns must be addressed. This paper presents a comprehensive survey
of LLM-based approaches for security testing and evaluates three
open-access LLMs, including Mistral-7B, Qwen3-8B, and Llama3.1-8B,
demonstrating their ability to enhance security alert analysis. Our findings
suggest that LLMs can improve alert clarity and usability, making
them more accessible to non-experts while providing valuable insights
for developers.