As cybersecurity operations scale, more teams become involved, making the timely assignment of incoming tasks increasingly important. A key challenge is ensuring that each request reaches the right team quickly and consistently.
At CERT-EU, this challenge is very concrete. Analysts triage hundreds of tickets every day, covering everything from cyberattacks and threat intelligence reports to vulnerability disclosures. Every minute a ticket remains unrouted in a generic inbox is a minute in which a potential threat may go uninvestigated. Ticket routing may sound routine, but in practice it is an important part of cybersecurity operations: each incoming ticket needs to reach the specialist queue best equipped to handle it.
To address this challenge, CERT-EU developed and deployed an in-house adaptive ticket-routing system, which routes tickets, on average, more than five times as fast as manual triage. The system runs entirely on CERT-EU's own infrastructure, and continuously adapts by evaluating analysts' routine decisions without requiring a separate manual labelling process.
This post is about how it works, what it has delivered in production, and what we learned from deploying it in an operational cybersecurity environment. The research behind it, “Adaptive Ticket Routing through Retrieval-Augmented In-Context Learning from Analyst Behaviour at CERT-EU”, was presented at KDD 2026, the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. KDD is one of the world's premier venues for data science and machine learning. This year's edition took place in August, on Jeju Island, South Korea.
The work was carried out by the CERT-EU AI Team, which builds and deploys AI systems to strengthen cybersecurity operations for CERT-EU and its constituents. The team aims to accelerate the responsible integration of AI into cybersecurity by turning advances in machine learning and other AI-related fields into practical tools that help analysts work faster and more consistently.
Why ticket routing is harder than it looks
Traditional approaches struggle to keep up. Rule-based systems need constant maintenance as queues and categories evolve. Supervised classifiers drift and must be retrained. Additionally, in environments handling sensitive operational data, relying on external AI services may not be an option.
Large language models (LLMs) do not solve this on their own either. In a zero-shot setting, prompted with no examples, a model has no knowledge of how this particular organisation operates: which queue handles which kind of request, and where the boundaries between them sit. In a few-shot setting, prompted with a fixed set of examples, it inherits a snapshot of routing conventions that goes stale as soon as those conventions change, and in a cybersecurity organisation, they change often. In our evaluation, both settings landed well below our deployed system.
For CERT-EU, this meant that good routing performance alone was not enough. Any solution also had to keep sensitive data under our control, adapt as our operational environment changed, and do so without creating a new annotation workload for analysts. Those constraints shaped the system from the start.
Under the hood: organisational memory instead of retraining
Instead of training a static model, the system treats every successfully resolved ticket as a piece of organisational memory. When a new ticket arrives, the system retrieves the most similar previously routed tickets and uses them as examples, a technique known as retrieval-augmented in-context learning. In plain terms, the system asks, “How did we handle similar cases before?” and applies that pattern to the ticket in front of it.
Take a ticket that opens with "Suspicious login on Exchange server…". Deciding which queue this should be assigned to depends on details that are rarely spelled out anywhere: the urgency, the expertise required, and conventions that live mostly in the heads of experienced analysts. Rather than trying to encode those conventions as rules, the system looks up what happened the last time tickets like this one arrived.
The implementation runs entirely on-premises, on CERT-EU's own infrastructure and using open-source models hosted locally, so no sensitive data is ever transmitted to external services. The design also makes the system resilient to prompt injection. Ticket text is untrusted input, but the model's only job is to pick one queue from a fixed list: its output grants no access and triggers no action, so a manipulated prediction is just a case of misrouting, the kind of error analysts already catch and correct in their normal workflow.
Learning without annotation
The most novel part of the system is how it keeps improving and does so without anyone setting aside time to teach it.
Traditional machine learning depends on annotation: someone must label examples, usually through a dedicated interface separate from their normal work. Our system removes that overhead entirely by reinterpreting routine analyst activity as implicit labelling. The insight is simple but powerful: what matters is not what an analyst clicks, but whether they correct the system at all.
When an analyst quickly moves a ticket to a different queue, that correction is captured as an error signal. When a prediction is left untouched for an extended period, despite analysts regularly reviewing their queues, the system interprets that silence as implicit validation. We call this temporal feedback learning, and it turns everyday workflow into ongoing model improvement at zero additional annotation cost.
The waiting period is not arbitrary. Across the evaluation period, 95% of all analyst corrections arrived within seven days, so that is the window the system waits before treating an untouched prediction as confirmed. Shorter windows would misread pending corrections as approval; longer ones would delay learning without capturing meaningfully more signal.
Results: faster, more consistent, and continuously improving
The results below come from the evaluation reported in the paper: a six-month window of production deployment, measured on a representative sample of 1900 tickets.
Accuracy
The system achieves an 86.3% weighted F1 score, outperforming traditional supervised classifiers by 4.5 percentage points and zero-shot LLM inference by 15 points. Every one of these differences is statistically significant. Dynamically retrieved historical examples supply essential organisational context that general-purpose AI reasoning alone cannot replicate.
Speed
Routing velocity improved 5.5× overall compared with fully manual triage. Tickets that previously took a median of nearly 23 minutes to reach a specialist queue now arrive in a median of just over 4 minutes. For autonomously routed tickets, the speedup reaches 22×: high-confidence tickets are assigned in roughly one minute, effectively eliminating triage bottlenecks for predictable ticket types.
Adaptability and consistency
When a completely new ticket queue was created during the evaluation period, the system learned to route to it reliably within three weeks, with no retraining and no manual annotation; traditional machine learning approaches would typically need two to three months for a comparable adaptation.
Routing also became more uniform: intra-queue semantic cohesion rose by 2.5 percentage points, meaning tickets within each specialist queue are now more thematically alike, reflecting a more consistent application of routing criteria.
Why this matters: responding faster when it counts
In cybersecurity, speed matters. By automating the triage of predictable ticket types and accelerating recommendations for the more complex ones, CERT-EU's analysts can focus their expertise where it is needed most: on the nuanced, high-impact cases that require human judgement.
The acceptance of our paper at KDD 2026 is a recognition that the approach behind this work: building practical AI grounded in real operational needs is not only effective but represents a meaningful contribution to the broader AI and data science community.
But perhaps the most lasting benefit is one that has nothing to do with speed. When experienced analysts rotate to other roles or leave the organisation, their hard-won routing judgement usually leaves with them. Here, it does not. Every decision they make persists as an indexed example in the system's organisational memory, so the tacit expertise of the people who contributed to building CERT-EU's operational knowledge continues to guide every ticket that follows.
That is AI in the service of cybersecurity: practical, transparent, and designed not to replace human expertise, but to preserve and extend it.
The full paper, "Adaptive Ticket Routing through Retrieval-Augmented In-Context Learning from Analyst Behaviour at CERT-EU", is published in the Proceedings of KDD 2026.