AAAI Spring Symposium on Challenges Requiring the Combination of Machine Learning and Knowledge Engineering

Offline RL+CKG: A hybrid AI model for cybersecurity tasks

, , and

AI models for cybersecurity have to detect and defend against constantly evolving cyber threats. Much effort is spent building defenses for zero days and unseen variants of known cyber-attacks. Current AI models for cybersecurity struggle with these yet unseen threats due to the constantly evolving nature of threat vectors, vulnerabilities, and exploits. This paper shows that cybersecurity AI models will be improved and more general if we include semi-structured representations of background knowledge. This could include information about the software and systems, as well as information obtained from observing the behavior of malware samples captured and detonated in honeypots. We describe how we can transfer this knowledge into forms that the RL models can directly use for decision-making purposes.

  • 1327361 bytes

ai, cybersecurity, knowledge graph, reinforcement learning



The input to the CKG is a piece of intelligence describing some characteristics of a malware ‘X’. These characteristics are translated to state-action pairs. If these state-action pairs correspond to out-of-distribution (OOD) state-action pairs for the CQL algorithm, we can use it to re-estimate the Q-value for the OOD state-action pair.

Downloads: 198 downloads

UMBC ebiquity