Data Analytics for the Cyber Security of an Information System Based on a Markov Decision Process Model

Abstract

Intrusion detection is an important research topic in information systems and cyber security. Both a defender and an attacker detect and learn about each other during an intrusion process. The defender can expel the attacker as soon as the attacker is detected or wait and observe to know more about the attacker for the detection and prevention of other attacks in the future. An optimal decision is often required in this situation. Data analytics is conducted to achieve an optimal decision for the cyber security of an information system based on a Markov Decision Process (MDP) model in this study. The state of the information system is completely observable in the model. The model is validated using various algorithms that include policy iteration, value iteration, and Q-learning. Data analytics over a finite planning horizon and an infinite planning horizon is conducted, respectively. The expected total cost for each state is analyzed at various parameters of the transition probability and various parameters of the transition cost.