DNIDS: A dependable network intrusion detection system using the CSI-KNN algorithm

Abstract

The dependability of an Intrusion Detection System (IDS) relies on two factors: ability to detect intrusions and survivability in hostile environments. Machine learning-based anomaly detection approaches are gaining increasing attention in the network intrusion detection community because of their intrinsic ability to discover novel attacks. This ability has become critical since the number of new attacks has kept growing in recent years. However, most of today’s anomaly-based IDSs generate high false positive rates and miss many attacks because of a deficiency in their ability to discriminate attacks from legitimate behaviors. These unreliable results damage the dependability of IDSs. In addition, even if the detection method is sound and effective, the IDS might still be unable to deliver detection service when under attack. With the increasing importance of the IDS, some attackers attempt to disable the IDS before they launch a thorough attack. In this thesis, we propose a Dependable Network Intrusion Detection System (DNIDS) based on the Combined Strangeness and Isolation measure K-Nearest Neighbor (CSI-KNN) algorithm. The DNIDS can effectively detect network intrusions while providing continued service even under attacks. The intrusion detection algorithm analyzes different characteristics of network data by employing two measures: strangeness and isolation. Based on these measures, a correlation unit raises intrusion alerts with associated confidence estimates. In the DNIDS, multiple CSI-KNN classifiers work in parallel to deal with different types of network traffic. An intrusion-tolerant mechanism monitors the classifiers and the hosts on which the classifiers reside and enables the IDS to survive component failure due to intrusions. As soon as a failed IDS component is discovered, a copy of the component is installed to replace it and the detection service continues. We evaluate our detection approach over the KDD’99 benchmark dataset. The experimental results show that the performance of our approach is better than the best result of KDD’99 contest winner’s. In addition, the intrusion alerts generated by our algorithm provide graded confidence that offers some insight into the reliability of the intrusion detection. To verify the survivability of the DNIDS, we test the prototype in simulated attack scenarios. In addition, we evaluate the performance of the intrusion-tolerant mechanism and analyze the system reliability. The results demonstrate that the mechanism can effectively tolerate intrusions and achieve high dependability.