A comprehensive flow-based anomaly detection architecture using entropy calculation and machine learning classification

Abstract

The network behavior analysis relies on the understanding of normal or acceptable behavior characteristics in the network communication, in order to efficiently detect the anomalous traffic patterns and deviations that could cause performance issues or indicate a breach, thus allowing near real-time alerting and visibility of the potential network security threats. In contrast to the signature based intrusion detection systems, this approach is extremely beneficial not only for identifying unknown threats, zero-day attacks, and suspicious behavior regardless the used cryptographic methodology, but also to identify and allow the performance optimization opportunities. We propose a comprehensive architecture for practical implementation of the flow based anomaly detection solution for real life use cases, which is based on the combination of the entropy calculation and machine learning techniques, with the ability to model the attacks and generate representative labelled training data set.