Network traffic anomaly detection and analysis – from research to the implementation

Abstract

With a constantly increasing amount of encrypted network traffic and a new type of attack (“zero-day”), network traffic anomaly detection shows significant benefits over traditionally used signature-based packet inspection methods for cybersecurity attack detection. Using NetFlow or similar protocols is an attractive approach to providing accounting information about network communications due to its simplicity and applicability in a real-life network environment. Even though the basic set of information in flow data is not sufficient for efficient machine learning techniques, they are quite suitable for the application of entropy-based anomaly detection techniques. In this paper, we present comprehensive work in research, development and implementation of network traffic anomaly detection solutions based on the entropy of flow data. Starting from the well-known entropy-based approach, we reveal the results of our methodic work in solving the main challenges in designing an efficient anomaly detection solution empowered with the original classification method. Since the proof of concept was achieved in the laboratory environment using offline datasets, the solution has been implemented relying on the existing NetFlow Analyzer software product NetVizura. Even at the minimum viable product stage, the application confirms high performances and great applicability of the anomaly detection and classification method in real-life network environments.