NeTraMark: A Network Traffic Classification Benchmark

Suchul Lee, Hyunchul Kim, Dhiman Barman, Sungryoul Lee, Chong-kwon Kim, Ted Kwon, and Yanghee Choi
Appears in: 
CCR January 2011

Recent research on Internet traffic classification has produced a number of approaches for distinguishing types of traffic. However, a rigorous comparison of such proposed algorithms still remains a challenge, since every proposal considers a different benchmark for its experimental evaluation. A lack of clear consensus on an objective and scientific way for comparing results has made researchers uncertain of fundamental as well as relative contributions and limitations of each proposal. In response to the growing necessity for an objective method of comparing traffic classifiers and to shed light on scientifically grounded traffic classification research, we introduce an Internet traffic classification benchmark tool, NeTraMark. Based on six design guidelines (Comparability, Reproducibility, Efficiency, Extensibility, Synergy, and Flexibility/Ease-of-use), NeTraMark is the first Internet traffic classification benchmark where eleven different state-of-the-art traffic classifiers are integrated. NeTraMark allows researchers and practitioners to easily extend it with new classification algorithms and compare them with other built-in classifiers, in terms of three categories of performance metrics: per-whole-trace flow accuracy, per-application flow accuracy, and computational performance.

Public Review By: 
R. Teixeira

The area of network traffic classification, which aims at labeling network traffic according to application or application type, is constantly evolving. When classification based on port inspection got deployed, applications started using dynamic ports. Then, the deployment of deep-packet inspection caused some applications to use encryption or variable length padding. Every new application development triggers new traffic classification techniques. Often each technique is tested in different environments and using proprietary network traces making it hard to reproduce the results, compare techniques, and fully understand the limits and benefits of each technique.
This paper presents a tool to benchmark traffic classification techniques, called NeTraMark. NeTraMark is extensible, so researchers can plug-in their techniques to compare to other classification algorithms. NeTraMark already includes implementations of eleven existing classification algorithms ranging from port and deep-packet inspection to graph-based classifiers. It also implements a number of evaluation metrics and a visualization module. Researchers can easily compare the results of classification techniques under the same metrics. Since publicly available full-payload traces are rare, NeTraMark can be deployed at different sites to run on locally available data sets. In summary, NeTraMark combines a number of features that should facilitate the life of developers of traffic classification techniques. The source code is available, so we should all contribute with our own algorithms and techniques. A community effort should lead to better standards for evaluating traffic classification techniques.