Getting started with network security monitoring

Network Security Monitoing (NSM) can be broadly classified into three areas being collection, detection and analysis [11]. There are plenty of tools that are designed exclusively for each field. Tools like tcpdump [14], wireshark [13], tshark[16], DNSstats [15], Bro [17], Chaosreader [18] come under collection tools. Their main job is to collect data from an interface and save as a dump. They are also called sensor tools. The next category is detection, which consists of two categories: signature based as well as behaviour based detections. These are tools that are used for detecting malicious behaviours/signatures in pcaps. Snort [19] and suricata [20] are the main signature based detection tools available. They work mainly based on the signature hashes developed by analysing a malware. The main inefficiency with this approach is that only ‘known’ malwares are detected. Zero days and new versions of previous malwares can easily evade such detections. Behaviour based detection are discussed in detail in the following sections. The third category analysis comes to play after an incident (security breach) is reported. These tools allows more deep packet analysis and wide inspection of packet data allowing marking, easy traversal and so on. Sguil [21], squert [22], wireshark and ELSA [23] are examples in this category. In total an NSM tool should contain a collector, a detector and then an analysis interface that allows to analyse the data. The research problems associated with each category is different. My analysis was mainly on collection and detection.


The two main questions to ask here is 1) What data needs to be collected (indexing, filters used), 2) What line rate is supported.  Some of the important libraries to consider in this area are libpcap [24] (winpcap for windows) used in Tcpdump & tcpdump-netmap, n2disk10g [25] from ntop, and FloSIS [2].  FloSIS [2] and n2disk10g [25] follow a different approach compared to libpcap. The first two are designed to keep up with multi-Gigabit speeds on commodity hardware (10Gbps). The design make efficient use of  parallelism capabilities in CPUs, memory and NIC. They also maintain two stage indexes in real time to increase query response (bloom filters). Libpcap, being the most used library is not efficient with high speed networks. (works well with 400-500 Mbps rate). But libpcap is the best library available for free and ready to use. N2disk10g [25] requires a license. FlosSIS is not open source and is not available.


There are plenty of research based on statistical data and session data. Below I am discussing only those areas that are relevant to HTTP protocol analysis and corresponding detection mechanisms.  This is related to HTTP headers and payload analysis without deep packet inspection.

There are correlation techniques where length of URL, request type (GET/POST), average amount of data in payload, hosts that rely on the same Web-based application (which could be the C2 server) etc are used for clustering malware based on HTTP. TAMD [5] (traffic aggregation for malware detection) talks about a technique on hashing blocks of payload and performing matches to identify correlations. Also capturing syntactic similarities in the payload. But this is applied in spam emails and not application data bytes.  PAYL [6] describes about a payload inspection technique based on profiling normal payloads. It calculates deviation distance of a test payload from the normal profile using a simplified Mahalanobis distance. A payload is considered as anomalous if this distance exceeds a predetermined threshold. A very similar approach is by SLADE: Statistical PayLoad Anomaly Detection Engine as used in BotHunter [7]. Here they use n-byte sequence information from the payload, which helps in constructing a more precise model of the normal traffic compared to the single-byte as used in PAYL [6]. Another approach is where the payload of each HTTP request is segmented into a certain number of contiguous blocks, which are subsequently quantized according to a previously trained scalar codebook [3].

I hope you enjoyed the reading. More on network security monitoring later.


[1] Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces

[2] FloSIS: A Highly Scalable Network Flow Capture System for Fast Retrieval and Storage Efficiency

[3] Measuring normality in HTTP traffic for anomaly-based intrusion detection


[5] Classifying HTTP Traffic in the New Age

[6] K. Wang and S. Stolfo. Anomalous payload-based network intrusion detection. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID), 2004.

[7] BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation

[8] Know your digital enemy

[9] Comparison on dump tools:


[11] Sanders. Chris, Smith. Jason,  Applied network security monitoring- collection, detection, and analysis.

[12] Bejtlich. Richard, The practise of network security monitoring.

[13] Wireshark:

[14] Tcpdump:

[15] Dnstats:

[16] Tshark:

[17] Bro:

[18] Chaosreader:

[19] Snort:

[20] Suricata: ttps://

[21] Sguil:

[22] Squert:

[23] ELSA:

[24] Libpcap:

[25] n2disk:

[26] TRE:

[27] A. K. Jain, M. N. Murty, and P. J. Flynn. Data clustering: a review. ACM Comput. Surv., 31(3):264–323, 1999


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s