Network Security Monitoring: Detecting Threats in Real-Time
Your network generates signals of both normal operations and potential threats. Learn how to identify suspicious patterns and detect security incidents early.
Why Network Security Monitoring?
Perimeter defenses aren't enough. Modern threats bypass firewalls through phishing, compromised credentials, and supply chain attacks. Network Security Monitoring (NSM) assumes breaches will happen and focuses on detection and response.
The NSM Philosophy
Collect network evidence continuously. When an incident occurs, you'll have the data to understand what happened, when it started, and what was affected.
Data Sources for Security Monitoring
Flow Data (NetFlow/sFlow)
Metadata about every connection: source, destination, ports, bytes, duration. Reveals communication patterns and anomalies without capturing content.
DNS Logs
Every DNS query and response. Malware often uses DNS for C2 communication or data exfiltration through encoded queries.
Firewall Logs
Allowed and denied connections. Patterns in denied traffic reveal scanning, brute force attempts, and policy violations.
IDS/IPS Alerts
Signature-based detection of known threats. Useful for catching known exploits but misses novel attacks.
Indicators of Compromise (IOCs)
Network behaviors that suggest security incidents:
| Indicator | Possible Meaning | Detection Method |
|---|---|---|
| Beaconing traffic | Malware phoning home at intervals | Flow timing analysis |
| DNS to unusual TLDs | C2 via obscure domains | DNS query monitoring |
| Large outbound transfers | Data exfiltration | Flow volume tracking |
| Internal port scanning | Lateral movement | Connection pattern analysis |
| Non-standard ports | Tunneling, evasion | Protocol anomaly detection |
Baseline and Anomaly Detection
Effective detection requires knowing what's normal:
- 1.Establish baselines: Document typical traffic patterns, connection volumes, and data transfer sizes for each network segment.
- 2.Define thresholds: Set alert triggers for deviations. A workstation suddenly uploading 10GB is suspicious; a backup server doing the same is normal.
- 3.Account for patterns: Traffic varies by time of day, day of week, and business cycles. Your model should too.
# Example: Detect unusual outbound volume
# Alert if outbound bytes exceed 3 standard deviations
avg_daily_out = 50_000_000_000 # 50GB typical
std_dev = 10_000_000_000 # 10GB variation
threshold = avg_daily_out + (3 * std_dev) # 80GB
if current_outbound > threshold:
alert("Unusual outbound data volume detected")DNS Security Monitoring
DNS is a goldmine for security monitoring:
Domain Generation Algorithms
Malware generates random-looking domains. Detect high-entropy domain names or many NXDomain responses.
DNS Tunneling
Data encoded in DNS queries/responses. Look for unusually long subdomain strings or high query volumes to single domains.
Newly Registered Domains
Phishing and malware often use recently registered domains. Flag connections to domains less than 30 days old.
Lateral Movement Detection
Once inside, attackers move laterally. Watch for:
| Pattern | Description |
|---|---|
| New internal connections | Workstation suddenly talking to servers it never accessed before |
| Admin protocol from user VLANs | RDP, SSH, WinRM from unexpected sources |
| SMB scanning | Connections to port 445 across many hosts |
| Pass-the-hash patterns | NTLM auth to multiple systems in short time |
Tip: Build a connection graph of normal server-to-server and workstation-to-server patterns. Alert on edges that don't exist in the baseline graph.
Building a Detection Pipeline
A practical security monitoring architecture:
- -Collection: Flow exporters on network devices, DNS logging on resolvers, centralized firewall logging.
- -Storage: Time-series database for metrics, log aggregation platform for events. Retain raw data for forensics.
- -Analysis: Real-time correlation rules, scheduled threat hunting queries, ML anomaly detection.
- -Response: Alert routing, automated containment actions, integration with SOAR platforms.
Alert Tuning and False Positives
Too many false positives lead to alert fatigue. Tune your detections:
Whitelist Known Good
Exclude legitimate security scanners, backup systems, and known admin tools from generic alerts.
Require Multiple Indicators
Single anomalies often have innocent explanations. Alert when multiple suspicious behaviors correlate.
Time-Based Suppression
Expected maintenance windows and batch jobs can trigger alerts. Suppress known patterns during scheduled times.