NetGraph Pro
Cloud

AWS VPC Flow Logs: Complete Guide to Network Traffic Analysis

VPC Flow Logs capture network traffic metadata in your AWS environment. Learn how to enable, analyze, and extract security and operational insights from this data.

What Are VPC Flow Logs?

VPC Flow Logs capture metadata about IP traffic flowing through your VPC network interfaces. They don't capture packet contents, but they record who talked to whom, on what port, and whether the traffic was accepted or rejected.

Use Cases

Security monitoring and forensics, network troubleshooting, capacity planning, compliance auditing, anomaly detection, and cost attribution by traffic type.

Flow Log Record Format

Each flow log record contains these fields (default format): 

# Default flow log record format
version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status

# Example record (accepted traffic)
2 123456789012 eni-1234567890abcdef0 10.0.1.45 10.0.2.67 52834 443 6 25 20480 1609459200 1609459260 ACCEPT OK

# Example record (rejected traffic)
2 123456789012 eni-1234567890abcdef0 203.0.113.5 10.0.1.45 49321 22 6 1 40 1609459200 1609459260 REJECT OK
Field Description
srcaddr/dstaddrSource and destination IP addresses
srcport/dstportSource and destination ports
protocolIANA protocol number (6=TCP, 17=UDP, 1=ICMP)
packets/bytesNumber of packets and bytes in the flow
actionACCEPT or REJECT (based on security groups/NACLs)

Enabling Flow Logs

Flow logs can be created at three levels:

VPC Level

Captures all traffic in the VPC. Best for complete visibility but generates the most data. Start here for security monitoring.

Subnet Level

Captures traffic for all ENIs in a subnet. Useful for monitoring specific network segments or tiers.

ENI Level

Captures traffic for a single network interface. Most granular, lowest cost. Use for troubleshooting specific instances. 

# AWS CLI: Create flow log to CloudWatch
aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-ids vpc-12345678 \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name /vpc/flow-logs \
  --deliver-logs-permission-arn arn:aws:iam::123456789012:role/flowlogsRole

# AWS CLI: Create flow log to S3
aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-ids vpc-12345678 \
  --traffic-type ALL \
  --log-destination-type s3 \
  --log-destination arn:aws:s3:::my-flow-logs-bucket

Storage Destinations

Destination ProsCons
CloudWatch LogsReal-time queries, integrated alertingHigher cost at scale, limited retention
S3Cheapest storage, long retention, Athena queriesNot real-time, requires additional tooling
Kinesis Data FirehoseStream processing, multiple destinationsAdditional cost, more complex setup

Cost tip: For long-term storage, use S3 with lifecycle policies. Transition to S3 Glacier after 30-90 days. CloudWatch Logs can cost 10-50x more than S3 for the same data.

Analyzing Flow Logs with Athena

For S3-stored logs, Athena provides SQL-based analysis: 

-- Create table for flow logs
CREATE EXTERNAL TABLE vpc_flow_logs (
  version int,
  account string,
  interfaceid string,
  sourceaddress string,
  destinationaddress string,
  sourceport int,
  destinationport int,
  protocol int,
  numpackets bigint,
  numbytes bigint,
  starttime int,
  endtime int,
  action string,
  logstatus string
)
PARTITIONED BY (dt string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' '
LOCATION 's3://my-flow-logs-bucket/AWSLogs/';

-- Top talkers by bytes
SELECT sourceaddress, SUM(numbytes) as total_bytes
FROM vpc_flow_logs
WHERE dt = '2026/01/14'
GROUP BY sourceaddress
ORDER BY total_bytes DESC
LIMIT 20;

-- Rejected connection attempts
SELECT sourceaddress, destinationport, COUNT(*) as attempts
FROM vpc_flow_logs
WHERE action = 'REJECT' AND dt = '2026/01/14'
GROUP BY sourceaddress, destinationport
ORDER BY attempts DESC;

Security Monitoring Queries

Common security-focused analysis patterns:

Port Scanning Detection

Find IPs hitting many different ports in a short window:

SELECT sourceaddress, COUNT(DISTINCT destinationport) AS ports WHERE action='REJECT' GROUP BY sourceaddress HAVING ports > 20

Data Exfiltration Candidates

Large outbound transfers to unusual destinations:

SELECT destinationaddress, SUM(numbytes) WHERE NOT destinationaddress LIKE '10.%' GROUP BY destinationaddress ORDER BY SUM(numbytes) DESC

SSH Brute Force

Many rejected SSH connections from same source:

SELECT sourceaddress, COUNT(*) WHERE destinationport=22 AND action='REJECT' GROUP BY sourceaddress HAVING COUNT(*) > 100

Custom Flow Log Formats

AWS allows custom formats with additional fields:

Field Value
vpc-idVPC identifier for multi-VPC environments
subnet-idSubnet identifier for traffic segmentation
instance-idEC2 instance generating/receiving traffic
tcp-flagsTCP flags (SYN, ACK, FIN) for connection analysis
pkt-src-aws-serviceAWS service name if traffic from AWS service

Best Practices

Enable on All Production VPCs

Flow logs are essential for security investigations. You can't analyze traffic retroactively. Enable before you need them.

Use S3 for Cost Efficiency

Store in S3 with Parquet format for efficient Athena queries. Use CloudWatch only if you need real-time alerting.

Partition by Date

Configure S3 delivery with date-based prefixes. This makes Athena queries faster and cheaper by scanning less data.

← Back to Blog