Configure ERSPAN (Encapsulated Remote SPAN) in virtual Distributed Switch

ERSPAN (Encapsulated Remote SPAN) in a vSphere Virtual Distributed Switch (VDS) is a network traffic-mirroring feature that allows you to copy VM or uplink traffic and send it to a remote system over IP for monitoring or analysis.

ERSPAN (Encapsulated Remote SPAN) mirrors VM traffic and sends it over IP (GRE) to a remote analyzer (Wireshark, IDS, SIEM, etc.).

Unlike local port mirroring, ERSPAN:

• Works across hosts / data centers
• Does not require L2 adjacency
• Encapsulates traffic in GRE packets

Below is a clear, step-by-step guide to configure ERSPAN on a vSphere Virtual Distributed Switch (VDS).
This works for vSphere 7.0 + and is commonly used for remote packet capture / network analysis.

Lab Environment Overview

Lab Infrastructure:

• ESXi 7.0 Host: 192.168.91.129
• vCenter Server: 192.168.91.130
• Network: 192.168.91.0/24

Lab Objectives:

1. Create a vSphere Distributed Switch (VDS)
2. Configure distributed port groups
3. Set up Distributed Port Mirroring
4. Configure Remote Port Mirroring (RSPAN)
5. Implement ERSPAN (Encapsulated Remote SPAN)
6. Test and verify all mirroring types
7. Troubleshoot common issues

Prerequisites:

Remote analyzer VM or appliance (for RSPAN/ERSPAN testing)

Administrative access to vCenter Server

At least 2-3 virtual machines for testing

Network packet analyzer (Wireshark or tcpdump)

Part 1: Environment Preparation

Step 1.1: Access vCenter Server

1. Open a web browser and navigate to your vCenter Server:

   https://192.168.91.130

2. Log in with your administrator credentials
   • Username: administrator@vsphere.local
   • Password: [Your vCenter password]
3. Navigate to the vSphere Client interface

Step 1.2: Verify ESXi Host

1. In vCenter, click on Menu → Hosts and Clusters
2. Verify your ESXi host (192.168.91.129) is visible and connected
3. Check host status shows “Connected” with a green icon
4. Note the host version (should be ESXi 7.0.x)

Step 1.3: Prepare Test Virtual Machines

You’ll need at least three VMs for comprehensive testing:

• VM1 (Source): The VM whose traffic we’ll mirror
• VM2 (Local Monitor): For Distributed Port Mirroring
• VM3 (Remote Analyzer): For RSPAN/ERSPAN testing

Recommended VM Configuration:

• Small Linux VMs (Ubuntu Server 20.04 or CentOS 8)
• 1 vCPU, 2GB RAM each
• 20GB disk space
• Single network adapter initially

VM IP Assignments:

• VM1 (Source): 192.168.91.201/24
• VM2 (Local Monitor): 192.168.91.202/24
• VM3 (Remote Analyzer): 192.168.91.203/24

Part 2: Create vSphere Distributed Switch

Step 2.1: Create the Distributed Switch

1. In vCenter, click Menu → Networking
2. Right-click on your datacenter → Distributed Switch → New Distributed Switch
3. Name and Location:
   • Name: Lab-VDS-01
   • Click Next
4. Select Version:
   • Choose 7.0.0
   • Click Next
5. Configure Settings:
   • Number of uplinks: 4 (for better flexibility)
   • Network I/O Control: Enabled
   • Create a default port group: Checked
   • Default port group name: Lab-VDS-01-PG
   • Click Next
6. Ready to Complete:
   • Review and click Finish

Step 2.2: Add ESXi Host to Distributed Switch

1. Select Lab-VDS-01 → Actions → Add and Manage Hosts
2. Select Add hosts → Next
3. Add ESXi host 192.168.91.129 → Next
4. Manage physical adapters → Assign vmnic1 to Uplink 1
5. Complete the wizard

Step 2.3: Create Port Groups for All Lab Scenarios

Create multiple port groups for different mirroring types:

Port Group 1: Source Traffic

1. Right-click Lab-VDS-01 → Distributed Port Group → New
2. Name: Lab-Source-PG
3. VLAN: None (VLAN 0)
4. Finish

Port Group 2: Local Monitor

1. Create new distributed port group
2. Name: Lab-Monitor-PG
3. VLAN: None
4. After creation, edit settings:
   • Security: Promiscuous mode = Accept
   • Security: MAC address changes = Accept
   • Security: Forged transmits = Accept

Port Group 3: RSPAN VLAN

1. Create new distributed port group
2. Name: Lab-RSPAN-PG
3. VLAN type: VLAN
4. VLAN ID: 100
5. After creation, edit settings:
   • Security: Promiscuous mode = Accept

Port Group 4: Remote Analyzer

1. Create new distributed port group
2. Name: Lab-RemoteAnalyzer-PG
3. VLAN: None
4. Security: Promiscuous mode = Accept

Step 2.4: Migrate VMs to Distributed Switch

1. VM1 → Edit Settings → Network adapter 1 → Lab-Source-PG
2. VM2 → Edit Settings → Network adapter 1 → Lab-Monitor-PG
3. VM3 → Edit Settings → Network adapter 1 → Lab-RemoteAnalyzer-PG

Step 2.5: Configure VM Network Settings

On VM1 (Source):

sudo ip addr add 192.168.91.201/24 dev eth0
sudo ip link set eth0 up
sudo ip route add default via 192.168.91.1

On VM2 (Local Monitor):

sudo ip addr add 192.168.91.202/24 dev eth0
sudo ip link set eth0 up
# Install tcpdump
sudo apt-get update && sudo apt-get install -y tcpdump wireshark

On VM3 (Remote Analyzer):

sudo ip addr add 192.168.91.203/24 dev eth0
sudo ip link set eth0 up
# Install tcpdump
sudo apt-get update && sudo apt-get install -y tcpdump wireshark tshark

Test connectivity between all VMs:

ping 192.168.91.201
ping 192.168.91.202
ping 192.168.91.203

Part 3: Lab Exercise 1 – Distributed Port Mirroring

This is the most common type of port mirroring, where source and destination are on the same distributed switch.

Step 3.1: Create Distributed Port Mirroring Session

1. Navigate to Networking → Select Lab-VDS-01
2. Click Configure tab → Settings → Port Mirroring
3. Click Add (+)

Step 3.2: Configure Session – Basic Settings

Session Configuration:

• Session name: DPM-Session-01
• Session type: Distributed Port Mirroring
• Normal I/O on destination ports: Not Allowed
• Session enabled: Checked
• Mirroring packet length: 65535 bytes
• Sampling rate: 1 (no sampling)
• Description: Local distributed port mirroring - VM1 to VM2

Traffic Direction:

• Select: Ingress and Egress

Click Next

Step 3.3: Select Source Ports

1. Click + Add under Sources
2. Select Ports
3. Expand Lab-Source-PG
4. Find and select the port connected to VM1
   • Look for the port showing VM1’s name
   • Port format: Lab-Source-PG-dvPort-##
5. Click OK → Next

Pro Tip: To easily identify VM ports:

• Look at the “Connected To” column
• Check VM name in the port list
• Or note the port ID from VM’s network adapter settings

Step 3.4: Select Destination Port

1. Click + Add under Destinations
2. Select Ports
3. Expand Lab-Monitor-PG
4. Select the port connected to VM2
5. Click OK → Next

Step 3.5: Complete and Verify

1. Review configuration
2. Click Finish
3. Verify session appears with Enabled status

Read More about virtual standard switch

Step 3.6: Test Distributed Port Mirroring

On VM2 (Monitor) – Start Capture:

# Start capturing all traffic
sudo tcpdump -i eth0 -nn -v

# Or save to file for later analysis
sudo tcpdump -i eth0 -w /tmp/dpm_capture.pcap -v

On VM1 (Source) – Generate Traffic:

# ICMP traffic
ping 8.8.8.8 -c 50

# HTTP traffic
curl http://www.google.com

# DNS queries
nslookup www.vmware.com

# TCP connections
telnet 192.168.91.130 443

# Generate continuous traffic
ping 192.168.91.1 -i 0.5

Expected Results on VM2: You should see ALL traffic from VM1, including:

15:23:10.123456 IP 192.168.91.201 > 8.8.8.8: ICMP echo request
15:23:10.145678 IP 8.8.8.8 > 192.168.91.201: ICMP echo reply
15:23:11.234567 IP 192.168.91.201.54321 > 172.217.1.46.80: Flags [S], seq 123456
15:23:11.256789 IP 192.168.91.201.54321 > 8.8.8.8.53: 12345+ A? www.vmware.com

Step 3.7: Analyze Captured Traffic

Stop capture on VM2:

# Press Ctrl+C to stop tcpdump

# Analyze the capture file
sudo tcpdump -r /tmp/dpm_capture.pcap | head -50

# Count packets by protocol
sudo tcpdump -r /tmp/dpm_capture.pcap -n | awk '{print $3}' | sort | uniq -c

# Filter specific traffic
sudo tcpdump -r /tmp/dpm_capture.pcap 'icmp'
sudo tcpdump -r /tmp/dpm_capture.pcap 'tcp port 80'

Or use Wireshark for GUI analysis:

# Transfer file to your workstation
scp user@192.168.91.202:/tmp/dpm_capture.pcap .

# Open in Wireshark
wireshark dpm_capture.pcap

Part 4: Lab Exercise 2 – Remote Port Mirroring (RSPAN)

Remote port mirroring allows you to mirror traffic to a destination on a different VLAN, useful when the analyzer is on a different network segment.

Step 4.1: Understanding RSPAN Architecture

RSPAN Uses:

• Mirrors traffic across VLANs
• Destination is on a dedicated RSPAN VLAN
• Does not encapsulate packets (Layer 2 only)
• Limited to same Layer 2 domain

Our Setup:

• Source: VM1 (Lab-Source-PG, VLAN 0)
• RSPAN VLAN: VLAN 100
• Destination: VM3 with additional interface on VLAN 100

Step 4.2: Add Second Network Adapter to VM3

1. Power off VM3
2. Edit VM3 settings
3. Click Add New Device → Network Adapter
4. New Network Adapter 2:
   • Network: Lab-RSPAN-PG (VLAN 100)
   • Adapter Type: VMXNET3
5. Click OK
6. Power on VM3

Step 4.3: Configure Second Interface on VM3

On VM3:

# Check interfaces
ip link show

# Configure second interface (usually eth1)
sudo ip addr add 192.168.100.203/24 dev eth1
sudo ip link set eth1 up
sudo ip link set eth1 promisc on

# Verify promiscuous mode
ip link show eth1
# Should show: BROADCAST,MULTICAST,PROMISC,UP

Step 4.4: Disable Previous Mirroring Session

1. Navigate to Lab-VDS-01 → Configure → Port Mirroring
2. Select DPM-Session-01
3. Click Edit
4. Uncheck Session enabled
5. Click OK

Step 4.5: Create RSPAN Session

1. Click Add (+) to create new session

Basic Configuration:

• Session name: RSPAN-Session-01
• Session type: Remote Mirroring Source
• Normal I/O on destination ports: Not Allowed
• Session enabled: Checked
• Mirroring packet length: 65535
• Sampling rate: 1
• Description: RSPAN to VLAN 100

Traffic Direction:

• Select: Ingress and Egress

Click Next

Step 4.6: Configure RSPAN Source

1. Click + Add under Sources
2. Select Ports
3. Select VM1’s port on Lab-Source-PG
4. Click OK → Next

Step 4.7: Configure RSPAN Destination

1. Under Destination, click + Add
2. Select Uplinks
3. Choose an uplink (e.g., Uplink 1)
4. Destination uplink VLAN: 100
5. Click OK → Next

Note: RSPAN sends traffic to an uplink tagged with the RSPAN VLAN. The physical switch would need VLAN 100 configured, but in our lab, we’re simulating this within the distributed switch.

Step 4.8: Complete RSPAN Configuration

1. Review settings
2. Click Finish
3. Verify session status is Enabled

Step 4.9: Test RSPAN

On VM3 – Start Capture on RSPAN Interface:

# Capture on eth1 (VLAN 100 interface)
sudo tcpdump -i eth1 -nn -v

# Or save to file
sudo tcpdump -i eth1 -w /tmp/rspan_capture.pcap -v

On VM1 – Generate Traffic:

# Generate varied traffic
ping 8.8.8.8 -c 100
curl http://www.example.com
nslookup www.cisco.com
wget http://ipv4.download.thinkbroadband.com/5MB.zip

# Continuous traffic
ping 192.168.91.130 -i 0.2

Expected Results on VM3:

16:45:23.123456 IP 192.168.91.201 > 8.8.8.8: ICMP echo request
16:45:23.145678 IP 8.8.8.8 > 192.168.91.201: ICMP echo reply
16:45:24.234567 IP 192.168.91.201.52345 > 93.184.216.34.80: Flags [S]

Verification:

• Traffic arrives on eth1 (VLAN 100 interface)
• Packets maintain original source/destination
• All protocols are captured

Step 4.10: RSPAN Analysis

Compare RSPAN vs Distributed Port Mirroring:

# On VM3 - Analyze RSPAN capture
sudo tcpdump -r /tmp/rspan_capture.pcap -n | head -100

# Count total packets
sudo tcpdump -r /tmp/rspan_capture.pcap | wc -l

# Protocol distribution
sudo tcpdump -r /tmp/rspan_capture.pcap -n | \
  awk '{print $4}' | sed 's/:.*//' | sort | uniq -c

Key Differences:

• RSPAN uses VLAN tagging for transport
• Destination must be on configured RSPAN VLAN
• No encapsulation (vs ERSPAN which uses GRE)
• Limited to Layer 2 domain

Part 5: Lab Exercise 3 – ERSPAN (Encapsulated Remote SPAN)

ERSPAN is the most advanced mirroring type, using GRE encapsulation to send mirrored traffic across Layer 3 networks.

Step 5.1: Understanding ERSPAN Architecture

ERSPAN Features:

• Uses GRE (IP Protocol 47) encapsulation
• Works across Layer 3 networks (routable)
• Preserves original packet information
• Industry-standard (supported by Cisco, VMware, etc.)
• Most flexible mirroring option

Our Setup:

• Source: VM1 (192.168.91.201)
• ERSPAN Tunnel: GRE encapsulation
• Destination: VM3 (192.168.91.203)

Step 5.2: Prepare VM3 for ERSPAN

On VM3 (Remote Analyzer):

# Install GRE module (if not already loaded)
sudo modprobe ip_gre

# Create GRE tunnel interface to receive ERSPAN traffic
sudo ip link add gre1 type gre remote 192.168.91.129 local 192.168.91.203
sudo ip link set gre1 up
sudo ip link set gre1 promisc on

# Verify GRE interface
ip link show gre1

# Check loaded modules
lsmod | grep gre

Alternative – Use any interface to capture ERSPAN:

# ERSPAN packets arrive as GRE-encapsulated
# You can capture on eth0 and filter for GRE
sudo tcpdump -i eth0 proto gre -w /tmp/erspan_raw.pcap

Step 5.3: Disable Previous Sessions

1. Navigate to Lab-VDS-01 → Configure → Port Mirroring
2. Disable both DPM-Session-01 and RSPAN-Session-01
3. Or delete them if no longer needed

Step 5.4: Create ERSPAN Session

1. Click Add (+) for new session

Basic Configuration:

• Session name: ERSPAN-Session-01
• Session type: Encapsulated Remote Mirroring (L3) Source
• Normal I/O on destination ports: Not Allowed
• Session enabled: Checked
• Mirroring packet length: 65535
• Sampling rate: 1
• Description: ERSPAN to remote analyzer at 192.168.91.203

ERSPAN Specific Settings:

• ERSPAN ID: 1 (session identifier, 1-1023)
• IP address: 192.168.91.203 (VM3’s IP)
• Encapsulation VLAN ID: Leave blank or 0 (no VLAN tagging needed)

Traffic Direction:

• Select: Ingress and Egress

Click Next

Step 5.5: Configure ERSPAN Source

1. Click + Add under Sources
2. Select Ports
3. Select VM1’s port on Lab-Source-PG
4. Click OK → Next

Step 5.6: ERSPAN Destination

Note: For ERSPAN source sessions, destination is the uplink through which GRE packets exit.

1. Click + Add under Destinations
2. Select Uplinks
3. Choose the uplink connected to your physical network (e.g., Uplink 1)
4. Click OK → Next

Step 5.7: Complete ERSPAN Configuration

1. Review all settings:
   • Session type: Encapsulated Remote Mirroring (L3) Source
   • ERSPAN ID: 1
   • Destination IP: 192.168.91.203
   • Source: VM1’s port
   • Destination: Uplink 1
2. Click Finish

Step 5.8: Verify ERSPAN on ESXi Host

SSH to ESXi host:

ssh root@192.168.91.129

# List port mirroring sessions
net-dvs -l

# Check ERSPAN configuration
esxcli network vswitch dvs vmware list

# Verify GRE traffic
esxtop
# Press 'n' for network view
# Look for GRE protocol traffic

Step 5.9: Test ERSPAN Capture

On VM3 – Method 1: Capture GRE Traffic

# Capture GRE-encapsulated packets
sudo tcpdump -i eth0 -nn proto gre -v

# Or save to file
sudo tcpdump -i eth0 proto gre -w /tmp/erspan_gre.pcap -v

On VM3 – Method 2: Capture on GRE Tunnel Interface

# Capture decapsulated traffic on GRE interface
sudo tcpdump -i gre1 -nn -v

# Save to file
sudo tcpdump -i gre1 -w /tmp/erspan_decap.pcap -v

On VM1 – Generate Test Traffic:

# ICMP traffic
ping 8.8.8.8 -c 200

# HTTP requests
for i in {1..10}; do
  curl http://www.google.com > /dev/null 2>&1
  sleep 1
done

# DNS queries
for domain in google.com vmware.com cisco.com; do
  nslookup $domain
done

# Large file transfer
wget http://speedtest.tele2.net/1MB.zip

# Continuous ping
ping 192.168.91.1 -i 0.3

Expected Results on VM3:

If capturing on eth0 (GRE packets):

17:30:15.123456 IP 192.168.91.129 > 192.168.91.203: GREv0, length 102: IP 192.168.91.201 > 8.8.8.8: ICMP echo request
17:30:15.145678 IP 192.168.91.129 > 192.168.91.203: GREv0, length 102: IP 8.8.8.8 > 192.168.91.201: ICMP echo reply

If capturing on gre1 (decapsulated):

17:30:15.123456 IP 192.168.91.201 > 8.8.8.8: ICMP echo request
17:30:15.145678 IP 8.8.8.8 > 192.168.91.201: ICMP echo reply

Step 5.10: Analyze ERSPAN Traffic

Examine GRE Encapsulation:

# View GRE packet details
sudo tcpdump -i eth0 -vvv proto gre -c 10

# Extract inner (original) packets
sudo tcpdump -r /tmp/erspan_gre.pcap -vvv | grep "IP 192.168.91.201"

# Use tshark for detailed analysis
tshark -r /tmp/erspan_gre.pcap -V | grep -A 20 GRE

Wireshark Analysis:

1. Transfer capture file to workstation
2. Open in Wireshark
3. Apply display filter: gre
4. Right-click GRE packet → Follow → GRE Stream
5. View encapsulated packet details

Verify ERSPAN Headers:

# ERSPAN packets should show:
# - Outer IP: ESXi host (192.168.91.129) to Analyzer (192.168.91.203)
# - GRE protocol: 0x88be (ERSPAN Type II) or 0x88be (ERSPAN Type III)
# - ERSPAN header: Session ID = 1
# - Inner packet: Original traffic from VM1

tshark -r /tmp/erspan_gre.pcap -T fields \
  -e ip.src -e ip.dst -e gre.proto -e erspan.sessionid

Step 5.11: ERSPAN Performance Testing

Generate High Volume Traffic:

On VM1:

# Install iperf3
sudo apt-get install iperf3

# Start iperf server on another VM
# On VM2: iperf3 -s

# Generate traffic from VM1 to VM2
iperf3 -c 192.168.91.202 -t 60 -b 100M

On VM3 – Monitor ERSPAN Performance:

# Count packets per second
sudo tcpdump -i eth0 proto gre -nn | \
  awk '{print $1}' | uniq -c

# Monitor bandwidth
sudo iftop -i eth0 -f 'proto gre'

# Check for packet drops
watch -n 1 'ifconfig eth0 | grep -E "RX|TX"'

Check ESXi Host Load:

# On ESXi host
esxtop
# Press 'n' for network
# Monitor GRE traffic on uplinks

Part 6: Comparison and Advanced Scenarios

Step 6.1: Mirroring Type Comparison Table

Step 6.2: Multi-Source ERSPAN

Configure ERSPAN to mirror multiple VMs simultaneously.

Create Multi-Source Session:

1. Edit ERSPAN-Session-01 or create new session
2. In Sources section, click + Add multiple times
3. Add ports for VM1, VM2, and other VMs
4. All traffic mirrors to same destination (VM3)

Test Multi-Source:

# On VM3
sudo tcpdump -i gre1 -nn -v

# On VM1
ping 8.8.8.8 &

# On VM2  
ping 8.8.4.4 &

# VM3 should show traffic from BOTH VMs

Step 6.3: ERSPAN with Traffic Filtering

Using Sampling to Reduce Overhead:

1. Edit ERSPAN session
2. Change Sampling rate to 10
   • Captures 1 out of every 10 packets
   • Reduces bandwidth by 90%
3. Useful for high-traffic environments

Using Packet Length Limits:

1. Change Mirroring packet length to 128 bytes
2. Captures only headers, not full payload
3. Reduces storage and bandwidth requirements
4. Sufficient for flow analysis

Step 6.4: ERSPAN to Multiple Destinations

vSphere 7 supports multiple concurrent sessions.

Create Second ERSPAN Session:

1. Keep ERSPAN-Session-01 enabled (to VM3)
2. Create ERSPAN-Session-02
   • Destination IP: Different analyzer (e.g., 192.168.91.204)
   • ERSPAN ID: 2 (must be unique)
   • Same source (VM1)
3. Traffic now mirrors to two analyzers simultaneously

Step 6.5: Directional Filtering

Capture Only Ingress Traffic:

1. Edit any mirroring session
2. Traffic direction: Ingress only
3. Captures only incoming traffic to VM1
4. Reduces capture volume by ~50%

Capture Only Egress Traffic:

1. Traffic direction: Egress only
2. Captures only outgoing traffic from VM1
3. Useful for monitoring VM’s outbound connections

Test Directional Filtering:

# On VM3 - capture
sudo tcpdump -i gre1 -w /tmp/ingress_test.pcap

# On VM1 - from external host, ping VM1
ping 192.168.91.201

# On VM1 - ping external
ping 8.8.8.8

# Analyze which traffic was captured
sudo tcpdump -r /tmp/ingress_test.pcap -nn

Part 7: Production-Ready Scenarios

Scenario A: IDS/IPS Integration

Setup ERSPAN to Security Appliance:

1. Deploy security VM (Suricata, Snort, or commercial IDS)
2. Configure ERSPAN to security appliance IP
3. Security appliance analyzes all mirrored traffic
4. Alerts on suspicious activity

Example with Suricata:

# On security appliance VM
sudo apt-get install suricata

# Configure Suricata to monitor GRE interface
sudo vi /etc/suricata/suricata.yaml
# Set: af-packet: - interface: gre1

# Start Suricata
sudo systemctl start suricata

# Monitor alerts
sudo tail -f /var/log/suricata/fast.log