Packet Analysis with tcpdump and Wireshark¶

Overview¶

When logs and curl are not enough, packet capture shows exactly what crosses the wire — SYN packets that never get ACKed, TLS handshakes failing on cipher mismatch, DNS responses with wrong TTL, or HTTP requests hitting the wrong virtual host. tcpdump captures packets on Linux servers where GUI tools cannot run. Wireshark provides deep protocol dissection, TCP stream reassembly, and visual timelines for analysis.

Packet analysis is a core SRE and security skill. It confirms whether a problem is network, firewall, or application — with evidence, not guesses. This tutorial covers capture techniques, tcpdump filter expressions, transferring captures to Wireshark, reading TCP streams, and the filter library you will use in every production incident.

This is Tutorial 15 in Module 5: Troubleshooting of the REBASH Academy Networking series.

Prerequisites¶

• Complete Network Troubleshooting Methodology
• Complete TCP and UDP Deep Dive — understand SYN/ACK/FIN and port concepts
• Linux VM or cloud instance with sudo and outbound network access
• Wireshark installed on your workstation (wireshark.org) for GUI analysis

Learning Objectives¶

By the end of this tutorial, you will be able to:

• Capture packets on specific interfaces with tcpdump and write to pcap files
• Construct tcpdump filter expressions for host, port, and protocol filtering
• Transfer and analyze captures in Wireshark with proper dissection
• Follow TCP streams to reconstruct HTTP conversations
• Identify common failure patterns: SYN retransmits, RST, TLS failures, DNS issues
• Apply a library of production-ready tcpdump and Wireshark display filters

Architecture Diagram¶

flowchart LR
    subgraph Capture["Capture Point"]
        NIC["eth0 / any interface"]
        TD["tcpdump / dumpcap"]
        PCAP["(capture.pcap")]
    end

    subgraph Analysis
        WS[Wireshark GUI]
        FILT[Display Filters]
        STREAM[TCP Stream Follow]
    end

    subgraph Evidence
        SYN[SYN retransmits]
        RST[RST packets]
        TLS[TLS handshake]
        HTTP[HTTP requests]
    end

    NIC --> TD
    TD --> PCAP
    PCAP --> WS
    WS --> FILT
    FILT --> STREAM
    STREAM --> HTTP
    FILT --> SYN
    FILT --> RST
    FILT --> TLS```

## Theory

### When to Capture Packets

Use packet capture when:

- **Connection timeouts** — no application log entry; need to see if SYN reaches server
- **Intermittent failures** — capture during the failure window
- **TLS/SSL mysteries** — handshake failures before HTTP logs exist
- **DNS discrepancies** — verify actual queries and responses on the wire
- **Performance debugging** — retransmissions, zero-window, high latency segments
- **Security incidents** — unauthorized traffic, data exfiltration patterns

Avoid capture when:

- Volume would overwhelm disk (busy production NIC — use filters and ring buffer)
- Legal/compliance prohibits capture (PII-heavy traffic without approval)
- Simpler tools suffice (`curl -v`, `ss`, application logs)

### Capture Points Matter

Where you capture determines what you see:

| Location | Sees |
|----------|------|
| Client host | Outbound requests, DNS to resolver |
| Server host (INPUT) | Packets arriving at server — post-NAT source may be LB IP |
| Load balancer | Client IP (if preserved), backend connection |
| Mirror/SPAN port | Copy of traffic through switch — non-intrusive |
| Container network namespace | Pod-to-pod traffic (`ip netns exec`) |

Capturing on the **server** during a timeout: if you see SYN arrive but no SYN-ACK leave, the local firewall or app is the problem. If no SYN arrives, the problem is upstream.

### tcpdump Fundamentals

**tcpdump** reads packets from interfaces and optionally writes **pcap** format for Wireshark.

Basic syntax:

```bash
tcpdump [options] [filter expression]

Permission: Capturing requires CAP_NET_RAW or root (sudo tcpdump).

tcpdump Filter Expressions¶

Filters use Berkeley Packet Filter (BPF) syntax — evaluated in kernel for efficiency.

Primitives:

host 10.0.1.5              # src or dst
src host 10.0.1.5
dst host 10.0.1.5
net 10.0.1.0/24
port 443
src port 1024
dst port 8080
portrange 8000-8010

Protocols:

tcp
udp
icmp
arp

Combinators:

and    # both conditions
or     # either condition
not    # negation

Production examples:

# HTTPS traffic to/from web server
sudo tcpdump -i any -nn 'host 10.0.2.10 and port 443'

# SYN packets only (connection attempts)
sudo tcpdump -i any -nn 'tcp[tcpflags] & tcp-syn != 0 and tcp[tcpflags] & tcp-ack == 0'

# DNS queries
sudo tcpdump -i any -nn 'udp port 53'

# Exclude SSH noise during capture
sudo tcpdump -i any -nn 'not port 22'

# Traffic between two hosts
sudo tcpdump -i any -nn 'host 10.0.1.5 and host 10.0.2.10'

Wireshark Analysis Workflow¶

1. Open pcap — File → Open, or drag-and-drop
2. Apply display filter — narrows view without re-capturing (non-destructive)
3. Follow TCP stream — right-click packet → Follow → TCP Stream
4. Expert Information — Analyze → Expert Information (warnings, retransmits)
5. IO Graphs / Flow Graph — Statistics menu for timing visualization
6. Export — Save filtered packets or export objects (HTTP files)

Display filters (Wireshark syntax — different from tcpdump):

ip.addr == 10.0.1.5
tcp.port == 443
http.request.method == "GET"
tls.handshake.type == 1
dns.qry.name contains "example.com"
tcp.flags.syn == 1 && tcp.flags.ack == 0
tcp.analysis.retransmission

Reading TCP Streams¶

A TCP stream is the conversation between two endpoints (IP:port ↔ IP:port). Wireshark reassembles segments into readable byte streams.

Healthy HTTP over TLS flow:

1. TCP three-way handshake (SYN → SYN-ACK → ACK)
2. TLS Client Hello → Server Hello → Certificate → Finished
3. Encrypted application data (HTTP inside TLS after decryption if key available)

Failure patterns:

Follow TCP Stream shows red (client→server) and blue (server→client) text — ideal for debugging HTTP headers, SMTP dialogs, and Redis commands.

Decrypting TLS in Wireshark¶

Production HTTPS appears encrypted in Wireshark unless you provide keys:

• Server private key — import in Preferences → Protocols → TLS (rarely available in prod)
• PREMASTER-SECRET log — set SSLKEYLOGFILE env var on client or server
• Mitigation — capture on loopback before TLS (curl http://localhost:8080) or use staging with key access

For most ops work, analyze TLS handshake metadata (cipher, SNI, cert CN) without decrypting application data.

Hands-on Lab¶

Run these exercises on a Linux lab host. Install tcpdump if needed: sudo apt install tcpdump.

Step 1 – Identify interfaces and baseline capture¶

Command:

ip link show
sudo tcpdump -i any -nn -c 5

Explanation: any captures all interfaces — useful when unsure which NIC carries traffic. -c 5 stops after five packets.

Expected output:

tcpdump: data link type LINUX_SLL2
15:04:01.123456 IP 10.0.1.5.54321 > 93.184.216.34.443: Flags [S], seq ...
...
5 packets captured

Step 2 – Capture DNS and HTTP to pcap file¶

Command:

# Terminal 1: start capture
sudo tcpdump -i any -nn -s 0 -w /tmp/lab.pcap 'host 93.184.216.34 or port 53' &
TCPDUMP_PID=$!
sleep 1

# Terminal 2: generate traffic
dig example.com +short
curl -sI http://example.com/ | head -3

# Stop capture
sleep 2
sudo kill $TCPDUMP_PID
ls -lh /tmp/lab.pcap

Explanation: Write to pcap for Wireshark analysis. Filter limits file size to relevant traffic.

Expected output:

-rw-r--r-- 1 tcpdump tcpdump 4.2K ... /tmp/lab.pcap

Step 3 – Read pcap with tcpdump (command-line analysis)¶

Command:

sudo tcpdump -nn -r /tmp/lab.pcap | head -20
sudo tcpdump -nn -r /tmp/lab.pcap 'tcp[tcpflags] & tcp-syn != 0' 
sudo tcpdump -nn -r /tmp/lab.pcap -A 'tcp port 80' 2>/dev/null | head -30

Explanation: -r reads file offline. SYN filter shows connection attempts. -A prints ASCII — useful for cleartext HTTP.

Step 4 – Capture SYN-only handshake to example.com:80¶

Command:

sudo tcpdump -i any -nn -c 10 'tcp port 80 and host 93.184.216.34' &
sleep 1
curl -s http://93.184.216.34/ -H "Host: example.com" -o /dev/null
wait

Expected output:

IP 10.0.x.x.XXXXX > 93.184.216.34.80: Flags [S], seq ...
IP 93.184.216.34.80 > 10.0.x.x.XXXXX: Flags [S.], seq ..., ack ...
IP 10.0.x.x.XXXXX > 93.184.216.34.80: Flags [.], ack ...

Observe three-way handshake: [S], [S.], [.].

Step 5 – Filter DNS queries¶

Command:

sudo tcpdump -i any -nn -vv 'udp port 53' -c 6 &
dig example.com @8.8.8.8 +short
dig google.com @8.8.8.8 +short
wait

Explanation: -vv shows DNS query names in tcpdump output when formatted as DNS packets.

Transfer /tmp/lab.pcap to Wireshark on your workstation (scp or open locally). Apply display filter dns || http || tcp.flags.syn == 1, then use Follow → TCP Stream and Statistics → Conversations.

Commands & Code¶

Common Mistakes¶

Best Practices¶

Troubleshooting¶

Summary¶

• tcpdump captures on Linux servers; write to pcap and analyze in Wireshark
• BPF filters (host, port, tcp, combinators) limit capture to relevant traffic
• Capture point determines whether you see client IP, NAT'd IP, or LB IP
• Follow TCP Stream in Wireshark reconstructs application conversations
• SYN retransmits imply filtering or no route; RST implies closed port or rejection
• Use ring buffers, snaplen limits, and privacy policies for production captures
• Keep a filter cookbook for tcpdump and Wireshark display filters — speed matters during incidents

Interview Questions¶

1. When would you use packet capture instead of application logs?
2. Explain the tcpdump command: tcpdump -i any -nn -w out.pcap 'host 10.0.1.5 and port 443'
3. What does it mean when you see repeated SYN packets without SYN-ACK?
4. How do tcpdump capture filters differ from Wireshark display filters?
5. How would you capture traffic for a specific Kubernetes pod?
6. What is the TCP three-way handshake, and how does it appear in a capture?
7. Why should you avoid full payload capture in production?
8. How do you follow an HTTP conversation inside a pcap?

Related Tutorials¶

• Networking – Category Overview
• Network Troubleshooting Methodology (previous in Module 5)
• Cloud Networking — VPCs and Subnets (next — Module 6)
• TCP and UDP Deep Dive
• Linux Networking Essentials

References¶

• tcpdump man page
• Wireshark User's Guide
• BPF syntax overview