Protocol Reverse Engineering

Decode and document unknown network protocols through systematic analysis

✨ The solution you've been looking for

Verified
Tested and verified by our team
25450 Stars

Master network protocol reverse engineering including packet analysis, protocol dissection, and custom protocol documentation. Use when analyzing network traffic, understanding proprietary protocols, or debugging network communication.

protocol-analysis network-security reverse-engineering packet-capture binary-analysis wireshark vulnerability-research forensics
Repository

See It In Action

Interactive preview & real-world examples

Live Demo
Skill Demo Animation

AI Conversation Simulator

See how users interact with this skill

User Prompt

I have a packet capture from a proprietary IoT device. Help me identify the protocol structure and message types.

Skill Processing

Analyzing request...

Agent Response

Step-by-step analysis revealing protocol headers, message formats, and communication patterns with documented findings

Quick Start (3 Steps)

Get up and running in minutes

1

Install

claude-code skill install protocol-reverse-engineering

claude-code skill install protocol-reverse-engineering
2

Config

3

First Trigger

@protocol-reverse-engineering help

Commands

CommandDescriptionRequired Args
@protocol-reverse-engineering proprietary-protocol-investigationAnalyze unknown network traffic from a custom application to understand its communication protocolNone
@protocol-reverse-engineering malware-communication-analysisReverse engineer C&C communication protocols used by malware samplesNone
@protocol-reverse-engineering api-protocol-documentationCreate comprehensive documentation for undocumented network APIs through traffic analysisNone

Typical Use Cases

Proprietary Protocol Investigation

Analyze unknown network traffic from a custom application to understand its communication protocol

Malware Communication Analysis

Reverse engineer C&C communication protocols used by malware samples

API Protocol Documentation

Create comprehensive documentation for undocumented network APIs through traffic analysis

Overview

Protocol Reverse Engineering

Comprehensive techniques for capturing, analyzing, and documenting network protocols for security research, interoperability, and debugging.

Traffic Capture

Wireshark Capture

 1# Capture on specific interface
 2wireshark -i eth0 -k
 3
 4# Capture with filter
 5wireshark -i eth0 -k -f "port 443"
 6
 7# Capture to file
 8tshark -i eth0 -w capture.pcap
 9
10# Ring buffer capture (rotate files)
11tshark -i eth0 -b filesize:100000 -b files:10 -w capture.pcap

tcpdump Capture

 1# Basic capture
 2tcpdump -i eth0 -w capture.pcap
 3
 4# With filter
 5tcpdump -i eth0 port 8080 -w capture.pcap
 6
 7# Capture specific bytes
 8tcpdump -i eth0 -s 0 -w capture.pcap  # Full packet
 9
10# Real-time display
11tcpdump -i eth0 -X port 80

Man-in-the-Middle Capture

 1# mitmproxy for HTTP/HTTPS
 2mitmproxy --mode transparent -p 8080
 3
 4# SSL/TLS interception
 5mitmproxy --mode transparent --ssl-insecure
 6
 7# Dump to file
 8mitmdump -w traffic.mitm
 9
10# Burp Suite
11# Configure browser proxy to 127.0.0.1:8080

Protocol Analysis

Wireshark Analysis

# Display filters
tcp.port == 8080
http.request.method == "POST"
ip.addr == 192.168.1.1
tcp.flags.syn == 1 && tcp.flags.ack == 0
frame contains "password"

# Following streams
Right-click > Follow > TCP Stream
Right-click > Follow > HTTP Stream

# Export objects
File > Export Objects > HTTP

# Decryption
Edit > Preferences > Protocols > TLS
  - (Pre)-Master-Secret log filename
  - RSA keys list

tshark Analysis

 1# Extract specific fields
 2tshark -r capture.pcap -T fields -e ip.src -e ip.dst -e tcp.port
 3
 4# Statistics
 5tshark -r capture.pcap -q -z conv,tcp
 6tshark -r capture.pcap -q -z endpoints,ip
 7
 8# Filter and extract
 9tshark -r capture.pcap -Y "http" -T json > http_traffic.json
10
11# Protocol hierarchy
12tshark -r capture.pcap -q -z io,phs

Scapy for Custom Analysis

 1from scapy.all import *
 2
 3# Read pcap
 4packets = rdpcap("capture.pcap")
 5
 6# Analyze packets
 7for pkt in packets:
 8    if pkt.haslayer(TCP):
 9        print(f"Src: {pkt[IP].src}:{pkt[TCP].sport}")
10        print(f"Dst: {pkt[IP].dst}:{pkt[TCP].dport}")
11        if pkt.haslayer(Raw):
12            print(f"Data: {pkt[Raw].load[:50]}")
13
14# Filter packets
15http_packets = [p for p in packets if p.haslayer(TCP)
16                and (p[TCP].sport == 80 or p[TCP].dport == 80)]
17
18# Create custom packets
19pkt = IP(dst="target")/TCP(dport=80)/Raw(load="GET / HTTP/1.1\r\n")
20send(pkt)

Protocol Identification

Common Protocol Signatures

HTTP        - "HTTP/1." or "GET " or "POST " at start
TLS/SSL     - 0x16 0x03 (record layer)
DNS         - UDP port 53, specific header format
SMB         - 0xFF 0x53 0x4D 0x42 ("SMB" signature)
SSH         - "SSH-2.0" banner
FTP         - "220 " response, "USER " command
SMTP        - "220 " banner, "EHLO" command
MySQL       - 0x00 length prefix, protocol version
PostgreSQL  - 0x00 0x00 0x00 startup length
Redis       - "*" RESP array prefix
MongoDB     - BSON documents with specific header

Protocol Header Patterns

+--------+--------+--------+--------+
|  Magic number / Signature         |
+--------+--------+--------+--------+
|  Version       |  Flags          |
+--------+--------+--------+--------+
|  Length        |  Message Type   |
+--------+--------+--------+--------+
|  Sequence Number / Session ID     |
+--------+--------+--------+--------+
|  Payload...                       |
+--------+--------+--------+--------+

Binary Protocol Analysis

Structure Identification

 1# Common patterns in binary protocols
 2
 3# Length-prefixed message
 4struct Message {
 5    uint32_t length;      # Total message length
 6    uint16_t msg_type;    # Message type identifier
 7    uint8_t  flags;       # Flags/options
 8    uint8_t  reserved;    # Padding/alignment
 9    uint8_t  payload[];   # Variable-length payload
10};
11
12# Type-Length-Value (TLV)
13struct TLV {
14    uint8_t  type;        # Field type
15    uint16_t length;      # Field length
16    uint8_t  value[];     # Field data
17};
18
19# Fixed header + variable payload
20struct Packet {
21    uint8_t  magic[4];    # "ABCD" signature
22    uint32_t version;
23    uint32_t payload_len;
24    uint32_t checksum;    # CRC32 or similar
25    uint8_t  payload[];
26};

Python Protocol Parser

 1import struct
 2from dataclasses import dataclass
 3
 4@dataclass
 5class MessageHeader:
 6    magic: bytes
 7    version: int
 8    msg_type: int
 9    length: int
10
11    @classmethod
12    def from_bytes(cls, data: bytes):
13        magic, version, msg_type, length = struct.unpack(
14            ">4sHHI", data[:12]
15        )
16        return cls(magic, version, msg_type, length)
17
18def parse_messages(data: bytes):
19    offset = 0
20    messages = []
21
22    while offset < len(data):
23        header = MessageHeader.from_bytes(data[offset:])
24        payload = data[offset+12:offset+12+header.length]
25        messages.append((header, payload))
26        offset += 12 + header.length
27
28    return messages
29
30# Parse TLV structure
31def parse_tlv(data: bytes):
32    fields = []
33    offset = 0
34
35    while offset < len(data):
36        field_type = data[offset]
37        length = struct.unpack(">H", data[offset+1:offset+3])[0]
38        value = data[offset+3:offset+3+length]
39        fields.append((field_type, value))
40        offset += 3 + length
41
42    return fields

Hex Dump Analysis

 1def hexdump(data: bytes, width: int = 16):
 2    """Format binary data as hex dump."""
 3    lines = []
 4    for i in range(0, len(data), width):
 5        chunk = data[i:i+width]
 6        hex_part = ' '.join(f'{b:02x}' for b in chunk)
 7        ascii_part = ''.join(
 8            chr(b) if 32 <= b < 127 else '.'
 9            for b in chunk
10        )
11        lines.append(f'{i:08x}  {hex_part:<{width*3}}  {ascii_part}')
12    return '\n'.join(lines)
13
14# Example output:
15# 00000000  48 54 54 50 2f 31 2e 31  20 32 30 30 20 4f 4b 0d  HTTP/1.1 200 OK.
16# 00000010  0a 43 6f 6e 74 65 6e 74  2d 54 79 70 65 3a 20 74  .Content-Type: t

Encryption Analysis

Identifying Encryption

 1# Entropy analysis - high entropy suggests encryption/compression
 2import math
 3from collections import Counter
 4
 5def entropy(data: bytes) -> float:
 6    if not data:
 7        return 0.0
 8    counter = Counter(data)
 9    probs = [count / len(data) for count in counter.values()]
10    return -sum(p * math.log2(p) for p in probs)
11
12# Entropy thresholds:
13# < 6.0: Likely plaintext or structured data
14# 6.0-7.5: Possibly compressed
15# > 7.5: Likely encrypted or random
16
17# Common encryption indicators
18# - High, uniform entropy
19# - No obvious structure or patterns
20# - Length often multiple of block size (16 for AES)
21# - Possible IV at start (16 bytes for AES-CBC)

TLS Analysis

 1# Extract TLS metadata
 2tshark -r capture.pcap -Y "ssl.handshake" \
 3    -T fields -e ip.src -e ssl.handshake.ciphersuite
 4
 5# JA3 fingerprinting (client)
 6tshark -r capture.pcap -Y "ssl.handshake.type == 1" \
 7    -T fields -e ssl.handshake.ja3
 8
 9# JA3S fingerprinting (server)
10tshark -r capture.pcap -Y "ssl.handshake.type == 2" \
11    -T fields -e ssl.handshake.ja3s
12
13# Certificate extraction
14tshark -r capture.pcap -Y "ssl.handshake.certificate" \
15    -T fields -e x509sat.printableString

Decryption Approaches

 1# Pre-master secret log (browser)
 2export SSLKEYLOGFILE=/tmp/keys.log
 3
 4# Configure Wireshark
 5# Edit > Preferences > Protocols > TLS
 6# (Pre)-Master-Secret log filename: /tmp/keys.log
 7
 8# Decrypt with private key (if available)
 9# Only works for RSA key exchange
10# Edit > Preferences > Protocols > TLS > RSA keys list

Custom Protocol Documentation

Protocol Specification Template

 1# Protocol Name Specification
 2
 3## Overview
 4
 5Brief description of protocol purpose and design.
 6
 7## Transport
 8
 9- Layer: TCP/UDP
10- Port: XXXX
11- Encryption: TLS 1.2+
12
13## Message Format
14
15### Header (12 bytes)
16
17| Offset | Size | Field   | Description             |
18| ------ | ---- | ------- | ----------------------- |
19| 0      | 4    | Magic   | 0x50524F54 ("PROT")     |
20| 4      | 2    | Version | Protocol version (1)    |
21| 6      | 2    | Type    | Message type identifier |
22| 8      | 4    | Length  | Payload length in bytes |
23
24### Message Types
25
26| Type | Name      | Description            |
27| ---- | --------- | ---------------------- |
28| 0x01 | HELLO     | Connection initiation  |
29| 0x02 | HELLO_ACK | Connection accepted    |
30| 0x03 | DATA      | Application data       |
31| 0x04 | CLOSE     | Connection termination |
32
33### Type 0x01: HELLO
34
35| Offset | Size | Field      | Description              |
36| ------ | ---- | ---------- | ------------------------ |
37| 0      | 4    | ClientID   | Unique client identifier |
38| 4      | 2    | Flags      | Connection flags         |
39| 6      | var  | Extensions | TLV-encoded extensions   |
40
41## State Machine

[INIT] –HELLO–> [WAIT_ACK] –HELLO_ACK–> [CONNECTED] | DATA/DATA | [CLOSED] <–CLOSE–+


## Examples
### Connection Establishment

Client -> Server: HELLO (ClientID=0x12345678) Server -> Client: HELLO_ACK (Status=OK) Client -> Server: DATA (payload)

Wireshark Dissector (Lua)

 1-- custom_protocol.lua
 2local proto = Proto("custom", "Custom Protocol")
 3
 4-- Define fields
 5local f_magic = ProtoField.string("custom.magic", "Magic")
 6local f_version = ProtoField.uint16("custom.version", "Version")
 7local f_type = ProtoField.uint16("custom.type", "Type")
 8local f_length = ProtoField.uint32("custom.length", "Length")
 9local f_payload = ProtoField.bytes("custom.payload", "Payload")
10
11proto.fields = { f_magic, f_version, f_type, f_length, f_payload }
12
13-- Message type names
14local msg_types = {
15    [0x01] = "HELLO",
16    [0x02] = "HELLO_ACK",
17    [0x03] = "DATA",
18    [0x04] = "CLOSE"
19}
20
21function proto.dissector(buffer, pinfo, tree)
22    pinfo.cols.protocol = "CUSTOM"
23
24    local subtree = tree:add(proto, buffer())
25
26    -- Parse header
27    subtree:add(f_magic, buffer(0, 4))
28    subtree:add(f_version, buffer(4, 2))
29
30    local msg_type = buffer(6, 2):uint()
31    subtree:add(f_type, buffer(6, 2)):append_text(
32        " (" .. (msg_types[msg_type] or "Unknown") .. ")"
33    )
34
35    local length = buffer(8, 4):uint()
36    subtree:add(f_length, buffer(8, 4))
37
38    if length > 0 then
39        subtree:add(f_payload, buffer(12, length))
40    end
41end
42
43-- Register for TCP port
44local tcp_table = DissectorTable.get("tcp.port")
45tcp_table:add(8888, proto)

Active Testing

Fuzzing with Boofuzz

 1from boofuzz import *
 2
 3def main():
 4    session = Session(
 5        target=Target(
 6            connection=TCPSocketConnection("target", 8888)
 7        )
 8    )
 9
10    # Define protocol structure
11    s_initialize("HELLO")
12    s_static(b"\x50\x52\x4f\x54")  # Magic
13    s_word(1, name="version")       # Version
14    s_word(0x01, name="type")       # Type (HELLO)
15    s_size("payload", length=4)     # Length field
16    s_block_start("payload")
17    s_dword(0x12345678, name="client_id")
18    s_word(0, name="flags")
19    s_block_end()
20
21    session.connect(s_get("HELLO"))
22    session.fuzz()
23
24if __name__ == "__main__":
25    main()

Replay and Modification

 1from scapy.all import *
 2
 3# Replay captured traffic
 4packets = rdpcap("capture.pcap")
 5for pkt in packets:
 6    if pkt.haslayer(TCP) and pkt[TCP].dport == 8888:
 7        send(pkt)
 8
 9# Modify and replay
10for pkt in packets:
11    if pkt.haslayer(Raw):
12        # Modify payload
13        original = pkt[Raw].load
14        modified = original.replace(b"client", b"CLIENT")
15        pkt[Raw].load = modified
16        # Recalculate checksums
17        del pkt[IP].chksum
18        del pkt[TCP].chksum
19        send(pkt)

Best Practices

Analysis Workflow

  1. Capture traffic: Multiple sessions, different scenarios
  2. Identify boundaries: Message start/end markers
  3. Map structure: Fixed header, variable payload
  4. Identify fields: Compare multiple samples
  5. Document format: Create specification
  6. Validate understanding: Implement parser/generator
  7. Test edge cases: Fuzzing, boundary conditions

Common Patterns to Look For

  • Magic numbers/signatures at message start
  • Version fields for compatibility
  • Length fields (often before variable data)
  • Type/opcode fields for message identification
  • Sequence numbers for ordering
  • Checksums/CRCs for integrity
  • Timestamps for timing
  • Session/connection identifiers

What Users Are Saying

Real feedback from the community

Environment Matrix

Dependencies

Wireshark 3.0+
Python 3.8+
Scapy library
tshark command-line tools

Framework Support

Scapy ✓ (recommended) Boofuzz ✓ mitmproxy ✓ Burp Suite ✓

Context Window

Token Usage ~3K-8K tokens for protocol analysis sessions

Security & Privacy

Information

Author
wshobson
Updated
2026-01-30
Category
debugging