Skip to content

Instantly share code, notes, and snippets.

@patocarr

patocarr/TCPdump on ICMP

Last active February 6, 2017 21:30
Show Gist options
  • Save patocarr/81d549debbc10077ed71fcf892863824 to your computer and use it in GitHub Desktop.
Save patocarr/81d549debbc10077ed71fcf892863824 to your computer and use it in GitHub Desktop.
Download ZIP
Multihomed gateway using iptables & iproute
Raw
ip route
Main
default via 21.226.0.113 dev wwp1s0u1i2
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1
10.0.2.0/24 dev wlp2s0 proto kernel scope link src 10.0.2.1
10.0.3.0/24 dev wlp2s0_1 proto kernel scope link src 10.0.3.1
10.0.4.0/24 dev wlp3s0 proto kernel scope link src 10.0.4.1
10.0.5.0/24 dev wlp3s0_1 proto kernel scope link src 10.0.5.1
10.8.0.1 dev tun0 proto kernel scope link src 10.8.0.108
10.208.236.164/30 dev wwp1s0u2i2 proto kernel scope link src 10.208.236.164
21.226.0.112/29 dev wwp1s0u1i2 proto kernel scope link src 21.226.0.115
100.75.166.144/29 dev wwp1s0u3i2 proto kernel scope link src 100.75.166.147
169.254.0.0/16 dev enp4s0 scope link metric 1004
Table 100
default via 21.226.0.113 dev wwp1s0u1i2 proto static
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1
10.0.2.0/24 dev wlp2s0 proto kernel scope link src 10.0.2.1
10.0.3.0/24 dev wlp2s0_1 proto kernel scope link src 10.0.3.1
10.0.4.0/24 dev wlp3s0 proto kernel scope link src 10.0.4.1
10.0.5.0/24 dev wlp3s0_1 proto kernel scope link src 10.0.5.1
21.226.0.112/29 dev wwp1s0u1i2 proto kernel scope link src 21.226.0.115
Table 101
default via 10.208.236.165 dev wwp1s0u2i2 proto static
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1
10.0.2.0/24 dev wlp2s0 proto kernel scope link src 10.0.2.1
10.0.3.0/24 dev wlp2s0_1 proto kernel scope link src 10.0.3.1
10.0.4.0/24 dev wlp3s0 proto kernel scope link src 10.0.4.1
10.0.5.0/24 dev wlp3s0_1 proto kernel scope link src 10.0.5.1
10.208.236.164/30 dev wwp1s0u2i2 proto kernel scope link src 10.208.236.164
Table 102
default via 100.75.166.145 dev wwp1s0u3i2 proto static
10.0.1.0/24 dev enp4s0 proto kernel scope link src 10.0.1.1
10.0.2.0/24 dev wlp2s0 proto kernel scope link src 10.0.2.1
10.0.3.0/24 dev wlp2s0_1 proto kernel scope link src 10.0.3.1
10.0.4.0/24 dev wlp3s0 proto kernel scope link src 10.0.4.1
10.0.5.0/24 dev wlp3s0_1 proto kernel scope link src 10.0.5.1
100.75.166.144/29 dev wwp1s0u3i2 proto kernel scope link src 100.75.166.147
Raw
ip rule
0: from all lookup local
31900: from 21.226.0.115 lookup rt_link1
31901: from 10.208.236.164 lookup rt_link2
31902: from 100.75.166.147 lookup rt_link3
32000: from all fwmark 0x1 lookup rt_link1
32001: from all fwmark 0x2 lookup rt_link2
32002: from all fwmark 0x3 lookup rt_link3
32766: from all lookup main
32767: from all lookup default
Raw
iptables
The idea is to load-balance outgoing traffic over the 3 WAN gateways (ie. wwp1s0u?i2) using the BALANCE target.
PREROUTING and OUTPUT try to maintain the connections on their respective gateway by restoring the mark, set originally on NEW.
PREBALANCE is for NEW packets and tries to filter traffic not intended to be balanced, such as local traffic and without a mark.
BALANCE is flushed and modified from python with different statistical weights to balance the gateways. Depending on which rule is hit based on the weights, a mark value is set corresponding to a gateway.
SETMARK sets the mark on the connection, and routes traffic based on the mark rules which split it in 3 tables.
SAVEMARK saves the set mark for next packet in connection, and ACCEPTs the packet.
# Generated by iptables-save v1.4.21 on Sun Feb 5 18:20:55 2017
*raw
:PREROUTING ACCEPT [85125:8624307]
:OUTPUT ACCEPT [58815:9395247]
-A PREROUTING -p udp -j TRACE
-A OUTPUT -p udp -j TRACE
COMMIT
# Completed on Sun Feb 5 18:20:55 2017
# Generated by iptables-save v1.4.21 on Sun Feb 5 18:20:55 2017
*mangle
:PREROUTING ACCEPT [67792:6879346]
:INPUT ACCEPT [50234:4026196]
:FORWARD ACCEPT [14030:2444745]
:OUTPUT ACCEPT [49190:4512575]
:POSTROUTING ACCEPT [64092:7049287]
:BALANCE - [0:0]
:PREBALANCE - [0:0]
:SAVEMARK - [0:0]
:SETMARK1 - [0:0]
:SETMARK2 - [0:0]
:SETMARK3 - [0:0]
-A PREROUTING -m conntrack --ctstate RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -i wwp1s0u1i2 -m conntrack --ctstate NEW -j SETMARK1
-A PREROUTING -i wwp1s0u2i2 -m conntrack --ctstate NEW -j SETMARK2
-A PREROUTING -i wwp1s0u3i2 -m conntrack --ctstate NEW -j SETMARK3
-A PREROUTING -s 10.0.0.0/16 -m conntrack --ctstate NEW -j PREBALANCE
-A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A OUTPUT -o wwp1s0u1i2 -m conntrack --ctstate NEW -j SETMARK1
-A OUTPUT -o wwp1s0u2i2 -m conntrack --ctstate NEW -j SETMARK2
-A OUTPUT -o wwp1s0u3i2 -m conntrack --ctstate NEW -j SETMARK3
-A OUTPUT -s 10.0.0.0/16 -m conntrack --ctstate NEW -j PREBALANCE
-A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1300
-A BALANCE -m statistic --mode random --probability 0.33000000007 -g SETMARK1
-A BALANCE -m statistic --mode random --probability 0.33000000007 -g SETMARK2
-A BALANCE -m statistic --mode random --probability 0.33000000007 -g SETMARK3
-A PREBALANCE -d 10.0.0.0/16 -j RETURN
-A PREBALANCE -m connmark ! --mark 0x0 -j RETURN
-A PREBALANCE -m conntrack --ctstate RELATED,ESTABLISHED -j RETURN
-A PREBALANCE -j BALANCE
-A SAVEMARK -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A SAVEMARK -j LOG --log-prefix "[netfilter] SAVEMARK " --log-level 6
-A SETMARK1 -j CONNMARK --set-xmark 0x1/0xffffffff
-A SETMARK1 -j MARK --set-xmark 0x1/0xffffffff
-A SETMARK1 -j SAVEMARK
-A SETMARK2 -j CONNMARK --set-xmark 0x2/0xffffffff
-A SETMARK2 -j MARK --set-xmark 0x2/0xffffffff
-A SETMARK2 -j SAVEMARK
-A SETMARK3 -j CONNMARK --set-xmark 0x3/0xffffffff
-A SETMARK3 -j MARK --set-xmark 0x3/0xffffffff
-A SETMARK3 -j SAVEMARK
COMMIT
# Completed on Sun Feb 5 18:20:55 2017
# Generated by iptables-save v1.4.21 on Sun Feb 5 18:20:55 2017
*nat
:PREROUTING ACCEPT [3533:273855]
:INPUT ACCEPT [507:72474]
:OUTPUT ACCEPT [1344:121877]
:POSTROUTING ACCEPT [41:8100]
-A POSTROUTING -o wwp1s0u1i2 -j MASQUERADE
-A POSTROUTING -o wwp1s0u2i2 -j MASQUERADE
-A POSTROUTING -o wwp1s0u3i2 -j MASQUERADE
COMMIT
# Completed on Sun Feb 5 18:20:55 2017
# Generated by iptables-save v1.4.21 on Sun Feb 5 18:20:55 2017
*filter
:INPUT ACCEPT [56476:4418933]
:FORWARD ACCEPT [21104:3389287]
:OUTPUT ACCEPT [58816:9395331]
COMMIT
# Completed on Sun Feb 5 18:20:55 2017
Raw
TCPdump on ICMP
12:04:04.367369 IP (tos 0x0, ttl 128, id 18024, offset 0, flags [none], proto ICMP (1), length 60)
10.0.1.103 > 216.58.219.46: ICMP echo request, id 1, seq 122, length 40
12:04:04.368241 IP (tos 0x0, ttl 127, id 18024, offset 0, flags [none], proto ICMP (1), length 60)
22.3.60.33 > 216.58.219.46: ICMP echo request, id 1, seq 122, length 40
12:04:05.371982 IP (tos 0x0, ttl 54, id 0, offset 0, flags [none], proto ICMP (1), length 60)
216.58.219.46 > 22.3.60.33: ICMP echo reply, id 1, seq 122, length 40
12:04:06.572694 IP (tos 0xd8, ttl 64, id 19175, offset 0, flags [none], proto ICMP (1), length 109)
10.208.236.164 > 8.8.8.8: ICMP 10.208.236.164 udp port 64929 unreachable, length 89
IP (tos 0x58, ttl 56, id 18792, offset 0, flags [none], proto UDP (17), length 81)
8.8.8.8.53 > 10.208.236.164.64929: UDP, length 53
12:04:07.042543 IP (tos 0xd8, ttl 64, id 19345, offset 0, flags [none], proto ICMP (1), length 270)
10.208.236.164 > 8.8.8.8: ICMP 10.208.236.164 udp port 52723 unreachable, length 250
IP (tos 0x58, ttl 56, id 19244, offset 0, flags [none], proto UDP (17), length 242)
8.8.8.8.53 > 10.208.236.164.52723: UDP, length 214
12:04:09.310171 IP (tos 0x0, ttl 128, id 18027, offset 0, flags [none], proto ICMP (1), length 60)
10.0.1.103 > 216.58.219.46: ICMP echo request, id 1, seq 123, length 40
12:04:09.310436 IP (tos 0x0, ttl 127, id 18027, offset 0, flags [none], proto ICMP (1), length 60)
22.3.60.33 > 216.58.219.46: ICMP echo request, id 1, seq 123, length 40
12:04:14.309592 IP (tos 0x0, ttl 128, id 18028, offset 0, flags [none], proto ICMP (1), length 60)
10.0.1.103 > 216.58.219.46: ICMP echo request, id 1, seq 124, length 40
12:04:14.310299 IP (tos 0x0, ttl 127, id 18028, offset 0, flags [none], proto ICMP (1), length 60)
22.3.60.33 > 216.58.219.46: ICMP echo request, id 1, seq 124, length 40
12:04:14.711896 IP (tos 0xc0, ttl 64, id 39163, offset 0, flags [none], proto ICMP (1), length 181)
22.3.60.33 > 8.8.8.8: ICMP 22.3.60.33 udp port 49375 unreachable, length 161
IP (tos 0x0, ttl 58, id 20828, offset 0, flags [none], proto UDP (17), length 153)
8.8.8.8.53 > 22.3.60.33.49375: UDP, length 125
12:04:14.713411 IP (tos 0x0, ttl 54, id 0, offset 0, flags [none], proto ICMP (1), length 60)
216.58.219.46 > 22.3.60.33: ICMP echo reply, id 1, seq 123, length 40
12:04:14.713412 IP (tos 0x0, ttl 54, id 0, offset 0, flags [none], proto ICMP (1), length 60)
216.58.219.46 > 22.3.60.33: ICMP echo reply, id 1, seq 124, length 40
12:04:19.308978 IP (tos 0x0, ttl 128, id 18029, offset 0, flags [none], proto ICMP (1), length 60)
10.0.1.103 > 216.58.219.46: ICMP echo request, id 1, seq 125, length 40
12:04:19.309471 IP (tos 0x0, ttl 127, id 18029, offset 0, flags [none], proto ICMP (1), length 60)
22.3.60.33 > 216.58.219.46: ICMP echo request, id 1, seq 125, length 40
12:04:24.073583 IP (tos 0x0, ttl 54, id 0, offset 0, flags [none], proto ICMP (1), length 60)
216.58.219.46 > 22.3.60.33: ICMP echo reply, id 1, seq 125, length 40
12:04:24.075711 IP (tos 0xc0, ttl 64, id 47510, offset 0, flags [none], proto ICMP (1), length 133)
22.3.60.33 > 8.8.8.8: ICMP 22.3.60.33 udp port 64852 unreachable, length 113
IP (tos 0x0, ttl 58, id 26887, offset 0, flags [none], proto UDP (17), length 105)
8.8.8.8.53 > 22.3.60.33.64852: UDP, length 77
12:04:24.077416 IP (tos 0xc0, ttl 64, id 47511, offset 0, flags [none], proto ICMP (1), length 139)
22.3.60.33 > 8.8.8.8: ICMP 22.3.60.33 udp port 57273 unreachable, length 119
IP (tos 0x0, ttl 58, id 56481, offset 0, flags [none], proto UDP (17), length 111)
Raw
Trace - ping from LAN client
Feb 6 12:04:04 localhost kernel: TRACE: raw:PREROUTING:policy:2 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:PREROUTING:rule:5 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:PREBALANCE:rule:4 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:BALANCE:rule:1 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK1:rule:1 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK1:rule:2 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK1:rule:3 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:rule:1 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:rule:2 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:return:3 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK1:return:4 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:BALANCE:rule:3 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK3:rule:3 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:rule:1 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:rule:2 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SAVEMARK:return:3 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:SETMARK3:return:4 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:BALANCE:return:4 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:PREBALANCE:return:5 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:PREROUTING:policy:6 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: nat:PREROUTING:policy:1 IN=enp4s0 OUT= MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: mangle:FORWARD:policy:1 IN=enp4s0 OUT=wwp1s0u1i2 MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Feb 6 12:04:04 localhost kernel: TRACE: filter:FORWARD:policy:1 IN=enp4s0 OUT=wwp1s0u1i2 MAC=00:30:64:14:a4:69:00:05:1b:b0:e0:1e:08:00 SRC=10.0.1.103 DST=216.58.219.46 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=18024 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=122 MARK=0x1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment