(Ab)using OpenVPN to make two unrealiable Internet connections act asone reliable connection (Failure #2)
We mirror our hard disks, we mirror RAM on servers. Why not mirror network traffic as well?
Next I tried to send packets back to self, mark them and nat with different outgoing addresses. Like previous attempt with tc even if this had worked it's horrible hack and there's still major problem of how to handle packets on server side. Especially how to duplicate return packets when client side IP addresses are dynamic.
Create un-numbered IPIP tunnel via loopback interface. You can create more than one as long as local and remote IP addresses are unique. Also adds route to 172.31.31.31/32 via ipip10, our outgoing IPIP dummy interface.
- Create MAKEDUPE iptables chain. Logs packets before and after processing.
- TOS flag is abused as "mark". This was part of my troubleshooting procedure but it turned out to fail exactly same way with "-j MARK --set-mark" and "-j TOS --set-tos".
- In short this sends copy of outgoing packets without TOS 0x08 to IP 172.31.31.31
- Since 172.31.31.31 is looped back to us we receive same packet back for further processing.
First ping packet is lost with 'Operation not permitted' error for some reason.
# With tcpdump we can see 192.168.0.8 source address used for both packets that have been cloned (tos 0x8, ttl 63) and original packets (tos 0x10, ttl 64). Those original packets should have 192.168.0.10 based on iptables rules.
But kernel log claims packets were properly processed by TOS08NAT and TOS10NAT. Weird.
That should be enough failures for you. In next episode you'll see how to make it actually work.
Next I tried to send packets back to self, mark them and nat with different outgoing addresses. Like previous attempt with tc even if this had worked it's horrible hack and there's still major problem of how to handle packets on server side. Especially how to duplicate return packets when client side IP addresses are dynamic.
2. Using iptables with ipip-tunnel over loopback interface
Start by allowing "send to self" i.e. don't drop packets with our own IP as source. Add these lines to /etc/sysctl.confnet.ipv4.ip_forward=1 net.ipv6.conf.all.disable_ipv6=1 net.ipv4.conf.all.log_martians=1 net.ipv4.conf.default.rp_filter=0 net.ipv4.conf.all.rp_filter=0 net.ipv4.conf.default.accept_local=1 net.ipv4.conf.all.accept_local=1
ip tunnel add ipip10 mode gre local 127.10.0.10 remote 127.10.0.11 ttl 255 ip link set ipip10 up ip tunnel add ipip11 mode gre local 127.10.0.11 remote 127.10.0.10 ttl 255 ip link set ipip11 up ip route add 172.31.31.31/32 dev ipip10 ip route flush cache
- TOS flag is abused as "mark". This was part of my troubleshooting procedure but it turned out to fail exactly same way with "-j MARK --set-mark" and "-j TOS --set-tos".
- In short this sends copy of outgoing packets without TOS 0x08 to IP 172.31.31.31
- Since 172.31.31.31 is looped back to us we receive same packet back for further processing.
# MAKEDUPE chain, logs packet before, during and after processing # tos flag is used to differentiate original and cloned packet # copy of packet with tos 0x08 sent to 172.31.31.31 which is then received back for processing from ipip11 iptables -t mangle -N MAKEDUPE iptables -t mangle -F MAKEDUPE iptables -t mangle -A MAKEDUPE -j LOG --log-prefix 'MAKEDUPE Input: ' --log-level info iptables -t mangle -A MAKEDUPE -j TOS --set-tos 0x08 iptables -t mangle -A MAKEDUPE -j LOG --log-prefix 'MAKEDUPE Clone: ' --log-level info iptables -t mangle -A MAKEDUPE -j TEE --gateway 172.31.31.31 iptables -t mangle -A MAKEDUPE -j TOS --set-tos 0x10 iptables -t mangle -A MAKEDUPE -j LOG --log-prefix 'MAKEDUPE Output: ' --log-level info # Send outgoing packets to MAKEDUPE chain unless packet is clone with tos 0x08 iptables -t mangle -A OUTPUT -d 172.16.0.1 -o eth0 -m tos \! --tos 0x08 -j MAKEDUPE # NAT rules for original packet iptables -t nat -N TOS10NAT iptables -t nat -F TOS10NAT iptables -t nat -A TOS10NAT -j LOG --log-prefix 'TOS10NAT: ' --log-level info iptables -t nat -A TOS10NAT -j SNAT --to-source 192.168.0.10 # NAT rules for cloned packet iptables -t nat -N TOS08NAT iptables -t nat -F TOS08NAT iptables -t nat -A TOS08NAT -j LOG --log-prefix 'TOS08NAT: ' --log-level info iptables -t nat -A TOS08NAT -j SNAT --to-source 192.168.0.8 # Call proper NAT chain depending on tos value iptables -A POSTROUTING -t nat -d 172.16.0.1 -m tos --tos 0x08 -g TOS08NAT iptables -A POSTROUTING -t nat -d 172.16.0.1 -m tos --tos 0x10 -g TOS10NATBut as I said this fails. What happens is that packets are always natted with TOS08NAT rule. Could be something with my iptables rules but result is it doesn't work as intended. See below for tcpdump capture and kernel log showing how it fails.
First ping packet is lost with 'Operation not permitted' error for some reason.
root@VPN-Client:~# ping 172.16.0.1 PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data. ping: sendmsg: Operation not permitted ^C --- 172.16.0.1 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 3012ms
# With tcpdump we can see 192.168.0.8 source address used for both packets that have been cloned (tos 0x8, ttl 63) and original packets (tos 0x10, ttl 64). Those original packets should have 192.168.0.10 based on iptables rules.
root@VPN-Client:~# tcpdump -vv -nn -i eth0 icmp tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 14:25:55.142008 IP (tos 0x8, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 1, length 64 14:25:56.146343 IP (tos 0x8, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 2, length 64 14:25:56.146565 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 2, length 64 14:25:57.146178 IP (tos 0x8, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 3, length 64 14:25:57.146371 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 3, length 64 14:25:58.146088 IP (tos 0x8, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 4, length 64 14:25:58.146287 IP (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.0.8 > 172.16.0.1: ICMP echo request, id 1340, seq 4, length 64
But kernel log claims packets were properly processed by TOS08NAT and TOS10NAT. Weird.
[ 116.326987] device eth0 entered promiscuous mode [ 120.524492] MAKEDUPE Input: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=1 [ 120.524742] MAKEDUPE Clone: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x08 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=1 [ 120.527406] MAKEDUPE Output: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=1 [ 120.529129] TOS08NAT: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x08 PREC=0x00 TTL=63 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=1 [ 120.531472] TOS10NAT: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=1 [ 121.533004] MAKEDUPE Input: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=2 [ 121.533026] MAKEDUPE Clone: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x08 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=2 [ 121.533087] MAKEDUPE Output: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=2 [ 122.531224] MAKEDUPE Input: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=3 [ 122.531240] MAKEDUPE Clone: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x08 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=3 [ 122.531277] MAKEDUPE Output: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=3 [ 123.529486] MAKEDUPE Input: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=4 [ 123.529502] MAKEDUPE Clone: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x08 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=4 [ 123.529537] MAKEDUPE Output: IN= OUT=eth0 SRC=192.168.224.21 DST=172.16.0.1 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=1340 SEQ=4 [ 129.689781] device eth0 left promiscuous mode
That should be enough failures for you. In next episode you'll see how to make it actually work.
Comments
Post a Comment
Got something to say?!