Ubuntu 13.10 with TCP-IR (TCP Instant Recovery / FEC) enabled kernel v3.4.83

How would FEC (Forward Error Correction) enabled TCP/IP stack for Linux sound like? Yep, I know you're interested and want it. Start by checking these two links.
http://www.ietf.org/proceedings/87/slides/slides-87-tcpm-8.pdf
http://tools.ietf.org/html/draft-flach-tcpm-fec-00



There's patch out to implement TCP-IR with Linux kernel 3.2.36 available on https://github.com/tflach/tcp-fec/tree/linux-stable-3.2.36. Kernel version is bit dated by now, but we can apply same patch to more recent 3.4 series kernels such as 3.4.83. Anyway, exact version hardly matters for experiments like this. Go!

# Let's download and install 3.4.83 without TCP-IR support
# so we can more easily narrow cause if TCP-IR enabled one acts odd
mkdir -p /opt/kernel_3.4.83
cd /opt/kernel_3.4.83
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/linux-headers-3.4.83-030483-generic_3.4.83-030483.201403111935_amd64.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/linux-headers-3.4.83-030483_3.4.83-030483.201403111935_all.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/linux-image-3.4.83-030483-generic_3.4.83-030483.201403111935_amd64.deb
dpkg -i linux-*.deb

# Reboot now and select 3.4.83 from grub to make sure your PC works properly
# with older kernel.

# Deps for compiling kernel
apt-get -y install git-core kernel-package fakeroot build-essential ncurses-dev

# Create build environment
adduser bobbuilder --shell /bin/bash \
        --disabled-password --gecos bobbuilder

# Switch to non-priviledged user for build
sudo su - bobbuilder

# Download clean kernel tarball
mkdir -p ~/build/tcpir
cd ~/build/tcpir
wget ftp://ftp.kernel.org/pub/linux/kernel/v3.x/linux-3.4.83.tar.gz

# Download TCP-IR patch
wget https://github.com/tflach/tcp-fec/commit/9e56a3d2bab809eb36eb062c6ee3ab1f11f68565.patch

# Download Ubuntu patches for 3.4.83 kernel
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/0001-base-packaging.patch
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/0002-UBUNTU-SAUCE-highbank-export-clock-functions-for-mod.patch
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/0003-debian-changelog.patch
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.83-quantal/0004-configs-based-on-Ubuntu-END.patch

# Extract and patch with Ubuntu changes
tar xvzf linux-3.4.83.tar.gz
cd linux-3.4.83
patch -p1 <../0001-base-packaging.patch
patch -p1 <../0002-UBUNTU-SAUCE-highbank-export-clock-functions-for-mod.patch
patch -p1 <../0003-debian-changelog.patch
patch -p1 <../0004-configs-based-on-Ubuntu-END.patch

# Apply TCP-IR patch
patch -p1 -F3 <../9e56a3d2bab809eb36eb062c6ee3ab1f11f68565.patch

# Following hack is to enable TCP-IR by default without requiring
# patching and recompiling of all applications with new TCP_FEC sockopt.
cat<<'_EOF_'|patch -p0 -l
--- net/ipv4/tcp_output.c~      2014-03-20 01:04:36.000000000 +0200
+++ net/ipv4/tcp_output.c       2014-03-20 23:55:35.263873726 +0200
@@ -2639,6 +2639,9 @@
        struct tcp_sock *tp = tcp_sk(sk);
        __u8 rcv_wscale;
 
+        // Try to use TCP-IR if /proc/net/ipv4/tcp_fec is 1 (non-interleaved) or 2 (interleaved)
+        if (sysctl_tcp_fec >= 1) tp->fec.type = sysctl_tcp_fec;
+                
        /* We'll fix this up when we get a response from the other end.
         * See tcp_input.c:tcp_rcv_state_process case TCP_SYN_SENT.
         */
_EOF_

# Use default ubuntu kernel config for our custom kernel
cp /boot/config-3.4.83-030483-generic .config
yes "" | make oldconfig

# Compile kernel and generate new dpkgs
CONCURRENCY_LEVEL=16 fakeroot make-kpkg \
  --initrd --append-to-version=-tcpir kernel_image kernel_headers

# Optional: Hack to enable TCP-IR debug printk and compile another kernel 
# package. This WILL flood your logs but will also let you easily see what
# TCP-IR does.
sed -i.bak Makefile -e's/-Wall/-Wall -DFEC_DEBUG/g'
CONCURRENCY_LEVEL=16 fakeroot make-kpkg \
  --initrd --append-to-version=-tcpirdebug kernel_image kernel_headers

# Rest as root

# Make sure ohci-pci USB driver is loaded during initramfs
echo "ohci-pci" >>/etc/initramfs-tools/modules

# Install new kernel
cd /home/bobbuilder/build/tcpir
dpkg -i linux-image-3.4.83-tcpir_3.4.83-tcpir-10.00.Custom_amd64.deb \
        linux-headers-3.4.83-tcpir_3.4.83-tcpir-10.00.Custom_amd64.deb
 
# Prevent future kernel updates breaking things
echo "linux-image-3.4.83-tcpir hold"            | dpkg --set-selections
echo "linux-headers-3.4.83-tcpir hold"          | dpkg --set-selections
echo "linux-image-3.4.83-030483-generic hold"   | dpkg --set-selections
echo "linux-headers-3.4.83-030483 hold"         | dpkg --set-selections
echo "linux-headers-3.4.83-030483-generic hold" | dpkg --set-selections
 
# For new Ubuntu versions with silly grub submenu layout
# https://help.ubuntu.com/community/Grub2/Submenus#Setting_a_Submenu_entry_as_the_default
sed -i.bak /etc/default/grub \
    -e's|^GRUB_DEFAULT=.*|GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.4.83-tcpir"|g'
 
# For older versions such as 12.04
sed -i.bak /etc/default/grub \
    -e's|^GRUB_DEFAULT=.*|GRUB_DEFAULT="Ubuntu, with Linux 3.4.83-tcpir"|g'
 
# and run update-grub
update-grub
 
# Reboot

And that's about it. All outgoing TCP connections will use TCP-IR if remote end supports it. Which is of course extremely unlikely unless it's host under your control with custom kernel. :) You can turn TCP-IR off and toggle between regular and interleaved mode with sysctl. Just make sure to use same mode on remote side for FEC to work.

# Disable TCP-IR
sysctl net.ipv4.tcp_fec=0

# Regular mode
sysctl net.ipv4.tcp_fec=1

# Interleaved mode
sysctl net.ipv4.tcp_fec=2

So we have error correcting TCP stack now and any TCP connections to another TCP-IR enabled host will be FEC enabled. But is it any good? It does indeed repair most missing packets without retransmissions, but like it said on that presentation linked on top of this article it's no good for long duration TCP connections. Perhaps bit surprisingly regular TCP retransmission logic does better, especially on lossy high latency links. In best cases with TCP-IR I've got around half of regular Linux TCP performance.