Setting up XBMCbuntu 12 based HTPC - Part 5 - Recompile XBMC

Now it's getting more interesting. XBMC is notoriously bad in buffering content. There's various patches floating around but none of them really does what I'm looking for:

1. Buffer entire file or at least most of it on background, not just 10 seconds or so ahead of playback.
2. Don't be too conservative with bandwidth use.
3. If there's plenty of free memory or disk use it for caching currently playing media.

#1 is easy, it's just one extra line in XBMC advancedsettings.xml (cachemembuffersize).

#2 requires patching XBMC source and recompiling it. Sorry, no way around this. Have fun, it's going to take quite a while on measly HTPC with Atom. If precise bandwidth capping is required to prevent other uses of same Internet connection from suffering I'll do it elsewhere. How about OpenWrt router or XBMCbuntu HTPC itself using built-in Linux features?

Remember to mention about using patched binaries if you ever need to request support. Even better, revert to stock binaries first, reproduce problem and only then file support ticket, do forum post or call your 84 year old granny for help.

#3 can also be solved by OS without changes to XBMC code. Simply use tmpfs for XBMC temp directory. XBMC thinks it's writing to disk while in reality it's ramdisk backed with SSD. Gotcha!

Lets start with #2 fix as #1 will be covered on my next post. Follow closely instructions shown below - as usual.

# Recompiling XBMC on XBMCbuntu 12 RC3 is quite easy. 
# Perhaps not as easy as it should be, but close enough.

# Fix broken configuration file
# Without this you'll get old xbmc version from Ubuntu 12.10
echo "deb-src http://ppa.launchpad.net/team-xbmc/unstable/ubuntu quantal main" >> /etc/apt/sources.list.d/xbmc.list

# Add PPA containing build depends
# Should be included by default in my opinion but devs seem to disagree
add-apt-repository -y ppa:team-xbmc/xbmc-ppa-build-depends

# Add missing symlink, without this compilation will fail with
# "/usr/bin/ld.bfd.real: cannot find -lcurl-gnutls" error.
ln -sf /usr/lib/i386-linux-gnu/libcurl-gnutls.so.4 \
       /usr/lib/i386-linux-gnu/libcurl-gnutls.so

# Refresh package list
apt-get update

# Install rest of xbmc build dependencies
apt-get -y build-dep xbmc

# Grab source package
mkdir -p /opt/xbmcbuild
cd /opt/xbmcbuild
apt-get source xbmc

# Download and apply buffering patch
wget http://dump.asiantuntijakaveri.fi/le_bueno_dumpo/a46ea2b2807f57f3780156bcfae6ffc6afd53cd1.patch
# or
# wget https://github.com/classicspam/xbmc/commit/a46ea2b2807f57f3780156bcfae6ffc6afd53cd1.patch
cd xbmc-*
patch -p1 -l <../a46ea2b2807f57f3780156bcfae6ffc6afd53cd1.patch 

# Make few changes to new caching logic patch applied 
# Buffer at 4 * bitrate instead of default 1.25 * bitrate 
# Maximum download rate 60Mbit/s instead of 40Mbit/s 
# Maximum memory cache size 2GB instead of 1GB 
# Forward/backward memory cache split 80%/20% instead of 75%/25% 
# Disk cache size is limited by free disk space 
sed -i.bak xbmc/cores/dvdplayer/DVDPlayer.cpp \
     -e's|\* 1.25|\* 4|g' -e's|40000000|60000000|g'
sed -i.bak xbmc/filesystem/FileCache.cpp \
     -e's|1024 \* 1000|1024 \* 2000|g' -e's|\* 0\.75|\* 0\.80|g' 

# Fix problem with first seek on mkv playback clearing
# cache buffer content.
sed -i.bak ./xbmc/cores/dvdplayer/DVDPlayer.cpp \
     -e's|if(starttime > 0)|if(starttime >= 0)|g'

# Disable unnecessary refresh rate changes when stopping playback
cat <<'__EOF__' | patch -l -p0
--- xbmc/guilib/GraphicContext.cpp    2012-11-18 16:15:06.000000000 +0200
+++ xbmc/guilib/GraphicContext.cpp       2013-01-06 01:16:49.666906377 +0200
@@ -281,10 +281,10 @@
     bool allowDesktopRes = g_guiSettings.GetInt("videoplayer.adjustrefreshrate") == ADJUST_REFRESHRATE_ALWAYS;
     if(m_bFullScreenVideo || (!allowDesktopRes && g_application.IsPlayingVideo()))
       SetVideoResolution(g_renderManager.GetResolution());
-    else if(g_guiSettings.m_LookAndFeelResolution > RES_DESKTOP)
-      SetVideoResolution(g_guiSettings.m_LookAndFeelResolution);
-    else
-      SetVideoResolution(RES_DESKTOP);
+    //else if(g_guiSettings.m_LookAndFeelResolution > RES_DESKTOP)
+    //  SetVideoResolution(g_guiSettings.m_LookAndFeelResolution);
+    //else
+    //  SetVideoResolution(RES_DESKTOP);
   }
   else
     SetVideoResolution(RES_WINDOW);
__EOF__

# Build new binary
dpkg-buildpackage -j5 -b -us -uc -tc

# Install newly built binary
cd ..
dpkg -i xbmc_12*.deb xbmc-bin_12*.deb

# Prevent updates from repository from overwriting our version
echo "xbmc hold" | dpkg --set-selections
echo "xbmc-bin hold" | dpkg --set-selections

You can download precompiled binaries with above patches applied from my website http://dump.asiantuntijakaveri.fi/le_bueno_dumpo/frodorc3/.

# Skip instructions below in case you compiled and installed
# your own patched binaries.

# Download hacked binaries
mkdir -p /opt/xbmchack
cd /opt/xbmchack
wget -r -np -nd http://dump.asiantuntijakaveri.fi/le_bueno_dumpo/frodorc3/

# Install
dpkg -i xbmc_12*.deb xbmc-bin_12*.deb

# Prevent updates from repository from overwriting our version
echo "xbmc hold" | dpkg --set-selections
echo "xbmc-bin hold" | dpkg --set-selections

Then #3. If you have slow Internet and limited amount of memory on HTPC you can use disk backed ramdisk instead. Example below uses 25GB swap. If you're watching high bitrate HD streams or extremely big MKV files you can run out of memory and crash XBMC with this. I think it's rather unlikely scenario but still possible so be aware of this. At 4Mbit/s bitrate 25GB is enough for around 16 hours. Stopping playback will clear cache instantly but pausing won't.

You can skip everything below if memory caching is enough for your needs. It's entirely up yours. Caching to disk will work just fine without tmpfs layer between so even with disk caching this part isn't mandatory.

# Create 25GB file to be used for swap on root of SSD
dd if=/dev/zero of=/swapfile bs=1G count=25

# Fix permissions and format it as swap
chmod 0600 /swapfile
mkswap /swapfile

# Enable both extra swap and tmpfs on boot
mkdir -p /home/xbmc/.xbmc/temp
echo "tmpfs /home/xbmc/.xbmc/temp tmpfs rw,size=25G,mode=0777,nr_inodes=10k" >>/etc/fstab
echo "/swapfile none swap sw 0 0" >>/etc/fstab