Tuesday, April 17, 2012

Fix blue people on Flash videos without losing HW acceleration

Adobe announced that they abandon Flash on Linux and 11.2 will be the last release after which they will provide 5 years support limited to security issues only.

11.2 was released with a terrible bug: Flash Player sends U and V components in the wrong order which results in incorrect colors. They are not going to fix it - all reports are being closed as CannotReproduce or FeatureRemoved.

This bug affects only users with NVidia cards but still: I can hardly accept that such a bug slips through all the quality checks and that they refuse to fix it. Users have not much choice: earlier versions are full of security holes, the latest renders the videos unwatchable.

Since Flash Player is closed source and Adobe doesn't care, we can just workaround this problem, and libvdpau is one possible layer where we can do this. VDPAU is an open-source library which allows programs to offload portions of the video decoding process and video post-processing to the GPU video-hardware. This is the only open-source component I can think of in the chain so this will have to be abused. While this goes against all my software engineering principles here is the deal:

We modify the VDPAU trace library. It's meant for tracing VDPAU functionality and not for working around Adobe's incompetence but that's our best shot. In this library we can swap the U and V color components so they will be in the correct order. The patch is not mine, the credit goes to Pierre-Loup A. Griffais. The patch stubs out all the tracing routines and only does color swapping. If you need to trace the VDPAU functionality remember to use the unmodified version!

I applied the patch on the 10.04 (previous LTS version of Ubuntu), 11.10 (current stable version) and 12.04 (upcoming LTS release). All of them have been built on Launchpad and reside in my Personal Package Archive (PPA).

If you are reading this and eager to use the patched version you will know how to create your own PPA and copy my packages there. You can also add my PPA to your package manager (instructions on the PPA site) and upgrade the libvdpau1 package. Downloading the debs and installing manually is also an option. Be aware that my ppa has other packages which you may not want to upgrade so make sure you know what you are doing.

Having the patched VDPAU library is not enough: you have to activate it by enabling tracing. You can do so by using the VDPAU_TRACE=1 before starting your browser, e.g. you can modify the Exec line(s) in your firefox.desktop or google-chrome.desktop file to start with env VDPAU_TRACE=1

Not an easy ride. While it was possible to workaround this bug that's not always the case. Which proprietary, closed-source software comes next? Try to find the open alternatives, which, in case of flash, is using HTML5.

In the meantime, enjoy YouTube videos, on Linux, even with NVidia cards!

Update 1: I have backported the official fix for Precise (12.04) and it is in my ppa now. Other versions have been removed in order to reduce maintenance burden.

Update 2: The fixed version is in the official archive now - no need for 3rd party ppa's anymore.